Conor Cruise O'Brien (1917-2008) was an Irish politician and author of histories who took a critical look at Thomas Jefferson and came to the conclusion that Jefferson was anything but a suitable icon for the cause of racial tolerance.
From Conor Cruise O'Brien, The Long Affair: Thomas Jefferson and the French Revolution, 1785-1800 (University of Chicago Press, 1996):
The factor, however, that is bound eventually to eliminate a personal cult of Thomas Jefferson from the civil religion of a multiracial America is not his record in relation to slaves and slavery, but the policy laid down by him in relation to "free Negroes." Jefferson's vision of the future America -- after the hypothetical abolition of slavery by the slaveowners themselves -- is a lily-white one. All the ex-slaves are to be deported to Africa. In the meantime, free blacks have to be eliminated from Virginia. Jefferson's proposals for their elimination were too draconian to be stomached even by his fellow slaveowners (above, chapter 7). His proposed (and rejected) amendments to the Virginian legal code included a recommendation for the penalization of what Virginian slaveowners called "miscegenation": by which they always mean sexual intercourse between black men and white women, never between white men and black women, an event of frequent but unmentionable occurrence. Jefferson made provision for the case of a white woman who might bear a mulatto child. Both the mother and her child were to leave Virginia, immediately after the birth. In the event of their failure to do so, mother and child were declared to be "beyond the protection of the law." In the circumstances, that proposition was a license for lynching: for the physical destruction of mother and child by any Virginian who might care to do the job. Volunteers would not be lacking.
Mr. O'Brien makes the legitimate point that there has been a double standard for males and females in regard to miscegenation, but there are good biological reasons for that, including the fact that when a Negress bears a mixed-race child, it is the Black, not the White gene-pool, that is thus corrupted. This too can have bad consequences for Whites as the differences between the races become less stark, and as Blacks are provided with relatively intelligent mixed-race leaders like Frederick Douglas and Booker T. Washington, but that is a longterm and indirect danger, whereas the effect of introducing Negro genes into the White gene-pool is direct ruination.