Nothing tells like being contemptuous,” Charles Francis Adams Jr. once counseled his brother Henry. The grandson and great-grandson of presidents, Adams lived by that curious maxim his entire life. Although not the oldest male of his generation, Charles Francis Jr. was the child his father, politician and diplomat Charles Francis Sr., “lean[ed] upon the most.” As America’s first political dynasty, the Adamses were expected—both by the nation and by their forefathers—to step forward selflessly and serve their country. But in the years after John Quincy Adams’s death in 1848, his grandchildren began to exhibit both the best and worst of the family’s traits, and while talented and highly educated, they all grew to dislike themselves, detest one another, and loathe their lineage. Late in life, Charles Francis Jr. even condemned his father in print, deriding him as “a cold man outwardly.”

When in early 1861, Charles Francis Sr. was tapped by President-elect Abraham Lincoln to serve as his minister to Great Britain, the elder Adams took his third son, Henry, with him to London, leaving Charles Francis Jr. behind to manage the family’s investments. But after the debacle at Bull Run in Virginia, young Adams believed it was his duty to enlist. With the brutal honesty so typical of his family, Adams admitted to his diary that the army was not his “vocation; and that, in deserting the law for it,” he was abandoning a profession to which he was “little adapted for one to which I was adapted even less.” Nevertheless, on November 26, Charles Francis sat down to write his father. “I don’t know whether you will be surprised or disgusted or annoyed or distressed by the information that I have gone into the army,” he observed, “but such is the fact.” Generations of Adamses had built America, and “it cannot be otherwise right for me to fight to maintain that which my ancestors passed their whole lives in establishing.” How wrong it would be were he to sit “idle while history is to be written,” he noted. The decision was final: “You know it now and I am glad of it!”

When the news arrived in London, the horrified minister simply declined to reply. Instead, he poured his emotions into his journal: “My son Charles after long doubt and hesitation has at last accepted a commission as an officer in the cavalry regiment now forming in Massachusetts,” adding that he feared Charles Francis had “thrown himself away simply from family pride.” Because “none of his predecessors have been soldiers,” Adams wondered, “why should he?” But he finished his diary entry that evening with a blessing his son longed to hear yet never would: “As he has decided upon high grounds of duty I am content to abide by it. God protect him in the midst of this agony.”

Commissioned as a first lieutenant in the First Massachusetts Volunteer Cavalry in late December 1861, Adams was promoted to captain a year later. He fought with distinction at the June 1863 Battle of Aldie, Virginia. “What with men shot down and horses wounded and plunging, my ranks were disordered,” Adams marveled. “How and why I escaped I can’t say, for my men fell all around me.”

In September 1864, Massachusetts Governor John Andrew offered Adams the second place in the newly formed Fifth Regiment Volunteer Cavalry, the first U.S. black cavalry unit, and after the regiment’s original commander, Henry Russell, was wounded in battle, Adams advanced to the rank of colonel. But where other northern commanders of black regiments, such as Colonel Robert Gould Shaw, quickly shed themselves of their genteel racism, Adams never had a charitable word to say about the men under his command. In endless letters to his brothers—but never to his father—the grandson of John Quincy Adams habitually used the crudest racial epithets as synonymous with “negro” or “colored,” and his tone suggested disdain for his troopers’ working-class origins as well as their pigmentation. “I’m sick and out of sorts and hearts,” he told John Quincy II on one occasion, as “the n_____s bore me.” Aware that a majority of his cavalrymen had begun life in bondage, Adams even found humor in the fact that he was now in charge of so valuable a commodity. “N_____r driving seems to be a good business, and I think I shall turn my training here to practical advantage on a small well-stocked plantation as soon as this cruel war is over,” he joked. “These men are worth any day a thousand dollars apiece right along. Am I fit to be trusted with the care of $1,200,000 in n_____s alone?” Although one of his corporals was Charles Douglass, Adams had no interest in getting to know his junior officers, and he evidently never discovered that he served with the son of Frederick Douglass, America’s most famous black abolitionist.

After leading the Fifth into Richmond in early April 1865, Adams briefly thought better of his men, for once forgoing racist terms when describing the moment to his family. “What a piece of luck it was,” Adams mused. “That I, after all these years of fighting and toil and danger, of doubt, discouragement and almost despair, should as an emblem of the results of this war, lead into Richmond a regiment of black cavalry.” That decency quickly passed, however, after white Virginians complained that some of his cavalrymen were guilty of theft, a charge Adams was inclined to believe. Characterizing his troopers as “curious cattle,” impervious to “threat or punishment,” Adams suspected that black soldiers lacked basic morality, and he complained that at least one-third of his recruits were “the damnedest thieves and rascals alive.”

Adams devoted his postwar years to a career in railroads, eventually serving as president of the Union Pacific. Although Adams proved to be an honest robber baron, the position was a curious one for a family bred to public service, and after being forced out by financier Jay Gould in 1890, Adams’s thoughts returned to the war and race. In 1902 he published a collection of essays, under the title Lee at Appomattox and Other Papers. Adams’s article on the Confederate general was part of the emerging Lost Cause mythology that Lee was a reluctant secessionist and an antislavery Virginian who wished only to heal the nation’s wounds after 1865.

The piece, together with Adams’s insistence that a statue to Lee be erected in Washington, caught the attention of influential Virginians. President George Denny of Washington and Lee University invited Adams to come south as the keynote speaker for the January 1907 centennial celebrations of Lee’s birth. In his lengthy speech, Adams claimed that the foolish “African-and-brother doctrines” of the “Uncle Tom period,” had promoted the “sheerest of delusions” during the Reconstruction era. As he concluded, the all-white audience rose in applause, with many standing in line to shake his hand. It was gratifying, Adams thought, to have pleased “so many good people—so simple, straightforward, and genuine.” Mary Custis Lee, the general’s aged, unmarried daughter, wrote that people were praising Charles Francis Adams as one of the “greatest men” in America. Lyon Tyler, the president of the College of William and Mary and one of President John Tyler’s fifteen white children, wrote to Adams that were the matter “left to the South, the Adams family would have the honor of having another representative in the President’s Chair.”

Toward the end of his life, Adams and his brothers, Henry and Brooks, reflected on the fact that of all his siblings, only John Quincy II had sought political office. “Since the Civil War, I think we have produced not one figure that will be remembered [in] a life-time,” Henry mused. Brooks agreed, writing: “We are appendages only” to a once great family. To that, Charles Francis Jr. consented. Their illustrious ancestors “had all that political preferment could give. Were they happy or contented by it?” The question answered itself, yet by then Adams was sadly aware that his life and career had provided him with wealth but not with happiness: “I am tired; and the prizes I am after, well, if I gained them all, are not worth having.”