Over the past two weeks, national attention has been focused on the Confederate battle flag and rightly so. Critics have demanded that it is finally time for the flag to be removed from public buildings in the South, and a groundswell of popular support for its removal has pushed corporations like Walmart and Amazon to stop selling flag-based items. But there are other trappings of white supremacy that will be more difficult to dislodge from their place in our national culture.

Before white supremacists in the post-emancipation United States ever reached for the Confederate battle flag, they reached for their guns. During Reconstruction, when paramilitary groups like the Ku Klux Klan and White Leagues emerged to drive freedpeople as far back into slavery as they could, guns became both a tool and a symbol of their struggle to restore white power to the defeated South. Reconstruction precipitated an arms race in the former Confederacy that destroyed entire communities and cost innumerable black lives.

The Civil War, and in particular black men’s service in the Union army, changed the racial dynamic of gun ownership in America. Before the war, southern state laws prohibited slaves from having weapons of any kind, although slave owners often allowed a trusted slave to have access to firearms for hunting. Nearly 200,000 black soldiers gained familiarity with firearms while serving in the Union army, however, and after emancipation, many black veterans purchased their military-issue rifles and took them home. They took to heart Frederick Douglass’s wartime message about the symbolic and practical power of guns for the cause of black civil rights. “Once let the black man get upon his person the brass letters, U.S., let him get an eagle on his button, and a musket on his shoulder, and bullets in his pocket,” Douglass proclaimed, “and there is no power on earth which can deny that he has earned the right to citizenship in the United States.”

But white southerners worked tirelessly to deny that right. Immediately after the war, most southern state legislatures passed Black Codes that made it illegal for freedpeople to own firearms, and from the moment of its inception in 1867, the Ku Klux Klan’s primary goal was to disarm ex-slaves. When Congress gathered testimony in 1870 about the Klan’s activities in the South, witness after witness came forward to tell how the hooded vigilantes searched their homes and destroyed any weapons they found. These violent intrusions and seizures reflected more than just a rudimentary albeit effective attempt to disempower black farm hands who pressed landowners for better wages and working conditions. By making freedmen break their own guns into pieces and throw them into the fire, or by raping a Union veteran’s wife with the barrel of a pistol, Klansmen forced ex-slaves to perform their own vulnerability.

Lest anyone be tempted to argue that Reconstruction might have turned out differently had freedpeople had more weapons, it is important to note that black southerners, both individually and collectively, routinely defended themselves with arms. But they were almost always outnumbered by white vigilantes with better guns and faster horses. Indeed, the very idea of armed black men was enough to drive whites into a deadly panic, and they used the perennial specter of “race war” to justify pre-emptive assaults against black people whether they were armed or not. Even in areas where black men held positions in law enforcement, they were no match for the impressive organizational strength of white paramilitaries, many of whom were ex-Confederate soldiers.

In September 1874, for example, white supremacists in Louisiana, organized under the banner of the Crescent City White League, led a coup d’etat against state officials in New Orleans when the city police intercepted a shipment of guns the League had tried to smuggle into town. For several days, the paramilitaries placed the city under siege, attacked the racially integrated Metropolitan Police, killed a number of officers, and prompted Louisiana’s governor to hole up in the federal Custom House. Eventually President Grant dispatched federal troops to restore order and rescue the governor. The “September Rebellion” may have been short-lived, but it became a rallying cry for white paramilitaries in other parts of the South, especially South Carolina.

In the Palmetto State, the integration of the state militia by Republican governor and Union veteran Robert K. Scott ignited the cries of “race war” from native whites who viewed the move as a tyrannical attempt to put the “bottom rail on top.” White men refused to serve alongside black men and instead formed their own “rifle clubs.” The clubs advertised their meetings as opportunities for members to practice their shooting and riding skills, and they were ostensibly civic organizations that posed no danger to the state’s black majority state government. In truth, club members were training for attacks on state government officials. Sensing the impending danger, the governor outlawed private armed organizations, but they simply changed their names, becoming organizations such as the Allendale Mounted Baseball Club or First Baptist Church Sewing Circle. Members would often drill outside black political meetings in an effort to intimidate potential voters. They broke up rallies organized by their opponents and they escorted white political candidates like future governor Wade Hampton, effectively serving as his private army.

Rifle club members were required to provide their own weapons. Martin Gary, the leader of South Carolina’s White League, informed rifle club captains that their men should be “well armed and provided with at least thirty rounds of ammunition.” But money was tight, and not everyone had a reliable weapon. So rifle clubs pooled their money to purchase the newest state-of-the-art firearms. The Colleten Mounted Rifle Club collected $116, enough to buy 38 new Enfield rifles and 23 boxes of bullets. The St. Martin’s club failed to collect enough donations, so they forged their own ammunition.

These weapons enabled the rifle clubs to intimidate African American voters and public officials, but they also became important emblems of white manhood. According to a young journalist from Charleston who covered the rifle clubs, a gun became a part of a man’s self-presentation and a sign of his racial and political allegiances. The bigger the gun, the better. A “.38 was the very smallest caliber tolerated in respectable society,” the journalist wrote. Without a gun, one could feel exposed and vulnerable. “A man without a revolver,” he explained, “felt undressed and embarrassed, as a man would walking the public streets at noon lacking his trousers.”

South Carolina’s arms race came to a head over the Fourth of July holiday in 1876 in a little town called Hamburg. Located across the Savannah River from Augusta, Georgia, Hamburg became home to a mostly black community of freedpeople after the war. There a branch of the state militia met to drill on the morning of July 4 on the town’s main street. Two white farmers brought a formal complaint against the militia for obstructing a public road after militiamen had briefly blocked the farmers’ passage, but the militia captain, an ex-slave named Doc Adams, refused to disband his unit and surrender its weapons. In response, members of various rifle clubs descended upon the town ready for battle. Several hundred armed white men faced off against an estimated thirty to forty black militiamen. The rifle clubs not only had guns but also cannon, which they used to demolish the warehouse where about two dozen militiamen had taken cover. Doc Adams later told Congressional investigators that his men did not get off more than five or six rounds. As some of the militiamen attempted to flee the burning building, white paramilitaries shot them in the back. Several others who surrendered were executed by the mob. In all, at least seven black militiamen were killed along with one white belligerent. “By God! We will carry South Carolina now!” shouted the rifle clubs as they rode through the town victorious. Indeed, the Hamburg Massacre became a rallying cry for Wade Hampton in his gubernatorial campaign as he pledged to restore peace, order, and white supremacy to the state. Slowly, the freepeople moved away from what remained of Hamburg until it became a ghost town. Today, only a few broken piers of an old railroad bridge remain to mark what was once a vibrant place.

When members of the Carolina rifle clubs rode into Hamburg they did not carry the Confederate battle flag. The most potent symbol of their racial supremacy was instead the one they carried on their shoulders and hips. And after the Confederate flag is gone from public spaces, those weapons will remain. Walmart may pull the Confederate flag from its shelves, but it will continue to be the largest gun retailer in the United States. Will removing the flag improve our nation’s racial dialogue if guns are still the language through which so many of us speak our identity as Americans? Will increasing access to guns make our nation safer and more democratic? These are questions that arose out of the tumultuous era of Reconstruction and remain as relevant today as they were 150 years ago.