CHAPTER ELEVEN
Earlier in the day, Brig. Gen. Alfred Pleasonton received permission from Maj. Gen. Joseph Hooker to withdraw from the field at his discretion. As evening approached, Gregg’s division had already disengaged from Fleetwood Hill and was heading back to Kelly’s Ford, where it would re-cross the Rappahannock River. Pleasonton decided that his men had done all they could for one day and ordered John Buford to pull his men back across the Rappahannock. The staff officer sent by Pleasonton found the Kentuckian “entirely isolated from the rest of the command under Pleasonton … but paying no attention and fighting straight on.”
Buford wrote later that as the firing waned on Fleetwood Hill, “I was ordered to withdraw. Abundance of means was sent to aid me, and we came off the field in fine shape and at our convenience. Capt. [Richard S.C.] Lord with the 1st U.S. came up fresh comparatively with plenty of ammunition and entirely relieved my much exhausted but undaunted command in a most commendable style. The engagement lasted near 14 hours.”
With the 1st U.S. and the added protection of Brig. Gen. Adelbert Ames’ infantry covering their withdrawal, Buford’s men leisurely splashed their way across Beverly Ford to the Fauquier County side. Defiantly, Buford himself “came along serenely at a moderate walk.” The 33rd Massachusetts Infantry served as a rear guard. When all of the horse soldiers were across to the north bank, the Bay Staters followed. The gray-clad cavalry did not hinder the crossing.
Back on the north side of the Rappahannock, Buford joined Pleasonton and a number of staff officers to observe the crossing as the sun set. “Our cavalry fell back across the river that night,” wrote one of Buford’s unbowed troopers, surprised at the abandonment of the field. “It was a mystery to the boys why they fell back.”
Pleasonton, for his part, seemed satisfied. “General Buford withdrew his command in beautiful style to this side,” he later wrote, “the enemy not daring to follow, but showing his chagrin and mortification by an angry and sharp cannonading.” A reporter traveling with Pleasonton correctly observed, “the fact that the rebels did not take a step toward following the command, though vastly superior in numbers, indicates very clearly that they had had quite enough of the Yankees for one day.”
* * *
“I did what you wanted, crippled Stuart so that he can not go on a raid …” Pleasonton wrote in a 9:30 p.m. note to Hooker at army headquarters. “My own losses were very heavy, particularly in officers. I never saw greater gallantry … exhibited then on the occasion of the fierce 14 hours of fighting from 5 in the morning until 7 at night.”
Pleasonton wrote that “Buford’s cavalry had a long and desperate encounter, hand to hand, with the enemy, in which he drove handsomely before him very superior forces. Over two hundred prisoners were captured, and one battle flag.” Most importantly for the Union cavalry, “the troops are in splendid spirits, and are entitled to the highest praise for their distinguished conduct.”
Buford’s loss amounted to 36 officers and 435 enlisted men killed, wounded, and missing, for total casualties of 471. This was more than 50 percent of the total Union casualties for the day, which numbered 866. The 6th Pennsylvania Cavalry suffered the worst, losing 108 men, including eight officers. The 2nd U.S. Cavalry lost 66 killed or wounded out of 225 present for duty during the day’s fight. Along with these losses, a large number of horses were killed or wounded, leaving many troopers dismounted.
“No regiment engaged that day on the Union side had more of it than ours …” remembered a trooper in Grimes Davis’ 8th New York. “It was first in and last out in our division. It was not later than 4:30 a.m. in going in, and was rear-guard at the Ford.”
But John Buford was rightfully very proud of the performance of all of his troopers. “The men and officers of the entire command without exception behaved with great gallantry,” he wrote.
* * *
After Duffié’s arrival, Gregg had moved to realign his men with Buford’s position. The Confederates also shifted, but as Gregg prepared to attack again, he received orders from Pleasonton to withdraw. Brigadier General David Russell’s brigade, which had spent the day lightly skirmishing with Brig. Gen. Beverly Robertson’s North Carolinians, covered the Pennsylvanian’s retreat. “While Pleasonton was defeated at Brandy Station, he made a masterly withdrawal of his forces,” a Virginian remembered.
Stuart remembered that Buford’s attack on the northern end of Fleetwood Hill “made it absolutely necessary to desist from our pursuit of the force retreating toward Kelly’s particularly as the infantry known to be on that road would very soon have terminated the pursuit.”
With that, the largest cavalry fight ever seen in the Western Hemisphere came to an end.
Gregg may have lost the best opportunity for victory that day. He suffered high casualties, including a brigade commander (Wyndham), two regimental commanders, a third field-grade officer wounded, two line officers killed and 15 wounded, 18 enlisted men killed, 65 wounded, and 272 missing. His men captured eight commissioned officers and two sets of colors. Gregg singled out Cols. Percy Wyndham and Judson Kilpatrick for particular praise, and also praised the performance of Capt. Joseph Martin’s New York artillerists.
Gregg, however, laid the failure to carry Fleetwood Hill squarely at the feet of Duffié, who had encountered issues with reaching his river crossing and arrived on the battlefield late. Realizing that Stuart’s whole force was in front of him, Gregg should have sent for Duffié. He certainly could have used the additional division in a sort of hammer-and-anvil maneuver, with Duffié acting as the hammer driving Stuart’s troopers against Gregg’s anvil.
At the same time, the cautious Pleasonton, who held Buford in check for several hours, could have released the First Division, freeing further troopers to try to catch Stuart in a trap on Fleetwood Hill.
For all the boldness he expressed in his memo, Pleasonton’s assertion that the Union cavalry had dealt a serious blow to Stuart was false. Brandy Station cost the Confederates 51 killed, 250 wounded, and 132 missing. The Federals suffered 484 killed and wounded and 372 taken prisoner. These losses illustrate the intensity of the fighting—but even though Confederates received several well-placed blows, their cavalry was as formidable as ever.
The opportunity to destroy the Confederate cavalry had slipped through the fingers of the Union command.
* * *
Holding the field at the end of the day, Stuart ordered that his headquarters be established once again on Fleetwood Hill, symbolizing his successful defense of the position. While this was a moment of triumph for Stuart, dead and wounded men and horses littered the hill. The flies were “swarming so thick over the blood stains on the ground” that there were very few spots for the Confederates to pitch their tents. Exhausted and hungry, many of the Southern troopers simply collapsed where they stood.
Stuart remembered the fight for Fleetwood Hill as “long and spirited.” He generally praised all of his brigade commanders, but especially praising Brig. Gens. Grumble Jones and Wade Hampton while severely criticizing Robertson for failing to delay Gregg’s advance. Major Henry McClellan received particular praise: if not for his quick thinking on Fleetwood Hill, Gregg’s division may have seized and held the hill, thereby altering the outcome of the battle.
Robert E. Lee also praised Stuart, declaring, “the dispositions made by you to meet the strong attack of the enemy appear to have been judicious … the troops were well and skillfully managed, and … conducted themselves with marked gallantry. The result of the action calls for our grateful thanks to Almighty God, and is honorable alike to the officers and men engaged.”
If Lee realized how close his cavalry division came to defeat that day, he certainly did not let on.