Swinging his line to the left, Col. Sidney Burbank moved his brigade into the Wheatfield, perpendicular to its original reserve position, to face down Anderson’s and Semmes’ advancing brigades. The 2nd and 7th U.S. formed the right of his brigade in the Wheatfield, while the other regiments occupied the northeastern portion of Rose’s Woods. Burbank believed the section of woods in front of him had to be filled with the enemy, so he ordered his men to fire a volley into it. When no reply was forthcoming, he ordered his men to cease firing. Meanwhile, Col. Hannibal Day moved his brigade forward to occupy Burbank’s former position on the periphery of Rose’s Woods. The 5th New Hampshire and a portion of the 148th Pennsylvania continued moving to the rear to rejoin Colonel Cross’ battered brigade. Colonel Sweitzer continued moving survivors of his beleaguered brigade out of the Wheatfield.43
By now, matters were utterly confused (much more so than battle reports indicate or convey). Elements from Semmes’ and Anderson’s brigades charged through the woods against Burbank’s Regulars, but an effective volley stopped them in their tracks. According to the commander of the 2nd U. S., “a fresh column of the enemy at this time appearing upon our right, we were ordered to retire. The word was scarcely given when three lines of the enemy, elevated one above the other on the slope to our right, poured in a most destructive fire, almost decimating my regiment.” The firing was from Wofford’s men and perhaps some of Kershaw’s South Carolinians. According to a soldier in the 11th U.S., the order to withdraw was to “face about and wheel to the right at the double quick and form on the general line of battle.” In his report, Colonel Burbank wrote that this withdrawal was made “as rapidly and in as good order as the nature of the ground would permit.”
Some of the men, particularly those in the center, did not see the threat on the right and complained about the order. The sheer noise and confusion was another problem. According to Captain William Clinton of the 10th U.S. in the center of the brigade, “the roar of musketry was so extensive that a great portion of our command did not hear the order to fall back until some minutes after it had been given. The enemy at this time was in front and on both our flanks.”44
General Romeyn Ayres, commander of the division, could see that his two remaining brigades were in danger of annihilation and ordered their retreat. A young Federal lieutenant recalled that the Regulars “moved off the field in admirable style, with well-aligned ranks, facing about at times to deliver their fire and check pursuit … in this action the regulars sustained severe losses, but gave ample evidence of the fighting qualities, discipline, and steadiness under fire which made them the pattern and admiration of the entire army.”45
The retreat heartened Anderson’s and Semmes’ exhausted men, who fired several volleys into the backs of the withdrawing Regulars and inflicted heavy casualties doing so. An officer in the 11th U.S. characterized it as an “almost semi-circle of fire … almost a sheet of fire … the slaughter was fearful.” The survivors reformed near the signal station on Little Round Top. In an odd twist, Colonel Day’s brigade never engaged the enemy because it was ordered to follow Burbank’s brigade to the rear.46
Three of Burbank’s regiments—the 7th, 10th, and 17th U.S.—lost more than fifty percent of their men in the short time they spent in the Wheatfield; the others lost in excess of thirty-four percent. Burbank’s total losses amounted to forty-seven percent. Given the Confederate encirclement of the brigade, the losses could have been much higher. The same could be said for Sweitzer’s brigade, which lost more than four of every ten men engaged. The 4th Michigan, hit on two sides, lost nearly half of its men. Day’s brigade, which occupied a relatively safe supporting line and so was never directly engaged, still lost twenty-five percent of its men. Most of these losses occurred during the retreat from the Wheatfield sector.47
After some of the bloodiest and hardestfought combat of the war, the Confederates had finally wrested control of the Wheatfield and Stony Hill from their adversaries.