Arriving at the bus station in downtown Detroit for their departure to New York City, Leila Smith and her children, with their cardboard suitcases, could have been mistaken for a group of vagabonds. In preparation for the long bus ride, Leila had prepared a good supply of bologna sandwiches. Two pieces of Silvercup bread with a smear of Hellmann’s mayonnaise or mustard spread across a thin slice of bologna was the nutrition for thousands during the Depression, and for millions even such a meager meal was not possible. Sugar said his knickers were patched and his sisters’ dresses were faded, but they were clean. In Buffalo, Leila wanted something hot, and when she went to purchase a cup of soup, Sugar took the opportunity to strike up a conversation with the bus driver, who, upon learning they were headed for New York City, gave him a quarter for good luck. Later, Sugar gave the quarter to his mother, which she added to the fifty cents she had in her pocketbook.
In the winter of 1932, the Smiths settled into their three-room flat at 419 West 53rd Street, between Ninth and Tenth Avenues. This section of the city was known as Hell’s Kitchen, and the several blocks occupied mainly by blacks were an atoll surrounded by Italians and the Irish. (The black population was concentrated in Harlem.) Sugar would learn quickly it was not safe to venture outside the enclave, unless you were ready to fight or fast enough to outrun the gangs of white toughs who preyed upon little “niggers” like Sugar. On several occasions, Sugar, seeking a free lunch at the nearby Salvation Army office, was cornered outside his zone and had to use his wits to avoid getting his “black ass kicked,” as he recalled. But a good meal, he’d apparently decided, was worth the risk of getting his nose bloodied or his lips busted.
The always industrious Leila soon secured work as a seamstress at a linen-supply company. It was similar to the job she had in Detroit, but it paid two dollars more a week. Even so, with the rent twenty-three dollars a month, it was barely enough to feed and clothe her children. Somehow, though, she managed to squirrel away a few cents to purchase a bottle of gin every Saturday night, and to pay for dancing lessons for Sugar and Evelyn. Sugar’s imitation of Bill “Bojangles” Robinson, the best tap dancer of the day, convinced his mother to spend fifty cents a week for his lessons—both to invest in his talent and to keep him out of mischief. But the lessons, at the Roy Scott Studio, around the corner from his home on Ninth Avenue, came to an abrupt halt when Leila discovered that neither Sugar nor Evelyn was going to the studio, but rather spending the money on candy. Because it was Sugar’s idea, he got two whippings—one for him and one for Evelyn.
With no more money coming from his mother, Sugar resorted to making a little change dancing outside theaters on Broadway and Times Square. Improvising around the one step or two he had learned at the studio, Sugar joined several other young black dancers who’d gather at the theaters to entertain patrons who would go outside to smoke and mingle during intermission. When the dancing was over one day, outside the Alvin Theater, Sugar was the lone dancer, and the doorman asked him if he would like to perform for a smoker. He had no idea what a “smoker” was, but it was an opportunity and he leaped at it. A smoker, he was to learn, was an informal gathering of young men that usually featured female dancers. Sugar did his Bojangles routine, for which he earned two dollars. After the performance he hurried home to give the money to his mother. When he told her how he had earned it, she took it and told him: “It’s about time you paid me back for all that cheatin’ you did on dancing school.”
To fill in the slow days when dancing wasn’t bringing in the money, Sugar made a few dollars collecting driftwood along the docks of the Hudson River, which was only a couple of blocks from his home. He would chop up the driftwood and stuff it into bushel baskets. A junk dealer gave him twenty-five cents for each bushel. The man never told him what he did with the wood, and Sugar never asked. Maybe he sold it to artists; there were several notable sculptors who used driftwood during the Depression to make furniture and even to construct inns. Alternatively, it could have been stored and used as kindling during the winter. What the man did with the driftwood didn’t concern Sugar. Nor did the future boxer express any keen interest in what was going on inside nearby Madison Square Garden, where one day he would be a headliner. He was only interested in exploring the city, which with its tall buildings and endless motion stood in stark contrast to Detroit’s downtown section. He found himself moving from one landmark to the next, including the Empire State Building. A trip up the elevator there sickened his stomach, and it would be his first and last elevator ride. For the rest of his life, he would climb the stairs, no matter what floor his destination.