How much of the purse after the Fullmer fight actually belonged to Sugar is left to speculation, but a good percentage was usually required to take care of the crowd of well-wishers who, when they weren’t applauding his every move, had their hands out for donations and monetary contributions. Edna Mae had more than a passing interest in the welfare of his income. He was often behind in his promised allotments to take care of their son, and with bill collectors pulling him every which way, the likelihood of his keeping his commitments were slim to none. Moreover, there was the abiding tax dilemma, which Edna Mae’s records show she tried valiantly to fix. Then there were Gainford’s salary and publicity costs, each just under fifteen thousand dollars.
When Sugar finally got back to New York, he went to see his wife and son. “He was totally miserable while he told me how happy he was,” Edna Mae said. “He confided in me that he had started smoking marijuana and did not want his mother to know it.” She was stunned by this revelation, since Sugar didn’t smoke cigarettes or drink much. “I knew Sugar was in great emotional trauma to now be indulging in vices so detrimental to his health and career,” she concluded.
Sugar prevailed on her even further, asking her to allow him to come to her house to smoke marijuana while their son was at school. He needed to calm his nerves, he told her. As usual when it came to Sugar’s sweet talk, Edna Mae gave in. He would come to the house, get in bed (without her, at her insistence), and light up his joints, tossing the butts out the window, where they landed below the front door. One day Ray II came into the house with a handful of butts and showed them to his mother. He told her he had seen his father smoke those kinds of cigarettes. Sugar confessed to Edna Mae that Ray, then eleven, had seen him smoking a joint once before he left for school. She was furious, and that ended his privilege. But it was only the beginning of Sugar’s fury.
When Edna Mae and Ray II returned home the next day after staying all night with her son’s godmother, Sugar soon arrived and began ringing the bell and banging on the door. “Ray was in the bathtub, so I answered the door,” she recounted. Sugar took one step inside the house and began calling her filthy names and smacking her upside the head, until she nearly passed out. Ray II came to the top of the stairs and started screaming for his father to stop hitting his mother. “You’ll kill Mommy!” he cried. Sugar stopped, and went upstairs to his son to comfort him. Edna Mae used this moment to hurry to the kitchen to call for help. Aunt Blanche called Sugar’s mother, who lived only four blocks away, and told her what was happening and threatened to call the police. Sugar’s mother begged her not to do that, and promised she would be right over. She was there in a few minutes and quickly interceded, repelling another attack against Edna Mae by Sugar. While Sugar was in his mother’s arms, Edna Mae grabbed Ray and hustled over to a neighbor’s house.
Once Sugar left, his mother went to see about Edna Mae and Ray. She cringed at the sight of her daughter-in-law’s face. Dr. Arthur Logan was summoned, and he took one look at her battered face and rushed her to the hospital for treatment and X rays. She was diagnosed with two concussions. After almost twenty years of marriage, infidelity after infidelity, years of abuse, and separation, she said, “I decided to get a divorce.”
After losing to Fullmer, Sugar fought once a month over the last third of 1961. On September 25, the twentieth birthday of Ronnie, his oldest son, whom he rarely saw, Sugar won a ten-round decision over Wilf Greaves in Detroit. In October he out-pointed Denny Moyer in New York City, and for the first time in his career, because of a less than exciting performance, he heard a chorus of sustained boos. “I was at the weigh-in for his first Denny Moyer fight,” said fight authority Clint Edwards. “Sugar strolled in, bumped past everybody, stood on the scales, was weighed, and then turned to find out who he was fighting. When he looked back and saw it was Moyer, he commented, ‘I’m fighting this baby-face kid?’ It was another indication of how old he was getting.”
“The contest with Moyer taught me many things,” Sugar told Les Matthews of the Amsterdam News, who was popularly known as “Mr. 125th Street” because of the wide range of information he possessed about the community. “I will not forget them. Yes, the punches did hurt me. The only time a punch did not hurt me during my ring career was when I got under it or blocked it. I’m human.” He wasn’t exactly sure why his performance was so unusually blah. “My legs were in fine shape; so was my wind. I was in excellent condition. I must sharpen my punches. I was disgusted with myself when I missed. The boos never hurt me because I know the paying customers are always right. I love the cheers.”1
The cheers returned in his next fight, against Al Hauser, whom he obliterated in Providence in November; then Sugar plastered Greaves again in December in Pittsburgh. Sugar had plummeted to the has-been circuit, hammering it out in almost meaningless bouts with nameless club fighters, many of them over the hill and, like him, trying to earn a buck without losing their teeth or their sanity.
It got so bad that one day when Sugar was training for a return match in February 1962 with Denny Moyer, whom he had defeated just months before, he was told he was no longer a special nonpaying customer at Harry Wiley’s Gym. Sugar stared at the attendant incredulously with a you-gotta-be-kidding expression on his face. The next day Wiley patched up what Sugar had assumed all along—and fervently hoped—was a mistake. Indeed, according to actor Johnny Barnes, a longtime Harlemite who gave a realistic performance as Sugar in the movie Raging Bull, Sugar himself had bought the gym and then given it to Wiley in order to keep him from begging and pestering him for money. “Ray used it for tax purposes,” Barnes explained, “and he allowed Harry to keep all the money paid through dues.”
The courtesies he expected at Wiley’s, Sugar also expected each time he went by Edna Mae’s under the pretext of wanting to see his son. She never denied him these opportunities, though she was ever on guard for an outbreak of his sudden anger. Sugar’s mood, she said, could roll in like a thunderstorm, ripping apart everything in its path. That was one reason she never said much to him during these visits. Her hope was that he would come, spend some time with his son, and then leave peacefully.
Keeping the peace with Edna Mae didn’t mean Sugar had to keep his peace with the IRS. He was still trying to recoup money owed to him from as far back as the first Basilio fight four years earlier in September 1957. Then, Sugar received a most miraculous windfall. The IRS sent him a check for $123,935.65.2 The full amount seized had been $313,449.82, but Sugar had used it each time he needed it as credit on his taxes and the check the IRS sent was the amount left over.
Sugar wasted no time and hurried to the bank and cashed the check. “The money filled two large shopping bags, which he walked with down Seventh Avenue while carrying it back to the office,” Edna Mae reported. Ray II was in Sugar’s office the day the money arrived. “I was there at the office that day and saw my dad and a few of his friends holding duffel bags full of money,” he recalled. “The manager of Chase Manhattan Bank had called them to say the government had made a mistake and there was a release on his money. And he wanted Dad to come in that day and pick up all the money owed to him. It was absolutely astounding to see all those duffel bags and shopping bags full of money, and there was no thought of anybody ripping them off or anything.”
Now, Sugar’s planned trip to Europe and his list of companions could be expanded, much to Edna Mae’s chagrin. He had not divulged his good fortune to his wife, maintaining his woeful tale about being broke and destitute. But Sugar was in for a very rude awakening—and Edna Mae would be no better off: The money he received was taxable (of course, Sugar had never entertained that possibility). Edna Mae would be informed later that she was equally responsible for the indebtedness, though she never received any more than the stipulated support money during separation. With Sugar laying plans for a European sojourn in the fall of 1962, Edna Mae booked passage to Mexico, for a divorce.
The summer of 1962 found Sugar in Los Angeles getting ready to tangle with the always tough Phil Moyer from Oregon. Back in
February, Sugar had lost a decision to Moyer’s brother, Denny; so giving Phil a shellacking would be a sort of revenge, he reasoned. But one Moyer was just as savvy and ring-wise as the other, and Phil too won a narrow decision over Sugar. Other than Millie and a few sweeties on the side who helped to salve his wounds, the only consolation in California was to be the company of Cassius Clay (not yet Muhammad Ali) and Joe Louis.
Photographer Howard Bingham caught the trio in a memorable pose one afternoon. Both Clay and Louis are wearing bow ties, while Sugar, seated between them, is informally attired in one of his typical striped knit short-sleeve sport shirts. It’s a classic shot of three of the greatest boxers ever to step through the ropes. “It was during this time that I accidentally met Ali for the first time,” said Bingham, who would become among the champ’s most devoted and loyal friends, as well as his personal photographer.3 Clay was in Los Angeles to do battle with Alejandro Lavorante, who, by the fifth round, had been bludgeoned to the point of collapse. Only Doug Jones would go the route with Clay as he pranced inexorably to that memorable match with Sonny Liston.
En route to Mexico, Edna Mae reclined in her seat on the train and began reminiscing about how she used to travel with Sugar, holding hands, taking turns feeding each other, looking lovingly at each other with unbroken gazes. They were so much in love, she thought as the train raced toward Mexico, recalling how because of their mutual fear of flying, both of them preferred traveling long distances by train or boat. “As I rode across the weary miles and sat gazing out of my bedroom window on the train, I was reliving and remembering how much fun train rides had been for our immediate family—for me, Sugar, and little Ray. We would really turn the train into a hotel on wheels.”
And then came a torrent of memories—Grand Central Station, darting redcaps, the streamlined Twentieth Century, the gallant porters standing at the head of each car, Penn Station and the cries of “All aboard!” “In those days we were often called the Robinson Caravan because of the large numbers of persons that traveled with us…Because of this we would have at least one whole car reserved for the Robinsons.” In her reverie, she recalled the parade of celebrities who beckoned at their door—Jimmy Durante, Joan Crawford…all of them seeking autographs or asking if it would be all right to have a picture taken with the champ. “It seemed that we had so few private times together that we turned our travel times into hours packed with joy, love, and happiness. We hoped somehow that it could carry us through the times that were shorn of tenderness and caring by the pressures of his demanding life and profession. Now, here I was rolling into Juárez to put an end to our marriage.”
Rather than sit in her room in the villa and sulk and cry, she decided to do the town with her lawyer and to see what sprawling Mexico City was all about. It was a splendid evening, she said, and everywhere they went they were serenaded by mariachi musicians dressed in colorful costumes and blessed with melodious voices. But despite the lavish dining, the dancing, and the touring of fabulous sites, there was no way she could dispel the marital trauma that gripped her. In the courtroom there was additional anxiety, since the entire proceeding was conducted in Spanish. Somewhere in the midst of the flurry of words she didn’t understand, she was asked to raise her right hand and to repeat after the judge. Several minutes later she was escorted to another room, and her lawyer informed her she was a free woman.
Back in her room, with her lawyer still in tow, the hysteria was so intense that she ran to the bathroom and heaved between sobs. After taking some time to console her, the lawyer left and Edna Mae pulled herself together and sat relaxing in a large chair. She was jolted from her meditation by the jarring ring of the telephone. Somehow the press had caught wind that she was there, even though she’d checked in under an assumed name. The place was a known haven for divorcées, and the press always had informants on the scene to tip them off. (Two years before, in 1960, Hazel Scott had probably endured the same hounding when she was there ending her fifteen-year marriage to Adam Clayton Powell, Jr., and earlier in 1962, in January, Marguerite Mays had made the same trip to the same place to untie her seven-year knot with her husband, baseball great Willie Mays.) Uppermost in the newsman’s mind was whether Sugar knew about the divorce, since he was then traveling in Europe. She told the reporter that Sugar was not aware that they were now officially divorced. By morning the news was everywhere: As of October 2, 1962, the Robinsons were no longer a unit.