Life out in the woods with my father was rough. We lived in a series of shacks all around the Barnwell and Elko areas. We lived in one as long as the people gave my father work. When he lost a job or tried to find better work, we moved on. The shacks were unpainted, didn’t have windows except for shutters that you could pull together; and there was no electricity or indoor plumbing. But we did have plenty of firewood for the stove. My father chopped it, and we threw in some kerosene or fatty pine to start the fire.
We ate black-eyed peas and lima beans, fatback and syrup, polk salad that we picked in the woods, and corn bread. Although the diet never varied, there was almost always enough. But I was unhappy because I was alone all the time. Daddy was gone a lot, working in the turpentine camps, and the various common-law wives he had to take care of me didn’t stay around very long, so I was left by myself a lot in the house or out in the yard. Every now and then I had a playmate, but we were so far out in the country I more or less had to be a loner. So I played with sticks and sang, I guess. Dug holes. Got up under the house. Played with the doodle bugs—“Bag, bag doodle”—that kind of thing. Years later I wrote and recorded an instrumental tune called “Doodle Bug.”
I don’t think you can spend that much time by yourself as a child and not have it affect you in a big way. Being alone in the woods like that, spending nights in a cabin with nobody else there, not having anybody to talk to, worked a change in me that stayed with me from then on: It gave me my own mind. No matter what came my way after that—prison, personal problems, government harassment—I had the ability to fall back on myself.
The best thing I remember from that time is the ten-cent harmonica—we called ’em harps—my father gave me. I started playing it real early, when I was about five years old. I played “Lost John,” “Oh, Susannah,” “John Henry,” and I sang. My father sang, too, but he sang blues songs he heard in the turpentine camps, things by Sonny Boy Fuller and Blind Boy Fuller, “Rattlesnakin’ Daddy,” things like that. I don’t remember whether I sang them, but I know I never liked them. This is going to surprise a lot of people: I still don’t like the blues. Never have.
My father also made home brew, and he was real good at it. Everybody wanted some. All the white people asked him to make it for them because he could beat anybody else making it. He made it out of apples. He’d let ’em sour and then peel ’em. Then he’d put ’em in a barrel and stir ’em and stir ’em and put sugar in. After that he let it sit a long time until it got real thick. You knew when it was ready because you could smell it. I guess it was really apple cider, but it was good stuff.
He had a capper, too. He washed out the bottles and scalded ’em and got the big old Co’-Cola caps and beat ’em flat. Then he’d put the bottles in the capper, put the caps on, and mash it down.
There was another thing my father had: a temper. He could be very mean, and a lot of times he gave me whippings I didn’t deserve. He’d be away from home, and when he’d come back somebody would tell him I needed a whipping, and he’d give it to me, no questions asked.
He had a temper about white people, too, but he never showed it to them. Where white people were concerned, I would say my father threw a rock and hit his hand. He’d call white people “crackers,” curse ’em and everything when they weren’t around, but when he was in front of them, he’d say, “Yessir, nawsir.” That’s when I lost respect for my father.
I will not accept what my daddy accepted. I will not accept being a boy. If you push me, you got a problem. I was a boy as a boy, but as a grown man I will be respected. That’s the reason I call everybody by his last name with a Mr. in front of it and insist on the same thing in return.
One of the things that probably makes me feel worse than anything in the world is to see a Caucasian walk up to my father and say, “How are you, Joe,” and then walk to me and say, “How are you, Mr. Brown.” I think the man who does that is more ignorant than my father.
I love my daddy to death, but he has never looked a man in the eye and told him he didn’t like him. That’s the difference between us: I’d tell a man to his face I didn’t like him. But I wouldn’t be mad with his brother. My daddy would be mad at all of ’em but tell ’em, “Yessir, nawsir,” and then be ready to kill ’em later.
People like that are dangerous. And that’s what we’re facing today: people who laugh with you and say you’re all right and then kill you later. It’s the same thing the Ku Klux Klan used to do. The same people you work with in the daytime come at night to lynch you.
My father gambled a lot, too. Never won. He’d gamble any place he could find. He played a lot of Georgia Skin. That’s where you shuffle the cards and deal one to each player. Say, I have a seven and you have a five. You flip over cards from the top of the deck, and if a seven comes up before a five, I caught you and win your money. You bet any amount you want on it.
I don’t know what Daddy had going for him. He’s been a strange man for a long time. The mystery could have been in me, though, I don’t know.
One thing about my daddy, he was always hardworking. I think I got a lot of my drive from him. He was never without a job for more than five days in his life. He did whatever work he could get. He did farm work. He did a lot of filling station attendant work, washing and greasing cars, and maintenance around the station. After all the turpentine work, he did a lot of highway work. He’s a heavy-equipment operator, but he never had formal teaching so he couldn’t get his certification papers. He stopped in the second grade in school. He is a jack of all trades and master of none because of the sheepskin he wasn’t able to get.
My father had a very hard time trying to bring me through. He worked hard just to take care of his child, but the system really went against him because he didn’t have very much knowledge. They deprived him of knowledge, and that’s probably the greatest sin in the world.
The social system back then was like it is right today: economic slavery. One thing you have to understand about slavery: A man never enslaved a man because he didn’t like him; he enslaved him because he wanted him to work for him. It’s about free labor. That’s all it’s ever been about. It works that way everywhere in the world.
My father did his best, but finally he had to get Aunt Minnie, who was really my great-aunt, to come take care of me—Minnie Walker, who first blew breath into me. The three of us were living around Robbins, South Carolina, in another one of those shacks, when my father decided that he could find better work across the Savannah River in Augusta. So the three of us moved into town, but he split from Minnie and me. He was still around, but from then on my father and I never lived in the same house again.
In Augusta, Aunt Minnie and I lived with another aunt of mine in a house at 944 Twiggs Street. That’s one place I will never forget. Outside, Highway 1 ran right by the door. You could go all the way to New York on that highway. Inside, there was gambling, moonshine liquor, and prostitution. I wasn’t quite six years old.