Chapter 8
You Must Learn How You'll Be Taxed
Let's pause for a moment and consider what your biggest future expense might be.
Will it be healthcare? Well, you'll likely pay a lot for medical expenses. Estimates vary, but it's safe to anticipate that a 65-year-old couple retiring now will spend somewhere between $285,000 and $385,000 during their retirement. That's a lot of money, but its not going to be your biggest expense.
What about travel? Maybe if you're lucky, and if that's your answer I salute your optimism and hope you're correct.
Housing could be a big expense, depending on whether you anticipate carrying a mortgage or paying rent for the majority of your retirement.
But for many people, the answer to the question of what their biggest future expense will be lies in the title to this chapter. That's right, taxes. The taxman always cometh, and over the course of your retirement taxes are quite likely to be your largest expense. This is also generally true when working, but as you'll soon see you probably have more control over the taxes you pay in retirement than during your working years.
Now, I don't know you personally so I'm not sure what your precise financial goals are. But at the end of the day, money can only go four places. When you have money, you can:
- Spend it on yourself
- Give it to your family
- Give it to a charity
- Give it to the government (taxes)
As I said earlier, I don't know what your goals are. Almost everyone wants to make sure they don't run out of money in retirement. Maybe you want to spend more money on yourself. Maybe you want to treat your family to a vacation, or help your children with a home purchase, or pay for a grandchild's college education, or leave an inheritance. Maybe your goals are charitable, and you want to increase your tithing, or leave a bequest to your favorite charity when you pass. Again, I don't know you, so I'm not sure exactly what it is that you want to accomplish.
But, I'm here to tell you that no matter what it is that you want to do, you're going to be far more successful at it if fewer of your dollars go to the government. Because the bottom line is that the less you pay in taxes, the more money you'll have to spend on yourself, or give to your family, or give to a charity.
As such, efficiently accumulating, investing, and distributing your assets in order to minimize your lifetime tax burden is a crucial step in securing your financial future and helping you live the lifestyle you desire.
However, and this is the crux of the matter, taxes are another area of your finances where you'll need to ignore the hype and pursue financial strategies outside of the mainstream. That is because, to be blunt, much of the common wisdom you've heard around saving for retirement is either incorrect, outdated, or both.
As with so much of your financial life, navigating your way to a tax efficient future is going to require a mindset shift, and you might have to adopt practices different than those you've used so far. Your reward for doing so could be a future in which you have more control of the taxes you pay, a higher likelihood of not running out of money, and a better chance of accomplishing your financial goals. It's up to you to decide if that reward is worth ignoring the hype and charting your own path.
If you've decided to chart your own course, let's take a closer look at how you can keep the taxman at bay.
But First, the Hype
Before we talk about what you should do to lower your future tax bill, let's take a moment to address some common misperceptions about what your future might look like. Some of these are addressed elsewhere in this book, but I want to summarize them here since its important to understand exactly where the prevailing wisdom comes from as well as why it might not make sense for you to follow that wisdom.
- You'll spend less in retirement: As discussed in Chapter 5, many people spend as much or more in retirement. This is particularly true if you want to live a comparable lifestyle in retirement as you did when working, which is a reasonable definition of a successful retirement.
- You'll be in a lower tax bracket in retirement: Maybe, if you spend far less money. But if you're living a similar lifestyle, you'll need a similar amount of money coming in which would lead to similar tax bills. Besides, ask yourself this: Over the next 10 or 20 years, do you think taxes are going lower? Do you think taxes are going higher?
- Defer, Defer, Defer: Prevailing wisdom, based in large part on the misconceptions mentioned earlier, is that you should defer as much of your tax bill as possible during your working years. This may not make sense, for reasons we will explore in much greater detail.
- Taxes are inevitable, and beyond your control: With proper planning you can control your taxes.
And Now, the Truth
If you're ready to ignore the hype and take control of your taxes, the starting point is to focus on diversifying the tax status of the assets you own and more importantly, the future income streams you will receive.
I know, I know. You probably read the words “tax status” and immediately tuned out. In fact, I'm impressed you're still reading. But I think your perseverance will be rewarded, because what I'm about to discuss isn't really about taxes. It's about quality of life later on in life. It's about living large while paying less to the taxman. And that, my friends, is a very exciting topic indeed.
Best of all, the taxation of your future income streams is something very much within your control. So, if the idea of maximizing your lifestyle and minimizing your future tax bills sounds appealing, read on!
Income Taxes: A Quick Primer
Living like a king while paying taxes like a pauper requires a basic understanding of how the U.S. tax system works. So, just like vegetables must be eaten before dessert, we'll have to cover some rather bland material before getting to the good stuff.
Picture a staircase. Staircases are great visuals for income taxes, because rates step up at successive income levels. Figure 8.1 shows the 2020 tax brackets for a single filer.
Tax rates are progressive, meaning that the higher your income, the higher the rate at which it's taxed.
There's a 10% rate, and 12%, 22%, 24%, 32%, 35%, and 37% rates – these are the federal rates. There are also state taxes, but since these vary widely, we'll focus on federal income tax for this discussion.
As an example, in 2020, the top of the 22% bracket was $85,525 of taxable income for a single filer ($171,050 for a married couple filing jointly). This is income after all deductions and can be found on line 11b of tax form 1040. A sample of tax form 1040 is shown in Figure 8.2. This number on line 11b represents your taxable income and determines your marginal tax rate.
Figure 8.1 Federal Income Tax Brackets for a Single Filer
SOURCE: Analysis by Brian Perry. Information sourced from Internal Revenue Service.
For instance, if you are married and filing jointly, and your taxable income on line 11b were $152,000, then your income would be taxed as shown in Figure 8.3.
As you can see, your income is taxed at marginal rates. This means that even though you're in the 22% tax bracket, you don't pay 22% of your taxable income to Uncle Sam. The total blended percentage you end up paying is what's called your effective tax rate.
To calculate your effective tax rate, you divide your income taxes by your taxable income. So, for the married couple earlier, you'd divide the taxes due ($25,020) by the taxable income ($152,000) and come up with an effective tax rate of approximately 16.5%.
The above refers to federal income taxes. There are different rates that apply for qualified dividends and capital gains, and these rates become very important as you move into retirement and begin to distribute income from your portfolio.
As a rough estimate, Figure 8.4 shows you what capital gains rate you might be subject to (keep in mind, these levels are approximate, not exact).
Figure 8.2 Example of U.S. Individual Tax Return Form 1040
SOURCE: Internal Revenue Service.
Figure 8.3 Example of Income Tax Calculation for a Married Couple
SOURCE: Analysis by Brian Perry. Tax brackets courtesy of Internal Revenue Service.
| If taxable income falls below the 22% tax bracket | 0% |
| If taxable income falls at or above the 22% tax bracket but below the 37% rate | 15% |
| If taxable income falls in the 37% tax bracket | 20% |
Figure 8.4 Federal Capital Gains Tax Brackets
SOURCE: Analysis by Brian Perry. Information from Internal Revenue Service.
Both your effective tax rate and your capital gains tax rate are important figures to incorporate into your overall tax planning. But for the discussion to follow, it's your marginal rate that is most important, because that is the rate you will pay on your next dollar of income.
Are Taxes Going Higher?
The truth of course is that no one knows for sure, but the general consensus among people I meet is that yes, taxes are eventually going higher. In fact, it seems inevitable that taxes will increase, given the challenges facing the country: funding for entitlement programs like Social Security and Medicare, a huge national debt, an aging population, and so forth.
But all of those circumstances existed a couple of years ago, and at that point, instead of going higher, tax rates actually fell. The Tax Cut and Jobs Act, put into place at the start of 2018, lowered both business and personal income tax rates. The corporate tax cuts are permanent, but the individual tax cuts are not.
Because the personal income tax reductions were forecast to add more than one trillion dollars to the national debt, they could not be made permanent without a supermajority of 60 votes in the Senate. The Bill passed along partisan lines (shocker there!) without a supermajority, and so the personal tax cuts are only temporary and sunset after 2025. In other words, we know for a fact that, absent an act of Congress, taxes are going higher after 2025.
Congress could act of course, and the reversion to the old tax rates may not occur as scheduled. But ask yourself this: If Congress does act to alter tax law, do you think it would be to lower taxes further? Or would the more likely outcome be a reversion to higher tax rates? Given that uncertainty, taking advantage of a bird in hand to do some strategic tax planning at a time when you have the lowest tax rates in your lifetime could make a lot of sense.
Tax Diversification
Just as diversifying your investments makes sense, it's also wise to practice tax diversification. There are three different sources of cash flow in retirement, and as you can see in Figure 8.5, they are all taxed differently.
Figure 8.5 How Retirement Income Sources Are Taxed
SOURCE: Analysis by Brian Perry.
One source of income gets distributed tax-free. This pool includes any assets held in a Roth account or a Health Savings Account (HSA) as well as municipal bonds. Income derived from the second pool includes capital gains and dividends received from stocks, bonds, real estate, and so forth. The tax treatment of these assets varies, but careful planning can often result in a 15% rate for these funds; some folks might even avoid taxes here completely.
The final pool consists of tax-deferred assets, and the defining characteristic of this bucket is that when you take money out, it's taxed at your ordinary income rate, which can be as high as 37%. This bucket includes most of your retirement accounts, such as IRAs, 401(k)s, TSPs, 403(b)s, and 457s. Income derived from pensions and Social Security, as well as from any part-time work you, do also falls in this bucket.
One of the keys to a successful retirement is to manage the tax efficiency of your income by drawing from the appropriate pool at the appropriate time. Doing so can slow down the rate at which your assets are depleted, or phrased differently, can increase the length of your retirement. Alternatively, tax efficiency can allow you to retire sooner, or perhaps enhance your lifestyle in retirement.
Do You Know Who Your Partner Is?
If I asked you if you have ever run an investment partnership, how would you answer that question? Before you give me your answer, let's define an investment partnership as follows:
An investment partnership is an arrangement wherein you do the investing and your partner receives a slice of your profits.
With that definition in mind, I'll ask you again whether you have ever run an investment partnership.
How'd you answer that question? If you replied no, let me ask you a couple of additional questions:
- Do you have an IRA?
- Do you have a 401(k), 403(b), TSP, or similar employer-sponsored plan?
If you answered yes to either of those questions, then you have in fact run an investment partnership. In fact, you're running one right now. Your investment partner is … the IRS!
Think about it. If the definition of an investment partnership is that you do the investing and someone else gets a share of your profits, then an IRA or 401(k) is the very definition of an investment partnership. You invest the money, and the IRS gets a share of any profits when you withdraw funds from the account at some future date.
Let's broaden our discussion beyond investment partnerships to talk about partnerships in general. When you enter into a business partnership of any sort, there are a couple of things you probably want to sort out right upfront. Chief among those would be who gets what share of the profits.
Think about it. The guys who started Google never would have waited until it was a multibillion-dollar business to determine who owned how much of the company. At the start of any business partnership, the partners need to determine who owns what. This way there are no future disagreements, and everyone involved has a clear idea of what the fruits of their labor might look like. The more important the partnership is to your future, the more important it is to nail down this ownership structure so that you know what to expect going forward.
If you believe (and I hope you do!) that saving for your retirement is pretty important to your future, then that only serves to heighten the following irony:
Your IRA or 401(k) is the only business partnership you'll ever enter into in which you've given the other side unilateral ability to change the partnership terms anytime they want!
Remember, the IRS owns a share of your account. But even worse is that with the flick of a pen they can raise tax rates, which effectively increases their ownership share in the partnership you are running with them.
And there's nothing you can do about that.
No wonder planning for retirement is so difficult. You don't know what the economy will be like, or what stock market returns will be, or what inflation might do to your future cost of living. You don't know if you'll stay healthy, or what your family situation will be, or how long you'll live.
And now you don't even know how much money you have saved for retirement.
Because you know that the IRS owns some portion of your savings, but you don't know how much, because you don't know what future tax rates will be.
That is why it's so important to build some tax diversification and take back control of your future financial situation.
Let me be clear that there is nothing wrong with deferring taxes. As with so much financial advice, the suggestion to defer taxes rests on a solid foundation, at least while you are accumulating assets. But as your portfolios grow in size, and as you move closer to retirement, your strategy should evolve, and your goal should shift from tax deferral to tax optimization. And by tax optimization, I mean that you should seek not just to minimize your taxes in the current year, but rather to pay the least amount of tax possible over the course of both your working years and your retirement.
And unfortunately, having the bulk of your assets tied up in a tax-deferred pool whose distribution will result in the highest of tax rates just isn't optimal, particularly when you don't know what those future tax rates will be. This situation is only exacerbated by the fact that taxes seem likely to increase in the years ahead.
Careful planning can help fix this situation. If you take only one of my suggestions, please do the following: Give as much care, effort, and attention to protecting and distributing your wealth as you did to working for and accumulating your wealth. That, dear reader, is the key to enjoying your golden years and to leaving the sort of legacy you desire.
As If That's Not Enough …
As if the lack of clarity around how much of your money will end up in your partner's pocket weren't bad enough, there are several other issues you'll face if the bulk of your assets are tied up in tax-deferred accounts.
The first challenge you'll face is that all of the one-offs in life will cost you significantly more than they otherwise would. Think about it this way. Your core lifestyle (housing, food, travel, etc.) is going to require a certain level of income. That income will generate taxes, and you'll find yourself in whatever tax bracket you fall into based on that income.
Then you decide to remodel the bathroom. Or you want to take a once-in-a-lifetime six-month cruise. Or you have a medical emergency or need long-term care.
Let's look at what occurs if you need a new car. Just because you're buying a new car doesn't mean that you can stop paying the mortgage or cell phone bill. And you probably won't stop eating, either. So your core lifestyle will remain similar, and hence your taxable income will remain consistent, too.
But now you need $30,000 for a new car. If all of your money is in a tax-deferred IRA or 401(k), where are you going to pull the money for the car from? The answer of course is that tax-deferred account.
The problem is that the $30,000 you pull from that account represents additional taxable income. So not only do you have to pull the $30,000 for that new car, you also need to pull money to pay the taxes on the money you pulled to pay the car. And to take that one step further, you also need to pull money to pay the taxes on the money you pulled to pay the taxes on the money you pulled to pay for the new car. The effect is not unlike a snowball gaining speed downhill, and the result is that your $30,000 new car can easily wind up costing $40,000 or more.
Oh, and by the way, this situation becomes even worse when you consider that the funds you're pulling from that deferred account might actually be pushing you into a higher tax bracket to boot.
Inflation causes a similar situation to occur with your core living expenses. With inflation, your cost of living increases as time goes by, which means that you need more dollars to live the same lifestyle. And, you guessed it. Those additional dollars needed mean you'll have to pull more funds from your tax-deferred accounts, leading to higher tax bills and declining account balances.
So, the bottom line is that a life paid for with funds from your tax-deferred accounts is likely to be a very expensive life indeed.
What's Certain in Life?
Let's summarize where we are so far. For starters, having all your funds in a tax-deferred account means you'll be carrying around the IRS as an investment partner for the rest of your life, which is a bit like running a marathon with a gorilla on your back. Then, your tax bills are going to increase over time as you withdraw money to keep pace with inflation or pay for the one-offs in life. But, that has to be the end of the bad news regarding the predicament you're facing.
Right?
Nope.
Just to make things even better, your tax-deferred investments come with a wonderful cocktail of rules and requirements designed to make you miserable. Let's take a look at three of them now:
Required Minimum Distribution (RMD) – Sure, the IRS owns part of your deferred account and they are going to collect taxes when you pull money from that account. But maybe you don't need the money and aren't ever planning on pulling it out. So you're good, right?
Wrong.
Remember, the IRS essentially gave you a loan when you put money in this account. You should have paid taxes at that point, but the IRS “loaned” you the money for the taxes and let you put it into your retirement account. Bu unfortunately, those pesky Internal Revenue Service agents aren't the forgetful sort. They lent you money, and they want it back.
Figure 8.6 Required Minimum Distributions
SOURCE: Information from Internal Revenue Service.
When you turn 72 the IRS decides its time to start collecting on what you owe them. They do this by forcing you to begin taking withdrawals from your deferred account, regardless of whether or not you actually need or want the money. And then of course they tax you on these forced distributions.
Required Minimum Distributions (RMDs) start when you turn 72. They then increase each year, as you get older, according to the following schedule in Figure 8.6.
So, at age 72 your RMD is approximately 4% (3.91%, to be precise). And then that withdrawal amount increases each year because the IRS wants you to deplete the assets before you die so that they can collect their tax money.
Required Minimum Distributions are inevitable, and if the majority of your assets are held in tax-deferred accounts, your 72nd birthday might start to look like a looming cliff.
There's only one real way to avoid those RMDs. However, it's quite a dramatic step, and I don't necessarily recommend it.
The reason I don't necessarily recommend avoiding your RMDs is because the best way to do so is to die!
That's right, if you die, you don't have to take RMDs. But it is, as I said, a draconian step, and one I can't really recommend.
And besides, don't forget that old saying about how the only two certainties in life are death and taxes. Because…
Income in Respect of a Decedent (IRD) – Even if you die, you still can't really avoid the taxman. Or, to be more precise, your heirs can't avoid him. That's because, in the event that you die while there are still funds in your retirement account, your heirs will be forced to pay taxes via Income in Respect of a Decedent, which effectively acts like RMDs for those who inherit IRAs.
That's right, you die and then the taxman comes along with a shovel and a crowbar. The taxman digs up your grave, uses a crowbar to pry open your casket, reaches into your pocket, and takes some more taxes in the form of IRD. Because even in death the taxman cometh!
There was a positive here, however. Remember, RMDs are based on life expectancy. And often times you wind up leaving your accounts to someone younger than you, like a son or daughter, or niece or nephew. And that younger person has a longer life expectancy, so their RMDs are less each year than yours would have been. For instance, if you pass away at 80, and your daughter is 40 at the time, the ensuing distributions she must take (assuming she sets this up correctly) would be much less than those you had been taking, since she has a much longer assumed life expectancy. This concept is known as a stretch IRA because the distributions are stretched across a greater number of years. This stretching results in more time for the funds to grow tax deferred with the result that your daughter receives more of an inheritance. This was a very good thing and could potentially save your heirs a great deal of money.
Let me pause for a moment and ask you a question. When you think of Congress, is the first word that comes to mind bipartisan? Probably not.
Which makes it truly impressive that in 2019 the House of Representatives voted 417 to 3 on a bill. Now, I'm not sure you could get that kind of consensus from the House around things as noncontroversial as:
- Having food poisoning isn't fun.
- Puppies are cute.
- Bell bottoms were a bad idea.
And yet that august body was able to join together in brotherly and nearly universal harmony as they agreed to a bill that included, among other things, the abolition of that lovely stretch IRA I discussed earlier.
So now, instead of taking those distributions over the next 40 years of her life, your daughter is forced to deplete the account you left her over a 10-year time frame. Which means far more of that money will go to the IRS, and far less will go to your loved ones.
But at least we finally got Congress to come together in bipartisan unity!
Penalties – So to summarize, the IRS wants you to get older (72) or they want you to die (IRDs). There's also a third thing they want you to do, and it's equally unappealing. Because basically, the IRS wants you to screw up!
That's right, your partners over at the Internal Revenue Service are hoping you make a mistake and miscalculate or forget to withdraw your RMD, because, if you do so, the IRS assesses a penalty of 50% of the amount you neglected to take.
In other words, if your RMD was supposed to be $20,000, and you neglected to take it, you'd owe a $10,000 penalty!
Making matters even worse, you'd still owe taxes on the entire $20,000 you were supposed to withdraw. In that scenario, you'd see your IRA balance decline by $20,000, but might only wind up with $5,000 in your pocket. The remaining $15,000 could easily have gone to your friends at the IRS in the form of penalties and taxes.
So, to summarize the details of the partnership you've entered into with the IRS:
- Your “partner” wants you to get older and reach age 72.
- Your “partner” wants you to pass away so they can tax your heirs.
- Your “partner” wants you to screw up so they can shift 50% of your partnership stake to their side of the ledger.
And of course, in any of the these scenarios, as well as in those instances in which you're withdrawing funds simply because you need the money to live on, the distributions are taxed as ordinary income, which represents the highest rates available.
When you look at it that way, you've unwittingly entered into quite the devil's bargain when it comes to how you're going to fund your retirement.
Fortunately, there is a solution.