Market Downturns & Tax Silver Linings

Benjamin Brandt:         Welcome back to the Retirement Tax Podcast. I’m your humble host as always, Benjamin Brandt. Joining me today as always, is the distinguished Steven Jarvis.

Steven Jarvis:     Oh, distinguished, I think that’s a new one. You’ve got your thesaurus out today, Ben.

Benjamin Brandt:         I literally do have a thesaurus up on my phone. I was hoping you couldn’t see that.

Steven Jarvis:     Oh no, I couldn’t see it. I just assumed. You’ve got so many great compliments for me, but I feel like that was a new one.

Benjamin Brandt:         It was. I hope that was new. I’ll have to cross that off the list. Steven, I’ve heard tales that the market’s not doing so hot this summer. What’s going on?

Steven Jarvis:     Yeah, you don’t have to go far to hear that message. That’s a pretty mild way to put it compared to a lot of the headlines I see. And we both take an approach with clients that we’re really … I mean, we’re talking about the long-term here.

Whether we’re talking about financial planning on your side or tax planning with Retirement Tax Services, I specifically look to work with clients who are open-minded; so, let’s look at the long-term, how do we get ahead over time. And it’s important to remember that when the market takes a little bit of a downturn like it is, but it can be a little challenging to keep that in mind.

So, really, today, we wanted to talk about from a tax perspective, what are some of the things that we should be thinking about as the market goes in the opposite direction of what we’re hoping for?

Benjamin Brandt:         Yeah, and we like to keep things light on the show. I mean, this is the confluence of retirement and tax topics. So, it can be a little bit … if you’re not a financial professional, it can get a little bit in the weeds.

So, we like to keep it light, but it’s difficult to keep it kind of light sometimes when you see in clients’ portfolios or when you talk with your clients, “Oh, we’re down 100,000, 200,000, 300,000 whatever it’d be — I think the average half stock, half bond portfolio might be down 10, 15, 20% kind of, depending on if you have more of a growth tilt or a value tilt.

So, we look at that account balance down 15% and we say from a retirement perspective, from a tax perspective, I think it’s a given that we’re going to stay at the course. But from a retirement and tax perspective, how can we potentially take advantage of this market downturn?

We’re going to always be positive, we’re going to buy the best companies in the world and never sell. That situation never changes. We’re going to build a Swiss army knife portfolio so that no matter what the market throws at us, we can have an amazing retirement.

But apart from that, assuming that the market’s going to bounce back, how could we take advantage of this situation so that maybe we can have a little less taxes to pay at the end of the year.

Steven Jarvis:     Yeah, definitely. And while we like to make light of some things on this show, we certainly get that this is a very emotional thing when we see our portfolios going down. We’re not trying to make light of that. We trying to give ourselves something else to focus on to take advantage of the positive. So, a couple-

Benjamin Brandt:         I think in the military, we would call that gallows humor, where things aren’t going so great, so you have to crack a joke. But-

Steven Jarvis:     That’s right.

Benjamin Brandt:         As financial professionals, we can look at this and say … we’ve been doing this for quite a long time. I’ve been an advisor about 15 years. I’ve seen two or three really bad downturns and a dozen sort of medium bad downturns. So, you just sort of get accustomed to it.

If you jump out of an airplane every day, eventually, it’s just your commute to the office. Something that’s thrilling and shocking and scary to other people, if you do it enough times, it just sort of becomes old hat.

So, yeah, again, we don’t want to take light of people losing money, but we know we’re extremely confident that it’s going to come back. So, we’re able to sort of crack the occasional joke but you were saying, Steven?

Steven Jarvis:     Yes, that’s spot on, Ben. So, in my experience, there’s two kinds of headlines that keep coming up when the market goes down, as it relates to taxes, as far as strategies we can implement. And so, we want to go a little bit beyond the headlines and kind of give some context to these two things.

And the ones that come to mind for me are tax loss harvesting, and then taking advantage of Roth conversions.

Now, Roth conversions, of course, something we’ve talked about before, they come up regardless what the market’s doing, but we want to talk a little bit about why there might be an extra advantage to doing it while the market’s down.

Benjamin Brandt:         Absolutely, absolutely. And I think we have a listener question from my show and I’ll read that because that that sheds light on the idea.

Question from Matt: last month, Morningstar’s, Christine Benz wrote an article about what types of investments to keep out of a taxable account. Do you agree with her recommendations? Also, what kind of investments should be in taxable accounts? What about after-tax accounts like Roths or HSAs? And then he also mentioned some individual stocks that we really can’t cover.

But how important is it to separate from his question, from Matt’s question — how important is it to separate your investments between the different taxability or tax rules that each of these accounts have to follow?

Steven Jarvis:     So, this is one of those areas where we can run analysis for days and some people do. And there’s financial professionals as well as DIY investors who are guilty of this, that we have seven different tools and 12 different spreadsheets, and we try to get this dialed into the perfect number where we know … we refer to this a lot of times as asset location and we’ve got the optimal balance of what that looks like.

And while I don’t really go to that level because it’s usually diminishing returns on the time we put in versus the benefit we get out, there are some general concepts we should be aware of, specifically around how these different types of accounts are taxed.

So, in general, we’ve got our taxable accounts, which is often is like a brokerage account. This is just the account you go set up by different securities in, there’s no tax advantage there from a how the money is going in standpoint.

Then we also have our tax deferred accounts: IRAs, 401(k)s, other accounts like that. And then we have our Roth accounts, whether that’s Roth IRA or Roth 401(k)s that are after tax accounts.

Benjamin Brandt:         So, some accounts are pay as you go. So, the money going in or the money coming out doesn’t necessarily decide when we’re paying taxes. It’s sort of pay as you go, profit and loss.

And then the other accounts are either something is happening tax-wise as we put the money in and then nothing as we go, or something’s happening as we take the money out at the end.

So, we have pay as you go or money in or money out. Is that kind of the two main things we’re dealing with?

Steven Jarvis:     Yeah, that’s spot on. So, we’ll start with the Roth accounts because it’s the quickest conversation for taxes, because the reason we want to get money into a Roth account is because then the IRS is done taking their portion.

So, the money we put in, the growth on it, that’s all after tax. We don’t have to get concerned. You should not be trying to do tax loss harvesting within your Roth account. That’s not a thing, that doesn’t work because it’s not going to be taxable anyway.

So, where there becomes a more important distinction is between the tax-deferred, which is we don’t have to pay tax along the way, but we’re going to pay taxes eventually, or our taxable account, which is the pay as you go. Because when we talk about tax loss harvesting, we’re talking about securities that are going to change in value.

Because in general, we have the opportunity to get better tax rates on the growth of a stock, as opposed to our W-2 wages. But that’s only true in a taxable account. In our tax-deferred account, we don’t have access to those preferential rates.

And so, that’s one of those distinctions that we need to keep in mind. And so, that’s where the question from the listener, from Matt, from a high level, we look at what types of account or types of investments should be in these different accounts.

Some of the kind of the broad strokes we can take are our highest growth assets, we are going to want to, as much as we can, put in our Roth accounts where we’re never going to be taxed on it or on the growth once it’s in there.

In our taxable account, we want potentially things that we can take advantage of those preferential rates, and then our tax-deferred account a lot of times will look at our income producing investments in there.

Benjamin Brandt:         Okay. Yeah, so as we’re looking at our highest growth assets, so people might be wondering why is my Roth IRA down hypothetically more than my IRA? Well, in many cases, people will take (and this is often something we recommend to clients) — we will take our more aggressive assets that we expect to produce a better return over time and we’ll put those in the Roth IRA because we expect that those will be some of the last assets that we access.

And so, thus, they have the longest time and we’ve already paid taxes on those, we want those to grow. So, maybe it would look something like my IRA that I’m living off of monthly, maybe that’s half stocks, half bonds, because during times like now when the market might be down 50% in some areas with some tech stocks, obviously, you don’t want to reduce your income by 50% during those times. So, we have bonds and stocks to kind of give us a nice balance.

But if we’re not living off a Roth IRA monthly, we might have 80%, 90%, 100% stocks in our Roth. Just because we’ve already paid the taxes, we’re not going to access that money for a long time. Maybe we never will. Maybe our kids will. And so, we can grow that long-term.

So, if you’ve been looking at your Charles Schwab statement or your Vanguard statement and say, “Boy, my Roth is down a lot more,” maybe you’ve been sort of following that same logic and now, you’re just realizing why that is the difference. And maybe not because both stocks and bonds are down, so maybe it’s equal across the board, but oftentimes, Roth IRAs are more aggressive and that’s why.

Steven Jarvis:     Yeah, so with that as kind of the framework of how these different accounts are taxed and what investments — again, from real high level, you might be looking at these different areas; let’s circle back to kind of what tax loss harvesting is and how it works.

Because we’re recording this in May, when this comes out in July, who knows where the market’s gone and then over those six or seven weeks, I can’t predict that, but probably still going to be a timely topic. And you’re probably seeing a lot of headlines of, “Hey, go out and do tax loss harvesting while the opportunity’s there.”

So, Ben, when your clients see one of those headlines and come to you with this, how do you simply explain here’s what tax loss harvesting is?

Benjamin Brandt:         Yeah, so tax loss harvesting just your general accounting rule — and you can correct me if this is wrong, Steven, since you went to accounting school and I didn’t. I actually changed majors from finance to econ because there was one less accounting class. I don’t know if that says anything about what I think about accounting or my study habits at 19, but I digress.

So, in accounting school, Steven, they probably taught you that the pluses and the minuses can balance each other out. So, when we think about tax loss harvesting, if you look at your Fidelity statement or your Vanguard statement, there’s probably some pluses left over because again, we had a lot of good years in the market, but there’s probably some minuses now.

So, if we’ve got minus 3000 in fund A or stock A, and we’ve got minus 3000 in stock B, well, we could sell both of those and they would kind of wash each other out. So, during good market times, if we sell a gain, we’ve got to pay taxes on that gain. Even if we bought it last year or the year before, the time that we realized that gain, we pay taxes that year.

So, what we can do is we can say, okay, well, I’ve got 30,000 in gains and I’ve got 30,000 in losses. If we sell those at the same time, they can kind of wash each other out. So, we can take advantage of that loss to wash out the gain that we’re going to pay taxes on anyway, going forward.

So, we’ve got to be careful that we don’t buy the asset back and there’s all kinds of different wash-sale rules that we’ve got to be careful of. Don’t get too cute with the IRS and that you’re going to sell your Vanguard fund A and replace it with Fidelity fund A that’s essentially identical because it’s an index fund.

The IRS might think you’re too cute by half and send you a really nasty letter in six years and we’ve long since forgotten we even did this. So, we want it to be genuine tax loss harvesting, but yeah, we can use that to balance our losses.

So, how we look at it is short-term gains can stand against short-term losses and vice versa. Long-term losses versus long-term gains and then income. So, if we’re in the unfortunate situation where we have a lot more losses than gains, we can write that off against our income tax, but not so fast, only to the tune of $3,000 a year.

So, if you have a $300,000 loss, I really hope you live a long time because it’s going to take us a while to work that. We won’t lose it, we won’t lose it — carries forward indefinitely, but we’ll have to 3,000 at a time.

So, hopefully, you’ve got gains and losses and we can kind of balance those against each other. Again, only in a brokerage account. Anything that has IRA at the end of it or HSA doesn’t work. This is just the pay as you go stuff.

Steven Jarvis:     Yeah, that’s a great reminder right there at the end, that when we’re talking about tax loss harvesting, this is only relevant in a taxable brokerage account. And I like that you highlighted the loss limitations because people get a little caught off guard by this, of you get really excited about the idea of great, let’s go harvest all these losses, I’ve got a hundred thousand dollars in losses, great. That a hundred thousand dollars I made at work this year, I don’t have to pay any taxes on it.

Okay, slow down. That’s not how it works. The IRS does put limits on how far you can go with that. And so, being able to offset gains is where we’d love to start and then that $3,000 number, that’s how I instantly know if there are additional losses we got to pay attention to, next year is when I’m looking at a tax return and I see a nice round $3,000 loss on the tax returns; okay, that got capped.

Which there’s value there from a tax standpoint, we just offset $3,000 of ordinary income. And we can keep doing that every year even if we don’t have additional capital gains in the future, but we want to pay attention to what that dollar amount is.

To your point, Ben, maybe it doesn’t make sense to be so aggressive with tax loss harvesting, that now we’ve got hundreds of thousands of dollars of unrealized losses and when are we actually going to be able to do something about that.

So, we want to make sure we’re keeping track of it and planning ahead. Maybe, we can use those … in fact, worked with a client and their advisor this year who had this loss has been carried over for a few years. And then in 2021, when some of their investments were up, they went ahead and intentionally recognized some capital gains because they knew they had the losses to offset it.

Benjamin Brandt:         Oh, perfect. That was going to be my next question, is let’s say we harvest 10,000 more in losses than gains. And so, we can only write off as much as our losses and our gains match, $3,000 next year. But the following year, because we’re optimistic, the market bounced back and we’ve got a balance of 7,000, we could just take that whole 7,000 if we offset a gain.

So, once we’re doing 3,000 a year, we’re not locked into that forever. We’re just locked into that until we have gains to offset.

Steven Jarvis:     Yeah, that’s a great point. And tax gain harvesting doesn’t get talked about as often, and it might not be relevant for most people this year with what the market’s doing, but as we-

Benjamin Brandt:         Yeah, where were you six months ago?

Steven Jarvis:     I was writing articles about tax gain harvesting.

Benjamin Brandt:         Oh good.

Steven Jarvis:     But as we move to the future and we’ve got those losses we’re carrying forward, again, Ben, you mentioned the wash-sale rules on losses, where if we sell a position, we can’t just immediately buy that same position or really even a substantially similar one.

That doesn’t apply to capital gains harvesting. And so, as we get into 12 months from now or 18 months from now, and the market’s back to where we had hoped it would be, we can sell a stock for a gain and then immediately, repurchase the same stock because we’re excited to have in our portfolio just so that we can offset those losses. And so, there’s some really great planning we can do if we’re intentional.

Benjamin Brandt:         Excellent. So, the next thing that people think about when they think about the market being down, this is … Steven and I have a lot of conversations with advisors. A lot of advisors are talking about Roth conversions now when the market is down.

So, Steven, maybe you can share with me a little bit about making the best of a bad situation, the market’s down: why are people talking so much about Roth conversions in a down market?

Steven Jarvis:     Yeah, I love hearing this come up because it’s really focusing on the positive or the silver lining, however, you want to look at this.

No, we don’t want the market to be down, but while it’s down, especially if Roth conversions were already on our radar, this is already something we’re interested in, already something that makes sense for us — it’s basically getting that Roth conversion at a discount because we’re going to pay taxes at the time of the conversion based on the value of the account at the time of the conversion.

And so, to use a really extreme example, if we had $150,000 we wanted to convert, and then between the time we started talking about it and the time we actually went to convert it, our investments took a huge dive, and now, it’s at a hundred thousand dollars — great, we can still get that same portfolio, those same holdings converted. But now, we’re only going to pay taxes on a hundred thousand dollars, not $150,000.

And as long as we still have this long-term approach and we’re still confident this is the right thing to be invested in, that over time this is going to go up, this is fantastic news for us because we paid less in taxes.

And the IRS doesn’t go back and do some kind of retroactive analysis to say, “Well, if you would’ve converted at this time, as opposed to that time …” that’s not how it works. Conversion happens at a point in time, we pay our taxes, and we move on.

Benjamin Brandt:         Yeah, and if that doesn’t totally make sense, what was 150,000 is now a hundred and that’s good; if you fast forward a couple of steps to think what we’re solving for, we’re trying to right size your IRA so that we don’t spike our income taxes down the road, mess up our Medicare, all those sorts of things.

If we’ve done sort of the portfolio analysis, the tax analysis let’s say, we’ve got to convert 4% of your IRA every year. Well, now, we can get the same value out of that IRA. And we only have to convert 3% or 4% is a smaller number or something like that.

So, if you think about 150,000, a hundred thousand, whatever that is, just think percent of the total, and then that is going to probably maybe make the math make a little bit more sense.

Steven Jarvis:     Yeah. So, I know that most people don’t really think in terms of percentages. I always have to remind myself of this, that we definitely talked a lot about percentages at accounting school. So, I’m going to use really round numbers, this is not precise math.

But if we said, okay, we were planning to convert at 150,000 and let’s just assume a 20% tax rate. So, if we convert at 150, we’re going to pay $30,000 in taxes. That’s my quick back of the napkin math.

So, great, it’s going to cost us $30,000 to get that into raw, which still might be the right approach for whatever our situation is. But if instead of having converted to 150, we take advantage of this market downturn and say, great. Now, my investments are only worth a hundred thousand, great.

Now, I’m only going to pay $20,000 in taxes. I’m taking a hundred thousand times at 20%. And of course, that’s oversimplifying how the brackets work, but just go with me on this so we can put some real numbers to it.

And then regardless if I converted 150 or at a hundred, at some point, that’s going to go back up and we’re going to end up in the same place long-term, as far as the value of the account. But now, I’ve paid taxes on a much smaller … I’ve only paid $20,000 of taxes instead of $30,000 of taxes.

And when I ask people if they’d be okay saving $10,000 in taxes, no one tells me no.

Benjamin Brandt:         Absolutely. Logistically, how we do it in our office is, it’s a little trickier if you’re a year from retirement or a year past retirement, because we don’t really have that income track record. Your income is going to change so much differently — the last year you worked, the first year you retired, maybe the second year you retired.

But let’s say you’re four years into retirement, we’ve got kind of this track record to look back on, and your income is kind of levelized. Maybe social security is different, maybe Medicare, there’s some things that are different that first decade of retirement. But for a lot of clients, they sort of settle into an income level and so, we’re adjusting their income every year based on the marginal bracket.

So, let’s say we’ve done $25,000 of Roth conversions every year for four years in a row, and this is the fifth year. Well, when the market goes down, we’ve got this great sort of track record to look back on and say, we don’t know if the market’s going to go down more or what have you, but we can take advantage of this bad situation by saying there’s a really strong likelihood we’re going to convert 25,000 in December of 2022. Why do we do half of that amount now?

Because if we have the track record of 25,000 a year for a couple years, we know that that’s not in particular going to mess up our income because we’ve proven that out in the past. Let’s do half of that amount now.

Maybe you don’t want to do the whole thing because if you wreck your car, you need to buy a new one or the water heater blows up or your roof blows off your house, we might need to dip into the IRA to get more. So, we don’t want to do the whole Roth conversion midyear because we might need more.

But then we’ll just reevaluate at the end of the year and say, well, great, nothing happened where we needed to take more money out of the IRA. We just convert the other half.

So, we can take a little bit of advantage of a poor situation by saying that thing we’re going to do in December, let’s do half of it now, and then let’s say the market goes down another leg of 10% — well, we could do another fourth, another 25%, whatever it is.

But it even gives you that little, I don’t know, if it’s serotonin or dopamine in your brain or whatever it is, saying I had a positive, I did something good in this negative situation. You get a little win, a little attaboy by doing that or attagirl.

So, that’s kind of logistically how we do it in our office if you can kind of picture that in your mind and where you’ve done a similar Roth conversion for years in a row, maybe you could look at that and say, well, I’ll do half of that now and then I’ll be in a little bit better of a situation.

Steven Jarvis:     Yeah, I think for me, the big takeaway for this conversation is kind of what you hit on there towards the end Ben, of we want to make sure that this fits in our overall plan. We’re not drastically changing what we are doing.

This isn’t going from I was never considering Roth conversions and now, I should convert my whole balance all at once. Like this still needs to fit in our plan, it’s making adjustments. But it certainly is a, “Hey let’s take the time to consider what our current opportunities are.”

And then while we’re dealing with the nick of emotions of watching our account balance go down, this can be that silver lining, that little hit of dopamine or serotonin (I’m with you, I don’t know exactly which one it is either). But that we can get some good out of this.

So, definitely a good time to say, is this a fit for me and maybe this is one of those times where you even say, okay, maybe I should take the time to talk to someone else, if I’m typically a DIY-er, make sure we get this right. Or if you’ve been through it before, great. Just make sure you’re taking the time to think about these different pieces we’re talking about.

Benjamin Brandt:         And just a nice little reminder of what we’re all doing in the first place, we’re investing because we long-term are optimists. So, we’re doing this Roth conversion, if that is appropriate for you, because we’re optimistic about the market, we’re buying … again, the best companies in the world are never selling.

So, we get to ride the market back on the friendly side of taxes. That’s really why we’re doing all of this. So, by doing this Roth conversion, just a little reminder that it is going to come back and we’re going to look back on this in four or five years and hopefully, barely remember it.

So, the tech crisis and the financial crisis and COVID, those are all really terrible things. But just a short couple years later, we’re all recovered and as long as we had a good plan in place ahead of time, nobody went bankrupt that was investing in index funds, 60/40 with a plan to reduce overtime if the market didn’t keep up. None of those people that stuck to the plan went bankrupt, and this time is not going to be any different.

So, with that being said, Steven, always a pleasure.

Steven Jarvis:     Ben, it’s been great.

Benjamin Brandt:         And until next time, don’t let the tax man hit you where the good Lord split you.