#So, you've tried...

• Confronting him to come to a mutual understanding of what's wrong and solve the issue. But he doesn't admit that he's done anything wrong or apologize.

• Putting up with his behavior and hoping it'll get better. But it clearly hasn't and things are getting out of control.

Given the above, I'd suggest you stop trying to deal with this together with him. Assume the worst--that he will not see reason on this--and let's work from there.

So what can you do?

#Take control before the confrontation starts

When he starts arguing that you're doing something wrong and you argue back, you're giving him the power in the situation. There's an old saying from Mark Twain:

Never argue with a fool, onlookers may not be able to tell the difference.

If you argue with a fool, you'll be dragged down to their level. A perfect example is the conversation on "gaslighting" you mentioned in the comments. If you take these arguments seriously he'll make you question yourself with these accusations to keep you in the argument.

It sounds like the one time you successfully prevented this was by shutting him down before the argument started, but how can you do that when he's not on "best behavior" with other people around?

#Enforce your stance

Find a time when he's calm and tell him that some of the conversations you have with him upset you. It sounds like you care dearly for him if you're willing to put up with all of this and still try to have him in your life, so tell him that! But also warn him that the next time an accusation/argument/conversation starts that makes you feel this way, you will end it immediately. And that if he tries to continue, you'll leave.

This could look something like:

Hey dad, I had a lot of fun hanging out with you like at rock climbing the other day. But sometimes we have conversations that really upset me. I want you to be a part of my life, but I don't like these kind of conversations. In the future if one starts to come up, I'm going to tell you "Dad, I'd rather not have this conversation. Please stop." I'd appreciate if we could drop what we're talking about when that happens. I love you and I want to see more of you!

And the next time you feel one of these conversations coming on, recite the line you gave him: "Dad, I'd rather not have this conversation. Please stop."

If he continues the conversation, or begins yelling at you for telling him to stop the conversation or does anything except quit the yelling and change topics, remain silent. If you're in the car, turn and look out the window. If you're in a place where you can, walk away.

If he stays silent or gives you some lip and then stays silent, take a deep breath and try to change the subject. It won't be easy in an awkward conversation like this, but it'll get better the more you do it!

#The effect this should have

One of two things should start happening here. Either A) he'll learn that he needs to being heeding your warning when you tell him to stop because you mean what you say or B) you'll find yourself remaining silent more and more and seeing him less and less.

I know you said that you don't want to remove him from your life, but if you set down this path, you need to stay strong and stick by what you say. If that means seeing him less until he learns, then that may be the only way.

#Conclusion

I know you said you wanted a simple patch that you could start using immediately to fix these situations, but I think laying a groundwork like this and enforcing it are your best bets at dealing with just these arguments.

This situation sucks, but if he's going to act like a child, you may have to treat him like a child. He's not responding to your attempts at reason, so you'll have to teach him through action.