If it's someone you know fairly well, I would recommend politely saying:
I'd rather text/email. I'llI'll be able to provide you with more details if I'm able to visually see my thoughts before giving them to you. IfIf it's a hassle for you to write me about this, would you have time to meet up and talk about these things instead? That way I can give you my full attention when talking about this matter.
If it's a stranger you've already answered the phone for, I would (and have) said something more along the following:
Now isn't a good time for me, and I would prefer to respond to your inquiry by text or e-mail, as I could respond much faster and with greater detail on the subject.
If it's a stranger who wants to contact you by phone, use the above but omit the part about "Now isn't a good time for me.
These responses deflect anyone from trying to convince you to enjoy talking on the phone, and instead refocus to you being able to provide a greater contribution to your interaction with them by communicating in a preferred way.