There are literally dozens of things you can do in public that are not technically displays of affection, but that in fact display affection. Early in our relationship, my partner used to stride out ahead of me when we were headed somewhere, leaving me trailing a few feet behind. If I pushed hard to catch up, he'd push hard to get back ahead. I hated it. (I'll omit the details of how it made me feel.) Eventually, he learned this. He learned to walk beside me whenever possible, and if we had to go single file, to look back and check on me frequently, or to let me go in front. There may be a similar mechanism here. When you go somewhere together, do you actually go there together or just at the same time?
You can examine other aspects of your behaviour too. Do the two of you chat as you do whatever it is you're doing in public, or are you super task-focused (getting to a destination, looking in store windows for something specific, etc) and pay no attention to your partner? Can you find some energy to spend pointing out interesting or pretty sights? Can you tell them how your task is going (ok I think it's 3 more blocks, we're going to turn up ahead, damn another shoe store why have all the bookstores closed) or otherwise engage them in conversation?
And of course, if you lavish attention (not just physical contact) on your partner in private: listening, asking interested questions, valuing -- even praising -- things your partner says and does, then the fact you're a little aloof in public should be easier to take. I accept the fact my partner is not going to take my hand as we walk down the street. That's not the only gauge I have to measure how he feels about me.