I'm going to preface my answer with a gentle warning. It simply may not work. Living with people encroaches on ones personal space and privacy. The more people you live with, the less personal space and privacy you get. That being said, you should be able to carve out some space for yourself, and that is how I would approach it.
This is a request for Personal Space and Privacy. Go into the conversation with Primary Roommate with that in mind. You have your own room. You pay your share for it. You would like to make it your space where you can make it your bubble, which is important to some people (myself included.) It has nothing to do with trust (even if some trust issues are involved omit that for this discussion, it's a red herring and can only be used to disarm your request.) When personal space is invaded it feels violating. Explain it as a request from that perspective.
It will benefit the conversation on both sides if you understand that different people have different needs in this regard. Your roommates are probably not flagrantly disregarding your boundaries; they probably don't see them the way you do. Obviously, Primary Roommate doesn't understand this need. It will be your job in the conversation to bridge that understanding between you and Primary Roommate.
One possible way to engage in this conversation would be as follows:
Hey {Primary Roommates Name}, I've been thinking about our previous
conversation about my room key and I'd like to clear some things up. I
can see why you feel like it is a lack of trust for me to do this, but
I want you to know I don't intend it that way. For me it is about a
sense of Privacy and Personal Space, And the only way I know how to do
that is with clearly defined boundaries. I feel uncomfortable just
knowing that others can enter my room without permission. I don't know
any other way to establish these kinds of well defined boundaries. If
you have another suggestion I am willing to at least consider it. But
moving forward it is important to me that the location to the spare
key not be shared knowledge. If everyone knows where the key is then
it is not a boundary any more.