I want to address what I see as the most important part of this question, and that's that you were wearing a branded shirt.
Branded shirts are a whole different issue than a random compliment on your outfit. Some sub-cultures have well defined and established rules for when you should wear their shirts and when you should not.
References below, but some examples:
Generally in the US it's expected that if you're wearing a university shirt you have some interest in that university (Went there, going there, want to go there, support their athletic team(s), have a child/sibling that goes there, etc.)
Overseas, it's very different. I frequently see people wearing university shirts that they didn't attend because it's very popular to do so. I'm an American living in Leamington Spa England, and when I see someone wearing a University of Arkansas shirt and I say 'Woo Pig' to them and they look at me like I'm crazy, it's a little offensive. However, culturally what they're doing over here is perfectly fine and acceptable (and they're very unlikely to run into many alumni.)
Running shirts are also another culture where there are set, defined rules. You shouldn't wear a race shirt on the day of the race (it's considered anything from offensive, to bad luck, to arrogant by other runners).
Not attire, specifically, but travelers frequently display flags of the countries they been on. Usually on a pack, but sometimes on a jacket or other article of clothing. It would not be unusual for me to have someone ask about my Morocco flag, if it was the only flag on my pack that they haven't been to. They might ask what I did there, whether I enjoyed it, and whether I'd recommend it to others.
Lastly, when it comes specifically to the growing 'nerd-culture' shirts, more and more it's becoming a similar sub-culture where wearing a shirt says, to many people, that you like said thing. I have a Firefly shirt (the TV show), that's a spaceship inside of a jar with holes in the top. I get more comments on it than any other shirt I own. Some people are curious what it means, some people just say "Nice shirt!", and others just look at me and say, "Man, I miss that show."
Rarely does this conversation go anywhere other than that, and I feel like you over explained by saying you're not a Star Wars fan. Shirts like that are a conversation starter, and many (but not all) people will assume that it means you have something in common. A simple "Thank you" would suffice for 99% of the people you come into contact with and they wouldn't say much more in response. If someone presses and really wants to get into a discussion about Star Wars, then what you said is completely appropriate, but I might turn it into more of a joke to smooth things over:
Thanks, it's a funny story really. I've not even seen Star Wars, but my aunt bought it for me because I wanted to be an astronaut when I was a kid and now it's one of my favorite shirts!
That way you can be honest, don't downplay the importance of their interest in the subject matter, and you get to share a laugh.
Running Shirt Etiquette: http://www.psychowyco.com/id74.html
Stack Exchange Discussion about University Shirts: https://academia.stackexchange.com/questions/41427/does-wearing-a-university-t-shirt-imply-that-the-wearer-is-affiliated-with-the-u
