As someone who (while being told of Santa) was never told anything of entities like the tooth fairy or Easter Bunny, I encountered this sort of attitude from adults when I was young. I would say that the approach would need to be adjusted basedonbased on who you were speaking to.
With strangers I think it would be best to simply smile and mention "Oh, we don't really go in for all that" or something similarly vague but still friendly. This helps in case there are other children in the vicinity and you aren't pushing your beliefs on anyone.
FriendsIt would be best to let friends/family if possible you are best letting know in advance whereif possible. WhereWhen it isn't I would again go with the first method if their kids are with them but just be direct if not. As long as you are pretty matter of fact about it, they should be quite understanding. Over-explaining your reasoning is unnecessary and can risk making you come across as "holier than thou", which you want to avoid.