How can I politely point this out and try to change the behavior?
This is exactly why meetings exist and what they are about!
Instead of having to explain yourself (bad idea) or ask why he's asking (bad idea too, because it'll increase the level of stress and may lead to damaging the relationship), call for a meeting.
This way, you acknowledge that some things (that he doesn't like) happen but not necessarily because of you. And that you're willing to help solving the problem. Other roommates have to be involved too, of course. Like a team. But it's to point out big problems, not small (still annoying) things. For the latter, you may have to suck it up though, like adults do. Let's just find a way out together. Talking and explaining is the key.
This must not be a "trial" or even passive-aggressive (like notes), just a meeting where people congregate and share thoughts and/or problems. It can easily be done around a couple of pizza. (1)
This way, you're the good guy, willing to listen and take your share of the burden, but not before everyone is aware. Important: instead of problems, you provide people with a solution. And that's why the others have to be involved. Because you say that, most of the time, it's not your fault. The goal of the meeting is that every individual within the house knows about the chores, the duties and the rights, the respect, both individually and collectively.
During the meeting, listen to what is said. Carefully. Don't argue, even if some aren't aware or are lying. Keep track of what is said. And get there with many ideas and arguments ready. Never go unprepared. People may then realize that they made mistakes they weren't even aware of, you never know! And that nothing wasn't done "on purpose" to "let it just go and others will do". Still, they can be lazy, so the purpose of the meeting is to separate the good and the bad: let them know, let them weight, let them decide, let them (dis)agree.
In the past, I've already helped someone in this situation, and it was based upon my experience with teammates. They tried, and it worked like a charm. Where business meets family/friends. Different people, different relationships, same problems sometimes...
(1) read more about Jeff Bezos running a meeting.