People do the things that get them what they want. They may not realize that's what they're doing or even realize what exactly it is they want, but ultimately that's what drives all behavior. You're expecting his respect for you not wanting to be annoyed to override his desire for whatever it is he wants, which is understandable, but obviously not what happens in reality. Since he's not doing these things for the purpose of annoying you, then the solution is to make it so the things you want him to do are the best ways to get what he wants.
An easy one is your brother not knocking before entering. What today happens after that? Do you always tell him to immediately get out, or do you remind him to knock and then give him whatever it is he wanted? I bet if you refuse to engage with him unless he leaves, shuts the door, knocks, and waits for an answer, this behavior will stop. If knocking is the fastest and easiest way to get what he wants then he'll do it. If it's not then he won't.
The lurking behind your back may take a bit more thinking on your part. If he's interested in what you're doing, is there an approach you'd prefer he took instead of simply watching from behind you? If so, you can ask him to do that instead, and if he lurks behind you, make him get out. Right now his desire is to see what you're up to, and the best way he's found to satisfy that is to stand behind you and watch. Make it so that approach doesn't work and your desired approach does, and he'll naturally change his ways.
For the boxing, I'd set expectations up front and get him to explicitly agree. If he doesn't keep to his agreement, immediately end the interaction. The next time he wants to box, tell him you're reluctant because he didn't keep to his agreement last time. Make it clear that it if he doesn't keep his word, you will simply stop boxing with him. You may have to start sometimes refusing if he doesn't change his ways. Refuse more often if he keeps ignoring your boundary. Eventually he'll either reform or you'll stop boxing him; either way he won't violate your boundary. It'll be more effective if you can offer him something he wants when he does stop as agreed, but I don't have any suggestions on what that might be. Perhaps you can think of something?
Your approach of asking him why he's doing these things is a good one, but of course he's not going to say, "because I care more about satisfying my curiosity than I do about your feelings". He probably isn't actively making that decision any more than you're deciding you care more about being annoyed than about what he wants. Sometimes you have to figure out people's motivations based on what they do rather than what they say. It can also help to ask your question in a different way. "How come you're standing back there?" is better than "why are you lurking behind me even though you know it annoys me?" The latter will make someone too defensive to answer honestly, but the former might just work if you can avoid sounding annoyed.
As an older sibling myself, I'm guessing he also wants more time and attention from you in general than you'd prefer to give, and these things might all be ways of getting that. If so, making a point of doing things with him could reduce all of these other undesirable behaviors. Whether that would be worth the effort is up to you, though.