Quick note: This answer is writing from my own experience. I fully acknowledge that there are (several?) men (maybe women too?) who are far less considerate than I am and just push out to get as much space as they want.
Why I spread my legs (and why your attempts did not work)
I'm a tall man. 1,94 m (6 foot 4.4 inches) to be precise. I also can't stand heat very well. Especially around the parts that biologically make me a man. To combat this heat I prefer to reduce the contact area between the chair and my say upper legs as much as possible. At home or in my office chair this means I'm sitting so far to the front of my chair that anyone gently tapping it could push it from under me sending me crashing to the floor. (Quick note: I'm already wearing shorts most of the time, so adjusted clothing can't help any further).
Of course, in public transport, there's no room for this extreme sagging. Instead, I'm already forced in a more uncomfortable position and the next best thing to keep my nether regions somewhat in the livable temperature range is to spread out my legs as far as my surroundings let me vent it at least a little bit.
I'll attempt to at least keep some space between our legs but if, after a few moments, you moved your leg to gently touch mine I would probably take that like you don't care that we're touching so I probably won't change my position. Depends mostly on how I noticed the touching if it's gentle enough I might not even know it happened at first.
The second attempt, pushing my knee in a small tap has exactly the effect as you noticed already. I'll move my knee a couple of centimeters further so they're not touching when you sit in resting position. Since I have happily given you the now required extra space to sit comfortably (at least physically) all is fine from my point of view.
The problem we're facing is that from my point of view, I'm already limiting my position so that you can at least physically sit comfortably. I know I might be invading your personal space but I consider that invasion about equal to my own discomfort of not taking more wanted space and touching you instead. A woman next to me who doesn't know anything about how uncomfortable it is to close my legs will value her personal space far higher instead and think this balance highly unfair instead. (Quick note: most often this doesn't happen with men since they take the same approach as I do, and thus we both end up somehow equally spread out without mentioning anything).
What does work
Asking me kindly without blame.
You cannot phrase the request in a way that blames me for taking too much space. I know I'm taking more space than you would prefer. A request like that would feel to me like you're not taking into account that it might possibly be uncomfortable for me to do so. So statements like "you're taking too much space" mean nothing to me and will push me into a defensive mode which creates exactly the kind of conflict you would rather avoid.
The only thing you can do is try to make me feel that I've been underestimating your personal space in my tradeoff. Questions like "Could you give me a bit more space please?" or "Can you move your legs a bit please?" will work if you keep the tone at least neutral (if friendly is impossible since you're agitated). I'll always give you a bit more room. Most likely 15~20 cm depending on how much space there is for me to move my legs. You can't expect me to really move my legs parallel to each other if there's over 10 cm between us though since that would unbalance things too much against me.
Alternative reasons
This answer is based on heat as main issue. Of course there can also be other reasons why it's preferable to spread your legs:
1) low seats: All tall people will have exprienced this regardless of gender. If the seat is too low and you relax your legs, your knees will automatically fall to the sides. It takes effort to keep your knees together in such a situation and men having an "obstruction" in between doesn't help staying motivated on doing so. Thus some involentary spreading becomes almost inevitable.
2) fat: No, I'm not pointing fingers here, nor am I telling people to lose weight. I'm merely stating the fact that if there's fat in the way, closing your knees might actually be physically impossible. Be mindful that overweight people might already be really self concious about their body and asking them to do the (almost) impossible may be far more uncomfortable for them then tolerating them into your own personal space (at least that's what I've been told).
3) personal space: Usually done to prevent people from sitting next to you in the first place, same reason as to put a bag or purse on the seat next to you. (I might have done this once or twice when I was in a really bad mood myself already). The idea is that by taking up too much space, other passengers will first take the other free seats instead of sitting next to you. The correct way to deal with someone like this is to just nicely ask "can I sit here?" to socially force them to make room for you (just don't expect too much cooperation afterwards though).
Extra notes:
Comments like "Why don't you stand if sitting is uncomfortable" really aren't helpful. This exact same reason can be given to a woman that wants more personal space which isn't what we want either. I would prefer a balance on how far you let me invade your personal space along with a mutual understanding that it's a bit uncomfortable for both of us.
"Why don't you see a doctor if heat is such a problem". A doctor isn't able to make it any less hot. I've been to several doctors to deal with certain symptoms if things got too bad but that doesn't help with the core problem (me being too tall and getting too hot easily).
"That's just a lame excuse" or "Just learn to deal with heat" really aren't helpful. Believe me, if I could do something to take away my discomfort with the heat I would. Until such a solution exists, we're going to have to learn to compromise both ways (not only blaming men for having sweaty balls).