This is from my experience talking with people who don't speak fluent English, so maybe it will help or maybe not.
I know you said you can only understand 5% but I don't know if that means you can understand a single word or anything. But if you can pick out a noun/verb, the best way I've found is to ask Yes / No Questions. If you understood 5% of what he's saying, you could try:
(mumble) weather (mumble)
"Oh, I think it's suppose to rain today, did you bring an umbrella?"
"It's a beautiful day today, sad to be stuck indoors today, right?"
Or if you can't understand from the tone of the sentence or any words at all, or read any context clues, you can try
"Sorry, I only heard the word "weather," were you asking what the
forecast was like today?"
wait for him to nod or shake his head. Or even just hand them a pencil and paper. Or whiteboard + marker.
Maybe try to pay attention to to the context, like if you're both in the break room about to make coffee, maybe he's warning you there aren't any more cups or telling you where the creme/sugar is, or offering to brew some decaf? But definitely try to use context clues if it's possible and ask based off that? But this is only if you feel like talking to him.