I actually used to have this issue, as I would frequently use incorrect words or phrase something incorrectly, even as a native English speaker. I still do, but I've learned to not let it affect me in conversations if I ever do slip up.
As AskMs.D stated already, I am assuming you are subconscious about not being fluent in English. That's perfectly fine, nobody should be judging you for that. My go to response for people mocking or pointing out my incorrect grammar is to joke about it or dismiss it as a non-issue. Point out how a second language is hard to master (especially considering it's English) and that you're not perfect yet. Be sure to thank them for correcting you, as the correction is usually warranted.
Here are some references that you can base what you actually want to use in conversation.
Jokes (Only if you're witty enough to pull these off.)
"There's always something about English that I mess up on."
"My brain's thinking too fast right now."
Dismissals (What I would recommend.)
"Ah, that's the word. Thanks for the reminder."
"That's what I meant."
The dismissals are short and get right to the point, so I would recommend starting with that. With jokes, YMMV (Your Mileage May Vary), which means it depends person to person. Some people are better at making jokes than others, so only do it if you're comfortable with it yourself.
Don't mind your colleagues reactions to what you said. Again, you are not a native English speaker, so there is only so much you can do, as you will not possibly know as much as someone who has grown up speaking English. Keep learning and just adapt your message with the recommendations that your colleagues give you.