This is a very tricky situation indeed.
On the one hand you don't want to upset Tom further, but on the other hand you shouldn't give up on having a relationship because someone is giving you ultimatums.
It's quite likely that you will have to choose between Tom and your co-worker (let's call him Jerry).
Before you decide what to do, put yourself in Tom's shoes.
(Please bear in mind that this is a partially hypothetical scenario.)
Presumably he thought the relationship was going well and perhaps didn't think it was as 'casual' as you did, perhaps he thought you two had a future. (Perhaps Tom doesn't even have a concept of 'casual' dating, some people don't.)
Then you broke up with him and you told him that this was because you had "lost feelings for him". You neglected to mention that you had found someone that you were more interested in because you believed this was somehow 'protecting him'. Tom is at this point probably very hurt.
He believes you have fallen out of love with him and he's most likely confused about why. He tries to find a reason for this, something to blame. Eventually he concludes that there must be someone that you're more interested in. He asks you about this and you admit that you concealed the truth from him.
Tom has just simultaneously discovered that not only was there someone that you were more interested in, but also that you concealed the truth from him (some might even class that as lying, either way it's a form of deception). Now Tom is even more hurt than before. You have hurt his trust by lying to him, and he now has an object on which to direct his hurt and hatred - Jerry.
Now that he knows you lied about it, his trust has been irreparably damaged. He's probably beginning to wonder whether you're still be hiding other things from him. He might even believe that there was more going on.
Like many people who feel wronged, he wants 'justice' - he wants those who he feels are responsible for his pain to be 'punished' for it. He is manifesting this with his ultimatum. He probably thinks that by preventing the relationship with Jerry from happening, it will make him feel better and will somehow provide him with 'justice'.
He is of course wrong to try to impose his will on other people, but when considering his point of view, it's understandable that he's letting his emotion get the better of him. Letting go of someone you like is hard.
Reluctance to let go of either person is the whole reason that you're asking this question. You don't want to let go of Tom or Jerry.
The fact of the matter is that there's nothing you can say here that will quell Tom's feelings. In his mind he's set you an ultimatum and he will enforce that ultimatum.
If you are sure that Jerry reciprocates your feelings then you absolutely should get together with Jerry. You should tell Tom that you want to remain friends but if he no longer wishes to remain friends then that's the way it will have to be.
The reason Tom sees his ultimatum as a tool is because he believes that "not being friends anymore" is a significant enough threat that you won't take that option. If you show him that you are willing to accept that condition, his threat is empty and his ultimatum backfires.
If Tom doesn't try to contact you to say that he regrets his ultimatum and wants to stay friends, then you must be prepared to let go of him. Perhaps someday you will reconnect, but until then you must try to forget about him and focus on your decision to be with Jerry.
If Jerry does not reciprocate your feelings then the objective changes somewhat. Forming a relationship with Jerry no longer becomes an option, and you're left facing a hurt possibly former friend who has forced an ultimatum upon you.
You must decide whether to try to reconcile and forgive his foolish threat, or to cut ties with him because of it. At this point you've both caused damage to your friendship, you by hiding your feelings for your co-worker, and him for threatening you with an ultimatum.
Either way you must make it clear to him that you aren't happy with his attempt to dictate your actions and that getting back together with him is out of the question even if you do remain friends.
He may end up deciding that if there's no hope of being 'together' with you (even when you aren't with someone else) then he might prefer to cut ties with you because it's less painful.
I'd ask that you make sure that Tom has someone who he can lean on for support after you've gone. The end of a relationship is a tough time that causes many people, men and women alike, to do stupid things that they regret. As many will readily predict, he might try to come back with a vengence (e.g. stalking, violence etc), but what many people forget is that he might alternatively fall into a deep depression and try to take his own life. Suicide is one of the biggest killers of young males.
Most likely he'll eventually get over it and move on with his life, but it pays to be aware of the less appealing alternatives. Good friends or a strong will/mind are imperative for preventing the negative outcomes.
Ultimately the thing to take away from all this is that secrets and lies are the best way to kill any relationship.
Be honest no matter how painful the truth is - if it's out in the open then you can deal with it openly and honestly.