I would look at the person speaking, or who is about to speak if there is a pause. If I am the one speaking, I look at one person for a few seconds and switch, except I am adressing one of them specifically, then I look at him/her.
If I am talking for a long time, I make open gestures, so I am not unintentionally blocking someone with my arms.
I would try not to talk about one person with the other person, e.g "Hey B do you remember when A messed up XY?", because A would feel left out. If you and another person are talking a lot and the third one is quiet, I would try to invite him, e.g "A, what do you think about XY?"
If giving a speach, I draw an eight with my head(slowly!) so everybody feels invited.
EDIT: So I look at the person speaking, because not looking at him is rude (in my opinion) and the other one will not feel left out, as he can look at the person speaking aswell. If you speak and no one looks at you, you will feel left out and awkward immediately.
If I am the one speaking, switching from one to another is a simple technique, so everybody gets attention, if I adress someone specifically, I look at him, because that undermines my intention to adress him. Supporting your look with your gestures is an easy way, of inviting everyone into the conversation. Imagine your arms draw two lines, and everyone between those lines is part of the conversation.
I am not talking about one person with another one, because he will feel left out very easily, this is based on personal experience (sorry). Inviting someone by adressing him, is my way of giving quiet people a chance to join the conversation, obviously if they do not want to join, do not do it :)
Drawing an eight is just one way to ensure, to look at the whole crowd. It does not matter if there are two or two hundred. If you draw an eight, you look at the center and each corner and your head moves smoothly. Same thought process when talking to two people and looking at everyone for a few seconds.