Well, sounds like you're interested in your friend's little sister and it also sounds like you're pretty set on pursuing a relationship.
So...
Let's take a moment to examine what sort of relationship you want to pursue with this young woman, cause if it was a young woman who meant something to me, that would be my first question. Are your intentions honourable? Do you intend to have some kind of respectful, lasting, and meaningful relationship, or is it just lust?
Admittedly I don't have a little sister, but I've had daughters, and if anything, that's probably a higher standard. You want the people who pursue the people you care about to treat them well. Any inclination that this pursuer's intentions aren't as pure as the driven snow, and all those nasty protective instincts kick in. You probably want to avoid those protective instincts. They may be bad for your friendships in the best case, and bad for your health in the worst.
Check your motives first. If you really want a real relationship with this woman, then proceed.
Next, think about your relationship with her brothers. Do they have a high opinion of you? Have they seen you behave questionably toward your previous partners? Effectively, would they think you're a respectable match for a young woman who means the world to them?
If you can check both of those boxes... Are you willing to risk all of these friendships for the relationship you might have with this woman? If that's even a question you have to think about, you probably shouldn't move forward. Because, doing this even in the very best circumstances, can and will, risk all of these friendships. Even if things go wonderful for years, if things turn bad, you should expect to have a falling out with all of these people.
Should you decide that your intentions are good, and that her brothers like and respect you well enough, and that it's worth the risks... Cause you will... Cause young love is like that... Brave, or foolish, or both... probably both. Talk to the young woman first. You're not asking her brothers for a date, you're asking her. She's your priority. If she likes you, and her brothers don't, they may get over it in time. If you ask her brothers first, she could and likely should feel somewhat demeaned and controlled.
(I realize that sounds counter intuitive given the opening paragraphs about the brothers respecting you, but bear with me.)
You'll be changing the context of all of these relationships. She'll need to see you as more than her friend, or her brothers' friend. And her brothers will need to see you as a potential brother-in-law. If you want this to go as smoothly as possible, and maintain those friendships, you'll need to seem like a good fit in both categories. But neither work if she doesn't like you, so that's going to be your first concern.
If she doesn't want to pursue something more with you, respecting that fully will be your best bet toward moving forward, without damaging all these friendships. (Another reason it'll be important to ask her before testing the waters with her brothers) Asking her brothers and then being shot down by her will likely lead them to question any and all interaction you have with her from then on.
why I was taking his sister to prom with some hostility in a joking manner
This is key to understanding the brothers' opinion, but you'd need to be more specific. Was it actual hostility played off as joking? Was is a joke about supposedly being hostile? Was there an assumption that you wouldn't have romantic intentions and it's more a matter of two friends with no other prom dates? There are a lot of options here and they're not easy to distinguish between, especially when we only read your interpretation of events.