They are definitely not unfriendly or rude or anything, but some of them can't even look me in the eyes, when I greet them in the hallway, and oftentimes it's just really, really awkward (for both sides I assume). In the beginning some of them tried to talk to me out of courtesy, asking about my studies and my project, but this stopped after about a month, when these subjects were exhausted.

As much as I'm not wanting to stereotype, don't underestimate the somewhat larger degree of social ineptitude that you find in what I'm going to generalize as "intellectual" fields.
In another vein, it may also be a consequence from them not wanting to come across as trying to garner the attention of the one female coworker, which can wrongly be inferred (by you or third parties) to be an attempt at flirting.

Based on your description, it's more than reasonable that this may simply be inexperience with casually interacting with women.

Note if this is in fact an ingrained company culture, then this answer definitely does not apply, and I genuinely wouldn't know how to overcome it. This may be incredibly hard or even nigh impossible to overcome.

I can actually use myself as an example here. I'm not the most social at the best of times, but I do notice that I feel slightly more comfortable talking to men than women (if they're strangers). Not because of any sexist ideas; it's simply a matter of statistically being more likely to find common ground, and due to social anxiety I tend to avoid situations that are more likely to lead to awkwardness (which ensues from not finding common ground).

The simplest fix here is to show that you won't bite. This means you'll initially be the one always taking the first step, but when you've created enough rapport to show that you're just "one of the guys", they'll be less guarded about interacting with you.

• Start conversations with them.
• Show interest in things the group does (e.g. ask if you could join them for lunch, or if there's a team building event, or ...)
• Strictly avoid coming across as wanting to prove a point, it makes your interactions feel disingenuous and it's hard to recover from it after it has happened. Stay true to yourself at all times.
• Try to match their usual conversations, both in topic and tone. However, don't force it. If the topic/tone does not come natural to you, find a middle ground between their preferred topic/tone and yours.
• Comedy works wonders. If you make people laugh, they tend to not feel guarded about you anymore. This is why icebreakers are so valuable for social purposes.
• (Subtly) disprove the specific stereotype they may have wrongly attributed to you. If they assume you don't engage in shop talk; show them that you do. If they assume you don't like coarse/dark humor; prove them wrong. If they're afraid of saying something wrong; laugh off your own mistake so that they know you don't get caught up on it.
• Rally behind a common enemy (within professional reason). Whether it's management, the code quality, or the procedures, there will generally always be a commonly agreed upon enemy. If you offhandedly (indirectly) talk about your frustation with something that just "happens to be", that has a bonding effect.
• Asking for their help may also lead them to feel less inept around you - as you've suggested to them that you find their input valuable (and therefore wouldn't dismiss them).

The core idea behind all these suggestions is that you genuinely show that there's common ground between you and them. If they realize that you're intentionally finding common ground, or that you knew that this would be common ground before you sad anything; then they might (wrongly) infer that you are forcing your way into social interaction.