Well, the justification for my answer is basically I have a similar background (wasn't popular during teenage / early adulthood, and also didn't have many friends growing up) and I'm simultaneously going through a move to a different job in a different company and different country and through a romantic relationship development, so my advice would be based on what worked for me and what didn't in the past, and also what's working now.
Justification is not an excuse, and the line between them is really thin
Don't get me wrong: knowing why you are the way you are, or why you react the way you react is good. It helps you deconstruct habits and behaviors you don't want to keep.
Bringing this up too much if unprompted makes you look like you're trying to come up with an excuse or shifting the blame, especially when the other part is not interested in the details or the context, and you indeed come across as whiny.
If you want to know why, it is because the other person can't do anything about it. This is why we get annoyed at spam mail, some push notifications and other unwanted bits of information that we received without requesting (little bit of insight from my therapist).
Bottom line: just say what you're feeling in that moment. You don't always have to provide a justification. If the person asks why, then you disclose the reason as much as you feel comfortable disclosing.
Recent example - this Friday I had to come forth with my cynophobia to my S.O. because we were getting close to an unleashed dog of a considerable size and I felt the onset of the panic attack. I just mentioned that I have cynophobia and I really needed to cross the street to get more distance. A few hours after, curiosity kicked in and I was asked more details since I tolerate small dogs rather well and then I filled in the blanks behind the phobia, giving the full context.
Unload and be at peace with yourself
This is the intrapersonal part that should not be addressed at IPS, but I'll add to the advice anyway. You mention a lot of baggage and feelings of loneliness and also worry about not meeting someone else's expectations. This is something you might want to look a little bit deeper, maybe with professional help (therapy is good for the soul, really).
The past more often than not doesn't matter. Who you are now is the relevant part