Key points to consider: What could the person do with the information? How could that be beneficial or detrimental to the person being asked? Unless the questioner has the ability to use the information to help the person being asked, and the person being asked knows this (e.g. bank officer evaluating a loan application), it is often against the person's best interest to answer.
Doing something that is against a person's best interests is at a fundamental level a breach or negative of trust, which is experienced as offensive. "Doing something" includes verbal moves that encourage the person to do something that is not in their own best interest. So, even if people do not consciously process or explicitly identify all this, these elements may be shaping their experience and reaction.
What are potential detriments?
I've seen it happen where once someone revealed a considerable earning potential (not even with detailed specifics), they became a target for overcharging, scams, and serious crimes by people who wanted that money.
This could be in the form of simple robbery, or a more detailed extortion scheme. If a scammer knows that you make X as salary and knows exactly what kind of false accusations or other actions the scammer could take that would cost you your job, the scammer can demand that you pay some amount less than X (ceiling may be reduced by some deductions for taxes etc.) and an economically rational person would probably pay it. Of course, it depends on the amount and credibility of the threats, but the scammer knowing the salary figure is purely helpful to the scammer and detrimental to the earner.
Further, those with higher earnings might be willing to pay more for the same thing than what a person of fewer resources would pay, and might be less likely to try to negotiate down, or not work as hard to seek lower prices. Thus, a person known as being richer can become a target for perfectly legitimate pricing discrimination, charity solicitations, telemarketing, etc. and the time/frustration/money costs of dealing with those adds up. If you want an example of price discrimination, check out tourist markets in developing countries and note the prices they charge Westerners who are perceived as rich (especially folks coming from a cruise ship) compared to what they charge locals.
If one person in a group makes even just a little more than others, s/he may be asked to make a disproportionately larger share of contribution to communal expenses that all benefit from equally (or that the more well-off person might even benefit less from). Being expected to do this with every group one is a part of (and facing social distancing/isolation for not doing so) can wind up making that person actually in a worse position financially and personally. Those issues can be avoided if salary information is not shared.
The recipient of salary information may also believe themselves to be an authority on what the salary recipient should spend the money on. These may be different priorities than the salary recipient's, and may not be as disciplined. For example, consider a teenager who knows his family's total income and decides to take actions that cause a large share of it to be spent on things like parties, clothing, electronics, trips, etc. or who grows upset when those things are not permitted, when the teenager might lack a good understanding of family financial obligations or priorities (e.g. housing expenses, saving up to be prepared for large necessary expenses, planning for retirement, contributing to college tuition, etc.). Trying to avoid such discussions may or may not be the best course of action within a family, but is more often a good idea outside of it.
If others don't know how much you make, it's harder for them to express strong opinions about how you should spend your money. Any of these discussions that still do happen must focus on whether or not the benefit of the expenditure is worth the cost, instead of how small a percentage of someone's total income something costs and how any specific "suggested" expenditure shouldn't be experienced as a material difference in financial capacity. Comparing marginal cost to marginal benefits is a much more useful focus.
On the other side, if someone doesn't make much, they can also be targeted for crime (on the suspicion they probably don't have a very good protection system in place) as well as scams, frauds, and overpricing that preys on weaknesses and vulnerabilities. People don't like feeling vulnerable. They also don't like signaling how precarious their situation is because that turns people off: they are seen as potentially less dependable, less able to bring benefits to interaction partners or groups, and less of a good choice for spending time with.
Finally, unless everyone earns the same salary, answering the question creates a social distinction between people, dividing them at least a little bit. Often, a good conversation aims to bring people together, to find common ground and points where interaction can be mutually beneficial (even if people are sharing different viewpoints that might provide insight toward a common goal). Someone asking this question communicates (even if unintentionally) an intent to create more differences and division, and as a conversation move this can be somewhat offensive.
Even if you don't think the person you're talking to directly would cause any of these detriments, what is really the benefit of providing that information? What about detriments that might result from anybody who is overhearing, or anyone any of them tell? It's just usually not in one's best interests to answer so casually, but people don't like violating social norms (e.g. by not answering a question) either. Answering or not answering, one is probably at least a tiny bit worse off than if the question had not been asked, and the question-asker gets the blame along with any resentment that may accompany the blame.
