I think it depends on the user and whether they agree with what you were telling them. We actually allow this on Stack Exchange as part of the limited messaging system.
Moderators send private "Mod Messages" to users - only users with diamonds can do so - and the user can reply in a similar private (to all site mods and CMs) message or, on occasion, they will decide to "air their grievances" in a public meta discussion, often quoting the Mod Message... which is fine.
One of the things that we always encourage from moderators is to be thoughtful in how they word messages for this very reason. If the person who feels like they're being asked to change their behavior doesn't like the way they were asked, they may make it public. As such, it's in your own best interest to put your best foot forward.
So, my advice to you is the same. If you feel the need to correct the behavior of someone in your email group - especially if you're not a "leader" or "moderator" of some sort or if the user is longer-standing than you - be absolutely certain that you address them politely and explain why they should toe the line and follow your request. If the email group has rules about what they're doing, cite them and link to any helpful guides.
Don't make personal remarks, don't say anything you'd be concerned about being revealed. You can always say "I'd prefer this stay between us" but you can't expect that - and depending on their personality, it may make them even more likely to reveal the contents of the email.
So, in that sense, I agree with the quote in your question.
If you don't want everyone to see it, you probably shouldn't be sending it.
Remember, you have no way to prevent them from sharing the email you sent. It's in your own best interest to restrict what you say, particularly when you're trying to change someone else's behavior.
What are the advantages and disadvantages of public replies to private messages in terms of building a friendly society? Is this a common practice?
The main advantage and disadvantage is that you get feedback from the entire group. This depends on the way both sides of the message are conveyed and the people who read the "outed" message.
If you follow my advice and make reasonable requests that are generally supported by the other people on the email group, you're likely to get a response supporting your request. If the opposite is the case, then you're probably going to get some negative response to your email.
Similarly, if their email is reasonable as is yours, you may start an honest debate which will lead to an interesting discussion of the situation and possibly some "decisions" about the appropriate way to act in a situation.
If their email is a rant, complaining about your overreach and slandering you when you've made a reasonable request, you're likely to be supported and their behavior condemned.
A secondary consideration is that this often depends on the relative experience level or participation level of the person on each side. If you're brand new and they're established, a rant on their side is more likely to be accepted, while if the situations were reversed or if you have the same standing, it may not.
You may also have to be concerned with a partial conveyance of the message you sent. It's not uncommon for someone trying to make their own point to pick and choose what they share, so it may require you to respond publicly to explain more completely... but don't be reactive or defensive. Respond after thinking about what they've said
This is what happens on Stack Exchange all the time. Rants get voted down and (occasionally) answered reasonably, though occasionally they're deleted entirely. Reasonable requests for discussion lead to discussion.