This is also dependent on your neighborhood, and the amount of contact you have with your neighbors. I come from one where there is a lot of talking, and I know at least 15 of my neighbors by name.
In my neighborhood, there are also cars parked on the street, since there are no private driveways. And it is also a quiet neighborhood, so as a kid I played in the streets a lot, and there are still very often children playing in the street. What would happen when one of our neighbors thought somebody had damaged something/was bothered by a particular behavior was a two-step process.
The first step was (and still is) always to confront the kids themselves, at the moment they were doing 'the wrong thing'. Just tell them the truth, that your car is parked there because you can not put the baby in it if it is in your driveway, and that you noticed some scratches you can't explain. But now you noticed they are playing very near your car and would they please remember that cars are expensive and not to damage yours. Always remember to stay calm, and avoid an accusatory tone of voice. The kids are just playing, and are probably not even noticing that their behavior might (have) cause(d) damage to your car.
The second step is to inform the parents when the kids are clearly not listening to you and your request. Just tell them what you asked their kids, and ask them if they could try to work with their kids so your car does not get more scratches. As always, be careful to be respectful, use a respectful tone of voice. Ask the parent to parent, do not go there telling them how they should parent. Asking them to punish their kids, or directly accusing their kids of the having made the scratches that are there already will probably do more harm than good. Same as above: Explain that you noticed their kids playing in the vicinity of your car, and that you noticed some scratches. You asked their kids to not play so near to your car again, but have seen no improvement, and would they please talk to their kids about it?
Also explain to them that with your baby, it is not really feasible for you to have your car in the driveway at all times. A great first step for you to take would be to park in your driveway whenever it is possible, this may create a certain amount of goodwill with your neighbors. It shows that you are willing to use your driveway whenever possible, and adds value to your argument that you can not always use it because of the baby.
Where I come from, it is definitely OK to go and have a talk with the parents, but only after you have tried to explain to the kids yourself. This might be different in other neighborhoods and cultures. But a respectful conversation with neighbors should in my experience never make an issue worse.