Should you be? It seems pretty obvious that, if you can avoid anger or offense, it's better for everyone. Typically, though, we don't get a choice about whether to be offended.
Now, if you get offended, you can borrow this move. I would love to say that I did this, but unfortunately, I was the one who had badly mimicked an accent here.
The other fellow smiled, shook his head, and said, "Don't do that. I promise, you sound nothing like a ____." He followed it with a little wink.
In all honesty, I truly had deluded myself into thinking that my accent was pretty good, so I was a bit disappointed. Nevertheless, I did not try the accent again. It was only years later that I found out that he had been a bit offended. His totally gracious response (a) spared my feelings and blocked me from embarrassment at that moment, and (b) prevented any bad feelings going forward. My friend and I eventually drifted apart, but I am fairly sure that the Atlantic Ocean between us was the reason for that, not my faux pas and his immensely graceful recovery.