This is such an important point that people overlook. You're right that when people try to erase the damage, it often makes it worse - although for example if "John Doe" makes a hateful tweet, it probably DOES make sense to delete it - not to pretend it never happened, but as a way of acknowledging the error in judgement.

This is why I use my real name as my web alias everywhere, even on places like Reddit - I know that anybody who wants to figure out who I am can easily do so. By using my real name, I never start to feel anonymous.