no, I usually just leave them. It looks kind of funny sometimes, especially when there were a lot in one story and it looks like a bunch of replies to nothing, but I figure people will realize that there used to be comments there.
It hasn't happened to me yet, so I have no policy in that regard. I probably won't delete mine, because it reflects my thoughts at that moment... but I do see your point, that the context is lost.
I just leave it alone. It doesn't do any harm. But then, I get mostly benign comments.
Everything you put into a computer data system is never deleted.