I see your point...
When you're throwing the first punch. Self defense is acceptable.
Always. Yeah, that's right, it's never ok to punch someone unless it's the only way you can make them stop punching you, and even then a non-violent solution is better if you can find one.