Could have come off as
"My toe is damaged"
"I have no self esteem"
Hmm. I wouldn't worry though, it's in the past now.
You handled it well in a jokey way. It would depend on the intonation in his voice when he said it to know what he meant.
Only he knows what he meant. If you want the relationship to advance, then ask him.
He wants you