It may not be what you said, but what you did. He may have been interested in you, and was hurt when you tried to pawn your sister off on him. Possible??
I don't think you said something wrong, but he thinks you're joking. And sometimes a joke hurts another person's pride. I'm way worse my words are just offensive to lots of people.