May be he needs some time.
There IS no point if you both want something different
If it troubles you then get away from him. There is no use putting yourself through that heartache and his face will be a reminder that keeps the wound open and until time heals it.
Then play it cool. And if you can't play it cool, drop him.
Sounds like he just wants to be friends.