You won't always have what you have now. One day you may not be able to just grab someone for comfort and fun. Only you know if this guy is the one you want to settle down with. I wouldn't keep throwing love back.
Give it a go- and be very open about the uncomfortable feelings, now and whenever they come up. Maybe it is something you can work through with the right person.
Take your time, you don't have to be married to do a live-in test. Set a time limit...
6 mo., a year, more...then decide.