We need a little more info from you to answer correctly. Is it your first time together? Are you stopping to put a condom on? His age and overall health? Is there plenty of foreplay from you toward him (oral sex, manual stimulation?
I was impotent or had ED. Mine was caused by a heart condition; however, when I tried methods to overcome it (due to heart Viagra and such was not allowed) I always had the question in the back of my mind, "Well, will it work this time?" ED is especially frustrating for the man, too. He wants nothing more than to prove his prowess and please you. As Idamae mentioned above, PLEASE, don't humiliate him about it. He's humiliating himself enough as it is! If it is a recurring problem have him see a good urologist. After struggling with it for approximately eight LONG years I was lucky enough to receive an inflatable penile implant. Now I squeeze a pump hidden in my scrotum (cannot be seen by the casual observer in the shower/locker rooms) and the two chambers implanted in my penis are filled with sterile water from a reservoir in my lower abdomen making for a very rigid usable erection. The plus side is that it does NOT go soft until I choose to release the fluid back into the reservoir. Even after climax! Sometimes,if wanted you might even reap the benefit of a second or third time in one night. Sound good? If it's a temporary occurrence it's not because he's not "into" you per say. It's most likely performance anxiety the master killer of an erect penis! Contact me if you need more info. Best of luck to you both.
There are endless reasons why this situation can develop. Don't humiliate him. Find a solution you baoth can live with.
Have you tried foreplay? Some women expect that all men always just get hard on signal. If he is distracted by other things like stress or he is tired he may need a little help in getting in the mood. Don't take it personally or make a big deal out of it. If it is a continuing health related problem he can always choose to talk to his doctor about it. However, performance pressure is the greatest turn-off for many men. If it takes a little more time, it is time well spent.
I know it's hard (no pun intended), but try not to take it personally. How old is he? The older a man gets the more difficult to get an erection...especially if he smokes or is out of shape. Or, if this is your first time he may be nervous about his skills if you're really hot. :D