Confused....somebody Please Help Me!

I'm so tired of being depressed and confused about life. I'm not enjoying my life at all and I'm only 24. I want to be happy with my ex of 5 yrs but I feel guilt for even talking to him because I met him through one of my friends. He says that they was never in a relationship. He said it was only a sexual thing. I met him through her and I was the type of person to say that I would never betray my friend. Him and I had fallen out and we just recently got in contact with one another. He wants to be with me but I'm letting my past keep me from being happy. I know him and my friend was sexual 6 years ago but why do I feel like I'm in the wrong. He's everything I want in a man he wants to spend time with me, he wants to take me out, he wants to do things for my family, he wants to treat me like a queen but I'm letting the situation about my friend stop me from being happy. It's like I'm afraid to be happy because no man has ever offered me what he's offering me, and it's like I'm pushing him away. Then i'm so insecurre with myself because I'm a plus size woman and to tell you the truth I don't love myself. He told me he would help me lose the weight and that he would be there with me every step of the way. Can somebody please help me because I am so stressed out and depressed am I in the wrong and should I give him a chance because he is a good man.