I Don't Know How To Let Him Go...

I have been having an affair with a married man for the last 2 1/2 years. I know that the end is near, I am staring straight at it. I just can't seem to let go though. Our relationship has been a roller coaster to say the least. I have cheated on him multiple times and been caught. I blame myself for the fact that this hasn't moved forward. He actually left his wife at one point and I wouldn't take him. I was so bitter and so angry at the time...I wanted him to know what it felt like to be alone for just one night but he didnt even do that. He went straight back to her that same day. I am so afraid that I missed my opportunity to finally start my life with him. I have so many regrets, so much heart ache. I am trying so hard to fix this broken mess I have made. I know I have screwed up on numerous occasions and I know I don't deserve him but damn it I love him so much that the thought of living my life with out him is absolutely devastating.
I realize the mistakes I have made and I have been changing. I am becoming a better person. In all honesty I do not want to become him. I do not want to end up in a marriage one day where I settle, where I sneak around and cheat and decieve. I want to be happy, I want to be happy with him. I am willing to sit here and except my roll as mistress.. I am willing to sacrifice a marriage and children and anything just to be with him. All I want is to dedicate my life to him and making him happy...I don't know how to fix this and I don't know how to let this go. I am actually living in pergatory right now..my own personal hell...