What Am I Doing?

Not only am I dating a married man, I am living with him! We were high school sweethearts, but ended up marrying other people. We ran into each other in our 30's, and had an affair for a while. We were both married. We reconnected about 9 months ago, He was living with a woman, other than his wife. The relationship with the girlfriend had run its course, and we started hanging out again. We are in our 50's now. He will never divorce his wife, and I know that. The wife is not the problem. The ex girlfriend is very bitter, and is doing child-like things: drive-by's, keying cars, etc. We are in love and have been for 40 years! As I type this, I wonder what am I doing? If you were to see us walking down the street, you would not take a second look. We are just two average looking hardworking, parents, and grandparents. We have ridiculously, insanely, GOOD sex, nothing is off limits between us. We do things regular married couples do, only minus the arguing. We both agree there is no sense in it, life is too short, and we want to enjoy the time we have together. We talk about a lot of what if's?? At our age it is important. We can talk about anything, I love him deeply, but what am I doing?