just be honest with him. The only way i got out of the very same situation was bite the bullet and do it. I was friends with this *** hole for 6 years and eventually i just told him "look, We had alot of great times and alot of great memories, but in the end i cant continue this relationship (insert reason here). Im sorry." And after that what every they say does not matter. It wont make a single difference. Just end it
what makes your relation ship unhealthy?