Not sure if I believe it's 100% true, but I read that men and women fall in love in two different ways. A woman's love for a man can grow, she can end up in love with someone she never expected to. A man feels a physical spark immediately, if he doesn't get the spark, he'll probably never get the spark.
You are the only one that can decide that. Do you think he loves you at all?
He may never get over the love he can't have and that will always be an issue. Is having only part of his heart what you want, because a part was given to this over woman and it doesn't seem he is ever going to give it to you fully.