You won't know when it's time to give up or to let go, because it happens on it's own over time. One day, you're going to wake up and you're going to feel free and you're going to have a good day. Those are the kind of days when you know you are moving on with your life.
walking away is rough.. part of you keeps screaming "what if, but, maybe it's all in my head"... etc. Don't listen to those voices.. they are only based in an unrealistic fear. There are so many fish in this ocean... too many to spend any length of time on something that leaves you feeling bad in any way. If your wondering it's probably already time... follow midlifemadness's advice.. it's solid
I think if you are asking this, you know the answer. You inner voice, your gut tells you.. it gives you clues. Follow it!!
Make a list of pros and cons and see how it ends up.
talk to a trusted friend
give yourself some space and time and see what happens then... sometimes being away for a short time gives you a new clearer perspective.