Are We Back To The Beginning Of The End?

Over this last year it's been pretty up and down (and here and there) within my marriage.  In the fall we started counselling, but I left right before Christmas.  There were too many things that were not ok.  (and lots of that stuff is in previous posts)   But my leaving was some sort of wake up call to my husband who finally starting DOING all the things he said were important and not just talking about making changes.  His commitment to me and our boys was strong and he worked really hard.  I decided to give things a chance.  First by spending time together, then by spending time together at the house as a family, and now more formally by taking a vacation together and talking about our future together.  Hard to lay it all out, but we are back to a marriage again.

But now I'm worried that I'm back to the same marriage that I left.  Not so terrible as it was, but that there's something I can't put my finger on that feels like the beginning of when things went wrong the first time.  What happened then I think was my husband felt sad, not appreciated, wasn't having his needs met, and lonely.  So he went and almost removed himself emotionally from the marriage and became intimate with friends....  not physically intimate, but emotionally.  It was about him reaching out and getting what he wanted from other people.  Women too.

And we've worked through this, on our own and in counselling, so that he sees that having those kinds of relationships was killing our marriage.  He respects that, but really doesn't see how come it hurt me.  Just that it hurt me so he stopped because he didn't want to hurt me.

So now he's doing things to be respectful of me - he doesn't have any women friends that he speaks with intimately (he says he just doesn't talk to anyone at all).  We were excited about our trip.  We are working on a few things around the house.  But he's become quite sad again.  I realize it's because it's hard for him -  all this work he's been doing for the last few months to prove it's worth it for me.  He's worn out.  But I know part of it is he feels like he's got needs that he's not allowed to have met.  

And so I'm feeling so much now too!  I'm feeling super scared now that he's withdrawn.  Am I am idiot for trying again?  Is it even possible for him to maintain changes if he doesn't get why it's a problem?  Why can't he turn to me to work through his sadness?  Is his sadness going to mean that he'll reach out for sympathy from other people again?  Are we right back to where we were?  Is his sadness because he's stuck with me and he'd rather just have all those other people in his life?  (oh, I know that given the chance he would like to have both - flesh and blood wife, but lots of affirmation from other women).

I get that there was a bit of a 'honeymoon' period to us coming together again.  I know marriage isn't always going to be exciting and fun.  There are annoyances and bumps on the road.  But how can I know if this is going to start it all over again versus this is him having to work through a new normal?  I know I'm strong enough to leave for good if it is bad.  It's just this withdrawal that has me so scared.  I don't want to go through that all again.