Infidelity Support Group
Any relationship in which one partner engages willfully in sexual relations with another outside of the partnership is considered to have experienced infidelity. This breach of trust is often traumatizing for the faithful partner as well as the relationship, and support is often needed to heal emotionally and to decide whether or not the relationship should continue after...
http://healinginfidelity.blogspot.com/2014/04/this-is-chapter-excerpt-from-our-book.html
One of the stages is anger, which sounds like were you're at. A natural reaction to a grave injustice done to you. For most people, they will need to get past this before they can forgive and heal, or come to the point of acceptance.
But some people do get stuck here, because they feel a sense of control, entitlement, moral superiority to use as leverage. For some it is hard for them to let that go, because it provides at least a measure of security in an out-of-control situation.
For me the key was in recognizing how many times in my life I've received mercy and forgiveness for wrongs I've done. They perhaps don't rate as high as infidelity, but some of them were bad choices I made and negatively affected people all the same who didn't deserve it.
But I think the key difference in our situation is fear of a reoccurrence. The past actions prove to us our spouse is capable, given the right circumstances, of betraying us. How do we know they won't do it again given the right circumstances? So I think the issue of not letting them off the hook is traceable to this fear. If we give them a pass, nothing will really change and it will happen again.
Which is why I say that forgiveness is not letting them off the hook. It is your refusal to punish them yourself. You still hurt over it. If they fail to straighten up and fly right, there will still be consequences. Forgiving doesn't erase the consequences, it means you're going to work through this together rather than as adversaries.
But the feelings take a while to turn around. Often they change very slowly, inch by inch, as you feel you are gaining a secure foothold. And that may take a few months, or 2-5 years. What you feel and what you know in your head logically will often not match up, especially right at first.
Keep working on yourself. In time this will get better if he does what he should, even if it doesn't seem like it right now.
But I think my point is because of your circumstances, you are feeling like he is only coming back to you because he can't have her. Which very well may be true. Only time will tell whether his decisions were due to irrational addiction to what she offered and he's come to his senses, or whether he really meant it and feels that way.
On top of that, we're also different individuals. I've seen some people on these forums put up with far more than I would have. It is possible you may have a real struggle getting past the rejection whereas others might take it in stride.
We all heal from this at different rates and struggle over some parts more than others, and that mix will be unique to each one even though we also all share a common experience.
IOW, my situation is also unique. But that doesn't make me alone, or that what others have gone through and learned doesn't apply to me necessarily. Some principles apply across the board.
For instance, the apparent rejection of me by Gabi didn't seem to hit me as hard, and one of the reasons, I think, is that my self-esteem wasn't as tied to her acceptance or rejection as is true in many cases. I saw what she did not as a problem with me, but with her. My only concern in moving forward with her was whether I could know she'd commit totally to me from then on and there would be no repeat performances.
That and the sense that during the As she became someone different from the woman I'd known for 28+ years. If I'd felt this was just who she was, I'd had a harder time getting past it.
But I've seen some who don't get past that. If you can't resolve that at some point, you may be forced to decide whether to stay in a substandard relationship or leave and start anew.
However you look at this mess we were unwillingly drug into, there are no "good" options, just what will be the best option for any one individual. Only you can decide that, based on as much data as you can gather.