Infidelity Support Group
Any relationship in which one partner engages willfully in sexual relations with another outside of the partnership is considered to have experienced infidelity. This breach of trust is often traumatizing for the faithful partner as well as the relationship, and support is often needed to heal emotionally and to decide whether or not the relationship should continue after...

If you are not trusting him, then he is doing something untrustworthy.
Do a little digging. You need to find answers right now.
If it turns out that nothing is going on, then HE has to work harder at your marriage. You need more comfort and reassurance.
I have said this many times - cheating is unforgivable. You do not need to forgive him. I would just let that go. I think it creates a lot of inner conflict.