Four Years And Still No Trust??

My fiancee and I had the mother of all fights last night.  I was just fed up, overwhelmed, and still hurting from the funeral.
A week or two ago, I got this text from a number I didn't know.  So I asked who it was, and they apologized and said that they had typed in the wrong number.  No problem.  But then he started talking to me and I found out he was a friend of my cousins who lived in another province.  I told Joey about him.  The kid was a few years younger then I am.  He started sending me texts I didn't like and I asked him to stop.  He didn't so whenever he text me I would blow him off with a reply like 'I'm going to bed with joey' or 'I'm spending time with my fiancee and daughter.'
The other day he text me and joey got the phone first.  He then, while I was cooking supper, proceeded to snoop through all the messages.  So he read what the guy had text, and my attempts to tell him to leave me be, and my blow offs.  But all he concentrated on was the texts from this stranger.  He freaked.
I never hid from him that this guy text me.  And I never did anything wrong.  I never sent dirty messages back.  I told him to leave me alone!  I included joey and my daughter in every text I sent back.  I even sent a text stating that I had a family I loved and that life would be grand if he would bugger off.
Joey never accussed me of cheating, but he hinted at it.  I was furious.  He saw everything! 
I was mad he would think such a thing, I was mad he snooped through my phone, I was mad that he was mad at me!
So we had a fight.  He claimed that he did trust me, but what would I think if I saw messages like that from a girl on his phone?  I told him A. I wouldnt snoop through his phone B. I'd ask him about it and trust his answer and C. He once attempted to go meet a girl through texts when we had had a fight!  So who was he to even accuse me of these things?
From there it branched out to how his ex cheated on him and I was showing the same signs as she was, I was distant and only wanted occasional sex.
So I gave him the what for right then and there.
Since we moved I have done ALL the housework, ALL the cooking, almost all the child care, I have been working my ass off so he can relax in a tidy home with a home cooked meal when he comes home.  And what has he done?  He has called me lazy, a bad mother, told me I have absolutley no reason to be tired because he's the one who does the real work in the relationship.  He says mean things about me, in front of me, to others to make them laugh and when I tell him they upset me he brushes it off and tells me to chill out, its just a joke.
I told him that he may whine about being a 'weekend' parent...but I have every right to get his help on weekends....because I do everything else with our daughter the other five days.  I was bawling but man did it feel good at the same time.
I told him that I wasn't being distant, that he was pushing me away as hard as he could.  And if he really wanted to be alone then I could go.  Marlow and I could make it just fine without him.  That I was with him because I wanted to be, not because I had to be.
In the end, he held me and told me he loved me and that he did trust me.  And at the time, it seemed like enough.  But this morning, thinking back.....
I realized he didn't say sorry, not once.  In our four years he has said sorry to me twice.  He always thinks hes right, and now I wonder if he was just saying that he trusted me and loved me to shut me up?  If this is still unresolved.  If nothing is going to change....
How much longer can I be with a man, who, I now know, does not trust me at all.  And I have never given him a reason not to trust me either.