Trying to let go...

My brother depended upon me, although he never realized it or would admit it, for the last 7 years, and he needs to depend upon himself. I took care of him in so many ways that he never appreciated, but now he is out on the streets, looking for places to stay.
In Maine I got him medical coverage through the state when I was getting coverage for my daughter and myself. It was a good thing, too, when he had his stroke and discovered that he had a heart condition, high blood pressure, and several other health conditions.
I think that the only real reason that I prolonged the agony of continuing to support him and put up with his bizarre habits and opinions was because of his health, because I was so worried about it.
But he has a lot of friends here, and is currently staying with one of them. He is yet to go down to the office of social services to apply for services, though, and the longer he puts it off, the harder things are going to be for him.
I cannot have him here, though, and continue to go through what I have been going through for the last 7 years. Yesterday, when he sat out in the car while I was in with social services instead of taking care of his own visit to social services, I was so irritated that I could hardly concentrate. He was talking the entire trip, and all I could think about was "why can't he just shut up", and it made me feel awful.
He's my brother, and I love him, but I am so very tired of him that I cannot breathe. He's like an albatros around my neck; a stain you can't get out. He cost me so much that I lost my house, and he just doesn't get it about taking care of himself.
And I feel like a terrible person for not wanting him around, but I don't. 
And I feel guilty.