Loneliness Support Group
Loneliness is an emotional state in which a person experiences a powerful feeling of emptiness and isolation. Loneliness is more than the feeling of wanting company or wanting to do something with another person. Loneliness is a feeling of being cut off, disconnected and alienated from other people. The lonely person may find it difficult or even impossible to have any...

You don't seem very emotionally involved with this guy, other than to say that you are better than him. I imagine that makes him unhappy. I imagine, for instance, that he doesn't share his feelings because nobody shares feelings with someone who judges them and doesn't love them.
If this guy is as bad as you say he is, go ahead and leave if you want. But if you have any care for this guy I recommend you find yourself a counsellor and investigate this situation more thoroughly.