Physical & Emotional Abuse Support Group
Abuse is a general term for the treatment of someone that causes some kind of harm (to the abused person, to the abusers themselves, or to someone else) that is unlawful or wrongful. No one deserves abuse, period. Abuse can be emotional, physical, or sexual.

The best thing you can do is go Total No Contact. That means no phone, texting, emailing. It also means blocking him from all social media. It's hard to heal from abuse when we stay in contact w/the abuser.
Only you can answer why it offends you and what it means to you. You know you should keep your distance, but you don't. That suggests that you were abused enough to be trauma bonded to him. (This is not a judgment.) Trauma bonding makes it hard to go & stay No Contact. No Contact is critical in the healing from abuse.
Please talk w/a qualified trauma release therapist. S/he can help you learn to set and keep healthy boundaries, process the trauma and all the losses associated w/the abuse. S/he can also help you come to terms with the fact that its your brother who is abusive. It's hard to process these things in a healthy way and begin to heal from the abuse without the help of a specially trained therapist.
When ppl in your family disrespect you by insisting you associate w/your abuser, then they are also guilty of abusing you. I know its hard when it comes to family, but when family hurts one of their own, in favor of the abuser, then, they are just as guilt of abuse as the abuser. I know I just repeated myself. And I'm going to say it a 3rd time. That's how important it is. Those who push another to associate w/their abuser(s) are guilty of abuse. I'm sorry to say that sometimes we have to cut ties w/family in order get and stay healthy.