Everybody Says I'm Fat

Was looking forward to my brother's family's visit today. And I took time off work and tried to muster the energy to go out, and the first thing he said was YOU GOT FAT.
It's not the first time I've gotten fat-shamed. I'm not obese--I just put on weight and might be around the 130 + lbs range . I used to be 100-105 lbs most of my life which was 30% underweight for my height. Now that I'm still within normal weight they all call me fat.
My looks are the only thing that matter for other people :(