Anxiety Support Group
Anxiety is a physical condition marked by intense and persistent feelings of distress, fear, angst or dread. General anxiety caused by routine day-to-day stresses usually passes quickly and is experienced by almost everyone at one time or another. However, such feelings that linger over time and are very difficult to cope with, and which lack a clear cause, may indicate...

I don't think you having pre-existing anxiety is what is chasing this guy away. From what you have written previously, he just doesn't seem to want a relationship in the same manner that you want a relationship. And that is okay. But, you have to accept that of him. We can't force anyone to want the same things we want, or act the way we wish they would act.
Do you have any other social outlets? I think seeing friends, family, maybe dating another guy..... would help you to not think so much what this one guy is thinking or doing, etc.... People show us who they are. It's not always fun or what we want to see though, and that's what makes us feel upset.
Backing off would probably really help the situation with him. Back way, way off. As in - go a week or more with NO contact at all. It seems to me that you are too available, and guys aren't usually into that (my opinion). I believe you said that some friends of yours also suggested giving him his space.
Keep trying to call your doctor regarding the dosage of your medication. But honestly, I think the stress of this guy is adding to your anxiety. And medication alone isn't going to fix that type of anxiety.
You two aren't in a relationship, so go ahead and date other guys. Finding a guy who is on the same page as you would really be a nice thing, don't you think?
Also - for him to say your pre-existing anxiety is "all in your head" is insensitive, insulting, and frankly mean, and really shows that he is not a good match for you. You don't have to settle!!
The thing is D makes me laugh and we have fun together. I miss that. And I know he does too. One time we didn't see each other for 3 or 4 days and he calls me to come over for some cake he made and he said he needed to catch up on his picking on me. He's a nut and I can't help liking him so much. I have my sister and when she is not working we do a lot together. But now she is back to work. That depresses me too. I have the old lady next to me that I help out. I go shopping or just go for a drive at times. I have a friend I go visit now and then. She has health problems too. I am more or less a homebody, always have been. I do like going out but love getting home too. It's not my life is centred on him, cus it's not. He has a huge family and is always going somewhere so he is not always home either. I just like being with him.