Your best and most economical course is to stick to whole, natural food, nothing exotic here, just your basic meat and vegetables.
Basic food that you have to prepare yourself without additives or processed food of any kind is the best prescription for a person who has celiac disease. Go one step further and consume only hormone free, antibiotic free, grass fed beef, or vegetable fed pork, i.e. not grain fed, not commercially raised. IOn our part of the country, you can buy such meat directly from the farmer, and it is less costly than the grocery store. If you have to rely on processed GF food, it will cost you more money.
Best of luck to you.

There are quite a few specialty GF sites out there that sell products. But none of them are truly inexpensive, particularly when it comes to shipping. I do order certain special items from Amazon in bulk; things like rice pasta, chebe bread mixes and enjoy life chocolate. By ordering in bulk I do at least save a little on shipping, but that is as good as it gets. Where I live there are only two grocery stores and the nearest health food store that stocks GF stuff is over an hour away, so the internet has become my friend over the years. Once I started figuring in gas and driving time the shipping didn't look quite so bad ;)