Worried about my cats ear

My cat has been having a droopy left ear for the past week. He paws and scratches and shakes his head very often and it clearly bothers him.
Today we took him to the vet to find out it if was some kind of infection. The vet cleaned them and swabbed them but couldn't find anything wrong.
When we got home, it seemed like it had gotten worse.
I want to take him back to the vet but I don't know if I can afford a second $150 bill this month. I'm looking for ANY advice as to what else it could be, or any tips as to how I can make him more comfortable.
.

If its an inner ear infection, it would not necessarily show.   Did they do any blood work to see if there were signs of an infection?  Did they offer you any antibiotic?  Theonly thing I can think of otherwise is allergies.  Many times cats' food allergies manifest in itchy ears, and they can scratch and dig at them so badly they get bloody.  I dont know what you feed him, but we had two brothers who had ears like that and the vet didn't know what it was.  We put them on a grain-free wet food and within a week the ears were better.  They have stayed on grain-free food in their adoptive home and they are fine, and they had terribly scarred ears from yearsand years of llergies that nobody had figured out.  We still don't know what grain it was, but we eliminated corn, wheat and soy and the allergies went away.

They didn't do blood tests or offer antibiotics, but said to come back if it doesn't improve by Tuesday. It was a hefty vet bill for such a small appointment, I couldn't even imagine how much they'll charge me for blood tests, but of course I'll do it if I need to. I hate seeing him so bothered
We have always had him on Grain Free food called Simply Nourish.

You didn't say whether dry or wet or both.  You really shouldn't feed ONLY dry food no matter how good.  The chicken pate version of Simply Nourish looks fine:

Ingredients:
Chicken, Chicken Broth, Chicken Liver, Ground Flaxseed, Cranberries, Potassium Chloride, Cassia Gum, Xanthan Gum, Vitamins (Vitamin E Supplement, Thiamine Mononitrate, Niacin Supplement, d-Calcium Pantothenate, Pyridoxine Hydrochloride, Riboflavin Supplement, Vitamin A Supplement, Biotin, Vitamin D3 Supplement, Vitamin B12 Supplement, Folic Acid), Taurine, Inulin, Minerals (Ferrous Sulfate, Zinc Oxide, Copper Proteinate, Potassium Iodide, Manganese Sulfate, Sodium Selenite), Choline Chloride, Salt

Guaranteed Analysis:
Crude Protein (Min.) 9.0%
Crude Fat (Min.) 6.5%
Crude Fiber (Max.) 1.0%
Moisture (Max.) 78.0%
Ash (Max.) 3.0%

Carbohydrates (calculated) <3.5%
Magnesium (Max.) 0.035%
Taurine (Min.) 0.05%
Omega-6 Fatty Acids* (Min.) 1.10%
Omega-3 Fatty Acids* (Min.) 0.11%
*Not recognized as an essential nutrient by the AAFCO cat food nutrient profiles.

The dry food Fish & Chicken

Ingredients:
Fish, Chicken Meal, Turkey Meal, Chicken, Dried Potatoes, Dried Peas, Canola Oil (Preserved with Mixed Tocopherols), Flaxseed Meal, Tomato Pomace, Dried Egg Product, Natural Flavor, Pea Fiber, Salt, Vitamins (Vitamin E Supplement, Niacin Supplement, L-Ascorbyl-2-Polyphosphate [Source of Ascorbic Acid], Vitamin A Supplement, Pantothenic Acid, Thiamine Mononitrate, Pyridoxine Hydrochloride, Riboflavin Supplement, Vitamin B12 Supplement, Vitamin D3 Supplement, Folic Acid, Biotin), Choline Chloride, Dried Chicory Root, Dried Cranberries, Dried Carrots, Dried Sweet Potatoes, Minerals (Zinc Sulfate, Ferrous Sulfate, Calcium Iodate, Copper Sulfate, Manganese Sulfate, Sodium Selenite), L-Carnitine, Taurine

Guaranteed Analysis:
Crude Protein (min) 39.0%
Crude Fat (min) 16.0%
Crude Fiber (max) 8.0%
Moisture (max) 10.0%

Carbohydrates (calculated) 27%

We feed him both wet and dry grain free food