One thing a lot of new pet owners don't realize is that you can buy cat food both dry and wet in bulk, and it is actually beneficial to hold onto the food for a bit since you'll not likely have fed it yet if there is a recall (and so far just about every brand has had recalls to date).
For dry, buy the biggest bags you can afford. If your cat doesn't like it, stores like Petsmart will take back already opened food for a full refund... well, within reason, don't bring it back almost empty.
Canned food also usually has a one to two year shelf life unopened (and realistically is probably good for a decade), so no worries. Prices vary by region, but pretty decent yet cost effective food for me has been the Nutro Natural Choice Complete Care line (5oz) wet. If you can use a whole 12oz can within three days max, there's also great deals for Innova at Petsmart. Manager's can also sometimes give you a discount for buying a whole case. If you have feed stores closer, these also usually carry quite a few cat foods sometimes at even lower prices than the big box places.
But shopping at Walmart and grocery stores IMO is going to very severely limit your options, and at my Krogers for example its not even cheap!