Hi Laura, welcome to TCS.
7 weeks is very very young. Kitten should ideally stay with the mumcat until they are 10-12 weeks old. Try mashing some wet kitten food with some warm water until you get a gruel-like mixture and offer him that.
Trying to eat the litter is normal in very young kittens. Using their mouth is the way they learn about the world. To avoid him becoming ill, use a wheat or corn based litter or clay litter (non-clumping). You could also make up some kitten glop and see if he likes that. It is very nutritious and yummy.
Home-made kitten formula (Kitten Glop)
8 ounces water (boiled then cooled)
1 envelope Knox unflavored gelatin
8 ounces whole evaporated milk (not skim)
1 tablespoons mayonnaise (not low fat)
2 tablespoons plain yogurt (not low fat)
1 large or 2 small eggs yolks (raw)
1 teaspoon clear Karo syrup
* 1-3 drops liquid pet vitamins
* 1 capsule acidophilus
* 1 drop Grapefruit Seed Extract (GSE)
* Optional, though very beneficial.
Boil the water, add the gelatin and mix well. Add the following ingredients in order, mixing well after each addition:
1/2 of the canned milk
Mayonnaise and Yogurt
Rest of the milk
All other ingredients
You can substitute canned goat's milk for the canned evaporated milk, if you prefer it. This mixture will keep in the fridge for up to four days. It is jello-like in consistency in the fridge so you can just scoop out as much as you need and warm. It can also be frozen in ice cube trays and defrosted as needed. You can find pet vitamins at a pet store or your vet, and acidophilus and GSE in the liquid form at a good health food store.
Good luck with the little one.