I hope the litter isn't the clumping kind. I've always used clumping litter (as do most cat folks that I know) but I've never heard of any kitty bothering with the clumping litter. However, I have heard of dogs getting quite ill from ingesting that type of litter (it can cause a blockage in the intestines). I think an animal would have to ingest a fair amount for it to cause a big problem though.
If you think your kitty is getting sick from the litter (or perhaps there is something wrong with him that is making him eat the litter?), then maybe a trip to the vet could help.
If the cat is older, my guess is Pica. Pica is a when they eat something out of the ordinary. It can be caused by a vitamin deficiency or it can be "just because". If he does it really regular, you might want to try giving him a vitamin suppliment and/or switch the litter to something that won't do any damage such as the pine.
there is a shop for cats where i live, and the guy that works there was telling me about a cat litter that is supposed to be really great. I think it is made from corn, and while I know that's not particularly good for cats, it is not as harmful for them to ingest than regular cat litter. I want to say that it's called the World's best Cat Litter or something like that-perhaps that would be a good thing to switch to?