Usually they begin try out solid food (or rather, moms food) at 4 weeks. So at latest, then, they must get litter. As their poo changes and mom not longer fancy as much to eat it up.
Use a shallow box of some sort, and some plain, non clumping litter (already now!). Small kittens, like human babies, tend to take samples of everything into mouth. And we dont want clumping sand to swell up in their intestines...
Food yes. The easiest way is they will probably imitate their mom. Just make sure the plates are shallow. In the beginning they will just go through the food...
Otherwise, mince meat is popular beginners food. Royal Canin makes a wellknown dry food for small kittens. Wet food is surely OK...