As soon as the kittens started getting out of the basket on their own, I put a shallow litter pan close to the basket. Of course, they didn't exactly figure out what it was for. So then I put a small pan next to each of the regular litter boxes. I think the older cats pretty much taught the babies. The first time I know for sure that one of them used the litter for its intended purpose (i.e.
not eating it), Cali was watching her mom use the litter box. As soon as Goldy finished, Cali hopped into her own little pan, dug around for a minute, then squatted down and peed.

It was so cute.

She was just about 4 weeks old at the time.
I would go ahead and set up litter pans for the kittens now. Don't wait until they've started peeing and pooping in inappropriate places. It's a lot harder to break bad habits than it is to train good habits. Also, use non-clumping litter. They do seem to think it's food before they finally figure out what it's really for.