The last time I had a cat who had kittens, of course the first thing is that they mew from birth on and they are loud enough to wake me up! But then all our cats gave birth under the bed, go figure . . .
Anyway, they seemed to begin purring within a few days, when they were nursing anyway, and that was surprisingly loud too! After the initial health check, I personally didn't pick them up until they were starting to get around on their own, so I haven't held one that young.
I think I read something that purring is about a cat needing or wanting attention of some kind - which would explain why sometimes injured cats purr. So kittens purr when they want mama's attention, and later they associate it with contentment so they purr when we pet them. I guess our hands feel a lot like mama's tongue! They also seem to do it to soothe themselves, assuming they aren't in a situation where they have to fight.