For that young? I wouldn't do it till at least closer to 12 weeks. Younger kittens have more trouble keeping themselves warm than older ones do.
The three kittens I took in back in October would have been left outside the entire night in weather that cold (plus it rained). They were 7 weeks and I knew if I didn't take them they probably would have been dead or close to it by morning.
Now if you're taking kitty out inside your coat or jacket so you can keep it warm, it would be perfectly fine.
