You can clean her with a warm, damp wash cloth. It's good to hear that she is doing better.
You can use human baby formula for her, but mix in twice as much powder as it says on the label. Kitty milk is stronger than people milk. Get regular formula---not soy, not iron fortified, not hydrolyzed, just plain, and only buy a respected brand. Mix some in with her canned food, also offer a bowl of it. She's old enough to not need a bottle but she does need the extra nutrition of formula for a few more weeks.
When you bathe her, make sure you dry her thoroughly and keep her somewhere warm until she's all the way dry. At that age they can catch a chill easily. Bathe her every day as long as you see fleas---fleas can kill small kittens so you have to get rid of them. So the risk of the bath is worth it.
If you can't find a flea comb you may be able to find a head lice comb made for humans. That will work too.
She will need to see the vet soon for de-worming and vaccinations (also spaying when she's old enough). De-worming is especially important because she can't grow if the worms are taking all her nutrients. So try to get her in as soon as you can!
the advantage II sounds amazing but it says for 8weeks kitten?mine is going onto 4 weeks now@MurMi
When you order the kitten formula and terramycin, also order Advantage II for kittens. You can put a tiny bit on her neck and it will kill the fleas.
You may have to reapply it again in 2 weeks until all fleas are gone. The bath helps, but there will be eggs hatching and new fleas. Advantage II works well.