If you know which dewormer to use its no problem. For example Pyrantel preparates against round worm, may be used as early as from 2+ weeks with healthy kittens . (unless its some peculiar brand).
Normally the kitten would be weaned from the mother about 5 to 6 weeks fully. starting about 4 weeks sometimes they can be encouraged to start eating on their own. Each kitten is different. as you did not have the mother you have had to be the mommy. I would seriously wonder why a Vet would give dewormer to a kitten under 6 weeks. @StefanZ and @catwoman707 know more about that end. The vets will normally encourage self feeding as it releaves you of the bottle feeding job. It is not the best to start the kitten before the kitten is ready.
Holding the kitten by the skuff of the neck is exactly how a mommy would carry the kitten. If you place your hand on the back of the kitten's back with your thumb and forefinger at the neck. Gently grabbing the lose skin and grabbing firmly at the back of the neck. When you have the grip right the kitten will hang in your hand. As the kitten get bigger placing your other hand under the butt to support the kitten. If you have watched any cat carry a kitten you will see the mother grab the back of the kittens neck and lift and the carry the kitten with no problem. I did a search for cat carrying kitten and found some videos.
The all round Milbomax boasts they can be used from 2+ weeks too. And a knowleable british breeder witnessed its true.
I would wait a little than this 2 weeks if I could, but if the kitten IS worm infested, it must get help. So 5-6 weeks is no problems, unless its some very peculiar dewormer.
You were perhaps thinking on vaccinating? THIS would be odd to vaccinate that early, and yet, some forumists told their vet did so.