Rescued feral kitten 3-4 weeks old starting to fear humans despite being handled by humans?

4 days ago, me and my cousin rescued a feral kitten around 3 weeks old, eyes open, ears open when we found him. Feral kittens being left here and there then dying are pretty common from where I live, and most people don't really like them. I know that mothers come back for their kittens but before taking this kitty in, a human handled him and placed him near a colony of feral tom cats. Its good that they didn't kill him but probably because they seem scared of the human presence and smell. I live in the tropics where its constantly somewhere around 80 F. I feed this kitten 100ml to 150ml of some homemade kmr (evaporated-like milf, egg yolk, mayonaise) every feeding, around 3 or 4 times a day. I keep him in a box outdoors while I work, and maybe feed him for a while then play with him and let him walk around the yard for 30 minutes or so.

Usually when I leave him at a distance, say 10 meters, then make a distinct sound I always repeat when near him he comes running (still a little wobbly) back.

Just today, Balai (kittys name) started moving his tail in curling directions as opposed to the stick-like tail position he always had. Then this evening, when I repeated the usual exercise of placing him 10 meters away then calling him, he seemed to ignore me, but when I repeated this exercise again he came over. Then when he sees my feet or human appendages its like he gets a little afraid (he recoils) unlike before, which I find really unusual since he always runs to my feet then I pick him up, and when I feed him I always pick him up.

I know he is a feral cat, so I am not really expecting that he becomes a cuddling/petting cat, but I would still like to maybe to have him indoors someday. I'm afraid though that his feral instinct might take over and will become afraid of me and other humans and might just run off when he becomes bigger, which will probably just end in his death since he lacks the necessary skills to survive in this area. What should I do? Please help me. Thanks in advance for any help.

Perhaps something in the environment other than yourself startled him. One thing that may be helpful is to sit down on the ground when interacting with Balai.  To a tiny three week old kitten a human is the size of a giant.  When he acts scared or startled sitting on the ground talking to him makes you looks smaller.  Also you can try making a fist and presenting your hand to him as you are calling him. Cats interact by head butting and this may help him feel more comfortable.

It is great that he is already letting you pick him up. Also I am wondering if you would be better off taking him inside sooner rather than later if you are able to pick him up.  

I do not have personal experience raising a feral kitten though.  There are other people on this site that do and I am hopeful that one of them will give you advice based on experience soon.

Here is a post I just came across in another thread that may be helpful to you.

http://www.thecatsite.com/t/323646/should-i-give-my-cat-away#post_4068010

The advice given by Kat in the post above this one may also be helpful to you.

I agree with @catlover73  ---- to a very tiny kitten (which this is) you're a giant! I've raised multiple feral kittens - and starting out this young you should be able to socialize him nicely with some love, affection, and food!!!!

Him curling his tail around is not likely a symbol of being angry - like an adult cat can be whipping their tail side to side. Kittens that small start to whip their tails side to side/curl them around at that point quite a bit - often in play - as their tails go from "stick straight flag tails" to more flexible, muscled tails. He's very likely just figuring out how to use it - and how to indicate a little emotion - but very likely it's anger or even fear. You can reduce his fear by getting down on his level, talking softly to him, getting him used to being petted and picked up (which I'm assuming you're doing a lot since you're feeding him formula!). 

Him recoiling at your feet is probably just be play - kittens charge and retreat as they test out their environment and those around them. If he's coming when you call - he's bonding already! 

And bonding this early with him means that you really shouldn't have an issue socializing him and bringing him inside. Hope you can take that step soon! Cats that are totally "outdoor cats" live much, much shorter lives - - so the sooner you can get him inside, the better! 

Keep us posted on Balai!