I don't get my ferals - any hope?

I have a semi-feral boy named G that I've had since he was just 3-4 months old. He's now almost 2. He was completely feral when I got him but has made some progress since. Now he's very odd in his behavior and I just don't get him. For the most part he's very skittish and will not let me come near him. He always stays some 3-4 feet away from me and won't let me touch him. He looks terrified if I stretch out my hand to him. But there is an exception and that's when he's with one of the other cats, Eli. When he's with Eli he's almost a normal cat. He lets me pet him, he rubs up against my legs and he's even let me pick him up. If Eli is next to me he will even come up to me to be petted. When Eli is not nearby though he acts like he thinks I'm going to kill him.
Clearly he feels very safe around Eli. When he first came to me Eli was the one that took him under his wing and took care of him so I understand why he feels safe around him. I just find it odd that he can go from wanting me to pet him one minute to being terrified of me the next when Eli has walked away. You would think that he would know that I'm not going to hurt him since I've never been anything but nice to him but that's clearly not the case. I just don't get how his brain works.

I suspect that he's going to be semi-feral for the rest of his life. Sometimes it seems like he's making progress but then go goes back to being scared. Obviously he has the ability to be okay with humans though so I'm thinking that there has to be a way to expand on that. Has anyone else dealt with similar situations and has success? I would really appreciate any tips or ideas anyone has.

[quote=StefanZ;2996631]oh, in cat language it is somewhat threatening - raised paw is for giving you a "teaching"...
QUOTE]

Oh, I didn't mean stretching my hand out over him. I meant like moving my hand, or any part of myself, in his direction. What's so funny though is that when Eli is around I can do pretty much anything with him, like a normal cat, but when Eli is not around he acts like I'm a monster. I just don't get what's going on in that little head of his. He has a brother who is even more feral. He would never come up to me to be petted with or without Eli. But when I catch him laying down relaxed I can sometimes pet him on his back. He will lay down and purr for a while then all of a sudden he looks terrified and takes off.

I just don't get them. They've lived with me since they were little, tiny kittens so you would think that they would know by now that I'm not going to hurt them but that's not the case. They surely are odd. I guess you just have to learn how they work and adjust your actions accordingly.

What people refer to as feral seem to be pretty broad. I realize that many people use the word feral to refer to a stray cat or any cat that is shy or skittish. The true defenition of feral, however, is an animal that is wild or has reverted to a wild state. A true feral cat is pretty much like a wild animal. Most cats that you see outside are not feral but socialized or unsocialized strays. You hear people saying things like "I took in a 2-year old feral and after a few months he was just like any cat". If that's the case, the cat you took in was not a feral but a once socialized cat that has not had contact with humans for a while. I've had several of these and they are fairly easy to get back to being a friendly cat. You just have to gain their trust and when you do you have it forever.
The two cats I'm talking about though are not just unsocialized cats but neither are they true ferals. I think describing them as semi-feral is quite appropriate. Their mom appeared feral, probably the offspring of a stray that never had contact with humans, and these boys did not have any contact with humans until they were 3-4 months old which in their case was too late.
They do fine living in a home and being in the same room with me but if I get within 6-7 feet of them they take off and hide. The exception is G who lets me pet him when he's with Eli. You would think this would make him a tame cat that is just shy and skittish as a part of his personality but that is not the case. I actually think he would be a very affectionate cat had I gotten him sooner. But as it is now he and his brother do not trust me and my husband and I don't think I will ever gain their trust. They don't make progress like an unsocialized stray that get better and better once they realize that you won't harm them. G and D may let me touch them one time and it seems like we're making progress, they've realized that I won't hurt them, but five minutes later they are just as fearful again and stare at me like I'm a monster. They are NOT tame cats. They are semi-feral.
I've had them for almost two years and the progress they made was in the first few months. After that they've stayed the same despite me trying to work with them. It is very much like a wild animal that can be trained to an extent but will never let go of their wild nature.
To me, it's just hard to understand how their brains work. I of course understand that it has a lot to do with instinct and deep rooted fear of humans since they didn't know humans early on. I understand that this IS how their brains work. It just doesn't make much sense from the standpoint of a regular tame cat that can be trained and tought new things and once it's learned it's there. But with G and D learning to trust me enough to touch them now does not mean that it is learned forever or even for five minutes. After two years with me they still don't know for sure if I'm going to hurt them or not even though you would think that their experience would tell them that I'm okay. But they just don't seem to function like that and to me that's hard to understand. That doesn't mean that I don't accept them for who they are. I do.