Sounds like you have a feral on your hands. He just isn't as trusting as cats raised in a home with people, and he probably isn't quite sure what to make of you.
Reading through this thread may give you some ways to work with your new friend, and help you understand what is going through that kitty head of his and let you know that you are not alone. A Friend Needs Help...
As for him attacking with no warning, they do put out warnings that to them are clear as day but to us feeble humans are harder to catch unless you are paying attention to them. His tail will start flicking back and forth more rapidly, his eyes will dialate, and his ears will shift from up and alert to back and lower. Pay attention to him when he next wants some pets and see if you can pick up on his particular body language. He will tell you in his own way that he is done.
I'm sure some of our resident experts will have more advice for you. Please keep us updated on how things progress with him!