Have you taken the cat to the vet to rule out any health problems? Sometimes cats will seek out a safe, secluded place (litter box, right!) to hide out when they are not feeling well.
If the cat is healthy - have you provided some other place for the him to hang out and feel secure? Sometimes a cat likes to be in an enclosed place just to feel safe. You could coax (treats, toys) the cat out of the box when he's not using the litter box for his business and encourage him to go to another secure location