Serious Cat Ennui--how can I help him?

Hi! My 1.5 year old male cat Myshkin (fixed) has been driving me batty recently because he seems to have a sense of chronic existential despair, but he doesn't seem to know what would make him feel better. Old favorite toys have totally lost their allure, even when I rotate them out or hide for a bit, new toys are completely uninteresting, even cardboard boxes, once the Greatest Pleasure of All, are not interesting to him.

The problem is that he still meows and trills and vocalizes FRANTICALLY at seemingly random times, but especially in the middle of the night. I've tried everything I can think of to tire him out, but he's just not interested. He still loves is being held (like an actual baby...it's weird), but after 10 or 20 minutes when I have to put him down, he gets upset again. I even tried that scarf wrap thing mothers sometimes use with babies so he could be near me while I did chores, but he wasn't into it. Even if I'm willing to hold him for an unlimited amount of time, eventually he wriggles out and still seems unhappy.

He's not a particularly scared or timid cat--I often let him outside my very safe neighborhood either on a lead or under really close supervision (he always comes back when I call), and he loves being outside. But I don't think he's just pining for the great outdoors now that he's had a taste, for two reasons: 1) He's not really interested in bird stalking/bug mauling/hiding/pouncing or exploring--he adores people--when he's let out, pretty much all he wants to do is lay on his back and get his stomach rubbed by literally anyone. 2) I will sometimes let him outside and he wants to be let back in again.

In the middle of the night he makes as much trouble as possible, knocking things over, pouncing on us (ouch), sprinting across the room for no reason. The only time he will play with old toys is if I'm half asleep. Then he'll bring me a favorite toy and play fetch with it, as long as I'm still half asleep and in bed. If I try and engage him with the same toy or the same activity or anything at all when I'm awake, he'll give a small start/vocalization the way he used to, but then just lets the game die.

I think it's pretty clear that he's bored, but I have no idea what to do to engage him. I get that cats are nocturnal, but he used to play during the day and that seriously minimized the Night Terrors. 

 I'm truly at my wits end--the frantic meowing in the middle of the night is unbearable and I feel terrible for my husband, who gets along with him great, but didn't really sign up for this. I can't tell if *I* don't know what he wants, or if he doesn't either. It's just creating a horrible cycle: I am cranky the next day after hours of nocturnal pouncing, I feel increasingly anxious and guilty about my husband's lack of sleep (kitty predates him by only a few months, so husband loves Myshkin, but his adoption wasn't a group decision), and I feel terrible for my cat because he seems so unhappy.

There are times when he's just trilling at me and there's nothing I can do: he won't play, he likes to be held, but it can only go on so long, he likes being outside, but I don't get the sense he's just waiting to be free. It's like having a colicky baby and I'm worried I will lose my temper and make everything worse. 

Does anyone have ANY IDEAS? Thank you so much!

My first thought would be that he's lonely (and bored) and needs a playmate, whether it's a cat friendly cat of his age or a new but energetic kitten he can chase and trill to, and who will talk back to him.  He doesn't know what he needs because he has not had it since he was a baby and had litter mates, but male cats in particular do tend to like the company of other (neutered) boys.   Also, if you had two cats and they got into nocturnal playtime, you could keep them out of the bedroom so you could get some sleep. 

If getting a kitten or cat is not an option, the only other thing I would recommend is crate training him, so that bedtime means bedtime and he gets used to the schedule.  You can use a large fabric crate or a regular large dog crate made of wire, which contains his litter box, water, a bed, and a few toys.  You will need to invest in some ear plugs for the first week, but they do eventually give up and accept bedtime, just as children do, if it is regular and you never give in to their complaints.  Cats are not dumb, and they do thrive with structure, so having a regular schedule agrees with them once they know what it is.