Grumpy Crumbly

Hello guys,
My cat Jessie has just recently had kittens, of which we have kept 2 of them.
They are about 3 weeks old now and doing great.
But unfortunately my oldest cat Crumbly isn't doing so good

It seems like he just doesn't want to know anymore
He hisses at Jessie and at the kittens and he is extremely skittish.
Most of his time is spent outside now and he only ever ventures into the house to eat (and maybe sleep, while we are out during the day)

I have tried to introduce him to the kittens but I'm not going to force it as it is plain to see that he isn't very happy.

We have tried to give him bucket loads of attention over the past week but nothing seems to get through,

He was always the alpha cat so I can understand maybe he feels undermined? but he was never this way before, even when we got Jessie, I'm hoping he will eventually come round.

Is this normal behaviour or do you think we may be losing his affections?

It's territorial, it's what cats go by, and his territory has been invaded by strangers (cats don't always distinguish between adults and kittens!). Is there some way (short term) to keep the kittens in a 'secondary' room so he feels a little less threatened, and gets used to the situation slowly, rather than all at once? It might help.