Not at all, they like to sleep in cavey like conditions next to our body to keep warm. I have four under the covers with me every night. Makes it hard to move, but wouldn't have it any other way
My cat Tori loves to sleep under the blankets and has been doing it for years. We have to constantly check the sheets by rubbing our hands over them in order to make sure we don't sit on her.
Albert makes himself a cave to sleep in out of the covers when we're not in bed. Other than that, he LOVES to crawl under the fitted sheet! It's so cute, there's this little bump running around under the sheet. He drives Cleo nuts when he does that!
Four of mine (not Persil - she prefers to be on top of the covers) sleep under my duvet for at least part of the night. I used to worry about it too, but they have always been fine. Sometimes they go right down to my feet and sleep on my toes.