I've always thought a pet skunk would be neat.
Especially an albino one.
But yes, you have to buy from a breeder, you have apply for permits in most states before you even buy the skunk. If there's not a breeder in the same state as you, you have to go through the proper route to import it to your state, etc.
There are sites that list each state's laws for skunks and racoons (which have the same restrictions). Some states don't even allow them to be kept as pets at all.
Years ago when I had an indoor/outdoor fluffy black and white kitty I would sometimes feed her and another cat of ours outside by the backdoor. Well I seen black and white and didn't think anything of it. Only after I had shut the screen door and turned around did I realize that wasn't my kitty I fed.