Depends on how allergic, honestly. I had severe allergies as a kid - 3 years of allergy shots - and still harbor a lot of allergies today. But I always kept cats around. I just have to be careful not to pet the boys and then rub my eyes or my face... although I give them lots of kisses, then find myself heading for the Visine-A (my best friend!). But if even airborne allergen/dander can affect them, then it may be difficult. And I've always kept shorthairs.
Please keep posting pictures... I'm loving the opportunity to watch these babies grow!