That is a good question, I'd remove them even if nothing seems visibly wrong - and ask your vet if you should have their kidney values checked or not. The quick surf I did said that in cats it causes weakness, listlessness and loss of kidney function. (and mentioned what it can do to toddlers who chew on the leaves...ouch.)
I tend to be assume the worst is possible, not the approach that everyone else takes, I know