I can't really tell what it is, possibly someone else can come along that does. I think it would be worth a trip to the vet though, if for not other reason than to make sure it doesn't get infected if she scratched it off. Not to mention that just sounds painful.
Did anything drain from it when she scratched it off last night? I looks like there is a small "hole" there. But that picture is before, right? Gosh, it's hard to say. Could be a bug bite (spider?), could be chin acne, although it's kind of far back for that, could be an abscess, could be so many things.
But I would definitely try to keep it clean, especially if it drained or is now open . You can use Hibiclens safely if you don't have anything already on hand specifically for cats. Seeing a Vet is never a bad thing