As difficult as it is, I would actually try to mend the friendship & keep visiting Patches, to hopefully make sure that if he gets tossed off again, he gets to return to you!! Plus, you could be a valuable teacher to the little boy by sharing with him your knowledge & love of cats
Otherwise, if you lose contact....