If I remember, or at least followed your story correctly. You're in the UK?, cats used to be unneutered because you feared the side effects of it and added a bit of your own feelings into it (wouldn't want that done to yourself, etc) and they used to be outside as much as they wanted until there was an issue with an unspay neighbor cat.
I think you've come pretty far since then!

I know the prevailing attitude over there is to let them go out if they wish and that's up to you. Inside is safer. I remember you have a rather large kitty (Bubbles), too, so you may need to look at plastic storage containers if you can't find a litter box to comfortably fit him.
Other than that, try getting as big of boxes as you can and try different litters. Even sprinkling a little dirt or leaves on top of the litter. Even if you do end up letting them out some, it's a good idea to have a litter box(es) inside.