Are they all neutered? That will help for starters.
Keep your two cats and his two cats separated for about a week, but make sure they know there are other cats around. They will probably sniff each other through the door. Slowly, start feeding them near each other in the same room and just move the bowls closer and closer to each other. Keep an eye on their interactions for a while. There will be hissing but that shouldn't discourage you, it's just how cats communicate and they have to figure out a new hierarchy among them. There are lots of little tips you can try, like putting vanilla extract under their chins so they all smell the same to each other. If you do a google search on "cat introductions", you will find more tips. The big thing with cat introductions is time.