Just to add, when a kitten doesn't seem to be moving or breathing after he is born (looks dead), you can sometimes jump-start things. Put him in a towel and rub him firmly with it. Still with him in the towel, hold him head-down between your legs as you're standing and, holding him firmly, swing him downwards 2-3 times, as if you're trying to empty the last drops from a bottle of water. Give him a rub with the towel again and see if he responds.
I'm so sorry about your loss. This is difficult to say gently..if you do bury him in the back yard, be sure to bury him deeply and then place a stone or something heavy over the area to prevent him from being dug up by another animal.
Dawn is right, as hard as it is to imagine. I'd advise you to keep his body for now and wait until she has finished giving birth ans is settled. If you don't want to bury him, you can take his body to a vet and they will cremate his body along with other animals that have died at the vet.
I buryed my chick out by the end of the fence by the pasture. My dogs don't go out there too often. So it was a safer place for her to rest in peace. I also put rocks ontop of it. So they would hae to get the rocks off of her grave and dig 2 feet to get to her body. And my dogs are really lazy, so I"m sure that they would not go through all that trouble.
I am terribly sorry for your loss though.