I was nervous as hell when I went for the first time about 12 years ago. I still don't enjoy the yearly visit. My doctor is male. You can request that a nurse be present for the exam if that makes you more comfortable. I know at my doctor's office, they have motivational posters on the ceiling that you can look at/read during the exam. I always tell myself just to read it from start to finish and he'll probably be done before I can finish reading. I never get to do that because he always talks to me during the exam.
My guess is that he can tell I'm nervous/uncomfortable and he's just trying to put me at ease.