Yet another nod to the "larger" divers.
The major issue between responses here isn't likely the size of their "manhood", but likely the mass of their skeletal muscle.
Muscle produces heat, and even fat cells do as well. Larger men will have more blood volume for retaining heat, and less surface area to volume ratio hense decreased heat radiation losses.
In the Puget Sound, the best diving is winter diving. We never see ice flows of course, but even in 30F air temps you'll see smaller women almost hypothermic with an "average" suit, while some obese men can dive in a farmer john... AND NO VEST. I'm serious, my PADI instructor swears hes had several clients go without sleeves or hoods to 60' without issues in the winter here. He's dove for 40+ years, worked at the PADI college, and would NEVER consider such a thing himself.
Heat generation (muscle mass), heat radiation (round better than "bean pole"), and heat retention (suit thickness AT DEPTH) will all play a piece of this.
Personally I can't imagine why ANYONE would go without a dry suit AND thermal underwear in anything but tropical waters... but I was born in winnipeg, raised in northern Sask, and have had more than my share of -40 weather thank you very much. Diving shouldn't be about proving you can handle a cold spell a FEW MINUTES long. If you really have to have such a challenge, go play pond hockey for a few hours in -20C weather, or live in -45C weather for a week or two every winter.
I'll stick to a dry suit.