I'm the same. When mostly drifting, I just get a bit chilly (especially fingers) halfway thru my second dive, even with a 3mm thermoprene. Everyone's different tho, my wife was fine in a worn 3mm shorty.
Btw, I was in Coz last week and while we had 28-29º water temps early in the week, after...