Yes, you'll feel 40-45 F cold water on your face. When you do a mask remove and replace, it can be shockingly cold, but you get used to it, and you try and be as efficient as possible.
I did my o/w in 38F/4 C in Tobermory years ago, but since it was early June and we spent a fair amount of time...