I'm a freediver who frequently dives in open-ocean conditions. This morning I swam up to a couple of scuba divers floating low on the surface (BCs not inflated) to see if they needed any assistance. One had his mask on his forehead, the other had his mask off entirely, and both appeared to be a bit confused and uncertain about something. They said they were OK and didn't need any help or advice, and they appeared to be managing alright, so I continued on.
Around these parts, mask on forehead or mask entirely off means a person is not very familar with and/or comfortable with their gear. When issues arise, off comes the bothersome gear! It's not so much a distress signal as a "newbie" signal.
When I get into a tough situation the last thing I would ever want to do is remove any of my precious, life-saving gear. Mask on face, snorkel in mouth and fins on feet is often the only way to survive in rough, challenging conditions. Rather than taking off my mask, I will tighten the strap to make sure it stays on.
Another issue is that your antifog treatment is more likely to be washed out if you take off your mask while you're still in the water.