I wear contacts as well and had the exact same problem. I am actually way more than legally blind without my contacts.
I am now absolutely fine with clearing my mask, regardless of depth, and I can do maskless, but it is not my favorite.
I am not a fan of the prescirption masks. First, if you are like me, the mask is not convenient to wear at all times, but I have to have either the mask or the glasses on to see anything. If you are on a boat, you have to leave the mask on until you can find your glasses, which you have put in a safe place - where was that and god forbid the boat is pitching any. Or, if you are on the surface for a while either doing some chit chatting or some surface swimming, there is no taking the mask off, because then you can't see anything.
Now my eyes are very bad. I can't read my gauges without my prescription, so if you aren't that bad then you may be okay with little bits between your mask and finding your glasses, but it just doesn't work out for me very well.
I love the contacts though, and there is the small chance I will lose them under water. I keep spare lenses in my dry box and I keep my buddy aware of the issue.
Good luck, you can get over the mask thing. Just hang in there.
Paula