Getting lost on a wreck is not all that likely if you have any kind of situational awareness at all (and are not narc'd out of your mind!) Just remember what feature of the boat the anchor is on/near, and you're cool.
On a reef, however, its a bit of a different matter.
In any event getting lost underwater is not a good thing. Around here at least there is a good chance of a ripping surface current - often, they are flatly unswimmable. The bottom can be completely devoid of ANY current, but at 10-15' you pick up a real mask-ripper.
The problem is that without a reference you don't KNOW when you pass the current zone - you might sense it with the thermocline, but if you're drifting there is a good chance you won't realize you're in a 1-knot+ current, because you're moving at the same speed - no relative motion.
This is big trouble; you ascend in "free water", and while on your safety stop you drift 1/2 mile away! Eeeeek!
Since you do not KNOW where the "breakpoint" is between the current and no current, you can't do the stop a bit deeper if necessary. Without a referance.....
Now if you have an upline you can use, then you can work around this. But even so, being unable to get back to the anchor line - if the boat is anchored - or the buoy line if you're live-boating - is very un-good.
If you can't be reasonably certain of getting back and there is a significant current then you need to think CAREFULLY about what you (and the surface support!) will do if you get lost. This is NOT so simple as "have a safety sausage."
Two weeks ago we had 2+ knot currents on one of our dive sites. It was flatly unswimmable on the surface - mask-ripping speeds. You had to pull yourself from the back of the boat to the anchor line via a granny line, and then pull yourself down the first 15' or so until the current went away - then you made your decent normally.
If anyone had been unable to find the line (and this was a reef dive!) for the ascent they would have been in a world of hurt. With a half-dozen people in the water, you CANNOT pull anchor and go get someone drifting off - that is simply not an option until everyone else is back on board. While I can sound a "recall" (engine revs) to the divers in the water, it takes time for them to ascend, do their safety stops, and reboard. During that time you're off to Cuba!
Give CAREFUL consideration to this if you dive where there are surface currents in particular.... although it applies to bottom currents as well.