How much harder is it to dive while wearing a suit? Are they really that buoyant when saturated? When snorkeling, we are constantly diving (down to 30') so, this could be a problem!
If you have a good dive technique you'll be able to get down to 30ft easy with a 3mm full suit.
Having 2-4lb on will allow you to stay there until you decide to kick back up.
As a little reference, I wear a 4/3mm in the school pool. With 4 lbs I'm a slight bit floaty at 8ft, but start sinking after 10ft.
I also have little to no body fat.