As others have said, if my wife had chosen to dive, and something went wrong with the pregnancy or baby's development I don't think we could live with ourselves. It's just not worth the risk.
That's another good point. I deal with a lot of children with congenital anomalies, and the first question that almost everyone asks is "why did this happen?". For the vast majority of cases, there isn't a good answer, we simply don't know enough about all of the interactions between environment, genetics and development to make a confident statement about that (with a few exceptions). And minor congenital anomalies are actually pretty common; there is a non-trivial chance of this, diving or not.
So this is more of a psychological point than actual medical advice. I certainly have no idea how the OP would react to any sort of congenital anomaly - major or minor - and it would be patronizing of me to tell her in advance how she would feel.
However, I can certainly imagine that if someone had a lot of people cautioning them about doing an activity that was completely elective and only done for fun, and then there was a problem with their child, even if there was no definitive proof of causality I would think that it would be VERY hard for that person to keep telling themselves that the activity had absolutely no contribution to this outcome.