Bookymad,
You have no reason to feel bad about yourself or your skill level. Breathing underwater is as strange to our bodies as hang gliding. Its not natural and can be frightening. What you feel is quite common and I personaly have my bouts with it still. I am a new diver myself and I still have a little anxiety when I first hit the water. I make myself calm down on the surface, I get my fear and nerves evened out and I submerge to the hangline or to the bottom in the shallows of a shore dive. Once Im there I make sure that my gear is all working, that Im in order, that im calm and mentaly ready, and then I descend. I use those two stops to work out the anxiety and nervousness. Each time I dive it gets less and less and difficult to get past it. I say you should keep at it, within reason. Get all the pool work in that you can. If you cant afford to rent the gear to dive, go snorkeling. Get as comfortable in and under the water as you can. Work on all of the emergency drills. When you can safely doff/don your gear, do mask removals, buddy breath, and use an alternate air supply with zero anxiety then you will be able to dive to depth with no problem. When you get on the next boat dive, make yourself go to the hang line. Once your off the surface and you have the freedom to float on your own, 90% of the fear goes right out the exhaust valve.
Now, I did say to keep at it within reason. Some people cant climb rock walls because of physical limitations, a fear of hights, or both . Diving is no different. Some people just are not meant to dive. I would rather see a person know their personal limits and throw in the towel than to get to depth and freak out. People like that get themsleves killed and even worse, they get others killed. Are you that person, I dont know. You had the sense to call your boat dive. I think that if you get to that point then you will be able to make that call yourself. I really hope that you can learn to trust your gear, let go of the fear and learn to love diving. But in the end its your call, and for the sake of your life and those diving with you, please make the right one. Good luck and by all means keep trying!!!!!