I'm not a doctor, but this a gauge you can use to evaluate your condition first. This is what I did.
Can you walk up at least 3 flights of stairs without being out of breath?
Should be able to answer Yes.
Can you walk at a brisk pace and still have a conversation after a 1/2 mile?
Should be able to answer Yes.
When you get your heart rate up to 75% max, how long does to take to drop 20bpm while resting?
Should be between 1 and 2 minutes
Are you shallow breathing or deep breathing with diaphragm when diving?
No shallow breathing. Exhale all the way out.
If you answer the affirmative to these, then get more dives in. At 25 to 49 dives most likely you just have not got things dialed in yet. Drop all the weight you can from gear. Learn how to trim out. Learn how to frog kick. Slow down. Forget about your SAC rate for now and just enjoy the dives. It will come.
Also being cold will make you breath more. Cold water can suck it out of you.