You've already gotten excellent advice, but I'm going to reprise some of it with my own viewpoint.
The bottom limit on how much you have to breathe is determined by your CO2 production. You need to move a certain amount of air in and out of your lungs to carry away the CO2. Here is where breathing PATTERN comes in -- rapid, shallow breathing exchanges the air in the trachea and bronchi (large airways), but that air isn't involved in carrying away CO2 at all. So all that ventilation is essentially wasted. This means you end up breathing faster and shallower, and blowing through your gas. Long, deliberate breaths end up ventilating the small air passages where gas exchange occurs, and therefore are much more efficient at removing CO2.
So a slow, deliberate breathing pattern helps REMOVE CO2, but is there anything you can do to minimize production of it? Yup. CO2 production is directly related to the amount of muscle activity present. If you are kicking wildly and waving your hands around, you are using a LOT of muscles, and all those muscles are releasing CO2. If you, on the other hand, are floating peacefully, you are using very little muscle energy, and your need to breathe is less.
So how can you get efficient? NWGratefulDiver and others have talked about correct weighting, but I'm going to talk about positioning those weights to give you an effortless, horizontal posture in the water. Many people, especially those diving in cold water where a lot of weight is required, load the weight pockets of their BC, or put on a big weight belt. This puts a lot of weight low on the body, and when you get in the water, you're quite likely to assume a feet-down position. Now you begin to kick. With your feet below you, your kick drives you upward. But, since you don't want to go up, you need to have an equal force driving you downward. So people who swim in a tipped-up position have to keep themselves negative to avoid ascending. Thus a significant part of every kick is completely wasted effort, because it gets you precisely nowhere. On the other hand, if you move some weight up onto your back and balance yourself out, then you can kick directly forward, and use a lot less energy.
Using a kick and glide pattern, rather than a continuous kick, also help conserve energy and reduce CO2 production. The frog kick is ideal for this, but you can really do it with any type of kick, if you just relax and glide for a while every so often. (This also gives you a great buoyancy check, because if you stop kicking and begin to sink, you're diving negative and wasting energy.)
For some people, it's difficult to stop using the hands. My Fundies instructor told us, "When you use your hands, it's INFORMATION. You're trying to fix something with them. Figure out what it is, and fix it another way." For example, if you are sinking, add some air to your BC until you are neutral. If you are out of balance to one side, figure out if your gear is off-center or loose to make you feel that way. My mentor (the above-mentioned NWGratefulDiver) made me swim with my hands clasped in front of me, and that pretty quickly broke me of the sculling habit.
All of this takes time, dives, and experimentation. The good news is that all the time you're working on this stuff, you are DIVING!