Not to deminish anything that has already been said (as I agree with this), keep in mind that not everyone has the same lung capacity. If you have a large lung capacity and take a deep breath, guess what? You will just consume more volume and exhaust your tanks that much quicker than the next person.
Therefore, fitness will help your body adapt to using O2 more efficiently - thus, reducing your body's demand for another breath. If you are like those of us that find getting in shape something that is rather difficult to do, you may want to compensate for this by moving to a larger tank size/volume.
Over time, as you get your dive routine down pat and more comfortable under water/or getting to the water, you may find that your air consumption will improve as well. Being calm under water helps, but if you're out of breath trying to get your fins on in the surf...