It isn't just how long you breathe in and out, it is how much air you exchange during that time. It is possible to inhale for 5 seconds and bring in very little air at all. It is possible to inhale for 2 seconds and fill your lungs.
If you ever have surgery that requires you to use what is called an
incentive spirometer, you will see that it forces you to inhale a large volume slowly enough to keep a little marker from rising too much. I have the fortune/misfortune of owning several of these, and I have sometimes used them in a scuba class. If you can inhale a reasonable lungful without raising the marker past the "best" area through the force of your breathing, you are doing it about right.
If you can get your hands on one, give it a try. If not, think about taking in a pretty normal breath in terms of volume--not so much that you change your buoyancy--but doing it slowly.