There have been several occasions when Aggressor Boats went out with a light load. I have read Captains Log submissions with as few as two divers (Utila last year I believe) on board. You are going to have a great time. The Belize Aggressor is my favorite boat with a great crew and great diving. Juan is a great divemaster.
If I may offer some advice for your trip. I always go down the day before and stay at the Georgetown Radisson. This allows for flight delays and eliminates the mad rush to get on the boat. The Belize Aggressor and the Sun Dancer tie up at the Radisson pier. The anchorage at the pier is shallow so the boat turns off it's supply pumps to keep them from getting clogged up with bottom debris. These same pumps run the air conditioning. My first trip on the BA III was in August and let me tell you it was stifling hot in the staterooms at night, almost everyone slept out on deck which wasn't much better. We learned a lesson. On the return stay most definitely stay at the Radisson Friday night. On the way in Friday we packed our gear, checked into the Hotel upon docking, had a nice shower, met everyone for dinner, slept in air conditioned comfort, didn't have to worry about getting rushed off the boat in the early AM, headed to the airport at 1 instead of hanging out at the airport from 9 to 1. Have a great trip.