Many people aren't aware that a BCD inflator can vent as fast as it fills. They're designed like that. In the event of a vigorously free-flowing inflator just hold the outlet open and it can't cause an uncontrolled ascent. That will also give you the time needed to unclip the inflator hose.
In the vast majority of cases a leaking inflator causes just the sort of thing the OP described. A slow filling of the BCD that's more irritating that life-threatening.
R..
Would you go on a trip knowing the inflator was leaking? I wouldn't, if I am spending $2000 or $3000 to travel, taking a week or more vacation I will have it fixed or replaced prior to leaving. It might be a minor inconvenience but it could easily turn into more. If I have a tire that is leaking slowly, I take it in and get it fixed before it becomes more than a minor inconvenience. YMMV