Interesting observations. I agree with Rhone Man that travel connections act as an impediment. I think that the diving is very good, and is actually underrated, but it does not have the distinguishing characteristics of the best diving destinations. For example, there are no dramatic walls like Cozumel, or Little Cayman, or the East End of Grand Cayman. With that said, the underwater topography is very nice. The health of most of the reefs (though not all) is very good. My guess is that the diving is not more of a draw because there is little that is identifiable about it like some of the other destinations.
With respect to the boat charter options, I agree with TS&M that this type of trip is a great option. I have done it twice (once captain only and once with a crewed charter). The second time, the crew (husband and wife) were both certified divemasters, and they had a compressor for filling tanks on the boat, so we did our diving from the boat. It was great, and I plan to do it again someday. The first charter trip we did was with another family who were not divers, and we just did snorkeling from the boat, although the rendezvous diving was an option we used, just meeting up with dive guides from local dive shops. That was also a lot of fun, and is easy to set up. It is not a boring trip at all, because the scenery is great, as is the snorkeling, and there is stuff to do at local stops.
I guess I am glad that the BVIs are not too popular, because they are not overrun, and that probably helps preserve the reefs, and keeps the atmosphere low key and fun.