It's never really bothered me. Then again, when I'm diving with a DM, I'm usually sightseeing, and don't mind being shown stuff that I probably wouldn't notice on my own. If I want to diverge from the main group and do my own dive with just my buddy, we always OK that with the boat first, in which case nobody ever tries to bang you.