The only reason that I’m hesitant to wear brightly colored or white gear these days is that I do a lot of diving with sharks. I believe that though sight is only one of the senses that sharks use to identify prey, some species such as Tiger Sharks see contrast quite well and find that attractive. I’d prefer not to have an article on my body that appears as “fish belly white”.
There are some local operations that offer baited shark dives, and at least one of them prohibits the use of white or bright, light colored suits or gear. I also cannot think of any of my hunter friends who wear anything white or light colored.
Conversely, I used to teach at a resort which supplied all of their instructors with bright white BCDs. These made us very easy to see while leading divers even in relatively poor visibility and I never had any qualms about wearing the vest that was issued to me. Hunting activities were not offered at the resort and as far as I’m aware, nobody had any issues at all regarding unexpected attention from the fauna. Likewise over the years I’ve been on dive boats with literally tens of thousands of divers, many of whom had yellow or neon green alternate second stages, and cannot think of a single occasion when these led to undesirable interactions.