I know a diver in our area who dives on Civil War era shipwrecks located in our area. The river in which these wrecks are located in has 1-2 foot viz on a good day. He has been doing salvage work on these wrecks for years, amassing a very large collection of artifacts and I've spent some time with him while he explained the different items which he had on display. I walked in one day and he even had the entire keel of a Civil War era boat which he and a team had raised and were in the process of drying out for display.
The people who have been fortunate enough to see his collection (it is open for anyone to come in and see, yet very few people know about it, I send anyone I can his way) get to see things that they would never, ever have been able to see - even if they were a diver, due to the conditions at the wreck sites.
Had these artifacts remained at their original site, they probably would have been lost to history. I think that some common sense has to be applied - if there is a near zero chance of people having access to artifacts and they are in danger of being lost forever - I think that salvage for display is fine, if the wreck is a diver accessible site and is in no immediate danger, then taking artifacts simply for the sake of taking artifacts should be strongly discouraged.