I thought I would tell a story of some divers that did this.
It was in South Florida, near the Hillsboro inlet. They had a single hunter in the water, and he attached his flag to the reef while he hunted, using the kind of hook you describe. After a while they were surprised to see a commercial dive boat approach while sounding its horn. That boat had to come to them because they did not have their radio on. The commercial dive boat told them that their hunter had strayed far, far, far from that flag and surfaced in the middle of their dive flags. The boat tore off and sped through those flags to get to their diver. It took a while to locate him and get him on the boat, at which time they went back to pick up their flag.
They tried to pull it up, but it was apparently too well affixed to the reef, so they got a good grip on it and held on tight while the skipper throttled to full speed in order to break off a chunk of reef so they could get the flag back. They ignored that damned commercial boat, which was once again sounding its horn for some reason. (They still hadn't turned on their radio.)
They must have realized that they had the wrong flag when they pulled in the line. It took awhile for the commercial boat to get to them to retrieve it because they first had to see if the dive team whose flag had been yanked away was OK.
I was the diver draging the flag on that trip. When I realized a boat was taking off with my flag, I had the sense to let it go and shoot a bag. I am glad I had not made the mistake of attaching it to my BCD.