I see we are getting to the psych part of this - I admit having my mother on the back of this all "Everybody gets to play or nobody gets to play" kinda attitude, sharing was ingrained hard. I am working on it!!
I think the things most affecting this poor choice on our part were:
a) inexperience - first time, we were taken aback anyone would even ask to join us, we should be the ones whining for bigbro to let us tag along, we didn't have our lines rehearsed
b) the trap of vagueness which led us 50ft from beach and starting to feel like we are backing off from an earlier yes. A yes that was to a totally different question but not in his mind. It just got too far before we really figured out what was going on. This probably equals to inexperience as well, we should have asked about his intentions more bluntly but how to clarify if you thought it was all clear. Then it was like, WE made him get all the way here just to tell him to back off...
("Excuse me, where is your buddy? So, you going solo? I am sorry, we didn't understand you were planning to dive with us but just use our flag. Sorry to tell you but I can't keep track of two buddies at the same time - so do you need help getting back ashore?" That's what should have come out of our mouths but we were dumbfounded.)