Our bodies are largely composed of water, bone, muscle and fat, with fat probably being the most variable.
Approximate specific gravities of:
fresh water 1.0
seawater 1.025
fat 0.94
muscle 1.06
bone 1.9
Except for bone, these are not huge differences, but we all know heavily muscled folks who have a more difficult time floating than their more adipose brethren, so they do have an effect.
However, water retention (extra water) will surely be a tiny percentage of our body mass, and it is close to the specific gravity of the medium in which we dive.
Therefore, I'll assert that the issue is not water retention, but almost certainly some or all of those factors already mentioned by captn and ro.
Or maybe you are wearing equipment that is less "floaty" than your fellow divers. e.g. wetsuits tend to lose some of their buoyancy over time.