(Disclosure: Just for the sake of discussion. I, too, prefer just some shorts and a rash guard)
In sub-tropical/tropical environments, you can definately get away with the minimal protection necessary. However, depending on your dive location and conditions, I would also recommend getting used...