That would only be true if there is something inside the device that is consuming power- such as a smart TV which is typically on standby but not completely "off". As far as most (or all?) electronic dive gear components such as lights, when they're off, they're off and it makes no difference whether the battery is inside it or on a shelf next to it.
But is that always true? I think the concern, whether real or imagined, is leakage of current through circuitry that is not powered. Unlike electronics of yore, there isn’t always a physical switch creating a perfect open circuit at the battery.