I agree with you on the O2 % for non-overhead environments, if it spikes to 1.6, but you're going straight up, meh. Cave penetrations, hell no, O2 needs to be matched for the depth and time to exit.
Narcosis on the other hand, you need to minimize that. My first hand experience was from when we used to switch from trimix to air at 200' because we were scared of the helium (this was in the mid-90s) and that switch was like being kicked in the head by a mule. Coming off helium to deep END's is much more dramatic than descending on air the whole way down -- so if you think you're gonna be fine switching to a bailout with a deep END because you've been to 180 on air before, think again buddy because you're wrong.