I dive a 19cf pony slung like a stage bottle.
Its big enough to get me back to the surface in a "brown water" situation, it can be handed off, and it "disappears" underwater. Finally, I nearly always am spearfishing, and buddies are less reliable in the pursuit of that sport than otherwise.
If I'm doing a "look-see" dive with a trusted buddy, I leave the pony on the surface.
Why not just dive doubles? Mass.
While "weight" disappears underwater, mass does not. Mass requires energy to accelerate, and area requires energy to keep moving, as it increases drag. Both contribute to exertion, which contributes to CO2 loading and increases gas consumption, neither of which is desirable.
However, you must NEVER treat a pony as "extra gas". Its there for emergencies only. And you need to practice "drawing" it - the best time to do so is on your last dive of the day, at your safety stop.
Do NOT ditch your "safe second" reg. That's asking for it. I know people who do remove their "safe second" with a pony, but I strongly recommend against it, because if your BUDDY needs gas, and you have a lot in your backgas tank, why would you want to give him the smallest reservoir of air that you have?!
(I dive a 7' primary anyway, so in my case removing my "backup" wouldn't do me any good - since in any OOA situation I get the backup and the OOA diver gets the one in my mouth!)
Most of the guys I dive with both have ponies and attach them to their primary tank. I don't like that arrangement, as it just smells of entanglement trouble behind you, where you can't see to disentangle yourself! Since the entire POINT of a pony is to get you back to the surface if you lose both a primary air source AND a buddy (your buddy's octo is better than a pony, as it comes with a spare BRAIN at no extra cost!), it seems to me that the absolutely worst thing to do is to add another tank but put it where you can't clear an entanglement personally when its primary mode of use is when your buddy is out of reach (or just plain "missing"!)