I have no cave training at all. But, I know I want to. I know this because I want to Mexico on vacation last year and did 2 cenote dives. They're basically cavern (not cave) dives and the dive operators there don't require any more training than OW. In fact, my cenote dives were numbers 5 and 6 for me after completing OW. But, they were awesome and convinced me that I definitely want to get training and do even more.
So, one option would be to take a vacation and go dive some cenotes. If you don't like it, you can always dive in the ocean for the rest of your vacation. I stayed at an all-inclusive resort in Playa del Carmen and was able to dive cenotes and Cozumel reefs. The cenotes were a bit of a van ride to get to, but the Coz reefs were a short boat ride. And the resort was cheap!
Plan B would be take a Cavern class. That's not a huge commitment and doesn't require much in the way of new equipment. I think maybe just a light and a reel? It would be more expensive than a "Try Dive" I guess. But, you'd get a lot more out of it, too, I think. Still without spending the huge chunk it would take to go all the way through full cave certification.