You can always do UNEXSO's Dolphin dive on Grand Bahama. It's not exactly rural, although Port Lucaya is not a large place, it is a fun place, and the dolphins aren't exactly wild, but I hear its still a blast. These are Atlantic Bottlenose Dolphins (like Flipper) that are contained in a lagoon, so its not quite like Sea World, but not a chance encounter either.
The only place I have encountered dolphin in the wild was off the coast of Lanai, when a pod of spinner dolphin swam by while we were finishing a dive at the Cathedrals. The Big Island and Maui are other possibilities for seeing Spinner dolphin. They are very small, but put on a big show as they leap high out of the water in a spinning motion.
Another possibilty is the snorkel with the dolphins in Bimini. Its not a dive and dolphins aren't guaranteed, but they have a good track record of finding the local pods of Atlantic Spotted dolphins.