All excellent choices. Sea Breeze & Taylor house are located on the Eastern/Northeastern side of Utila over in the Paradise Cove, Rocky Point, Red Cliff area if you're looking at a map, a bit away from town but absolutely beautiful snorkeling if the winds aren't up. Beaches along this area will range from sand, to sand & crushed coral, to coral & ironshore. Sea Breeze is on the beach, where Taylor house sits on the road across from the beach... not sure if that makes any difference to you or not. Of the rental houses in this area, my favorite is Tropico Vista Tranquilla, but it is a bit more expensive. This is a primarily a residential/rental community, so you'll most likely have to come into town to rent Kayacks, ATV's etc if they aren't included in your rental.
Bandu Beach, private beach/bar/vollyball etc is located between this area & town, fairly easy walking distance & is acessible for about $3.00pp.
The Round house is located in Blue Bayou, while still out of town, the road in/out is a bit easier than the road from Red cliff. It sits on the beach, and is near the public beach, as well as a smaller private beach in Blue Bayou that's accesible for a small fee. The snorkling in Blue Bayou is beautiful as well, so you can't go wrong with either option. Blue Bayou is a bit closer to activities in town, which might be a benefit to the non-divers? There's 2 restaurants on the public beach (Champas & Carlotta's), and 2 more (Driftwood & Fidi's) just past them, more are farther into town. Bike rentals are available just inside town if they're not included in your rental.
One thing to consider for yourself is getting back & forth to your diving. There are no operators in Paradise Cove, Red Cliff, Rocky Point area, so you will have to come into town to get your boat. Blue Bayou has one operator down at Coral View, but I'm not sure if they take folks outside their hotel guests, so you might check on that.
Other things to do would, for those so inclined, would include renting ATV's to explore the backroads all the way over to the Northside/Pumpkin Hill beach area. Guided hikes to Rock Harbor & up Pumpkin Hill are available through the Iguana Research Station, horseback riding, Fresh water cave exploration, picnic lunch/snorkling on Water Cay (really cool way to spend a day), fishing & more. Massages etc can also be arranged in town. My favorite activity is hammock time, but that's just me!
Yes we have Sea Mounds in Utila, in addition to sheer dramtac walls & overhangs on our North side, and more gentle sloping walls on our South. I'll let Andy, Steve or someone more qualified than I comment on the cave diving. While I have heard of it here, the stories were from the early days & scared the bejesus out of me, but I'm not a cave diver so that could have a lot to do with it
Steve/Andy any info you guys can share?