There are only two dive ops on Cayman Brac that I am aware of, Reef Divers and Brac Scuba Shack. Both make trips to Little Cayman on a regular basis. I have never used BSS, but have dove with their DMs when they worked for Reef Divers. Mikey and Vincent apparently dive with them quite frequently, so they can tell you what their policy is. The trick to diving LC from the Brac is that the crossing between the two can get a bit bumpy coming or going, especially with a moderate to strong east or north wind. Beyond a certain point, the boat captains won't make the trip if they are going to beat the tar out of their boat or their guests to make it there and back. It is generally up to the crew to decide that day whether they can or will make the trip, so there are no guarantees. It is usually done as a 2 dive morning trip and the time depends entirely on the sea conditions. In calm seas on Reef Divers boats, it is probably about a 45 minute trip one way.
The easiest way to get there is to stay at Brac Reef Beach Resort and dive with Reef Divers. I believe that they include a trip to LC once per week in any of their dive packages (weather permitting). That way, you have all week to wait out the weather to be sure to get a chance to go over and see it. They generally only send one boat over per day, but most people run packages from sat-sat, so if it gets to weds or thurs and they haven't been able to go all week, you might see 2-3 boats go on the same day to get everybody a chance to dive over there. If you aren't staying with them, but plan to shore dive the Brac for most of your trip, I suspect that you would be able to schedule in advance to jump on one of their boats depending on how busy they are when you are there. The trick for you will be if they are really crowded and you are not diving with them or staying with them other than those dives, you will have to be really flexible (or lucky) to pick the day, or days when weather allows them to make the crossing. It really just depends on the time of year...I have been there when you could make the crossing easily for 7 days straight, and then there are weeks when making the crossing was only possible one or two days out of seven. If you are shore diving, you might set something up with BSS instead where you would rent tanks from them and get fills from them, and if they have a trip across planned otherwise, you could get on with them and go.
Oh, and I missed your last question...worth it? I think so. On my trips to Cayman Brac, the best dive days were usually the ones spent on Little Cayman. Now I just go to Little Cayman instead. Now with that said, "worth it" really depends on the cost, your financial situation, and the conditions. Brac has excellent diving. So if I were going to have to pay more than say $50 extra for a couple of morning dives on Little Cayman, I probably wouldn't. Same situation if there was a strong north wind and it was very marginal conditions for a trip over. The vis on the Bloody Bay Wall can get washed out pretty quickly with a swell coming from the Northeast. I would not take a long and rough boat ride for an hour each way just to dive in low vis conditions. However, I say that as somebody who thinks $50 is a decent amount of money to part with, who has done probably close to 100 dives on the Bloody Bay Wall, and who knows I will be spending lots of future days on the Cayman Sister Islands. If I had never been there, was not planning to return for several years and thought nothing of dropping $50 on something, I would probably have a different perspective on whether the trip was "worth it" in marginal conditions.