Make sure you find a dive shop that you are comfortable with, and most importantly, an instructor. All of the reputable agencies offer good curriculae, but the key is how you are taught. I learned a lot more than the books contain (and a bit about the differences between what they contain, what advanced divers may choose to do additionally and/or differently, and, most importantly, why) by working with an instructor who cared about what his (and her) students learned. I'd avoid any place that boasts about the number of students that pass -- I'd rather know that they won't pass a student that hasn't mastered the skills! (Yes, the better shops will accomodate those who require additional time and training.)
Number two on comfort is how you feel in the water. Many people fit well into rental suits. For me, I'm not "normal" in shape, so the couple of hundred dollars to get a well-fitting wetsuit was well worth it. I don't know where you live, but here in NorCal there are some reputable custom shops that for $300-400 will fit you like a glove if one of the major manufacturers' suits can't. I was lucky that it only took me a month of looking to find the off-the-rack suit that fits. Wet suits are typically marked up 100% from cost, and are generally not under the same MSRP-based discounting policies as are regulators, BCs, and instruments, so you might get a much better overall price from your LDS on a package. There are also mail-order sources, if you are so inclined.
Some locales rent mask and fins, but a good seal around your face (and corrective lenses, if you need them) is, IMHO, well worth the investment. Most of the rental fins are functional, but nowhere as efficient as the better of fins out there on the market today.
Talk with people you respect about equipment. Find out not only what they like (or dislike), but also what kind of diving they do and why they hold the opinions. Extreme case-in-point: "I wouldn't dive in anything but my (insert $3000 model here) drysuit with argon fill." They wreck dive (many potential abrasive or cutting surfaces) in 34 deg. water. You may know that you'll never dive in anything but open water over 70 deg. F. Don't get me wrong, there is a lot to be said for a dry suit for cool or cold water, once you have the proper training and experience. Some people have a 3mm for tropical water and dive dry everywhere else.
One thing that I learned for other equipment-intensive activities (rock climbing, hang gliding, motorcycles) is that my gear needs changed with my experience. Would I buy, as an experienced enthusiast, what I bought as an intermediate? No. Would I use it and enjoy it, yes. Would I have wanted to gain experience with gear tailored for an advanced enthusiast? In many cases, no way! In any of these persuits, the quality of gear improved immensely in the few years that I was gaining experience. By the time I was ready for higher-performing gear, it was a lot better than what I would have bought a couple of years prior. Recreational scuba is being driven the same ways by the most advanced divers and their philosophies and practices; I expect the same advancement in the gear.
Remember that shops want to sell you what they have in the back, that is their job! (Though I have gret respect for a couple of salesmen who, when asked about a dive computer that they didn't carry said effectively, "Great computer, I'd dive with it. I can't sell you one that is markedly better.")
Other advice -- make sure that you are comfortable floating in the middle of the ocean. That doesn't mean that you have to be able to swim 10km, but knowing that you can keep your head out of the water for a long time without additional bouyancy is a good thing.
Finally -- don't feel it is a competition. Everyone learns at a different rate, and has trouble with different things. If you go at a comfortable pace, you actually get to see the scenery as go, rather than just some arbitrary destination. Enjoy the ride...
