Mask & fins: Probably got those for your cert class -- got to have a mask that fits or why bother?
Exposure suit: People have peed in those rental wet suits -- 'nuff said.
BC: Trim is a learned skill but a properly fit BC makes a world of difference. I concur with BP/W, and those can be picked up used for cheap as cave divers move to sidemount, and there's nothing wrong with a used backplate -- it's hard to damage a steel plate -- about the worst you could find would be the webbing trimmed to short for you and changing the webbing isn't too tough. Just make sure the wing you get isn't a doubles wing -- you don't want to be struggling with a 55lb wing in singles. You'll likely need a single-tank adaptor too.
Computer: You have to monitor depth and time. You can do this with an analog depth gauge and a dive watch, but why not just start out with a computer. There's a learning curve to dive computers so might as well start early so you can get to know yours. I you use rentals they'll always be slightly different and you have to relearn how to use the thing.
Regs: While there are a world of different regs with different hose configurations, there isn't much of a learning curve with them -- attach them and they're ready to go, and they all work pretty much the same. They are arguably the most critical piece of equipment, but the failure rate is low, so the basic uniformity of use is why they're at the bottom of my list. Of course with a rental reg you're putting a mouthpiece in your mouth that's been in uncounted other mouths, but hey, what's a little slobber among friends... (or dive buddies).
Plus all the add-ons, like knives, SMBs, lights, spools, etc.
Enjoy the shopping!