As a beginner, a simple but good point & shoot camera with a housing that's good to depth of 130 ft should be sufficient for you to bring some nice memories back from the water.
A strobe may help you get better color, but some camera's built-in flash works fine too if most of your dives are in tropical water with great vis. So that could be a later upgrade.
The housing price could change dramatically if you need something deeper than 130 ft.
A new set of P&S camera (like Canon G series or Olympus) with basic manufacture housing (<130ft) and accessories would cost about $600-700 without considering additional lens, strobes, etc. Later could easily cost another grand or two.
If you're looking for something less than that, then a used set might be an option (I might have one up for sell soon).