Hey there Riley,
Nitrox is a great tool for diving, and like any tool, it has it's place, upsides and potential downsides. Here's a quick overview:
Upside #1 - on all dives, reducing your exposure to nitrogen pressure reduces your body's uptake of nitrogen, making the every dive a safer one (not withstanding O2 toxicity issues discussed below). So whether you choose to dive longer or just wish to reduce your N2 exposure, nitrox is great.
Upside #2 - Nitrox gas is really great for extending bottom times in the recreational mid-range of depths. 40 feet ofr shallower, there is minimal gain, and of course when you hit 90 feet or deeper, then the O2 issue begins to play a bigger role in gas selection.
Downside #1 - Increased risk of oxygen toxicity. High partial pressures of oxygen, in combination with higher amounts of carbolic acid in our tissues and bloodstream, increase the potential for CNS oxygen toxicity and the seizures that accompany it. The seizures themselves won't hurt you; but the inability to hold your regulator in your mouth, the inability to control your arm to retrieve that regulator and put it back, and the inability to stop from sucking in copius amounts of water in our lungs ultimately tends to lead to a bad ending for this particular dive. So, make sure you know a bit about gas partial pressures, enriched air (nitrox) gas selection, and the amount of effort you plan to expend on the dive (resulting in greater production of carbolic acid) and you should do fine.
Downside #2 - higher O2 contents result in higher O2 partial pressures, ultimately limiting our depth on dives. Know the depth you plan to dive, choose your gas appropriately, don't deviate from your dive plan, and all should be good.
Downside #3 - Nitrox costs money, so unless you are diving in the recreational mid-ranges (50 to 100 ft), and/or planning repeptitive dives with reduced surfaced intervals, then don't waste the money.
And finally, the risks of handling and filling nitrox are very questionable, depending on where you seek your info. The scuba industry has chosen some random numbers such as 40% and have attached that to recreational limits, gear cleanliness and selection, and a few other things. There is absolutely no scientific basis to this number, in fact, the Compressed Gas Association (CGA) standards for dealing with enriched air start at 225 - there is no leeway for that 22-40% range. And they actually do have a scientific basis. But in all practicality, the important thing is how your tank if filled. If you (or your dive shop) are doing partial pressure fills, then you are putting 100% O2 in the tank, through the valve, and then topping it off with air. The velocities of the gases as they travel through the internal bends of the valve can approach 600 miles per hour - yes you read that right. And if we remember the "fire triangle" formula, you know you need heat, fuel and oxygen to have a fire. For the record, an "oxygen explosion" is really just a very fast growing fire; it is not an explosion in itself. So, when you are pumping that 100% O2 through that valve you want to make sure it is clean and has components and lubricants that are compatible with O2 to reduce the risk of a fire (explosion). If, on the other hand, you are filling the tank from banked nitrox, you have greatly reduced the risk by reducing the partial pressure of the O2 in the gas.
And when we breath the gas, it is coming out of the tank & valve at the reduced mix level, so again, the risk of fire (explosion) is reduced, but not eliminated.
I hope this has helped a little!