I agree with those who say the main factor is cost. A secondary factor is that it's just a good sport/hobby for older people, since it's not so physically demanding.
Sure, there are young people who fly off on ski vacations, etc. (or more likely snowboarding these days), but they are generally people with well-paying careers or who get help from mommy and daddy. There's a much larger segment of the 20-something population--I guess I should confine my comments to the US--that doesn't have that kind of disposable income. If those people live near a coast with good diving, they might very well take up the sport. But for most inland dwellers, the idea of diving in a quarry or lake is not very attractive, so those people will likely not take up diving until later in their lives when they have the money for tropical trips. When I got certified I was in the early years of a new and demanding career, so I hardly dived until I got into my late 30s and had more time and money. With each year, I have more time and (but for those morons in Washington) money to spend on diving. Ideally, I'd like to spend time diving when I retire. It's a good sport/hobby for older people, since it's not as physically demanding as, say, rock climbing, cycling, etc., yet not as sedate as, say, golf. Also, it's possible to combine diving with vacations that you'd possibly be taking anyway as a retiree.
No doubt there are plenty of exceptions to the types of people I'm referring to--I'm just making generalizations.