Downtown is a wider area than the financial core. It runs from Bathurst to the DVP and the lake up to north of Bloor.The wealthy executives and their cars coming down from Rosedale and the wealthy portions of North Toronto and York? Sort of reminds me of the time when drivers made a fit about removing the reversible lane on Jarvis because of the inconvenience it would cause.
Personally I would imagine driving into the Financial District in as a status of convenience, as parking rates are a bit too expensive to be sustainable for the average middle class.
Parking rates in much of that area run in the $150-400 range, which is expensive but manageable for an upper middle class family. Or, well, at least 20% of incomes, because 20% of commuters do it.