Actually the city itself is 4 times bigger, and the GTHA is closer to 3 times bigger, so yes, it's a bit surprising. Not to mention the fact that Toronto gets far more tourists and is just generally much more on the world stage. All that aside, I wouldn't say it always happens but it does seem to happen probably a lot more than it should. My gut feel is that flagship space is much more difficult to come by in Toronto, so it doesn't generally make sense for a brand to set up a 1,500 sq. ft shop on Bloor when it needs 3,000 to 10,000 sq. ft to make it wortwhile. Whereas in Vancouver, flagship may not be the name of the game and a brand can get away with a smaller store and use it as a test market. With the exception of LV, you don't find a lot of 6,000+ sq. ft high end boutiques in Vancouver.
That's my two cents.