Actually, those cities lost a lot too in those decades, but they weren't on the verge of national metropolis status. That isn't to excuse the lack of preservationist spirit in those years in Toronto or excuse choices like building an expressway next to the waterfront but the histories are quite different. Beyond the "Then and Now" thread, I highly recommend Eric Arthur's classic Toronto: No Mean City. It's a book of the architectural history of Toronto from the end of the 18th century to the beginning of the 20th century. Arthur revealed to many that our architectural heritage was in fact quite rich but being quickly destroyed and argued that 19th century Toronto saw the completion of a number of buildings comparable in quality to the best buildings of the time in European capitals.Many US cities, especially in the midwest literally seized all development since the 50's/60's. Definitely there are cities like that today; see Cincinnati/Cleveland/Baltimore/St Louis/Detroit/Buffalo ...list goes on. Most of these cities were not victims of urban renewal unlike Toronto. As if Toronto's historical stock was not strained enough, urban renewal really made a huge negative impact in respect to historic architecture (see Eaton's Centre, Yonge/Dundas, FCP, etc... ). I suggest you check out the "Then and Now" thread on the Picture forum and just see for yourself what urban renewal did to the city.
What is striking, though, is that because Arthur's research was so thorough in investigating the end of every building no longer standing, he revealed a lot about the life and death of Toronto architecture from the 19th century. The 50s/60s are demonized by anyone who appreciates the heritage before that era, but Arthur revealed that the Victorians destroyed a lot of the city as well. They razed a lot of Georgian Toronto, for instance. The Art Deco era saw a lot of demolition. Fires did a lot of damage to our stock of heritage buildings as well.