I fully agree, it's a truly horrible experience when compared to the rest of the world outside of North America. Compared to NYC, Chicago, Montreal and the other major North American cities I've experienced it holds its own... we simply don't value transit like the Europeans and advanced Asian countries seem to.
My year in Seoul showed me what can happen when transit is a priority. They have individual subway lines (line 1 comes to mind) with more stops then Toronto's entire system, above ground bus routes that go EVERYWHERE and comes every 3-4 minutes (no joke), regional commuter trains come in two flavours - slower and cheap (like a GO train) and high-speed (KTX, like nothing we have) giving transit riders the choice between speed and fares. They can get pretty much everywhere in Seoul by bus or subway. The fare is calculated based on distance traveled (averaging less than $1 per trip) using a smart T-Money card that even works in taxi's and variety stores and knows when you're transfering, the stations are always spotlessly clean and have ample washroom facilities (again, spotless) at EVERY STATION... not to mention the countless employees available to help at all times. Most cell phones are integrated into the network as well so you can calculate trip times, see any delays, and of course use your cell phone anywhere in the tunnels.
The government refuses to cut anything transit related, it's an absolute priority receiving massive investment by all levels of government. They do have a wider gap in classes than we do and have a lower portion of low-income earners the need the transit system to be what it is, but if you go on rush hour it's all well-dressed people and suits that most likely have a car in the driveway at home.
The real problem is the lack of funding from the feds and the province, the TTC's hands are tied. Transit on all levels in this country needs to be a priority, the TTC is just an example of how horribly it's all gone.