I've lived in Cabbagetown since 1998 when my newly married wife and I bought our semi. Except for three years in Fredericton, NB, we haven't lived anywhere since. We love CT, the walkability, the canopy of trees, and closeness to transit and the DVP, the parks, dogs, families, etc, etc. Yes, there are some shady folks, but that makes the neighbourhood stronger, since there's an almost siege mentality, where everyone is watching out for each other - for example, in the warmer months, you'll always find me an my neighbours on our front porches, wine in one hand, book or tablet in the other. In the 'burbs we'd all be in the basement or perhaps backyard.
It's not perfect of course. For example the local schools are hit or miss, but I would never choose a home based on the school. Instead we send our kids to a school out of district, and keep CT.