I know this thread is a couple of weeks old now, but since I live on Seaton St. in the block you describe, I thought I should pipe up. My wife & I bought a home here in 2007, after initially looking (like you!) in the East York (East Danforth) area. Given our price range at the time, we thought that East York was our best bet, but when we saw a possibility on Seaton (although slightly more expensive than the places in East York we were looking at) the possibility of living downtown, walking to work, not being chained to the TTC, and living in a house rather than a condo became a reality. We had looked at many, many places back then (going to open houses, etc. for over a year) and really thought hard about our options.
We were familiar with the South Cabbagetown area (having already looked at a place on Berkeley and frequented some of the restaurants in the area) and knew it was a somewhat colourful area. But in the end, we thought the area was tremendous value (in our opinion moreso than the other areas we had considered like East York & Leslieville) so we bought.
Three years later we are both extremely happy with our decision. We have loved living in the area, within short walking distance to the Distillery, Riverdale Park/Farm, St. Lawrence Market & the Rainbow Cinemas, Allen Gardens, restaurants/bars/stores on Parliament, many grocery options, all kinds of interesting/changing options in Corktown.
Some changes that we've seen in the last few years (apart from the obvious Regent Park revitalization):
-3 new banks have opened in the area and two large grocery stores. If this is not a good sign, I don't know what is.
-Several major condo developments stretching from the Sherbourne/Adelaide northward and Eastward.
-Eclectic retail opening from Queen/Sherbourne going east along Queen & many more restaurant options along here.
-The Shoppers Drug Mart at Queen & Parliament (although hideously suburban looking, I agree) is very convenient for purchasing staples (good deals on milk and frozen baguettes!)
You are living downtown and of course, you are going to see unsightly things. Hookers & johns do utilize the alleyways (as they do other alleyways in the city). Seeing an old bike messenger's penis while I was trying to retrieve my bike from the garage one morning was not something I wanted to wake up to. But I would say that the type of activity has steadily gone down over the last three years. And we feel really safe walking about. My wife frequently goes jogging in the area and isn't bothered.
There aren't a lot of families with kids in the area. I hope that will change. There are two demographics we're really seeing move into the area lately: professional 20-30 somethings (w/o kids) -- we fall into this group -- and more established same-sex couples.
The best thing about the area has been the community. The Cabbagetown South association is active in the area and the people who live in the area are always out working on their yards, etc. and up for a chat. They have helped us settle in and everyone looks out for each other. I haven't owned a condo but I have lived in several apartment high-rises and have never had this type of community experience in a tower. I can only imagine that within a few years many of those moving into the nearby condo developments will outgrow those digs and a good fraction of them will look for single-family housing nearby (which is in very short supply).
So of course, by no means is my opinion unbiased, but I feel very positively about the South Cabbagetown area. It's been a great few years here for us, and hopefully many more. Best of luck to you in your search.