I too have reservations about such a tall building on a midrise street. It certainly does not sound appropriate or desirable. That being said, I look forward to seeing the plans (and with the right setbacks, etc, it could work).
I'd rather it here than west of Spadina on King. It is pretty much next door to the SunLife Centre (or its mirror image building) which is what, 28-30 office storeys tall? It won't be much taller than that. It wont' take away from the mid-rise nature of the street because it is at the foot of the midrise portion. Everything west (save Lightbox) is midrise and everything east (until Church) is the tallest in the city. Seems like a perfect place and not a new place for a highrise.
I will wait for a render before I comment but I'm bordering on too much Clewes.