There are a whole bunch of other possibilities. There are some underused buildings on Adelaide and Camden that come to mind - in addition to those mentioned above. Unfortunately I'm unable to recall specific addresses.
West side of Charlotte Street (from south to north):
430 King Street West: Charlie Condos
15 Oxley Street: Glas
36 Charlotte Street: Charlotte Lofts
353 Adelaide Street: Langston Hall Residences
East side of Charlotte Street (from south to north):
300 King Street West: Mountain Equipment Coop 1-17 Charlotte Street: 2 story commercial building (most likely for redevelopment)
19 Charlotte Street: The Charlotte Room (Restaruant, bar, pool hall)
345 Adelaide Street: 7 story commercial building
There is also a parking lot on the southeast corner of Adelaide and Spadina, however it looks too small to build a condo building.