The residents aren't working with the developer either, if their request is basically to only allow economically infeasible development. For what the land must have cost them there's no way they could recoup their costs if they only built a low or even a mid-rise structure.
Perhaps the residents need to instead focus on getting the best possible addition to their neighbourhood. Make sure it meets the street well, and complements the existing feel and character of the other buildings.
As a for example, knowing the street and the area, I think it would be a huge plus if they could connect the underground garage to the one across the street that belongs to that white building on St. Mary, so that the underground ramp could be shared. Or, better yet, they could share a new ramp in the new building, so that that eyesore of a ramp that currently abuts St. Nicholas could be backfilled and replaced with a nice little parkette. I'm thinking cobblestones and a nice fountain in the middle would be a great fit for the area, maybe with some formal plantings and a wrought iron fence (All of which could be obtained under a section 37 agreement)