Sometimes I refer to Spire as Spire, and I hear other people do it too. (The name of the condo stands above the entrances in medium sized silver letters).
But yes, if I lived in a building with a particularly terrible or awkward name, I don't imagine I would ever say it out loud.