Nope - it's Mizrahi's loss. Need I remind you that Apple is a zillion dollar company? They have plenty of options. Their retail fortunes are hardly reliant on a single potential location in a single city in the North American market. The way I expect it went, Apple was willing to go along with their original plans until it became apparent that it would be foolish to passively stick around. But to evandyk's point, this withdrawal may also amount to a calculated bargaining chip on Apple's part. I guess we'll see.
As it stands, the loss of prestige from having Apple pull out is no small thing. Having said that, I want to see this building get finished.