Setbacks are an issue because it limits ground level outdoor space, especially for presumably family buyers of luxury homes in the Annex. If they are selling this project as a luxury product, then outdoor space is considered essential, so is some level privacy (set back from neighbours). Most buyers don't equate rooftop patios to match what a backyard extending out of bright kitchen can provide, especially for those seeking the idea of an "Annex home". There's also much competition for that in in neighbourhood, albite at a slight markup,, hence the cheapen remark.
These are reminding me slightly of the townhouses at U-condos on Bay /Charles. which continuously tend to lag on the market, and sell below asking because of some similar issues, although I'll admit they also suffer from an added two tower condos on top and being on a very busy intersection.