Right, but a bit more subtle. The Cafe was invited to apply for conditional use. It appears (from my limited research) that within the Financial District (and whatever overlays might apply) ground floor office use is not allowed
by right. Conditional use means that office use on the ground floor requires approval based on what is actually proposed. The Cafe might readily be approved as a conditional use or might be approved after a few adjustments. It's just not approved by right.
For what it's worth, reading the appeal reminds me why its worth paying a high dollar real-estate lawyer who deals with the planning department all the time. The Cafe's appeal didn't address the basis of the complaint: supplemental versus primary use in terms of floor area. Basically, there was no basis for overturning the zoning official's determination because the appeal did not address the basis for the zoning official's determination.