This is completely false. This might be surprising to learn, but for normal car dealerships (not buy-here, pay-here or used car dealerships) a huge amount of their compensation rides on receiving holdback payments from manufacturers, as well as per-unit bonuses that often have cliffs.
Cash buyers paying invoice price are welcomed (if they aren't too big of a headache) because they push a dealership over or at least closer to the next sales-volume bonus cliff.
Holdback alone is worth more than any realistic origination fee.