Yes, obviously it does cost compute (to them) but customers are not paying for the compute per-se, but rather for correct results/outcomes. This is a big alignment and accountability problem, relevant enough to some make companies like Lovable refund credits to customers when errors and token/credit mishandlings are too serious to ignore. That seems unsustainable in the long run though.
I expect this to become even bigger when the hype cools down and companies start looking for ROIs.
PS: I obviously talking about model errors (i.e. like use N tokens in a LLM solving a problem by just deleting/deactivating a test), and not end-user mistakes.
