That’s the tricky part about building LLM apps. I’d love to hear more from Indie devs because money is absolutely a bottle neck here.

For fun:

You don’t need an LLM for some of your calls I think. “Where is the Eiffel Tower”, Eiffel Tower is a NER that small NLP libraries can extract. Then it’s a simple long/lat lookup. You might be able to re-route 20% of your calls to a no-cost backend call.