A couple of thoughts off the top of my head:
1. Credit based pricing. Base service price + included "credits" per month w/ ability to purchase additional credits. I see this the most commonly. But it gets pretty confusing what a credit actually means. What if I want a follow up question, is that 0.5 credits? Or what about using a reasoning model, is that 2 credits? What if I offer multiple providers, does OpenAI cost 1.5 credits while Gemini costs 1 credit? Do credits rollover per month? Do they expire?
2. Same as above, but instead credits are actual $USD. Since every API returns how many tokens were used per query, it's easy to calculate how much each question costs. Essentially the same way any AI provider's API works. It would be easy to relay the cost to the end user and show an estimation of exactly how much each query might cost. This allows users to make as many queries as they'd like. If they run out of credits, they can just top up. However, seeing a usage meter and the cost per query might be off putting to the user seeing their balance drain with each question they ask-- as if they're losing something each time they ask a question.
3. Eat the cost and add generic limits. Base service price + avg cost of anticipated AI usage. Similar to how AI providers' chat bots work. You pay a base price with a token bucket rate limiter. Makes sense if you own the API, but gets confusing as soon as you have more than 1 provider with different pricing. This one seems like the best because you can impose arbitrary limits and adjust them as needed. The one drawback is that it punishes power users. If a user heavily relies on this application, I want them to be able to use it as much as they'd like without running into rate limits. Maybe have multiple plans for extended limits? Not my preferred approach, but might be the best option imo.
4. BYOK Hybrid - bring your own key in addition to #3 above (doesn't make sense for #1 or #2). Regular users can just use the application as needed as mentioned in #3 while power users can bring their own key. I'd love to be able to offer this, but this brings great responsibility to properly store the user's API key. Are there any other drawbacks to BYOK? The only one I can think of is that your system prompt can be leaked if a provider has logs. Luckily there isn't really anything special in my prompt; the bulk of it is just the context which is not easily replicable.
While #2 logically makes the most sense, it doesn't provide the best user experience. I am leaning towards #3/4 right now. Is there anything I missed or flaws with this approach? What has been working for you guys?
