1
AI builders, if you are using Llama 3 in backend, where do you host it or what API do you use? (For production usecases with good speed and rate limits close to ChatGPT or Claude)
-AWS sagemaker
-Self host on cloud GPUs
-Replicate API (just found them, 0.05/1m token, legit?)
-AWS bedrock (seems pricy)
-Others - pls comment
Any feedback is welcome!
I am working on an AI newsfeed app. It uses LLM to summarize and filter stories on any website or RSS. When exporting this custom AI feed to custom RSS (so that users on RSS readers can subscribe them), I am wondering which RSS field should I use for AI summary, the summary prompt or AI filter. Namely:
1. AI summary per top feed links (AI will summarize the page pointed by the link URL) - I am thinking of adding a new namespace within item: `channel > item > ai:summary`
2. User's summary prompt (.e.g "What are key points it covers" or "What are the key financial metrics") - What about `channel > item > ai:summaryPrompt`
3. Handle AI filtered feed and user's filter prompt (e.g. "Anything about open source") - I am thinking of including only the filtered feed items in RSS and adding a per channel tag like `channel > ai:filterPrompt`. Or alternatively include all the feed items but add `channel > item > ai:filterPrompt` only for the items filtered by AI (but I doubt RSS readers can handle it properly).
(I want to preserve existing RSS fields if possible, like item description `channel > item > description`)
Any feedback is welcome!