The most accurate and cheapest AI for scraping (opens in new tab)

(ortutay.substack.com)

20 pointsmarcell1y ago5 comments

5 comments

Your Gemini Flash score is low because Gemini Flash unlike gpt mini doesn't have system elaborate system prompts set by Google.

You need to write system prompts for Gemini and I bet it will blow smoke chatgpt mini in benchmarks.

marcellOP1y ago

I’ll give it a shot and update the results.

Unfortunately the bigger issue with Gemini is cost, which is too high for the scraping use case.

Gemini Flash API costs 50% less than gpt-4o-mini. It's not clear why your benchmark recorded 13x more token usage for Flash for the same input.

j / k navigate · click thread line to collapse

Your Gemini Flash score is low because Gemini Flash unlike gpt mini doesn't have system elaborate system prompts set by Google.

You need to write system prompts for Gemini and I bet it will blow smoke chatgpt mini in benchmarks.

marcellOP1y ago

I’ll give it a shot and update the results.

Unfortunately the bigger issue with Gemini is cost, which is too high for the scraping use case.

Gemini Flash API costs 50% less than gpt-4o-mini. It's not clear why your benchmark recorded 13x more token usage for Flash for the same input.

j / k navigate · click thread line to collapse