undefined | Better HN

0 pointsalecco10mo ago0 comments

The best and latest Gemini Pro model is not SOTA. The only good things it has are the huge context and the low API price. But I had to stop using it because it kept contradicting itself in the walls of text it produces. (My paid account was forced to pay for AI with a price hike so I tried for a couple of months to see if I could make it work with prompt engineering, no luck).

Google researchers are great, but Engineering is dropping like a stone, and management is a complete disaster. Starting with their Indian McKinsey CEO moving core engineering teams to India.

https://www.cnbc.com/2024/05/01/google-cuts-hundreds-of-core...

0 comments

3 comments · 3 top-level

jemmyw10mo ago

There are problems with every model, none of them are perfect. I've found Gemini to be very good but occasionally gets stuck in loops: it does, however, seem to detect the loop and stop. It's more cost effective than the Claude models, and Gemini has regular preview releases. I would rate it between sonnet and opus except it's cheaper and faster than both.

For whatever reason there are tasks that work better on one model compared to another, which can be quite perplexing.

mindwok10mo ago

It was the best model according to almost every benchmark until recently. It’s definitely SOTA.

navigate831010mo ago

No amount of big context window can stop the model from context poisoning. So in a sense, it's a gimmick when you start having the feel of how bad the output is.

j / k navigate · click thread line to collapse