undefined | Better HN

0 pointswordpad5mo ago0 comments

TLDR for others: * DeepSeek cutting edge models are still far behind * On par DeepSeek costs 35% more to run * DeepSeek models 12 times more susceptible to jail breaking and malicious instructions * DeepSeek models follow strict censorship

I guess none of these are a big deal to non-enterprise consumers.

0 comments

nylonstrung5mo ago

Saying Deepseek is more expensive is FUD

Token price on 3.2 exp is <5% what the US LLMs are and it's very close in benchmarks. Which we know that ChatGPT, Google, Grok and Claude have explicitly gamed to inflate their capabilities

ACCount375mo ago

And we "know" that how, exactly?

nylonstrung5mo ago

Read a study called "The Leaderboard Illusion" which credibly alleged that Meta Google OpenAI and Amazon got unfair treatment from LM Arena that distorted the benchmarks

They gave them special access to privately test and let them benchmark over and over without showing the failed tests

Meta got to privately test Llama 4 27 times to optimize it for high benchmark scores and then was allowed to report the only the highest cherry picked benchmark

Which makes sense because in real world applications Llama is recognized to be markedly inferior to models that scored lower

1 more reply

xpe5mo ago

> I urge everyone to go read the original report and _then_ to read this analysis and make up their own mind. Step away from the clickbait, go read the original report.

>> TLDR for others...

Facepalm.

j / k navigate · click thread line to collapse

0 pointswordpad5mo ago0 comments

I guess none of these are a big deal to non-enterprise consumers.

0 comments

nylonstrung5mo ago

Saying Deepseek is more expensive is FUD

Token price on 3.2 exp is <5% what the US LLMs are and it's very close in benchmarks. Which we know that ChatGPT, Google, Grok and Claude have explicitly gamed to inflate their capabilities

ACCount375mo ago

And we "know" that how, exactly?

nylonstrung5mo ago

Read a study called "The Leaderboard Illusion" which credibly alleged that Meta Google OpenAI and Amazon got unfair treatment from LM Arena that distorted the benchmarks

They gave them special access to privately test and let them benchmark over and over without showing the failed tests

Meta got to privately test Llama 4 27 times to optimize it for high benchmark scores and then was allowed to report the only the highest cherry picked benchmark

Which makes sense because in real world applications Llama is recognized to be markedly inferior to models that scored lower

1 more reply

xpe5mo ago

> I urge everyone to go read the original report and _then_ to read this analysis and make up their own mind. Step away from the clickbait, go read the original report.

>> TLDR for others...

Facepalm.

j / k navigate · click thread line to collapse