undefined | Better HN

0 pointsbredren2y ago0 comments

The speed of GPT-4 via chatgpt varies greatly on when you’re using it.

Could the data have been collected when the system is under different loads?

0 comments

2 comments · 2 top-level

Unless they captured many different times and days, that is very likely a factor. GPU resources are constrained enough that during peak times (which vary across the globe) the token throughput will vary a lot.

MacsHeadroom2y ago

The speed data is an average over 30 days.

Clearly OpenAI is throttling their API to save costs and get more out of fewer GPUs.

j / k navigate · click thread line to collapse