undefined | Better HN

0 pointstshaddox5y ago0 comments

That’s the point. Running multiple paginated queries in parallel is essentially circumventing the API provider’s intent to limit the number of items requested at one time.

0 comments

eyelidlessness5y ago

If your API doesn’t support a single client making ~6 concurrent requests, let me tell you about browsers since the last millennium.

tshaddoxOP5y ago

It’s not about what the API provider can physically support, it’s just about being a polite client. They probably can support 6 concurrent requests without it causing a problem. The point is that if they use offset or page number pagination with a maximum limit of 100, that’s probably a clue that you shouldn’t preemptively grab the first 100 pages in parallel.

eyelidlessness5y ago

Right, that’s my point. Don’t be overzealous obviously. But some concurrency should be expected (and in my opinion designed for and encouraged). There’s a huge gulf between grabbing 100 pages and grabbing the number of resources a browser grabs by default.

gkop5y ago

It’s not hard to imagine scenarios where an API provider might reasonably prefer n concurrent requests of m batch size over a single request of n * m batch size.

1 more reply

j / k navigate · click thread line to collapse

0 comments

eyelidlessness5y ago

If your API doesn’t support a single client making ~6 concurrent requests, let me tell you about browsers since the last millennium.

tshaddoxOP5y ago

eyelidlessness5y ago

gkop5y ago

It’s not hard to imagine scenarios where an API provider might reasonably prefer n concurrent requests of m batch size over a single request of n * m batch size.

1 more reply

j / k navigate · click thread line to collapse