undefined | Better HN

0 pointspas6y ago0 comments

How come there's no work stealing? Green threads are supposed to be backed by some N:M thread pool, no?

Also, isn't the problem that JSON decoding (or whatever computation) simply block the thread and the other green threads cannot proceed at all, because there are simply no safepoints (yield points) inside these low level functions?

And in all these cases shouldn't the application estimate work (eg. in case of JSON if the string is longer than 100K), and if it's too big just put it on a dedicated heavy compute N:N thread pool?

For Python it's best practice anyway because the GIL, no?

0 comments

ryanobjc6y ago

Is there a n:m green threading mode for python? What does gunicorn/gevent do? Sounds like “not that”.

Yes best practices and all that, but that really doesn’t sound like what’s happening.

pasOP6y ago

Python was always more about breadth than depth. (CPython is full of known inefficiencies, but it's with us since 1989, and basically the core dev who worked most on performance - Victor Stinner - thinks the best way is to introduce subinterpreters - https://github.com/vstinner/talks/blob/master/2019-EuroPytho... )

Oh, that PDF is interesting, Python 3.8 has shared memory for multiprocessing, no more pipe objects between processes.

Furthermore extension and internal stuff always had the ability to release the GIL and do its own thing (for example, on a threadpool, or using async/nonblocking I/O). But I have no idea about Gevent. I never liked it. (Just as Twisted/Tornado it was too much magic for too little benefit.)

Matthias2476y ago

I don’t know what gevent is doing. But if you have a global interpreter lock then M:N might not be worth it, since only thread will make progress (outside of the syscalls, which are non blocking).

j / k navigate · click thread line to collapse