Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Perhaps they meant that heavy long-running calculations could be offloaded to a worker pool with a help of concurrent futures and run_in_executor()

- https://docs.python.org/3/library/concurrent.futures.html

- https://docs.python.org/3/library/asyncio-eventloop.html#asy...



This will only help if the workers are separate processes. Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.


> Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.

Using cython:

    with nogil:
        # whatever you need to do, as long as it
        # doesn't touch a python object
If you're doing heavy calculations from python you should at least be considering cython.


sure, the pool only cares about concurrent.futures.Executor interface, an implementation could be processes or cloud resources.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: