英文:
Goroutines vs asyncio tasks + thread pool for CPU-bound calls
问题
Goroutines大致相当于Python的asyncio
任务,但有一个额外的特性,即任何CPU密集型任务都会被路由到ThreadPoolExecutor
而不是添加到事件循环中(当然,前提是我们使用没有GIL的Python解释器)。
除了并发作为Go的一个固有部分带来的效率和代码清晰度之外,我是否忽略了两种方法之间的任何实质性差异?
英文:
Are goroutines roughly equivalent to python's asyncio
tasks, with an additional feature that any CPU-bound task is routed to a ThreadPoolExecutor
instead of being added to the event loop (of course, with the assumption that we use a python interpreter without GIL)?
Is there any substantial difference between the two approaches that I'm missing? Of course, apart from the efficiencies and code clarity that result from the concurrency being an integral part of Go.
答案1
得分: 12
我认为我知道部分答案。我试图总结我对asyncio
任务和goroutine之间的区别的理解,按重要性排序:
1)与asyncio
不同,人们很少需要担心他们的goroutine会阻塞太久。然而,与asyncio
任务相比,goroutine之间的内存共享更类似于线程之间的内存共享,因为goroutine的执行顺序保证要弱得多(即使硬件只有一个核心)。
asyncio
只会在显式的await
、yield
和某些事件循环方法上切换上下文,而Go运行时可能会在更微妙的触发器上切换上下文(例如某些函数调用)。因此,asyncio
是完全协作的,而goroutine只是大部分协作(而且路线图表明它们将来会变得更不协作)。
一个非常紧密的循环(例如数值计算)仍然可能会阻塞Go运行时(以及它所在的线程)。如果发生这种情况,它对性能的影响将比在Python中小得多,除非它发生在多个线程中。
2)Goroutine具有开箱即用的并行计算支持,而在asyncio
下则需要更复杂的方法。
Go运行时可以并行运行线程(如果有多个核心可用),因此它有点类似于在没有GIL的Python运行时下在线程池中运行多个asyncio
事件循环,并在前面使用一个语言感知的负载均衡器。
3)Go运行时会自动在单独的线程中处理阻塞的系统调用;而在asyncio
下,这需要显式地完成(例如使用run_in_executor
)。
话虽如此,在内存成本方面,goroutine与asyncio
任务非常相似,而不是线程。
英文:
I think I know part of the answer. I tried to summarize my understanding of the differences, in order of importance, between asyncio
tasks and goroutines:
- Unlike under
asyncio
, one rarely needs to worry that their goroutine will block for too long. OTOH, memory sharing across goroutines is akin to memory sharing across threads rather thanasyncio
tasks since goroutine execution order guarantees are much weaker (even if the hardware has only a single core).
asyncio
will only switch context on explicit await
, yield
and certain event loop methods, while Go runtime may switch on far more subtle triggers (such as certain function calls). So asyncio
is perfectly cooperative, while goroutines are only mostly cooperative (and the roadmap suggests they will become even less cooperative over time).
A really tight loop (such as with numeric computation) could still block Go runtime (well, the thread it's running on). If it happens, it's going to have less of an impact than in python - unless it occurs in mutliple threads.
- Goroutines are have off-the-shelf support for parallel computation, which would require a more sophisticated approach under
asyncio
.
Go runtime can run threads in parallel (if multiple cores are available), and so it's somewhat similar to running multiple asyncio
event loops in a thread pool under a GIL-less python runtime, with a language-aware load balancer in front.
- Go runtime will automatically handle blocking syscalls in a separate thread; this needs to be done explicitly under
asyncio
(e.g., usingrun_in_executor
).
That said, in terms of memory cost, goroutines are very much like asyncio
tasks rather than threads.
答案2
得分: 1
我想你可以这样认为它在底层是这样工作的,当然。虽然不是非常准确,但足够接近。
但是有一个很大的区别:在Go中,你可以编写直线代码,所有的I/O阻塞都会自动处理。你可以在简单的直线代码中调用Read,然后Write,再调用Read。而在Python的asyncio中,据我了解,你需要排队一个函数来处理读取,而不仅仅是调用Read。
英文:
I suppose you could think of it working that way underneath, sure. It's not really accurate, but, close enough.
But there is a big difference: in Go you can write straight line code, and all the I/O blocking is handled for you automatically. You can call Read, then Write, then Read, in simple straight line code. With Python asyncio, as I understand it, you need to queue up a function to handle the reads, rather than just calling Read.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论