英文:
How are Go channels implemented?
问题
Go语言中的通道(channels)是一种特殊的数据结构,用于在不同的Go协程之间进行通信和同步。它们可以看作是一种线程安全的队列或数组。
通道的实现并不依赖于具体的架构,而是由Go语言的运行时系统提供支持。在底层,通道使用了互斥锁(mutex)和条件变量(condition variable)等机制来实现线程安全和同步操作。这使得多个协程可以安全地发送和接收数据,而不会出现竞态条件(race condition)等问题。
总之,Go语言的通道是一种高效且易于使用的并发原语,可以帮助开发者编写出安全可靠的并发程序。
英文:
After (briefly) reviewing the Go language spec, effective Go, and the Go memory model, I'm still a little unclear as to how Go channels work under the hood.
What kind of structure are they? They act kind of like a thread-safe queue /array.
Does their implementation depend on the architecture?
答案1
得分: 100
通道的源文件位于(从您的Go源代码根目录)/src/pkg/runtime/chan.go中。
hchan
是通道的中心数据结构,具有发送和接收的链表(保存指向它们的goroutine和数据元素的指针)以及一个closed
标志。有一个嵌入式结构Lock
,在runtime2.go中定义,它作为互斥锁(futex)或信号量,具体取决于操作系统。锁的实现在lock_futex.go(Linux/Dragonfly/某些BSD)或lock_sema.go(Windows/OSX/Plan9/某些BSD)中,根据构建标签选择。
通道的所有操作都在chan.go文件中实现,因此您可以看到makechan、发送和接收操作,以及select结构、close、len和cap内置函数。
要深入了解通道内部工作原理的详细解释,您可以阅读Dmitry Vyukov本人(Go核心开发人员,goroutine、调度器和通道等)撰写的Go channels on steroids。
英文:
The source file for channels is (from your go source code root) in /src/pkg/runtime/chan.go.
hchan
is the central data structure for a channel, with send and receive linked lists (holding a pointer to their goroutine and the data element) and a closed
flag. There's a Lock
embedded structure that is defined in runtime2.go and that serves as a mutex (futex) or semaphore depending on the OS. The locking implementation is in lock_futex.go (Linux/Dragonfly/Some BSD) or lock_sema.go (Windows/OSX/Plan9/Some BSD), based on the build tags.
Channel operations are all implemented in this chan.go file, so you can see the makechan, send and receive operations, as well as the select construct, close, len and cap built-ins.
For a great in-depth explanation on the inner workings of channels, you have to read Go channels on steroids by Dmitry Vyukov himself (Go core dev, goroutines, scheduler and channels among other things).
答案2
得分: 17
这是一个很好的讲解,大致描述了通道(channels)是如何实现的:
https://youtu.be/KBZlN0izeiY
讲解描述:
> GopherCon 2017: Kavya Joshi - 理解通道(Channels)
>
> 通道提供了一种简单的机制,用于协程之间的通信,并且是构建复杂并发模式的强大工具。我们将深入探讨通道和通道操作的内部工作原理,包括它们如何由运行时调度器和内存管理系统支持。
英文:
Here is a good talk that describes roughly how channels are implemented:
https://youtu.be/KBZlN0izeiY
Talk description:
> GopherCon 2017: Kavya Joshi - Understanding Channels
>
> Channels provide a simple mechanism for goroutines to communicate, and a powerful construct to build sophisticated concurrency patterns. We will delve into the inner workings of channels and channel operations, including how they're supported by the runtime scheduler and memory management systems.
答案3
得分: 11
你提出了两个问题:
- 它们是什么样的结构?
Go语言中的通道确实类似于“线程安全的队列”,更准确地说,Go语言中的通道具有以下特性:
- goroutine安全
- 提供FIFO(先进先出)语义
- 可以在goroutine之间存储和传递值
- 导致goroutine阻塞和解除阻塞
每次创建一个通道时,都会在堆上分配一个hchan结构,并返回指向hchan内存位置的指针,表示为通道,这是goroutine可以共享它的方式。
上述描述的前两个特性类似于带锁的队列实现。
通道可以传递给不同的goroutine的元素是作为hchan结构中的循环队列(环形缓冲区)实现的,索引表示缓冲区中元素的位置。
循环队列:
qcount uint // 队列中的数据总量
dataqsiz uint // 循环队列的大小
buf unsafe.Pointer // 指向包含dataqsiz个元素的数组
以及索引:
sendx uint // 发送索引
recvx uint // 接收索引
每当一个goroutine需要访问通道结构并修改其状态时,它会持有锁,例如:将元素复制到/从缓冲区中,更新列表或索引。某些操作被优化为无锁操作,但这超出了本回答的范围。
Go通道的阻塞和解除阻塞属性是通过两个队列(链表)实现的,它们保存了被阻塞的goroutine。
recvq waitq // 接收等待者列表
sendq waitq // 发送等待者列表
每当一个goroutine想要向满通道(缓冲区已满)添加任务,或者从空通道(缓冲区为空)中取出任务时,都会分配一个伪goroutine sudog结构,并将该sudog作为节点添加到发送或接收等待者列表中。然后,该goroutine使用特殊调用更新go运行时调度器,这些调用提示何时将它们从执行中取出(gopark
)或准备运行(goready
)。
请注意,这是一个非常简化的解释,隐藏了一些复杂性。
- 它们的实现是否依赖于架构?
除了锁的实现是特定于操作系统的,如@mna已经解释的那样,我不知道还有任何特定于架构的约束、优化或差异。
英文:
You asked two questions:
- What kind of structure are they?
Channels in go are indeed "kind of like a thread-safe queue", to be more precise, channels in Go have the following properties:
- goroutine-safe
- Provide FIFO semantics
- Can store and pass values between goroutines
- Cause goroutines to block and unblock
Every time you create a channel, an hchan struct is allocated on the heap, and a pointer to the hchan memory location is returned represented as a channel, this is how go-routines can share it.
The first two properties described above are implemented similarly to a queue with a lock.
The elements that the channel can pass to different go-routines are implemented as a circular queue (ring buffer) with indices in the hchan struct, the indices account for the position of elements in the buffer.
Circular queue:
qcount uint // total data in the queue
dataqsiz uint // size of the circular queue
buf unsafe.Pointer // points to an array of dataqsiz elements
And the indices:
sendx uint // send index
recvx uint // receive index
Every time a go-routine needs to access the channel structure and modify it's state it holds the lock, e.g: copy elements to/ from the buffer, update lists or an index. Some operations are optimized to be lock-free, but this is out of the scope for this answer.
The block and un-block property of go channels is achieved using two queues (linked lists) that hold the blocked go-routines
recvq waitq // list of recv waiters
sendq waitq // list of send waiters
Every time a go-routine wants to add a task to a full channel (buffer is full), or to take a task from an empty channel (buffer is empty), a pseudo go-routine sudog struct is allocated and the go-routine adds the sudog as a node to the send or receive waiters list accordingly. Then the go-routine updates the go runtime scheduler using special calls, which hints when they should be taken out of execution (gopark
) or ready to run (goready
).
Notice this is a very simplified explanations that hides some complexities.
- Does their implementation depend on the architecture?
Besides the lock implementation that is OS specific as @mna already explained, I'm not aware of any architecture specific constraints optimizations or differences.
答案4
得分: 0
一个更简单的理解通道的方法是,你可以在等待某个条件完成时暂停程序的执行,通常用于防止竞态条件的发生。竞态条件指的是一个线程可能在另一个线程之前结束,然后你后面的线程或代码依赖的某些内容可能无法完成。
举个例子,你有一个线程从数据库或其他服务器中检索一些数据,并将数据放入一个变量、切片或映射中,但由于某种原因它被延迟了。然后你有一个使用该变量的进程,但由于它还没有被初始化,或者还没有获取到数据,程序就会失败。
所以在代码中,一个简单的方式是这样的:
package main
import "fmt"
var doneA = make(chan bool)
var doneB = make(chan bool)
var doneC = make(chan bool)
func init() { // 程序启动时运行
go func() {
doneA <- true // 给doneA发送true
}()
}
func initB() { // 阻塞
go func() {
a := <- doneA // 在这里等待doneA为true
// 在这里做一些操作
fmt.Print(a)
doneB <- true // 表示完成
}()
}
func initC() {
go func() {
<-doneB // 仍然阻塞,但不关心值
// 在这里写一些代码
doneC <- true // 表示完成该函数
}()
}
func main() {
initB()
initC()
}
希望这能帮到你。这不是上面选中的答案,但我相信这应该有助于消除一些疑惑。我想知道是否应该提出一个问题并自己回答。
英文:
A simpler way to look at channels is as such, in that you may like to hold a program up while waiting for a condition to complete, typically used to prevent RACE condition, which means a thread might not finish before another, and then something your later thread or code depends on sometimes does not complete.
An example could be, you have a thread to retrieve some data from a database or other server and place the data into a variable, slice or map, and for some reason it gets delayed. then you have a process that uses that variable, but since it hasn't been initialised, or its not got its data yet. the program fails.
So a simple way to look at it in code is as follows:
package main
import "fmt"
var doneA = make(chan bool)
var doneB = make(chan bool)
var doneC = make(chan bool)
func init() { // this runs when you program starts.
go func() {
doneA <- true //Give donA true
}()
}
func initB() { //blocking
go func() {
a := <- doneA //will wait here until doneA is true
// Do somthing here
fmt.Print(a)
doneB <- true //State you finished
}()
}
func initC() {
go func() {
<-doneB // still blocking, but dont care about the value
// some code here
doneC <- true // Indicate finished this function
}()
}
func main() {
initB()
initC()
}
So hope this helps. not the selected answer above, but i believe should help to remove the mystery. I wonder if I should make a question and self answer?
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论