英文:
Cost of having a channel with very large limit
问题
基本上就是标题所说的。我想知道通道大小设置为很大时的缺点是什么。
k := make(chan int, 100000000)
英文:
Basically what the title says. I am wondering what the cons are for having a large limit for a channel size
k := make(chan int, 100000000)
答案1
得分: 8
这将分配大量内存。在makechan的实现中，我们可以看到在创建通道时它会完全分配发送缓冲区：
https://github.com/golang/go/blob/master/src/runtime/chan.go#L71
所以对于你的具体示例，你至少会一次性分配400MB（在32位系统上）或800MB（在64位系统上）的内存。
如果你发现自己需要超大缓冲区，可能有其他数据结构更适合满足你的需求吗？
英文:
It will allocate a lot of memory. In the makechan implementation, we can see that it fully allocates the send buffer when you create the channel:
https://github.com/golang/go/blob/master/src/runtime/chan.go#L71
So for your specific example, you would be allocating at least 400MB (on 32-bit systems) or 800MB (on 64-bit systems) of memory up front.
If you find yourself needing ultra-large buffers in your channels, is it possible that some other data structure might fit your needs better?
答案2
得分: 5
如果你确实需要一个大队列，但并非始终如此，那么你可以考虑创建一个自增/自减的队列。
func makeQueue() (enqueue chan<- int, dequeue <-chan int) {
enq, deq := make(chan int), make(chan int)
var queue []int
go func() {
for {
if len(queue) == 0 {
queue = append(queue, <-enq)
}
select {
case v := <-enq:
queue = append(queue, v)
case deq <- queue[0]:
queue = append([]int{}, queue[1:]...)
}
}
}()
return enq, deq
}
注意：目前实现中，出队操作会复制整个底层切片。虽然在Go中复制是惯用的做法，但是性能分析可能会显示使用链表（例如http://godoc.org/container/list）可能更快。你还可以安排以更大的块提取队列，并让出队操作从这些块中读取，以减少复制操作。
英文:
If you really need to have a big queue, but not all the time, how about making your own growing/shrinking one?
func makeQueue() (enqueue chan<- int, dequeue <-chan int) {
enq, deq := make(chan int), make(chan int)
var queue []int
go func() {
for {
if len(queue) == 0 {
queue = append(queue, <-enq)
}
select {
case v := <-enq:
queue = append(queue, v)
case deq <- queue[0]:
queue = append([]int{}, queue[1:]...)
}
}
}()
return enq, deq
}
http://play.golang.org/p/q_Uw-E9NQb
Caveat: as implemented, the dequeue operation copies the whole underlying slice. Although copying is idiomatic in Go, profiling may showcase that a linked list (like http://godoc.org/container/list) might be faster. You can also arrange to extract from the queue in bigger chunks, and have dequeue read from those chunks, reducing copying operations.
答案3
得分: 3
你可以从channels包中获取一些想法，它支持在通道满时不使用过多空间的通道。它还具有一个批处理函数，听起来可能对你的目的有用。
英文:
You may be able to gather some ideas from the channels package as well: https://github.com/eapache/channels
It supports channels that grow without using excess space when full. It also has a batching function that sounds like it might be useful for your purposes.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论