英文:
Simple concurrent queue
问题
有人能否提及队列实现中的缺陷和性能缺陷?
type Queue struct {
sync.Mutex
Items []interface{}
}
func (q *Queue) Push(item interface{}) {
q.Lock()
defer q.Unlock()
q.Items = append(q.Items, item)
}
func (q *Queue) Pop() interface{} {
q.Lock()
defer q.Unlock()
if len(q.Items) == 0 {
return nil
}
item := q.Items[0]
q.Items = q.Items[1:]
return item
}
我还有PopMany
和PushMany
等方法,我关心的是:重新切片是否会有太多的负面影响?
英文:
Could someone please mention the flaws and performance drawbacks in the Queue like implementation?
type Queue struct {
sync.Mutex
Items []interface{}
}
func (q *Queue) Push(item interface{}) {
q.Lock()
defer q.Unlock()
q.Items = append(q.Items, item)
}
func (q *Queue) Pop() interface{} {
q.Lock()
defer q.Unlock()
if len(q.Items) == 0 {
return nil
}
item := q.Items[0]
q.Items = q.Items[1:]
return item
}
I also have methods like PopMany
and PushMany
, and what I am concerned about is: Is too much re-slicing that bad?
答案1
得分: 10
你可以简单地使用带缓冲的通道。
var queue = make(chan interface{}, 100)
缓冲区的大小可以通过经验确定,以确保它足够大,能够处理推送速率与弹出速率之间的高水位线。理想情况下,它不应该比这个大太多,以避免浪费内存。
事实上,较小的缓冲区大小也可以工作,只要相互作用的 goroutine 不会因为其他原因而发生死锁。如果你使用较小的缓冲区大小,实际上是通过 goroutine 时间片引擎的运行队列来实现排队,这是 Go 运行时的一部分。(在许多情况下,缓冲区大小为零也可能有效。)
通道允许多个读取 goroutine 和多个写入 goroutine。它们的访问并发性由 Go 运行时自动处理。所有写入通道的操作都会交错进行,形成一个顺序流。所有读取操作也会交错进行,以按照入队的顺序顺序提取值。在这个主题上,这里有进一步的讨论:链接。
英文:
You could simply use a buffered channel.
var queue = make(chan interface{}, 100)
The size of the buffer could to be determined empirically to be large enough for the high-water mark for the rate of pushes vs rate of pops. It should ideally not be much larger than this, to avoid wasting memory.
Indeed, a smaller buffer size will also work, provided the interacting goroutines don't deadlock for other reasons. If you use a smaller buffer size, you are effectively getting queueing via the run-queue of the goroutine time-slice engine, part of the Go runtime. (Quite possible, a buffer size of zero could work in many circumstances.)
Channels allow many reader goroutines and many writer goroutines. The concurrency of their access is handled automatically by the Go runtime. All writes into the channel are interleaved so as to be a sequential stream. All the reads are also interleaved to extract values sequentially in the same order they were enqueued. Here's further discussion on this topic.
答案2
得分: 4
重新切片在这里不是一个问题。无论你使用线程安全还是不安全的版本,这基本上就是调整大小的方式。
你可以通过初始化队列的容量来减轻一些调整大小的开销:
func NewQueue(capacity int) *Queue {
return &Queue {
Items: make([]interface{}, 0, capacity),
}
}
这将初始化队列。它仍然可以超出容量,但在达到容量之前,你不会进行任何不必要的复制/重新分配。
可能会导致许多并发访问问题的是互斥锁。在某些情况下,你将花费更多的时间等待锁的释放,而不是实际工作。这是锁争用的一般问题,可以通过将队列实现为无锁数据结构来解决。
有一些第三方包提供了基本数据结构的无锁实现。
只有通过一些基准测试才能确定这是否对你有用。无锁结构可能有更高的基本成本,但在有许多并发用户时,它们的扩展性更好。当互斥锁的成本超过无锁方法时,就会出现一个截止点。
英文:
The re-slicing is not an issue here. It will also make no difference whether you have a thread-safe or unsafe version as this is pretty much how the re-sizing is meant to be done.
You can alleviate some of the re-sizing overhead by initializing the queue with a capacity:
func NewQueue(capacity int) *Queue {
return &Queue {
Items: make([]interface{}, 0, capacity),
}
}
This will initialize the queue. It can still grow beyond the capacity, but you will not be having any unnecessary copying/re-allocation until that capacity is reached.
What may potentially cause problems with many concurrent accesses, is the mutex lock. At some point, you will be spending more time waiting for locks to be released than you are actually doing work. This is a general problem with lock contention and can be solved by implementing the queue as a lock-free data structure.
There are a few third-party packages out there which provide lock free implementations of basic data structures.
Whether this will actually be useful to you can only be determined with some benchmarking. Lock-free structures can have a higher base cost, but they scale much better when you get many concurrent users. There is a cutoff point at which mutex locks become more expensive than the lock-free approach.
答案3
得分: 1
我认为处理这个问题的最佳方法是使用链表,你可以在标准包中找到一个可用的链表实现这里。
英文:
I think the best way to approach this is to use a linked list, there is already one available for you in standard package here
答案4
得分: 0
答案标记为正确的说重新切片不是问题。这是不正确的,重新切片是一个问题。Dave建议的是正确的,我们应该将该元素标记为nil。
在这里了解更多关于切片的信息:https://go.dev/blog/slices-intro
英文:
The answer marked correct says re-slicing is not an issue. That is not correct, it is an issue. What Dave is suggesting is right, we should mark that element as nil.
Refer more about slices here: https://go.dev/blog/slices-intro
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论