Proper way to gain access to a channel length in Go

huangapple go评论83阅读模式
英文:

Proper way to gain access to a channel length in Go

问题

我已经开始使用Go语言一段时间了,并且每天都在进步,但并不是一个专家。目前,我正在学习并发和goroutine,因为我认为这是我在Go工具包中的最后一个未知领域。我觉得我正在逐渐掌握它,但仍然是一个初学者。

我遇到的问题似乎对我来说相当基础,但是我尝试过的所有方法都不起作用。我想找出一种计算通道长度的方法。

根据我所了解,len()函数只适用于有缓冲的通道,所以在这种情况下对我没有帮助。我正在批量从数据库中读取值。我有一个生成器函数,如下所示:

func gen() chan Result {
  out := make(chan Result)

  go func() {
    ... 查询数据库
    for rows.Next() {
      out <- row
    }
    close(out)
  }()

  return out
}

然后我这样使用它:

c := gen()

...

// 做其他事情

我想要的是要么在out通道中返回计数,要么将所有内容都封装在一个结构体类型中并返回。

像这样:

c, length := gen()

或者:

a := gen()

fmt.Println(a.c)
fmt.Println(a.length)

我相信我已经尝试了除了使用原子操作之外的所有方法,使用原子操作似乎可以解决问题,但我查阅了一些资料,似乎不应该使用原子操作。除了得到0或无限阻塞的情况之外,我还有哪些其他选择呢?

谢谢!

英文:

I have been using Go for a little and still getting better everyday, but not an expert per se. Currently I am tackling concurrency and goroutines as I think that is the final unknown in my Go toolbelt. I think I am getting the hang of it as such, but still definitely a beginner.

The task I am having an issue with seems pretty basic to me, but nothing I have tried works. I would like to figure out a way to calculate the length of a channel.

From what I have gathered, len() only works on buffered channels so that won't help me in this case. What I am doing is reading values from the DB in batches. I have a generator func that goes like

func gen() chan Result {
  out := make(chan Result)

  go func() {
    ... query db
    for rows.Next() {
      out &lt;- row
    }
     close(out)
   }()

  return out
}

then I am using it as such

c := gen()

...

// do other stuff

I would either like to return the count with the out channel, or wrap all of it in a struct type and just return that.

like so:

c, len := gen()

or:

a := gen()

fmt.Println(a.c)
fmt.Println(a.len)

I believe I have tried all but using atomic, which I think would actually work but I read around and it apparently isn't the right thing to use atomic for. What other options do I have that either don't leave me with a 0 or blocks infinitely

Thanks!

答案1

得分: 2

len 内置函数可以返回通道的“长度”:

func len(v Type) int

len 内置函数根据其类型返回 v 的长度:

数组:v 中的元素数量。
数组指针:*v 中的元素数量(即使 v 是 nil)。
切片或映射:v 中的元素数量;如果 v 是 nil,则 len(v) 为零。
字符串:v 中的字节数量。
通道:通道缓冲区中排队(未读取)的元素数量;如果 v 是 nil,则 len(v) 为零。

但我认为这并不会对你有所帮助。

你真正需要的是一种新的方法来解决你的问题:通过计算通道中队列的项目数量,并不是处理“批处理”任务的合适方式。

你需要这个长度是用来做什么的?

英文:

The len built-in will return the "length" of a channel:

> ###func len(v Type) int
> The len built-in function returns the length of v, according to its type:

> Array: the number of elements in v.
> Pointer to array: the number of elements in *v (even if v is nil).
> Slice, or map: the number of elements in v; if v is nil, len(v) is zero.
> String: the number of bytes in v.
> Channel: the number of elements queued (unread) in the channel buffer;
if v is nil, len(v) is zero.

But I don't think that will help you.

What you really need is a new approach to your problem: counting the items in queue in a channel is not an appropriate way to handle "batches" of tasks.

What do you need this length for?

答案2

得分: 1

你正在使用缓冲通道。谢谢你的选择👍👏👌🎉
非缓冲通道不使用内存,因此永远不会为空!
非缓冲通道的唯一目的是通过在协程之间传递元素来实现同步。就是这样!

go func(){
    c := make(chan struct{})
    c <- struct{}{} // 肯定会被锁住
}()

另一个死锁的例子

go func(){
    c := make(chan struct{})
    <-c // 肯定会被锁住
    c <- struct{}{} // 永远不会执行到这里
}()

使用另一个协程来读取通道

go func(){
    c := make(chan struct{})
    go func(){ <-c }()
    c <- struct{}{}
}()

在你的情况下,你有一个生成器,这意味着你必须读取通道,直到生产者协程关闭它。这是一个很好的设计,可以确保你的协程不会悬空。

// 读取通道,直到注入器协程完成并关闭它。
for r := range gen() {
...
}
// gen() 内部的协程已经完成
英文:

You are using a not buffered channels. Thank you for that👍👏👌🙌
Unbuffered channel uses no memory. thus never contains nothing !
The only purpose of unbuffered channels are for achieving synchronicity between goroutine by passing an element from one to another. That's it !

    go func(){
        c:=make(chan struct{})
        c&lt;-struct{}{} // Definitely locked
    }()

another deadlock

    go func(){
        c:=make(chan struct{})
        &lt;-c // Definitely locked
        c&lt;-struct{}{} // Never get there
    }()

use another goroutine to read the channel

    go func(){
        c:=make(chan struct{})
        go func(){&lt;-c}()
        c&lt;-struct{}{}
    }()

In your case you have a generator, which means you have to read the channel until the producer goroutine will close it. It is a good design that ensures that your goroutine are not dangling.

// Read the channel until the injecter goroutine finishes and closes it.
for r := range gen() {
...
}
// goroutine inner of gen() as finished

答案3

得分: 0

我假设根据你后续的回答,你实际上想知道维持工作池和通道缓冲区在“最佳状态”下的“良好”值。

这非常困难,取决于工作人员在做什么,但是初步估计,我会考虑将缓冲通道的最小值设置为运行时的GOMAXPROCS(0),并设置一个工作池。如果你有很多资源,你甚至可以使用“无限”的工作人员。

英文:

I am assuming, from your follow on answers, you actually want to know "good" values for the workers pool and the buffer on the channel to keep everything working "optimally".

This is extremely hard, and depends on what the workers are doing, but at a first guess I'd look at a minimal value of buffered channel and a pool of workers at runtime.GOMAXPROCS(0). If you have a lot of resources then you could go as far as "infinite" workers.

huangapple
  • 本文由 发表于 2017年4月19日 04:42:04
  • 转载请务必保留本文链接:https://go.coder-hub.com/43481836.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定