长时间运行的进程中是否应该保留空闲线程?

huangapple go评论86阅读模式
英文:

Should idle threads be left around in long running process?

问题

我正在创建一个长期运行并监听工作的Go程序。当它收到请求时,它会在一个进程队列上运行工作。

我对Go语言和系统编程都不太熟悉,所以我的问题是:我应该在程序启动时启动进程队列(包括多个空闲的工作线程),让它们一直保持运行状态直到有工作到来,还是应该在工作到达时启动它们,并在完成后关闭它们?

我不清楚多个空闲线程对整个系统的影响,但我假设它们处于空闲状态时不会有任何影响,直到有工作到来。也就是说,我希望我的程序能够成为一个“好邻居”,尽可能高效。

--编辑--

为了澄清,这个“进程池”是一组等待在通道上工作的工作goroutine。它们应该在工作到达时启动/停止,还是在程序启动时启动并一直等待工作到来?

英文:

I am creating a go program that is intended to run long term and listen for work. When it receives a request, it runs the work on a process queue.

I am new to golang and systems programming, so my question is this: should I spin up the process queue (with it's multiple idle worker threads) at the program launch (they will just sit there until work comes in) or should I spin them up when work arrives and shut them down when finished?

I am unclear as to the overall system impact multiple idle threads will have, but I am assuming since they are idle there will be no impact until work arrives. That being said, I want to make sure my program is a "good neighbor" and as efficient as possible.

--EDIT--

To clarify, the "process pool" is a group of worker go routines waiting for work on a channel. Should they be started/stopped when work arrives, or started when the program launches and left waiting until work comes in?

答案1

得分: 1

首先,你不能使用标准的Go库来创建线程。在Go语言中,你应该使用所谓的"绿色线程",也就是goroutines。

通常情况下,你不应该创建"可重用"的goroutines。它们的创建成本很低,所以应该在需要时创建它们,并在工作完成后(从goroutine返回)立即结束。

此外,不要犹豫地创建嵌套的goroutines。一般来说,如果你觉得需要以并发的方式执行某些操作,就大胆地创建它们,不要试图重用它们,因为这没有意义。

英文:

First of all you can't create a thread using standard Go library. In Go universe you should use goroutines which are so called green threads.

Usually you shouldn't spawn "reusable" goroutines. They are cheap to create so create them on demand as work job arrives and finish (return from goroutine) as soon as work is completed.

Also don't hesitate to create nested goroutines. In general spawn them like crazy if you feel you should do something in concurrent manner and don't try to reuse them as it makes no sense.

答案2

得分: 1

无论哪种方式,成本都非常低。goroutine不需要单独的操作系统线程,在等待通道接收时几乎不消耗资源,但是启动它们的成本也非常低,所以没有太大的理由让它们保持开启状态。

我的代码很少使用工作池。通常,我的生产者会为每个工作单元生成一个goroutine,并直接将其与响应通道一起传递,然后生成一个“监听器”,对工作输出进行一些格式化,并将所有响应传回主线程。我经常使用的一种常见模式如下:

func Foo(input []interface{}) resp chan interface{} {
    var wg sync.WaitGroup
    resp := make(chan interface{})
    listen := make(chan interface{})
    theWork := makeWork(input)
    // 进行工作
    for _, unitOfWork := range theWork {
        wg.Add(1)
        go func() {
            // doWork的签名为:
            //   func doWork(w interface{}, ch chan interface{})
            doWork(unitOfWork, listen)
            wg.Done()
        }()
    }
    // 格式化listen通道的输出并发送到resp通道
    // 然后关闭resp通道以便主线程继续执行
    go func() {
        for r := range listen {
            resp <- doFormatting(r)
        }
        close(resp)
    }()
    // 在工作完成后关闭listen通道
    go func() {
        wg.Wait()
        close(listen)
    }()
    return resp
}

然后,我的主函数传递一些输入并监听响应通道:

func main() {
    loremipsum := []string{"foo", "bar", "spam", "eggs"}
    response := Foo(loremipsum)
    for output := range response {
        fmt.Println(output)
    }
}
英文:

There is very little cost either way. goroutines don't require a separate OS thread and consume practically no resources while blocking on a channel receive, but also cost very little to spin up, so there's no great reason to leave them open either.

My code rarely uses worker pools. Generally my producer will spawn a goroutine for every unit of work it produces and hands it off directly along with a response channel, then spawns a "listener" that does some formatting for the work output and pipes all the responses back to the main thread. A common pattern for me looks like:

func Foo(input []interface{}) resp chan interface{} {
    var wg sync.WaitGroup
    resp := make(chan interface{})
    listen := make(chan interface{})
    theWork := makeWork(input)
    // do work
    for _, unitOfWork := range theWork {
        wg.Add(1)
        go func() {
            // doWork has signature:
            //   func doWork(w interface{}, ch chan interface{})
            doWork(unitOfWork, listen)
            wg.Done()
        }()
    }
    // format the output of listen chan and send to resp chan
    // then close resp chan so main can continue
    go func() {
        for r := range listen {
            resp &lt;- doFormatting(r)
        }
        close(resp)
    }()
    // close listen chan after work is done
    go func() {
        wg.Wait()
        close(listen)
    }()
    return resp
}

Then my main function passes it some input and listens on the response channel

func main() {
    loremipsum := []string{&quot;foo&quot;, &quot;bar&quot;, &quot;spam&quot;, &quot;eggs&quot;}
    response := Foo(loremipsum)
    for output := range response {
        fmt.Println(output)
    }
}

答案3

得分: 0

在Go语言中,使用任务队列和等待工作线程的模式是很常见的。Goroutines是轻量级的,但执行顺序是不确定的。因此,如果你希望系统的行为是可预测的,最好通过在循环中请求无缓冲通道来控制工作线程与主例程的会合,或者以其他某种方式进行控制。否则,其中一些工作线程可能会被创建但保持空闲,这是合法的。

英文:

Pattern with tasks queue and waiting workers is common in Go. Goroutines are cheap, but order of execution is nondetermined. So if you want your system behavior to be predictable, you better would control workers rendezvous with main routine thru unbuffered channels requested in a loop or somehow else. Otherwise some of them can be spawned but remain idle which is legal.

huangapple
  • 本文由 发表于 2016年1月6日 00:10:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/34616169.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定