在Go中实现并发的生产者和消费者模式

huangapple go评论71阅读模式
英文:

Concurrent Producer and Consumer in Go

问题

我想在Go语言中创建一个生产者/消费者与管理器的程序。例如:我有5个生产者、5个消费者和一个管理器。生产者有自己的本地数组,它们遍历数组并将元素发送给管理器。消费者有自己的本地数组,其中包含要消费的元素信息;它们也将这些元素发送给管理器。管理器有自己的数组,用于存储元素及其数量(例如,如果生产者发送了1 1 2 3 1 2 0这些元素,管理器的数组将变为1 3 2 1,表示有一个0,三个1,两个2和一个3),管理器处理生产者和消费者的请求,将元素放入数组(生产)或从数组中移除(消费)。

在Go语言中是否可以实现这样的程序?我已经在JAVA + CSP中使用通道发送信息,并在管理器中使用保护条件来确定当生产者和消费者尝试处理相同元素时应该先执行哪个过程(例如,生产者想要将1添加到管理器数组,同时消费者想要消费1)。

欢迎提供任何示例或建议,因为我找不到关于我想做的事情的任何信息。如果需要,我可以提供我的JAVA+CSP代码。

更新。 如何进行同步(避免从空数组中取出元素)?例如,如果消费者想要从管理器数组中消费一个尚不存在的元素(例如,消费者想要消费'3',但管理器没有任何'3'),但生产者有这个元素,并且它将在几次迭代后生成-我如何让消费者一遍又一遍地检查管理器数组,直到生产者的工作完成?我是否需要为消费者元素创建结构体(或类),并标记它们是否已使用,或者Go语言有特定的方法来实现这一点?

英文:

I want to create a producer/consumer with manager program in Go. For example: I have a 5 producers, 5 consumers and manager. Producers have their own local arrays, they iterate over them and send the elements to the manager. Consumers have their own local arrays with info that elements consume; they send them to the manager too. The Manager has it own array, where it stores what and how many elements there are (for example - if the producer sends 1 1 2 3 1 2 0 elements, the manager array looks like 1 3 2 1 (one 0, three 1, two 2 and one 3 ) and it handles producers' and consumers' requests - placing an element into the array (produce) or removing it (consume).

Is it possible to make a program like this in Go? I already did this in JAVA + CSP with channels to send info and guards in the manager to determine which procedure should be done first when producer and consumer try to process the same element (for example, a producer wants to add 1 to the manager array and at the same time a consumer wants to consume 1).

Any examples or advice are welcome, because I don't find any info about what I want to do. If needed I can give my JAVA+CSP code.

UPDATE. How about synchronization (not to take from empty array)? For example - if consumer wants to consume element from manager array that does not exist yet (for example consumer wants to consume '3', but manager don't have any of thems) but producer has this element and it will be produced after few iterations - how can I make consumers to check manager array again and again until producers work is finished? Should I need to create structs (or classes) for consumers elements and mark that they are used or not, or Go has specific methods to do this?

答案1

得分: 2

我做了一个非常类似于你尝试做的例子,可以查看这个gist链接

我实现的方式是使用一个通道作为处理消费者,另一个通道作为产生项目的两个进程,for循环控制着从生产者到消费者的推送,当生产者不再推送任何内容时,循环将进入default状态。通过通道传递的对象是切片,进程会产生和消费每个切片中的标题。我相信你可以调整这段代码以适应你的例子。

英文:

I did a very similar example to what you're trying to do, check this gist github gist

The way I implemented that is using a single channel for my process consumer and another for my 2x processes that produce items, the for block controls the push from producers to the consumer and whenever the producers arent pushing anything the loop will default. The objects moved through the channels are slices, processes would produce and consume the headlines in each slice sent through the channel. I believe you can adjust this code to fit your example.

答案2

得分: 2

这是一个完整的示例,包括通道清理。在所有消费者和生产者完成后,Manager打印出结果(我使用了一个映射而不是切片,因为我认为这样的代码稍微容易一些)。

package main

import "fmt"

// Consume处理ns中的数字,在处理完后将它们发送到ch通道上。当例程完成时,在done通道上发出信号。
func Consume(done chan bool, ch chan int, ns []int) {
    for _, num := range ns {
        ch <- num
    }
    done <- true
}

// Produce在ns中“创建”数字,在生成后将它们发送到ch通道上。当例程完成时,在done通道上发出信号。
func Produce(done chan bool, ch chan int, ns []int) {
    for _, num := range ns {
        ch <- num
    }
    done <- true
}

// Manage为给定的整数切片创建消费者和生产者。一旦所有消费者和生产者都完成,它就会返回。
func Manage(cons, pros [][]int) {
    cch := make(chan int)
    pch := make(chan int)
    dch := make(chan bool)
    n := len(cons) + len(pros)
    data := make(map[int]int)
    for _, c := range cons {
        go Consume(dch, cch, c)
    }
    for _, p := range pros {
        go Produce(dch, pch, p)
    }
    for n > 0 {
        select {
        case c := <-cch:
            data[c] -= 1
        case c := <-pch:
            data[c] += 1
        case <-dch:
            n -= 1
        }
    }
    close(cch)
    close(pch)
    close(dch)
    fmt.Println(data)
}

func main() {
    cons := [][]int{{1, 3, 5}, {0, 1, 5}}
    pros := [][]int{{0, 1, 1}, {3, 5, 5, 7}}
    Manage(cons, pros)
}

希望对你有帮助!

英文:

Here's a full example, including channel cleanup. After all the consumers and producers are finished, the Manager prints out the result (for which I've used a map rather than a slice, since I think it makes the code a tad easier).

package main
import &quot;fmt&quot;
// Consume processes the numbers in ns, sending them on ch after they&#39;re
// processed. When the routine is finished, it signals that on done.
func Consume(done chan bool, ch chan int, ns []int) {
for i := range ns {
ch &lt;- i
}
done &lt;- true
}
// Produce &quot;creates&quot; the numbers in ns, sending them on ch after they&#39;re
// produced. When the routine is finished, it signals that on done.
func Produce(done chan bool, ch chan int, ns []int) {
for i := range ns {
ch &lt;- i
}
done &lt;- true
}
// Manage creates consumers and producers for the given int slices.
// It returns once all consumers and producers are finished.
func Manage(cons, pros [][]int) {
cch := make(chan int)
pch := make(chan int)
dch := make(chan bool)
n := len(cons) + len(pros)
data := make(map[int]int)
for _, c := range cons {
go Consume(dch, cch, c)
}
for _, p := range pros {
go Produce(dch, pch, p)
}
for n &gt; 0 {
select {
case c := &lt;-cch:
data[c] -= 1
case c := &lt;-pch:
data[c] += 1
case &lt;-dch:
n -= 1
}
}
close(cch)
close(pch)
close(dch)
fmt.Println(data)
}
func main() {
cons := [][]int{{1, 3, 5}, {0, 1, 5}}
pros := [][]int{{0, 1, 1}, {3, 5, 5, 7}}
Manage(cons, pros)
}

huangapple
  • 本文由 发表于 2013年12月23日 21:31:15
  • 转载请务必保留本文链接:https://go.coder-hub.com/20744619.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定