为什么 ZeroMQ 上下文(Context)不能在所有 goroutine 之间共享?

huangapple go评论72阅读模式
英文:

Why a ZeroMQ Context is not being shared amongst all the goroutines?

问题

在这个ZeroMQ的例子中,每个goroutine都有自己的ZeroMQ Context。然而,在ZeroMQ指南中,它提到:

在进程开始时创建一个ZeroMQ上下文,并将其传递给您想要通过inproc套接字连接的所有线程。

不要在线程之间共享ZeroMQ套接字。ZeroMQ套接字不是线程安全的。从技术上讲,将套接字从一个线程迁移到另一个线程是可能的,但需要技巧。唯一一个在线程之间共享套接字可能合理的地方是需要在套接字上执行类似垃圾回收的语言绑定。

我知道goroutine不是线程。

相反,它们存在于线程中。但我也读到过在goroutine之间可以共享对象的可能性。

那么,为什么在goroutine之间不共享上下文呢?

我认为这样做会占用更少的空间,因为一个包含套接字的上下文可能会很大。

但即使它不会占用太多空间,为什么在这个例子中上下文根本不共享呢?

英文:

In this ZeroMQ example,

// Multithreaded Hello World server.
// Uses Goroutines.  We could also use channels (a native form of
// inproc), but I stuck to the example.
//
// Author:  Brendan Mc.
// Requires: http://github.com/alecthomas/gozmq

package main

import (
    "fmt"
    zmq "github.com/alecthomas/gozmq"
    "time"
)

func main() {
    // Launch pool of worker threads
    for i := 0; i != 5; i = i + 1 {
        go worker()
    }

    // Prepare our context and sockets
    context, _ := zmq.NewContext()
    defer context.Close()

    // Socket to talk to clients
    clients, _ := context.NewSocket(zmq.ROUTER)
    defer clients.Close()
    clients.Bind("tcp://*:5555")

    // Socket to talk to workers
    workers, _ := context.NewSocket(zmq.DEALER)
    defer workers.Close()
    workers.Bind("ipc://workers.ipc")

    // connect work threads to client threads via a queue
    zmq.Device(zmq.QUEUE, clients, workers)

}

func worker() {
    context, _ := zmq.NewContext()
    defer context.Close()

    // Socket to talk to dispatcher
    receiver, _ := context.NewSocket(zmq.REP)
    defer receiver.Close()
    receiver.Connect("ipc://workers.ipc")

    for true {
        received, _ := receiver.Recv(0)
        fmt.Printf("Received request [%s]\n", received)

        // Do some 'work'
        time.Sleep(time.Second)

        // Send reply back to client
        receiver.Send([]byte("World"), 0)
    }
}

each goroutine has its own ZeroMQ Context. However, in the ZeroMQ guide, it says the following:

> Create one ZeroMQ context at the start of your process, and pass that
> to all threads that you want to connect via inproc sockets.
>
> Don't share ZeroMQ sockets between threads. ZeroMQ sockets are not
> threadsafe. Technically it's possible to migrate a socket from one
> thread to another but it demands skill. The only place where it's
> remotely sane to share sockets between threads are in language
> bindings that need to do magic like garbage collection on sockets.

I know that goroutines are not threads.

Instead, they live in threads. But I've also read that it's possible to have a shared object between goroutines.

So, why doesn't the context is shared between goroutines?

I think it'd consume a lot less space, because a context with a socket in it might be kinda big.

But even if it's not that consuming, why at all the context is not shared in this example?

答案1

得分: 1

为什么goroutine之间不共享上下文?(在示例中)

除了在某些充分理由和场合下确实共享上下文实例的技术选项之外,所示的代码片段专注于“零共享”原则,这是ZeroMQ非常著名的几个零原则之一。

使用一组不相交的Context实例(在同一线程中可以有多个这样的实例)有一些优势,包括性能扩展:

  • 在广义上,将工作负载分离在不同的实例之间
  • 在较窄的意义上,直接的IO线程扩展(在Context实例化时定义不同数量的IO线程,有些只有1个IO线程,而其他可能有2个、3个或10个IO线程,用于高性能的数据传输引擎),并使用基于setsockopt(ZMQ_AFFINITY ...)的套接字到IO线程的位掩码映射,可以将一些传输任务直接分配给不同的、不相交的Context实例,并且可以更具体地处理不同的传输流量优先级模式,通过使用独立的、专门的Context实例组的IO线程(自然地,这些IO线程具有更多和不同数量的底层ZeroMQ Context的IO线程)。这是性能扩展和几乎确定性流量策略处理实现的宝贵特性。

这可能会激发您进一步思考如何利用这种细粒度管理来更好地利用ZeroMQ框架的服务,以及这种零共享概念如何为您打开性能扩展和吞吐量限制的新世界。

如果有疑问,只需注意源代码中的初始备注:

...
// 使用Goroutines。我们也可以使用通道(一种本地形式的inproc),但我坚持使用示例。
...

inproc://传输类是一个明显的示例,说明如何从非共享、非阻塞的Context实例中受益,并享受几乎线性的性能扩展,直到达到峰值性能。

英文:

Q: why isn't the context shared between goroutines?<sup> ( in the example )</sup>

Besides the <sub>( rather extremist )</sub> technical option to indeed share a Context instance ( under some well justified circumstances / occassions ), the demonstrated piece of code has focused on a Zero-Sharing, as a general principle, the one of a few Zero-Maxims, the ZeroMQ is so well known for.

There are advantages of a habit to employ a group of disjunct Context instances ( which one may also have several of these, within the same thread, on purpose ). The performance scaling:

  • in a wide meaning, the separation of workloads, among instances

  • in a narrower meaning, a direct IO-thread scaling ( defining different number of IO-threads upon the Context instantiation, some having just 1 IO-thread, whereas other may have 2, 3 or 10 IO-threads for a high-performance Data-pumping engine ) and using a bit-mask mapping via a setsockopt( ZMQ_AFFINITY ... )-based socket-to-IO-threads mapping, which allows to directly separate some transport-tasks onto different, discjunct Context-instances, and also to more specifically handle different transport-traffic priority-modes, right by using the separate, specialised groups of Context-instances' IO-threads ( naturally, where knowingly equipped with more and different amounts of underlying ZeroMQ Context's IO-threads ). This is an invaluable feature for performance scaling and almost-deterministic traffic-policy handling implementations.

This may inspire your further thought on how such fine-grained management ( granularity ) may help your distributed applications better harness the ZeroMQ framework's services and how this Zero-Sharing concept opens you the new worlds of performance scaling and throughput throttling.

If in doubts, just notice the inital remark in the source code:

...
// Uses Goroutines.  We could also use channels (a native form of
// inproc), but I stuck to the example.
...

The inproc:// transport class is the clear example on how to benefit from the non-shared, non-blocking Context-instance and enjoy the almost linear scale-up, up to the peak performance.

答案2

得分: 0

你绝对可以在 Goroutines 之间共享对象,只要这些对象是线程安全的。你正在使用的 Go ZeroMQ 绑定是明确不支持多线程的。虽然一个 Goroutine 可能不对应一个操作系统线程,但无法保证你创建的 Goroutines 会在同一个操作系统线程上运行。它们的生命周期由 Go 运行时动态管理。

你可以通过使用通道来同步访问,或者使用互斥锁来“共享”你的 ZeroMQ 套接字,其中在 Goroutine 访问套接字时锁定对其的访问。

英文:

You can absolutely share objects between Goroutines, as long as those objects are threadsafe. The Go ZeroMQ bindings you are using is explicitly non-threadsafe. While a single Goroutine may not correspond to a single OS thread, it is not possible to guarantee that the Goroutines you spawn will live on the same OS thread. Their life cycle is managed dynamically by the Go run-time.

You can "share" your ZeroMQ socket between multiple Goroutines by using channels to synchronize access, or by using mutexes, where you lock access to your socket while a Goroutine is accessing it.

huangapple
  • 本文由 发表于 2017年2月4日 07:30:12
  • 转载请务必保留本文链接:https://go.coder-hub.com/42034641.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定