英文:
Golang ZeroMQ: REQ/REP senseless non-blocking
问题
在Python中,ZeroMQ的.recv()/.send()
操作是阻塞的,这对于REQ/REP
非常完美。在Golang中,我必须在.recv()
和.send()
操作中传递一个zmq.DONTWAIT
参数才能使其正常工作。
但问题是,流程需要保持同步,所以:
server.recv()
client.send()
client.recv()
server.send()
在第3步和第4步之间,由于它们是异步的,会出现一些奇怪的情况。
当客户端发送了一条消息,而服务器尚未接收到它,但客户端尝试接收响应时,同步就不再是同步了。
是否有一种与zmq.DONTWAIT
相反的zmq.DOBLOCK
选项?
或者我在这里理解错了什么?
编辑:
我正在使用C语言的这个ZeroMQ绑定:https://godoc.org/github.com/pebbe/zmq4#Type
如您在第二个参考链接中所见,.recv()
需要一个输入标志(flag),它是第二个参考链接中的其中一个:
Recv: https://godoc.org/github.com/pebbe/zmq4#Socket.Recv
要传递的标志:https://github.com/pebbe/zmq4/blob/master/zmq4.go#L403
这是我目前使用的代码,用于实现一种有些丑陋的解决方法:
package connection
import (
"zmq4"
"fmt"
"time"
)
const ERRTMPUNAV="resource temporarily unavailable"
func checkError(e error){
if e != nil {
panic(e)
}
}
func CreateRepNode(address string,onMessage chan<- string,send <-chan string,closeConn <-chan bool){
stop:=false
socket,err:=zmq4.NewSocket(zmq4.REP)
checkError(err)
err=socket.Bind(address)
checkError(err)
go func(socket *zmq4.Socket){
for {
msg,err:=socket.Recv(zmq4.DONTWAIT)
fmt.Println("server message"+msg)
if stop==true {
return
}
if err != nil {
rateLimit := time.Tick(100 * time.Millisecond)
<-rateLimit
continue
}
checkError(err)
onMessage<-msg
rep:=<-send
_,err=socket.Send(rep,zmq4.DONTWAIT)
}
}(socket)
<-closeConn
stop=true
}
func CreateReqNode(address string,onMessage chan<- string,send <-chan string,closeConn <-chan bool){
stop:=false
socket,err:=zmq4.NewSocket(zmq4.REQ)
checkError(err)
err=socket.Connect(address)
checkError(err)
go func(){
for {
msg:=<-send
if stop==true {
return
}
_,err:=socket.Send(msg,zmq4.DONTWAIT)
for {
msg,err=socket.Recv(zmq4.DONTWAIT)
fmt.Println("client got message "+msg)
if err!=nil {
if err.Error()==ERRTMPUNAV {
w:=time.Tick(100*time.Millisecond)
<-w
continue
}
}
break
}
onMessage<-msg
}
}()
<-closeConn
stop=true
}
英文:
On Python, the ZeroMQ .recv()/.send()
operations are blocking, which is just perfect for REQ/REP
.<br>In Golang, I must pass a zmq.DONTWAIT
to the .recv()
and .send()
operation in order to make it work.
But the thing is, the flow needs to be lock step, so:
server.recv()
client.send()
client.recv()
server.send()
And between 3 and 4 the weirdness starts, because they are async.
When the client has sent a message and the server has not received it yet but client tries to receive a response, the lock step is no lock step any more.
Is there some kind of zmq.DOBLOCK
in contrast to zmq.DONTWAIT
?
Or did I get something wrong here?
EDIT:
I am using this go binding in C for zeromq: https://godoc.org/github.com/pebbe/zmq4#Type
As you can see here the .recv()
needs a input flag
, which is one of the both on the second ref:
Recv: https://godoc.org/github.com/pebbe/zmq4#Socket.Recv
Flags to be passed: https://github.com/pebbe/zmq4/blob/master/zmq4.go#L403
This is the current code I got to make a workaround which feels somewhat ugly:
package connection
import (
"zmq4"
"fmt"
"time"
)
const ERRTMPUNAV="resource temporarily unavailable"
func checkError(e error){
if e != nil {
panic(e)
}
}
func CreateRepNode(address string,onMessage chan<- string,send <-chan string,closeConn <-chan bool){
stop:=false
socket,err:=zmq4.NewSocket(zmq4.REP)
checkError(err)
err=socket.Bind(address)
checkError(err)
go func(socket *zmq4.Socket){
for {
msg,err:=socket.Recv(zmq4.DONTWAIT)
fmt.Println("server message"+msg)
if stop==true {
return
}
if err != nil {
rateLimit := time.Tick(100 * time.Millisecond)
<-rateLimit
continue
}
checkError(err)
onMessage<-msg
rep:=<-send
_,err=socket.Send(rep,zmq4.DONTWAIT)
}
}(socket)
<-closeConn
stop=true
}
func CreateReqNode(address string,onMessage chan<- string,send <-chan string,closeConn <-chan bool){
stop:=false
socket,err:=zmq4.NewSocket(zmq4.REQ)
checkError(err)
err=socket.Connect(address)
checkError(err)
go func(){
for {
msg:=<-send
if stop==true {
return
}
_,err:=socket.Send(msg,zmq4.DONTWAIT)
for {
msg,err=socket.Recv(zmq4.DONTWAIT)
fmt.Println("client got message "+msg)
if err!=nil {
if err.Error()==ERRTMPUNAV {
w:=time.Tick(100*time.Millisecond)
<-w
continue
}
}
break
}
onMessage<-msg
}
}()
<-closeConn
stop=true
}
答案1
得分: 3
ZeroMQ的基本原型更像是一组构建模块,而不是针对任何需求的生产级解决方案。
Go语言是一种非常强大、现代化的语言,具有协程和其他智能工具,用于控制并发。因此,请原谅我提出以下建议:
-
尽量避免使用阻塞设计(非阻塞设计可以让您完全控制所有事物的到来,而不会在任何无限/不可控的等待循环中“挂起”,这在已经开发的死锁中尤为糟糕)。
-
避免依赖于仅具有单一、基本类型的形式通信模式的源代码示例,而应为可能出现问题的所有情况开发强大的生存能力处理策略(例如,在传输网络中丢失信号、消息丢失、资源超载等)。
重新设计提示-根本不要使用REQ/REP
。是的,永远不要使用...
ZeroMQ可扩展的形式通信模式REQ/REP
对于学习ZeroMQ来说是可以的,但在真正的生产级部署中是致命的。有关详细信息,请阅读这里。
接下来考虑内部无条件的模式,例如PAIR
(虽然标记为实验性的,但对于某些用例非常有效),XREQ/XREP
,PUSH/PULL
或一些复合的信令/传输多套接字自定义设计模式。
最好的下一步是什么?
我现在可以为您进一步的问题提供帮助,让您对这个主题有一个更全面的了解,其中包括更多的论点、一个简单的信令平面/消息平面示意图,以及指向Pieter HINTJENS的一本必读书籍的直接链接。
这本书值得花时间和精力阅读。如果您对分布式系统设计非常认真,您将喜欢它,以及Pieter对零共享、零阻塞(几乎)零拷贝等方面的热情。
英文:
ZeroMQ trivial elementary archetypes are more a set of building blocks than a production-grade solution to any need.
Go-lang is a very powerful, modern language with coroutines and other smart tools for a controlled concurrency, so forgive me to state the following list of recommendations:
-
avoid blocking designs wherever one can ( a non-blocking design leaves one in full control of all things as they come ... not "hanging" in any infinite/uncontrollable waiting loop, the worse in an already developed deadlock )
-
avoid relying on a SLOC examples with a single, elementary type of a Formal Communication Pattern, one shall rather develop a robust survivability-handler strategy for all the cases where something may go wrong ( Loss-of-Signal in transport network, Loss-of-Message, DDoS-level of resources overloads, ... )
Redesign hint - do not use REQ/REP
at all. Yes, never...
ZeroMQ Scaleable Formal Communication Pattern REQ/REP
is fine for learning ZeroMQ, but is lethal in real production grade deployment. For details, read here.
Next think about internally unconditional patterns, alike PAIR
( though marked experimental, for some use-cases it works great ), XREQ/XREP
, PUSH/PULL
or some composite signalling/transport multi-socket custom-designed own pattern.
The best next step?
What I can do for your further questions right now is to direct you to see a bigger picture on this subject with more arguments, a simple signalling-plane / messaging-plane illustration and a direct link to a must-read book from Pieter HINTJENS.
The book is worth one's time and efforts. If one is serious into distributed systems design, you will love it altogether with Pieter's passion for Zero-sharing, Zero-blocking, (almost) Zero-copy et al.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论