英文:
nsqjs clients, not receiving messages immediately from go-nsq server side
问题
尝试学习NSQ,并按照这里的Go语言示例(http://tleyden.github.io/blog/2014/11/12/an-example-of-using-nsq-from-go/)和这里的nsqjs示例(https://github.com/dudleycarr/nsqjs)进行操作。我在服务器端使用for循环和go协程发送消息。
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(x int) {
defer wg.Done()
chanName := fmt.Sprintf("import_progress_587e6442ff74889098498f6e")
m := map[string]interface{}{
"body": map[string]interface{}{
"progress": x,
},
}
msg, _ := json.Marshal(m)
req := NSQPubReq{
Topic: chanName,
Body: msg,
}
if err := producer.Publish(req.Topic, req.Body); err != nil {
}
utils.Info(fmt.Sprintf("sent msg=%v", string(msg)))
}(i)
}
wg.Wait()
但问题是在客户端。
// channel = 'import_progress_587e6442ff74889098498f6e'
let reader = new nsq.Reader(channel, channel, {
//lookupdHTTPAddresses: '<<IP>>:4161',
maxInFlight: 10000,
snappy: true
})
reader.connect()
reader.on('message', (msg) => {
var msgData = {
id: msg.id,
body: msg.body.toString(),
chan: channel
}
io.emit(channel, msgData)
msg.finish()
})
消息不会立即传递到客户端。我需要等待几秒钟,直到消息传递到Node.js客户端。是否有任何需要进行的设置?谢谢!
英文:
trying to learn nsq, and following the examples from here golang example and here nsqjs. I am sending messages in server side doing w/ a for loop and go routines
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(x int) {
defer wg.Done()
chanName := fmt.Sprintf("import_progress_587e6442ff74889098498f6e")
m := map[string]interface{}{
"body": map[string]interface{}{
"progress": x,
},
}
msg, _ := json.Marshal(m)
req := NSQPubReq{
Topic: chanName,
Body: msg,
}
if err := producer.Publish(req.Topic, req.Body); err != nil {
}
utils.Info(fmt.Sprintf("sent msg=%v", string(msg)))
}(i)
}
wg.Wait()
but the problem is, on the clientside.
// channel = 'import_progress_587e6442ff74889098498f6e'
let reader = new nsq.Reader(channel, channel, {
//lookupdHTTPAddresses: '<<IP>>:4161',
maxInFlight: 10000,
snappy: true
})
reader.connect()
reader.on('message', (msg) => {
var msgData = {
id: msg.id,
body: msg.body.toString(),
chan: channel
}
io.emit(channel, msgData)
msg.finish()
})
the message don't come up immediately to the client. i will wait for a couple of seconds until the message come to the nodejs client. is there any settings that I need to do? thank you!
答案1
得分: 4
有几个原因可能导致nsqjs客户端在接收刚发布的消息时变慢:
-
如果主题是新的,并且通过
nsqlookupd进行主题发现,那么默认情况下,nsqjs Reader将每30秒尝试发现新主题。从上面的示例中看,似乎你正在为每个导入创建新的主题。我相信如果你先从Golang客户端开始发布消息,然后再启动nsqjs客户端,你就不会看到延迟。
-
如果你有多个nsqd,并且
max-in-flight设置得太低,那么它会将nsqjs Reader置于饥饿模式,它会在一段时间内在nsqd之间移动RDY计数。我不确定这是否是问题的原因,因为我无法了解nsq的拓扑结构。只要你的max-in-flight设置高于你拥有的
nsqd实例数量,那么就没问题。
英文:
There are a couple of reasons why a nsqjs client will be slow to receive a message just published:
-
If the topic is new and the discovery of topics is via
nsqlookupd, then by default, the nsqjs Reader will attempt to discover new topics every 30 seconds.From the example above, it looks like you are creating new topics for every import. I believe that if you start publishing messages first from the Golang client and then start the nsqjs client, then you shouldn't see the delay.
-
If you have multiple nsqds with a
max-in-flightset too low, then it puts the nsqjsReaderin a starvation mode where it moves theRDYcount between nsqds for a set period of time.I'm not sure that's what's going on here since I can't tell anything about the nsq topology. As long as your max-in-flight is set higher than the number of
nsqdinstances that you have, then you'll be in good shape.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。


评论