英文:
nsqjs clients, not receiving messages immediately from go-nsq server side
问题
尝试学习NSQ,并按照这里的Go语言示例(http://tleyden.github.io/blog/2014/11/12/an-example-of-using-nsq-from-go/)和这里的nsqjs示例(https://github.com/dudleycarr/nsqjs)进行操作。我在服务器端使用for循环和go协程发送消息。
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(x int) {
defer wg.Done()
chanName := fmt.Sprintf("import_progress_587e6442ff74889098498f6e")
m := map[string]interface{}{
"body": map[string]interface{}{
"progress": x,
},
}
msg, _ := json.Marshal(m)
req := NSQPubReq{
Topic: chanName,
Body: msg,
}
if err := producer.Publish(req.Topic, req.Body); err != nil {
}
utils.Info(fmt.Sprintf("sent msg=%v", string(msg)))
}(i)
}
wg.Wait()
但问题是在客户端。
// channel = 'import_progress_587e6442ff74889098498f6e'
let reader = new nsq.Reader(channel, channel, {
//lookupdHTTPAddresses: '<<IP>>:4161',
maxInFlight: 10000,
snappy: true
})
reader.connect()
reader.on('message', (msg) => {
var msgData = {
id: msg.id,
body: msg.body.toString(),
chan: channel
}
io.emit(channel, msgData)
msg.finish()
})
消息不会立即传递到客户端。我需要等待几秒钟,直到消息传递到Node.js客户端。是否有任何需要进行的设置?谢谢!
英文:
trying to learn nsq, and following the examples from here golang example and here nsqjs. I am sending messages in server side doing w/ a for loop and go routines
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(x int) {
defer wg.Done()
chanName := fmt.Sprintf("import_progress_587e6442ff74889098498f6e")
m := map[string]interface{}{
"body": map[string]interface{}{
"progress": x,
},
}
msg, _ := json.Marshal(m)
req := NSQPubReq{
Topic: chanName,
Body: msg,
}
if err := producer.Publish(req.Topic, req.Body); err != nil {
}
utils.Info(fmt.Sprintf("sent msg=%v", string(msg)))
}(i)
}
wg.Wait()
but the problem is, on the clientside.
// channel = 'import_progress_587e6442ff74889098498f6e'
let reader = new nsq.Reader(channel, channel, {
//lookupdHTTPAddresses: '<<IP>>:4161',
maxInFlight: 10000,
snappy: true
})
reader.connect()
reader.on('message', (msg) => {
var msgData = {
id: msg.id,
body: msg.body.toString(),
chan: channel
}
io.emit(channel, msgData)
msg.finish()
})
the message don't come up immediately to the client. i will wait for a couple of seconds until the message come to the nodejs client. is there any settings that I need to do? thank you!
答案1
得分: 4
有几个原因可能导致nsqjs客户端在接收刚发布的消息时变慢:
-
如果主题是新的,并且通过
nsqlookupd
进行主题发现,那么默认情况下,nsqjs Reader将每30秒尝试发现新主题。从上面的示例中看,似乎你正在为每个导入创建新的主题。我相信如果你先从Golang客户端开始发布消息,然后再启动nsqjs客户端,你就不会看到延迟。
-
如果你有多个nsqd,并且
max-in-flight
设置得太低,那么它会将nsqjs Reader置于饥饿模式,它会在一段时间内在nsqd之间移动RDY
计数。我不确定这是否是问题的原因,因为我无法了解nsq的拓扑结构。只要你的max-in-flight设置高于你拥有的
nsqd
实例数量,那么就没问题。
英文:
There are a couple of reasons why a nsqjs client will be slow to receive a message just published:
-
If the topic is new and the discovery of topics is via
nsqlookupd
, then by default, the nsqjs Reader will attempt to discover new topics every 30 seconds.From the example above, it looks like you are creating new topics for every import. I believe that if you start publishing messages first from the Golang client and then start the nsqjs client, then you shouldn't see the delay.
-
If you have multiple nsqds with a
max-in-flight
set too low, then it puts the nsqjsReader
in a starvation mode where it moves theRDY
count between nsqds for a set period of time.I'm not sure that's what's going on here since I can't tell anything about the nsq topology. As long as your max-in-flight is set higher than the number of
nsqd
instances that you have, then you'll be in good shape.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论