nsqjs客户端无法立即从go-nsq服务器端接收消息。

huangapple go评论85阅读模式
英文:

nsqjs clients, not receiving messages immediately from go-nsq server side

问题

尝试学习NSQ,并按照这里的Go语言示例(http://tleyden.github.io/blog/2014/11/12/an-example-of-using-nsq-from-go/)和这里的nsqjs示例(https://github.com/dudleycarr/nsqjs)进行操作。我在服务器端使用for循环和go协程发送消息。

var wg sync.WaitGroup
for i := 0; i < 100; i++ {
    wg.Add(1)
    go func(x int) {
        defer wg.Done()

        chanName := fmt.Sprintf("import_progress_587e6442ff74889098498f6e")
        m := map[string]interface{}{
            "body": map[string]interface{}{
                "progress": x,
            },
        }
        msg, _ := json.Marshal(m)

        req := NSQPubReq{
            Topic: chanName,
            Body:  msg,
        }
        if err := producer.Publish(req.Topic, req.Body); err != nil {
        }
        utils.Info(fmt.Sprintf("sent msg=%v", string(msg)))

    }(i)
}

wg.Wait()

但问题是在客户端。

// channel = 'import_progress_587e6442ff74889098498f6e'
let reader = new nsq.Reader(channel, channel, {
    //lookupdHTTPAddresses: '<<IP>>:4161',
    maxInFlight: 10000,
    snappy: true
})
reader.connect()

reader.on('message', (msg) => {
    var msgData = {
        id:     msg.id,
        body:   msg.body.toString(),
        chan:   channel
    }
    io.emit(channel, msgData)
    msg.finish()
})

消息不会立即传递到客户端。我需要等待几秒钟,直到消息传递到Node.js客户端。是否有任何需要进行的设置?谢谢!

英文:

trying to learn nsq, and following the examples from here golang example and here nsqjs. I am sending messages in server side doing w/ a for loop and go routines

var wg sync.WaitGroup
for i := 0; i &lt; 100; i++ {
	wg.Add(1)
	go func(x int) {
		defer wg.Done()

		chanName := fmt.Sprintf(&quot;import_progress_587e6442ff74889098498f6e&quot;)
		m := map[string]interface{}{
			&quot;body&quot;: map[string]interface{}{
				&quot;progress&quot;: x,
			},
		}
		msg, _ := json.Marshal(m)

		req := NSQPubReq{
			Topic: chanName,
			Body:  msg,
		}
		if err := producer.Publish(req.Topic, req.Body); err != nil {
		}
		utils.Info(fmt.Sprintf(&quot;sent msg=%v&quot;, string(msg)))

	}(i)
}

wg.Wait()

but the problem is, on the clientside.

// channel = &#39;import_progress_587e6442ff74889098498f6e&#39;
let reader = new nsq.Reader(channel, channel, {
    //lookupdHTTPAddresses: &#39;&lt;&lt;IP&gt;&gt;:4161&#39;,
    maxInFlight: 10000,
    snappy: true
})
reader.connect()

reader.on(&#39;message&#39;, (msg) =&gt; {
    var msgData = {
            id:     msg.id,
            body:   msg.body.toString(),
            chan:   channel
    }
    io.emit(channel, msgData)
    msg.finish()
})

the message don't come up immediately to the client. i will wait for a couple of seconds until the message come to the nodejs client. is there any settings that I need to do? thank you!

答案1

得分: 4

有几个原因可能导致nsqjs客户端在接收刚发布的消息时变慢:

  1. 如果主题是新的,并且通过nsqlookupd进行主题发现,那么默认情况下,nsqjs Reader将每30秒尝试发现新主题。

    从上面的示例中看,似乎你正在为每个导入创建新的主题。我相信如果你先从Golang客户端开始发布消息,然后再启动nsqjs客户端,你就不会看到延迟。

  2. 如果你有多个nsqd,并且max-in-flight设置得太低,那么它会将nsqjs Reader置于饥饿模式,它会在一段时间内在nsqd之间移动RDY计数。

    我不确定这是否是问题的原因,因为我无法了解nsq的拓扑结构。只要你的max-in-flight设置高于你拥有的nsqd实例数量,那么就没问题。

英文:

There are a couple of reasons why a nsqjs client will be slow to receive a message just published:

  1. If the topic is new and the discovery of topics is via nsqlookupd, then by default, the nsqjs Reader will attempt to discover new topics every 30 seconds.

    From the example above, it looks like you are creating new topics for every import. I believe that if you start publishing messages first from the Golang client and then start the nsqjs client, then you shouldn't see the delay.

  2. If you have multiple nsqds with a max-in-flight set too low, then it puts the nsqjs Reader in a starvation mode where it moves the RDY count between nsqds for a set period of time.

    I'm not sure that's what's going on here since I can't tell anything about the nsq topology. As long as your max-in-flight is set higher than the number of nsqd instances that you have, then you'll be in good shape.

huangapple
  • 本文由 发表于 2017年1月18日 03:03:43
  • 转载请务必保留本文链接:https://go.coder-hub.com/41704986.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定