英文:
Getting too many channels error for a period of time
问题
I have 16 queues
& multiple consumer servers
for those queues. I have created
one dedicated
channel for each queue
to consume
messages. Consumer
& dispatch
channels
on each server share same connection
.
When I dispatch messages
to each queue, I do the following:
- create a new channel
- bind channel to the queue with proper routing
- dispatch the message
- close the channel
I have lots of incoming webhooks from Shopify & these webooks contents are dispatched to specific queues.
While processing each message, I need make an API call to Shopify. Shopify API has rate limit. If I hit rate limit once, I redispatch all messages from the consumer back to rabbitmq with a delay header of 1 minute (time required to clear the API rate limit). Now, when I have several consumers running with lots of messages in the queue & I re-dispatch those messages, I get
too many channels
error for a period of time. How can I avoid this error?
I tried to keep 2 dedicated channels per queue:
- for consumer purpose only
- for dispatch purpose only
For 16 queues, & around 11 consumer servers. This way, I always have to keep 352
channels open. This causes CPU utilization on rabbitmq host server to reach >90%
which is also an issue. As the server can crash any time.
英文:
I have 16 queues
& mulitple consumer servers
for those queues. I have created
one dedicated
channel for each queue
to consume
messages. Consumer
& dispatch
channels
on each server share same connection
.
When I dispatch messages
to each queue, I do the following:
- create a new channel
- bind channel to the queue with proper routing
- dispatch the message
- close the channel
I have lots of incoming webhooks from Shopify & these webooks contents are dispatched to specific queues.
> While processing each message, I need make an API call to Shopify. Shopify API has rate limit. If I hit rate limit once, I redispatch all messages from the consumer back to rabbitmq with a delay header of 1 minute(time required to clear the API rate limit).
Now, when I have several consumers running with lots of messages in the queue & I re-dispatch those messages, I get too many channels
error for a period of time. How can I avoid this error?
I tried to keep 2 dedicated channels per queue:
- for conusmer purpose only
- for dispatch purpose only
For, 16 queues, & around 11 consumer servers. This way, I always have to keep 352
channel open. This caues CPU utilization on rabbitmq host server to reach >90%
which is also an issue. As the server can crash any time.
答案1
得分: 1
我在查阅RabbitMQ文档
后找到了解决问题的方法。
与其为每次分发创建新通道,我创建了一个单一通道
并且在整个连接会话
期间保持它活动
。在创建
通道时,我声明了
所有我的队列将使用的交换机
。
然后,我只需将消息发布
到所需的交换机上,使用路由键
。由于我的队列已经与交换机绑定,并监听具有特定路由键的消息,因此消息最终进入了正确的队列!
这样,我可以仅维护01
个连接和每个服务器只有01
个通道!
英文:
I found the solution to the problem after digging through the RabbitMQ documentation
.
Instead of creating a new channel for each dispatch, I created a single channel
& kept it alive
for the entire
connection session
. When creating
the channel, I asserted
all the exchanges
that would be used by my queues.
Then I just publish
the messages to the desired exchange
with the routing key
. As my queues are already bonded with the exchanges & listen for messages with a given routing key, the messages end up in the correct queue!
This way I can maintain just 01
connection & only 01
channel per server!
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论