英文:
How to stop Redis queue backing up and timing out
问题
我在生产环境中遇到了一个问题,当我的应用程序尝试对Redis执行Get
操作时,出现了以下错误:
> Timeout awaiting response
(outbound=0KiB, inbound=0KiB, 30469ms elapsed, timeout is 30000ms),
超时时间为30秒,所以Get
调用等待的时间太长了!
StackExchange.Redis库的错误消息如下:
> inst: 0, qu: 0, qs: 299, aw: False, bw: SpinningDown, rs: ReadAsync,
> ws: Idle, in: 46478, in-pipe: 0, out-pipe: 0, mc: 1/1/0, mgr: 10 of 10
> available, IOCP: (Busy=0,Free=1000,Min=2,Max=1000), WORKER:
> (Busy=153,Free=32614,Min=2,Max=32767), POOL:
> (Threads=153,QueuedItems=522,CompletedItems=3890)
所以我的分析如下:
- 总共有299个调用在等待响应。
- 看起来没有利用IOCP池?
- 工作池似乎正常运行。
- "Pool"似乎有522个排队项目(我假设是等待执行的)。
我使用了这个页面(Redis超时文档)来尝试了解发生了什么情况。
我运行了SlowLog
命令来分析运行缓慢的查询,但它们似乎都在及时执行。
我正在使用略旧版本的库(1年前)V2.5.61。但不确定这是否会有太大的影响。
为什么我的池中有522个待处理项?为什么IOCP根本没有被利用?我需要在我的应用程序中进行一些更改吗,还是因为没有足够的CPU来处理需求而导致瓶颈?
英文:
So I am getting this error in a production environment when my application tries to do a Get
to Redis
> Timeout awaiting response
(outbound=0KiB, inbound=0KiB, 30469ms elapsed, timeout is 30000ms),
The timeout is 30 seconds.. so It's a long time for a Get
call to wait!
The error message from the StackExchange.Redis library says:
> inst: 0, qu: 0, qs: 299, aw: False, bw: SpinningDown, rs: ReadAsync,
> ws: Idle, in: 46478, in-pipe: 0, out-pipe: 0, mc: 1/1/0, mgr: 10 of 10
> available, IOCP: (Busy=0,Free=1000,Min=2,Max=1000), WORKER:
> (Busy=153,Free=32614,Min=2,Max=32767), POOL:
> (Threads=153,QueuedItems=522,CompletedItems=3890)
So my Analysis of this is the following:
- There are 299 calls awaiting a response in total
- It doesn't look like any of the IOCP Pool is being utlised?
- Worker pool seems to be operating correctly
- "Pool" seems to have 522 queuedItems (I assume waiting execution)
I have used this page (Redis Timeout documentation) to try see whats going on
And I have ran the SlowLog
command to analyse slow running queries, but they all seem to be executing in a timely manner
I am running a slightly older version of the library (1 year old) V2.5.61. But not sure that would make too much of a difference
Why has my pool got 522 pending items? and why is the IOCP not being utilised at all? Do I need to change something in my application or is this just bottlenecking because it doesnt have enough CPU to deal with the demand
答案1
得分: 0
我最终将线程池的 MinThreads 设置为 100,以解决了这个问题。
ThreadPool.SetMinThreads(100, 100);
MinThreads 的数量取决于系统。
关于设置最小线程值的支持信息:
- https://stackexchange.github.io/StackExchange.Redis/Timeouts#are-you-seeing-a-high-number-of-busyio-or-busyworker-threads-in-the-timeout-exception
- https://stackoverflow.com/questions/57932586/stackexchange-redis-timeouts-and-how-to-set-my-minimum-thread-pool-number-in-ne/59261867#59261867
英文:
I ended up setting the ThreadPool MinThreads to 100 which resolved the issue
ThreadPool.SetMinThreads(100, 100);
The number of minThreads depends on the system
Supporting information around setting min thread values
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论