多线程处理快速请求的编程范例。

huangapple go评论71阅读模式
英文:

Multi-threading paradigm for handling express requests

问题

I understand your instructions. Here's the translated text:

现在,我要求的内容可能有些奇怪,但我有我的理由,请耐心等待。

我正在使用Node.js和Express,默认行为是使用单线程并发处理请求,对大多数情况都适用。但不适用于我的情况,所以我想问,Express中是否有任何库/中间件可以使用工作线程的概念,每个线程处理一个请求?(传统的处理方式)

原因:
我的请求处理主要包括两件事,主要是与Redis/SSD通信(I/O绑定1秒),然后进行CRC校验、解压缩和压缩(计算绑定4秒)。问题是当并行度很高时,Node异步单线程方法开始延长请求时间,因为更多的并行请求开始击中服务器,从而使响应时间变得不可预测,甚至超时(也因为同步工作引起事件循环的饱和),所以我提出了这个问题。

我尝试过的方法:
1)集群模式应该和线程有相同的效果,唯一的缺点是“内存”消耗,我在集群模式中有应用级别的进程缓存,集群中的所有子进程都有它的理想副本,因此内存需求增加。
2)Piscina为Next.js框架提供了类似的功能,但不确定是否适用于Express。

所以,我的请求很简单,我们已经有了如何使用单线程异步操作处理获取设置数据库类型的REST服务的实现。我正在寻找相反的实现方式,或者说处理请求的老方法。如果有现成的库/中间件,我想使用。

英文:

Now what i request below some may find it weird, but i have my reasons so bear with me..

I am using node.js with express and the default behaviour of it is to serve requests concurrently using single thread works fine for most of the cases.. but not mine so the ask, Is there any library/middleware for express which will use worker thread concept and serve 1 request per thread?(old way of doing things)

Reasons:
My request processing consists of 2 things mainly talk to Redis/SSD(IO bound 1 sec) then do CRC, Unzip, Zip(compute bound 4sec) problem is when parallelism is high, node async single thread approach starts elongating request time as more parallel request starts to hit the server, thus making response time unpredictable or timming out alltogether(also saturation of eloop cause of sync work) so this ask..

What i have tried: 1) Cluster mode should have same effect like threads the only downside is "memory" consumption, I have app level in process cache in cluster mode all cluster child process have ideal copy of it thus shooting up mem req..
2)Piscina which offers something simillar for next framework but not sure since i am using express..

So finally the ask is simple we already have implementation for how we can use single thread async operations for get set db kind of rest services.. i am asking for the opposite implementation or the old school way of handling requests. A ready to use library/middleware if any..

答案1

得分: 1

Node.js集群是预先构建的方式,可以在处理Express请求时涉及多个CPU。我不知道其他预建机制适用于Node.js/Express。

你提到你认为它使用了太多内存,但Node.js线程不太可能更节省内存,因为它们为每个线程启动了单独的VM解释器。如果你有具体的内存要求,请分享出来。

根据你描述的情况,CRC和压缩是CPU密集型的部分。I/O部分可能在主线程中运行得很好,因为它是非阻塞的,不会占用CPU。

因此,你可以创建一个由多个线程处理的作业队列,然后将压缩和CRC工作提交给作业队列。这将使你的主Web服务器线程保持空闲,以响应其他请求,同时利用多个CPU来处理CPU密集型工作。主线程将请求放入队列,每个工作线程将获取一个任务,处理它,将结果返回给主线程,然后主线程可以发送响应,然后工作线程将获取作业队列中的下一个项目。编写自己的队列不需要太多的代码(我自己写过好几个),或者有许多第三方库提供带有各种功能的队列。

通常,我编写工作线程只是响应发送给它的作业。队列知道工作线程,每当它收到新作业时,它会检查是否有空闲的工作线程。如果有,它会将新作业发送给它。如果没有,它会将作业添加到队列中,并等待现有的工作线程完成它正在处理的作业。

英文:

Nodejs clustering is the pre-built way to involve more than one CPU in the processing of your Express requests. I'm not aware of some other pre-built mechanism for nodejs/Express.

You mentioned that you think that uses too much memory, but nodejs threads aren't going to be much more memory efficient since they start up a separate VM interpreter for each thread themselves. If you have a specific memory requirement, then please share it.

From what you've described, the CRC and compression are the CPU-bound pieces. The I/O stuff will likely work just fine in the main thread since it's non-blocking and not CPU-bound.

So, you could create a job queue that is handled by multiple threads and then feed the compression and CRC work to the job queue. This would keep your main web server thread free and responsive for other requests while using multiple CPUs for the CPU-intensive work. The main thread would put requests into the queue and each worker would grab a job, work on it, return the result back to the main thread where the main thread could then send the response, then the worker would grab the next item from the job queue. It's not much code to write your own queue (I've written several myself) or there are many third party libraries that offer queues with various features.

Usually, I code the workers to just respond to a job sent to them. The queue knows about the workers and whenever it gets a new job, it checks to see if it has an idle worker. If so, it sends it the new job. If not, it adds the job to the queue and waits for an existing worker to finish the job it's working on.

huangapple
  • 本文由 发表于 2023年4月20日 03:51:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/76058325.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定