强制 reqwest 定期重新建立连接以实现负载均衡

huangapple go评论49阅读模式
英文:

Force reqwest to periodically re-establish connections for load balancing

问题

I'm writing a high-throughput (>1000 requests per second) REST service in Rust, using reqwest to asynchronously connect to a load-balanced upstream service.
为了降低对上游服务请求的延迟,我正在使用一个长期存在的reqwest::Client,其中包含一个连接池。

The challenge is that reqwest's long-lived connections "break" the load-balancing of the upstream service, since - as long as the number of connections in the pool is sufficient - no connections will be established to machines added to that upstream service to increase its capacity. This leads to my service over-utilizing some machines of that upstream service and under-utilizing others (my applications is the main user of that upstream service, so I can't rely on other users of that service to balance my lopsided usage).
挑战在于reqwest的长期存在的连接“破坏”了上游服务的负载均衡,因为只要池中的连接数量足够,就不会建立到添加到上游服务以增加其容量的机器的连接。这导致我的服务过度利用了上游服务的某些机器,而对其他机器利用不足(我的应用程序是上游服务的主要用户,因此无法依赖该服务的其他用户来平衡我的不平衡使用)。

Is there any mechanism in reqwest to periodically close and re-establish connections, in order to ensure that they are balanced across all machines of the upstream service as evenly as possible (or any way to manually implement such behavior on top of reqwest)?
reqwest中是否有任何机制,可以定期关闭并重新建立连接,以确保它们尽可能均匀地分布在上游服务的所有机器之间(或者是否有任何方法可以在reqwest之上手动实现这种行为)?

In the reqwest documentation, I've seen ConnectionBuilder::timeout() to ensure connections are closed and later re-established when upstream machines are no longer accessible. There's also ConnectionBuilder::pool_idle_timeout() to eventually close connections from that pool if they are idle long enough. But I've found nothing on closing connections - either automatically or explicitly - that are behaving fine.
reqwest的文档中,我看到了ConnectionBuilder::timeout(),以确保在上游机器不再可访问时关闭连接,并稍后重新建立连接。还有ConnectionBuilder::pool_idle_timeout(),用于最终关闭连接池中的连接(如果它们处于足够长时间的空闲状态)。但我没有找到有关关闭连接的信息 - 无论是自动关闭还是显式关闭 - 这些连接正常运行。

英文:

I'm writing a high-throughput (>1000 requests per second) REST service in Rust, using reqwest to asynchronously connect to a load-balanced upstream service.
To reduce latency of requests to that upstream service, I'm using a long-lived reqwest::Client with a connection pool.

The challenge is that reqwest's long-lived connections "break" the load-balancing of the upstream service, since - as long as the number of connections in the pool is sufficient - no connections will be established to machines added to that upstream service to increase its capacity. This leads to my service over-utilizing some machines of that upstream service and under-utilizing others (my applications is the main user of that upstream service, so I can't rely on other users of that service to balance my lopsided usage).

Is there any mechanism in reqwest to periodically close and re-establish connections, in order to ensure that they are balanced across all machines of the upstream service as evenly as possible (or any way to manually implement such behavior on top of reqwest)?

In the reqwest documentation, I've seen ConnectionBuilder::timeout() to ensure connections are closed and later re-established when upstream machines are no longer accessible. There's also ConnectionBuilder::pool_idle_timeout() to eventually close connections from that pool if they are idle long enough. But I've found nothing on closing connections - either automatically or explicitly - that are behaving fine.

答案1

得分: 2

这听起来像是一个应该在上游服务中解决的问题。如果它希望在更多的机器上重新分配负载,它应该有意地关闭连接,以便它们可以重新建立到更大的连接池中。(特别是,如果它看到多个连接源自明显相同的源,关闭其中一个。)

这样,服务将平衡其负载,无论客户端做什么 — 它将与任何符合HTTP标准的HTTP客户端一起工作,而不仅仅是为特定服务调整的一个。

英文:

To me, this sounds like a problem that should be solved in the upstream service. If it wants to re-distribute load over more machines, it should purposefully close connections so they can be reestablished to the bigger pool. (In particular, if it sees multiple connections originating from apparently the same source, close one of them.)

That way the service will balance its load whatever the client does — it will work well with any conformant HTTP client, not just one tweaked for it in particular.

答案2

得分: 0

最终,我在reqwest::Client的基础上实现了所需的行为:

我实现了一个reqwest::Client池,其中每个Client与其创建时间戳一起存储。当从池中请求条目时,池将检查可用条目的年龄,将处理那些太旧的条目,并返回剩下的一个(或者如果没有剩下的话,返回一个新的)。

这并不完美,因为它实际上将每个reqwest::Client连接池缩减为单个连接。但它增加了所需的附加行为,而且简单、封装良好,在实践中似乎运行良好。

英文:

In the end, I've implemented the desired behavior on top of reqwest::Client:

I've implemented a pool of reqwest::Clients where each Client is stored along with its creation time stamp. When requesting an entry from the pool, the pool will check the age of available entries, will dispose of those that are too old and will return a remaining one (or a new one if none remains).

This isn't perfect, since it effectively reduces each reqwest::Client connection pool to a single connection. But it adds the desired additional behavior, is simple, well encapsulated and seems to work well in practice.

huangapple
  • 本文由 发表于 2023年3月9日 16:53:08
  • 转载请务必保留本文链接:https://go.coder-hub.com/75682278.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定