Python “requests”库针对多个冗余主机。

huangapple go评论72阅读模式
英文:

Python "requests" library targeting multiple redundant hosts

问题

我们有一个使用“requests”包的Python库,作为连接提供REST API的HW设备的客户端。在数据中心的设置中,有多个冗余的HW设备。客户端库可以使用其中任何一个,因为它们同步它们的数据。

我们想避免设置一个为这些冗余HW设备提供单一IP地址的负载均衡器,而是想要在Python客户端库中配置一个目标IP地址或主机名列表,以便它使用当前可用的任何一个。

在Python的“requests”库(或其底层的urllib3库)中是否有配置的方法?

英文:

We have a Python library that uses the "requests" package and acts as a client to a HW device that provides a REST API. In the setup of the data centers, there are multiple redundant such HW devices. Any one of them can be used by the client library because they synchronize their data.

We would like to avoid setting up a load balancer that provides a single IP address for those redundant HW devices, and instead want to configure the Python client library with a list of target IP addresses or host names of these redundant HW devices, so that it uses anyone of them that currently works.

Is there a way to configure that in the Python "requests" library (or its underlying urllib3 library)?

答案1

得分: 2

你可以配置请求库来使用一个目标IP地址或主机名列表,以用于冗余的硬件设备,方法是创建自定义的requests.Session对象,并配置HTTPAdapter以使用自定义的DNSResolver来动态解析目标IP地址或主机名。

以下是一个示例代码片段:

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
from requests.packages.urllib3.contrib.socks import SOCKSProxyManager
import socket

class DynamicDNSResolver:
    def __init__(self, targets):
        self.targets = targets
        self.resolved = set()
        self.socket_cache = {}

    def resolve(self, host, port, *args, **kwargs):
        if host in self.resolved:
            return self.socket_cache[(host, port)]
        for target in self.targets:
            try:
                addrinfo = socket.getaddrinfo(target, port, 0, socket.SOCK_STREAM)
                af, socktype, proto, canonname, sa = addrinfo[0]
                sock = socket.socket(af, socktype, proto)
                sock.connect(sa)
                self.socket_cache[(host, port)] = sock
                self.resolved.add(host)
                return sock
            except socket.error as e:
                print(f"failed to connect to {target}: {e}")
                raise e

targets = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
resolver = DynamicDNSResolver(targets)

session = requests.Session()
retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[500, 502, 503, 504])
adapter = HTTPAdapter(max_retries=retries, pool_connections=100, pool_maxsize=100)
adapter.poolmanager = SOCKSProxyManager(socks_version=socks.PROXY_TYPE_SOCKS5, proxy_url="socks5://localhost:9050")
adapter.poolmanager.resolver = resolver
session.mount("http://", adapter)
session.mount("https://", adapter)

response = session.get("http://stackoverflow.com")

(注意:这是你提供的代码的翻译部分,不包括任何问题的回答。)

英文:

You can configure the requests library to use a list of target IP addresses or hostnames for the redundant HW devices by creating custom requests.Session object and configuring the HTTPAdapter to use a custom DNSResolver that resolves the target IP addresses or hostnames dynamically.

Here's an example code snippet:

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
from requests.packages.urllib3.contrib.socks import SOCKSProxyManager
import socket

class DynamicDNSResolver:
    def __init__(self, targets):
        self.targets = targets
        self.resolved = set()
        self.socket_cache = {}


    def resolve(self, host, port, *args, **kwargs):
        if host in self.resolved:
            return self.socket_cache[(host, port)]
        for target in self.targets:
            try:
                addrinfo = socket.getaddrinfo(target, port, 0, socket.SOCK_STREAM)
                af, socktype, proto, canonname, sa = addrinfo[0]
                sock = socket.socket(af, socktype, proto)
                sock.connect(sa)
                self.socket_cache[(host, port)] = sock
                self.resolved.add(host)
                return sock
            except socket.error as e:
                print(f"failed to connect to {target}: {e}")
                raise e

targets = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
resolver = DynamicDNSResolver(targets)

session = requests.Session()
retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[500, 502, 503, 504])
adapter = HTTPAdapter(max_retries=retries, pool_connections=100, pool_maxsize=100)
adapter.poolmanager = SOCKSProxyManager(socks_version=socks.PROXY_TYPE_SOCKS5, proxy_url="socks5://localhost:9050")
adapter.poolmanager.resolver = resolver
session.mount("http://", adapter)
session.mount("https://", adapter)

response = session.get("http://stackoverflow.com")

huangapple
  • 本文由 发表于 2023年5月10日 19:48:11
  • 转载请务必保留本文链接:https://go.coder-hub.com/76218020.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定