英文:
Python "requests" library targeting multiple redundant hosts
问题
我们有一个使用“requests”包的Python库,作为连接提供REST API的HW设备的客户端。在数据中心的设置中,有多个冗余的HW设备。客户端库可以使用其中任何一个,因为它们同步它们的数据。
我们想避免设置一个为这些冗余HW设备提供单一IP地址的负载均衡器,而是想要在Python客户端库中配置一个目标IP地址或主机名列表,以便它使用当前可用的任何一个。
在Python的“requests”库(或其底层的urllib3库)中是否有配置的方法?
英文:
We have a Python library that uses the "requests" package and acts as a client to a HW device that provides a REST API. In the setup of the data centers, there are multiple redundant such HW devices. Any one of them can be used by the client library because they synchronize their data.
We would like to avoid setting up a load balancer that provides a single IP address for those redundant HW devices, and instead want to configure the Python client library with a list of target IP addresses or host names of these redundant HW devices, so that it uses anyone of them that currently works.
Is there a way to configure that in the Python "requests" library (or its underlying urllib3 library)?
答案1
得分: 2
你可以配置请求库来使用一个目标IP地址或主机名列表,以用于冗余的硬件设备,方法是创建自定义的requests.Session
对象,并配置HTTPAdapter
以使用自定义的DNSResolver
来动态解析目标IP地址或主机名。
以下是一个示例代码片段:
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
from requests.packages.urllib3.contrib.socks import SOCKSProxyManager
import socket
class DynamicDNSResolver:
def __init__(self, targets):
self.targets = targets
self.resolved = set()
self.socket_cache = {}
def resolve(self, host, port, *args, **kwargs):
if host in self.resolved:
return self.socket_cache[(host, port)]
for target in self.targets:
try:
addrinfo = socket.getaddrinfo(target, port, 0, socket.SOCK_STREAM)
af, socktype, proto, canonname, sa = addrinfo[0]
sock = socket.socket(af, socktype, proto)
sock.connect(sa)
self.socket_cache[(host, port)] = sock
self.resolved.add(host)
return sock
except socket.error as e:
print(f"failed to connect to {target}: {e}")
raise e
targets = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
resolver = DynamicDNSResolver(targets)
session = requests.Session()
retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[500, 502, 503, 504])
adapter = HTTPAdapter(max_retries=retries, pool_connections=100, pool_maxsize=100)
adapter.poolmanager = SOCKSProxyManager(socks_version=socks.PROXY_TYPE_SOCKS5, proxy_url="socks5://localhost:9050")
adapter.poolmanager.resolver = resolver
session.mount("http://", adapter)
session.mount("https://", adapter)
response = session.get("http://stackoverflow.com")
(注意:这是你提供的代码的翻译部分,不包括任何问题的回答。)
英文:
You can configure the requests library to use a list of target IP addresses or hostnames for the redundant HW devices by creating custom requests.Session
object and configuring the HTTPAdapter
to use a custom DNSResolver
that resolves the target IP addresses or hostnames dynamically.
Here's an example code snippet:
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
from requests.packages.urllib3.contrib.socks import SOCKSProxyManager
import socket
class DynamicDNSResolver:
def __init__(self, targets):
self.targets = targets
self.resolved = set()
self.socket_cache = {}
def resolve(self, host, port, *args, **kwargs):
if host in self.resolved:
return self.socket_cache[(host, port)]
for target in self.targets:
try:
addrinfo = socket.getaddrinfo(target, port, 0, socket.SOCK_STREAM)
af, socktype, proto, canonname, sa = addrinfo[0]
sock = socket.socket(af, socktype, proto)
sock.connect(sa)
self.socket_cache[(host, port)] = sock
self.resolved.add(host)
return sock
except socket.error as e:
print(f"failed to connect to {target}: {e}")
raise e
targets = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
resolver = DynamicDNSResolver(targets)
session = requests.Session()
retries = Retry(total=5, backoff_factor=0.1, status_forcelist=[500, 502, 503, 504])
adapter = HTTPAdapter(max_retries=retries, pool_connections=100, pool_maxsize=100)
adapter.poolmanager = SOCKSProxyManager(socks_version=socks.PROXY_TYPE_SOCKS5, proxy_url="socks5://localhost:9050")
adapter.poolmanager.resolver = resolver
session.mount("http://", adapter)
session.mount("https://", adapter)
response = session.get("http://stackoverflow.com")
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论