英文:
Go/Python asynchronous bridge
问题
我写了一个客户端来处理低级别的TLS连接参数，比如ClientHello等。
我使用Go语言来完成这个任务，因为在Go中更容易实现。我的主程序（网页爬虫）是用Python编写的。我通过ctypes将Go源代码连接到我的Python文件中的DLL。到目前为止，我的网页爬虫结构是异步的，可以同时处理多个连接。
不幸的是，我的Go客户端不是异步的。有没有办法在Python中对它进行更改，使其异步等待ctypes指针的响应，直到响应到达为止？目前它正在等待响应，但同时阻塞了所有其他代码的执行。
编辑：
下面是代码示例：
async def request(self, method, url, headers, body=None, rawBody=None, pseudoHeaderOrder=["method", "authority", "scheme", "path"]):global httpLibglobal initFuncglobal requestFuncglobal changeProxyFuncglobal freePointerFuncconfig = {"id": self.cid,"method": method.upper(),"timeout": 20000,"url": url,"pseudoHeaderOrder": pseudoHeaderOrder,"headers": headers}#Criticalif body:config["body"] = bodyif rawBody:rawBody = [b for b in bytes(rawBody, "utf-8")]config["rawBody"] = rawBodyconfig = json.dumps(config)#print(config)#THIS PART CASTS THE REQUESTptr = requestFunc(config.encode('utf-8'))string = ctypes.cast(ptr, ctypes.c_char_p).value.decode("utf-8")#THIS PART CLEARS THE POINTERfreePointerFunc(ptr)#...
英文:
I wrote a client to handle lower-level TLS connection parameters like ClientHellos etc.
I did this in Go since it's a lot easier there. My main program (webscraper) is in Python. I connected the Go source to my Python file through a DLL via ctypes. My webscraper structure was asynchronous so far, to handle multiple connections at a time.
Unfortunately, my Go client is not asynchronous. Is there a way I can change it in Python so it asynchronously awaits a response from the ctypes pointer until it is there? Right now it's waiting for a response but blocks all other code executions in the meantime obviously.
EDIT:
Code example below
async def request(self, method, url, headers, body=None, rawBody=None, pseudoHeaderOrder=["method", "authority", "scheme", "path"]):global httpLibglobal initFuncglobal requestFuncglobal changeProxyFuncglobal freePointerFuncconfig = {"id": self.cid,"method": method.upper(),"timeout": 20000,"url": url,"pseudoHeaderOrder": pseudoHeaderOrder,"headers": headers}#Criticalif body:config["body"] = bodyif rawBody:rawBody = [b for b in bytes(rawBody, "utf-8")]config["rawBody"] = rawBodyconfig = json.dumps(config)#print(config)#THIS PART CASTS THE REQUESTptr = requestFunc(config.encode('utf-8'))string = ctypes.cast(ptr, ctypes.c_char_p).value.decode("utf-8")#THIS PART CLEARS THE POINTERfreePointerFunc(ptr)#...
答案1
得分: 1
你可以使用执行器（executor）将阻塞调用移动到单独的线程或进程中。
类似这样的代码应该可以工作：
async def request(self, method, url, headers, body=None, rawBody=None, pseudoHeaderOrder=["method", "authority", "scheme", "path"]):global httpLibglobal initFuncglobal requestFuncglobal changeProxyFuncglobal freePointerFuncconfig = {"id": self.cid,"method": method.upper(),"timeout": 20000,"url": url,"pseudoHeaderOrder": pseudoHeaderOrder,"headers": headers}#关键部分if body:config["body"] = bodyif rawBody:rawBody = [b for b in bytes(rawBody, "utf-8")]config["rawBody"] = rawBodyconfig = json.dumps(config)# 将阻塞代码移动到单独的函数中def blocking_io():ptr = requestFunc(config.encode('utf-8'))string = ctypes.cast(ptr, ctypes.c_char_p).value.decode("utf-8")freePointerFunc(ptr)return string# 异步等待结果loop = asyncio.get_running_loop()string = await loop.run_in_executor(None, blocking_io)#...
你可以参考这里了解更多关于执行代码在线程或进程池中的信息。
英文:
You can move the blocking call to a separate thread/process using an executor.
Something like this should work,
async def request(self, method, url, headers, body=None, rawBody=None, pseudoHeaderOrder=["method", "authority", "scheme", "path"]):global httpLibglobal initFuncglobal requestFuncglobal changeProxyFuncglobal freePointerFuncconfig = {"id": self.cid,"method": method.upper(),"timeout": 20000,"url": url,"pseudoHeaderOrder": pseudoHeaderOrder,"headers": headers}#Criticalif body:config["body"] = bodyif rawBody:rawBody = [b for b in bytes(rawBody, "utf-8")]config["rawBody"] = rawBodyconfig = json.dumps(config)# Move blocking code to separate functiondef blocking_io():ptr = requestFunc(config.encode('utf-8'))string = ctypes.cast(ptr, ctypes.c_char_p).value.decode("utf-8")freePointerFunc(ptr)return string# Aschronously wait on the resultloop = asyncio.get_running_loop()string = await loop.run_in_executor(None, blocking_io)#...
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论