在Azure Functions中在运行时安装Python模块。

huangapple
go评论146阅读模式
英文:

Install python module during runtime in Azure Functions

问题

我正在尝试在Azure Functions运行时安装Python模块/包，但运气不太好。使用pip安装模块似乎工作得很好，但让函数（当前的Python解释器）意识到已经安装了新模块却相当棘手。

以下是重现此问题的最小代码：

  1. import io
  2. import json
  3. import logging
  4. import sys
  6. import azure.functions as func
  8. app = func.FunctionApp(http_auth_level=func.AuthLevel.ANONYMOUS)
  10. @app.route(route="InstallModule")
  11. def install_module(req: func.HttpRequest) -> func.HttpResponse:
  12.     logging.info("install_module was triggered")
  13.     data = req.get_json()
  14.     module_name = data.get("module")
  16.     output = None
  17.     try:
  18.         import subprocess
  19.         output = subprocess.run("pip install " + module_name, shell=True, check=True, capture_output=True)
  20.         logging.info(output)
  21.         __import__(module_name)
  22.         output = f"{output}\nwe have survived this"
  23.     except Exception as ex:
  24.         logging.error(f"Failed to execute got {ex}")
  26.     return func.HttpResponse(str(output))

第一次触发端点时，使用以下POST请求和JSON主体：

  1. {
  2.     "module": "numpy"
  3. }

输出看起来是这样的：

  1. Defaulting to user installation because normal site-packages is not writeable
  2. Collecting numpy
  3.   Downloading numpy-1.25.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
  4.   ...
  5. Successfully installed numpy-1.25.2

下一次：

  1. Defaulting to user installation because normal site-packages is not writeable
  2. Requirement already satisfied: numpy in /home/.local/lib/python3.10/site-packages (1.25.2)

但在两种情况下都将出现错误：

  1. [Error]   Failed to execute got No module named 'numpy'

在本地运行相同的Azure函数没有问题，因此这与Azure Functions的缓存或权限有关...有没有办法克服这个问题？

另外，看起来Azure Functions没有使用虚拟环境，并且pip安装目录在路径中：

  1.     "pip_V": "b'pip 23.0.1 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)\\n'",
  2.     "sys_path": [
  3.         "/tmp/functions/standby/wwwroot",
  4.         "/home/site/wwwroot/.python_packages/lib/site-packages",
  5.         "/azure-functions-host/workers/python/3.10/LINUX/X64",
  6.         "/usr/local/lib/python310.zip",
  7.         "/usr/local/lib/python3.10",
  8.         "/usr/local/lib/python3.10/lib-dynload",
  9.         "/usr/local/lib/python3.10/site-packages",
  10.         "/home/site/wwwroot"
  11.     ],

我尝试过检查Python路径，在之前和之后运行pip freeze，并验证模块确实被安装，但我无法让Python解释器意识到模块已安装。

编辑 2023年10月8日

嗨，@Sampath，非常感谢您尝试帮助我解决这个问题。不幸的是，问题仍然存在，我创建了一个全新的项目，并复制了一切（如果您想分享项目，请告诉我）。第一次运行时，我获得了：

在Azure Functions中在运行时安装Python模块。

但第二次我得到了：

在Azure Functions中在运行时安装Python模块。

我不禁想知道我们的设置有什么不同？是项目设置还是我创建Azure Functions App的方式不同吗？顺便说一下，我尝试了托管选项：消耗和应用服务，两种选项的行为都相同。

在Azure Functions中在运行时安装Python模块。

英文:

I'm trying to install python module/package during runtime in Azure functions without too much luck.
Installing the module with pip seems to work just fine, but getting the function (current python interpreter) to realize a new module has been isntalled is proving to be rather tricky.

This is minimum code to reproduce the issue::

  1. import io
  2. import json
  3. import logging
  4. import sys
  6. import azure.functions as func
  8. app = func.FunctionApp(http_auth_level=func.AuthLevel.ANONYMOUS)
  10. @app.route(route="InstallModule")
  11. def install_module(req: func.HttpRequest) -> func.HttpResponse:
  12.     logging.info("install_module was triggered")
  13.     data = req.get_json()
  14.     module_name = data.get("module")
  16.     output = None
  17.     try:
  18.         import subprocess
  19.         output = subprocess.run("pip install " + module_name, shell=True, check=True, capture_output=True)
  20.         logging.info(output)
  21.         __import__(module_name)
  22.         output = f"{output}\nwe have survived this"
  23.     except Exception as ex:
  24.         logging.error(f"Failed to execute got {ex}")
  26.     return func.HttpResponse(str(output))

The first time the endpoint is triggerd with: POST <azure url>/api/InstallModule w json body:

  1. {
  2.     &quot;module&quot;: &quot;numpy&quot;
  3. }

the output looks something like:

  1. Defaulting to user installation because normal site-packages is not writeable\nCollecting numpy\n  Downloading numpy-1.25.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)\n     \xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81 18.2/18.2 MB 42.9 MB/s eta 0:00:00\nInstalling collected packages: numpy\nSuccessfully installed numpy-1.25.2\n

next time:

  1. Defaulting to user installation because normal site-packages is not writeable\nRequirement already satisfied: numpy in /home/.local/lib/python3.10/site-packages (1.25.2)\n

But in both cases it will fail with error:

  1. [Error]   Failed to execute got No module named &#39;numpy&#39;

Runninc the same azure function localy works without issue, so it has to do something with the cacheing or permissions on the Azure Functions... Any ideas how to overcome this issue?

BTW it looks like azure functions are not using virtual environment, and pip install directory is in path:

  1.     &quot;pip_V&quot;: &quot;b&#39;pip 23.0.1 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)\\n&#39;&quot;,
  2.     &quot;sys_path&quot;: [
  3.         &quot;/tmp/functions\\standby\\wwwroot&quot;,
  4.         &quot;/home/site/wwwroot/.python_packages/lib/site-packages&quot;,
  5.         &quot;/azure-functions-host/workers/python/3.10/LINUX/X64&quot;,
  6.         &quot;/usr/local/lib/python310.zip&quot;,
  7.         &quot;/usr/local/lib/python3.10&quot;,
  8.         &quot;/usr/local/lib/python3.10/lib-dynload&quot;,
  9.         &quot;/usr/local/lib/python3.10/site-packages&quot;,
  10.         &quot;/home/site/wwwroot&quot;
  11.     ],

I have tried checking python path, running pip freeze before and after and verify that the module does get installed, but I couldn't make python interpterer realize that the module is installed.

EDIT 10.08.2023

Hi @Sampath thank you so much for trying to help me with this one. Unfortunately, the issue is still there, I have created an entirely new project and copied everything (your entire code if you would like me to share the project please let me know) and published it. The first time I run it I get the:
在Azure Functions中在运行时安装Python模块。

but the second time I'm getting:
在Azure Functions中在运行时安装Python模块。

I can't help but wonder what is different in our setups? Is it something in the project settings or in the way I have created Azure Functions App? btw I have tried hosting options: Consumption and App service and both options behave the same way.
在Azure Functions中在运行时安装Python模块。

答案1

得分: 1

  1. # 在 Azure Functions 中运行时安装 Python 模块
  3. v1
  4. ```python
  5. import  json
  6. import  logging
  7. import  sys
  8. import  azure.functions  as  func
  9. from  importlib  import  import_module
  11. def  main(req:  func.HttpRequest)  ->  func.HttpResponse:
  12.     try:
  13.         req_body = req.get_json()
  14.         if  'module'  in  req_body:
  15.             module_name = req_body['module']
  16.             # 尝试直接导入模块
  17.             try:
  18.                 import_module(module_name)
  19.                 return  func.HttpResponse(f"模块 '{module_name}' 已经可用。")
  20.             except  ImportError:
  21.                 pass
  22.             # 使用 subprocess 安装模块
  23.             output = install_module(module_name)
  24.             return  func.HttpResponse(f"模块 '{module_name}' 安装成功。\n{output}")
  25.         else:
  26.             return  func.HttpResponse("JSON 负载中缺少 'module' 键。",  status_code=400)
  27.     except  Exception  as  e:
  28.         return  func.HttpResponse(f"发生错误: {str(e)}",  status_code=500)
  30. def  install_module(module):
  31.     try:
  32.         import  subprocess
  33.         output = subprocess.run([sys.executable,  "-m",  "pip",  "install",  module],  capture_output=True,  text=True,  check=True)
  34.         return  output.stdout
  35.     except  Exception  as  ex:
  36.         return  f"执行失败: {ex}"

v2

  1. import json
  2. import logging
  3. import sys
  4. import subprocess
  5. import azure.functions as func
  6. from importlib import import_module
  8. app = func.FunctionApp()
  10. @app.function_name("HttpTrigger1")
  11. @app.route(route="req")  # 在这里指定路由
  12. def main(req: func.HttpRequest) -> func.HttpResponse:
  13.     try:
  14.         req_body = req.get_json()
  15.         if 'module' in req_body:
  16.             module_name = req_body['module']
  17.             # 尝试直接导入模块
  18.             try:
  19.                 import_module(module_name)
  20.                 return func.HttpResponse(f"模块 '{module_name}' 已经可用。")
  21.             except ImportError:
  22.                 pass
  23.             # 使用 subprocess 安装模块
  24.             output = install_module(module_name)
  25.             return func.HttpResponse(f"模块 '{module_name}' 安装成功。\n{output}")
  26.         else:
  27.             return func.HttpResponse("JSON 负载中缺少 'module' 键。", status_code=400)
  28.     except Exception as e:
  29.         return func.HttpResponse(f"发生错误: {str(e)}", status_code=500)
  31. def install_module(module):
  32.     try:
  33.         output = subprocess.run([sys.executable, "-m", "pip", "install", module], capture_output=True, text=True, check=True)
  34.         return output.stdout
  35.     except Exception as ex:
  36.         return f"执行失败: {ex}"

在本地:

在Azure Functions中在运行时安装Python模块。

在 Azure:

在Azure Functions中在运行时安装Python模块。

在 SSH:

在Azure Functions中在运行时安装Python模块。

  2. <details>
  3. <summary>英文:</summary>
  5. **Install python module during runtime in Azure Functions**
  7. v1
  8. ```python
  9. import  json
  10. import  logging
  11. import  sys
  12. import  azure.functions  as  func
  13. from  importlib  import  import_module
  14. def  main(req:  func.HttpRequest)  -&gt;  func.HttpResponse:
  15. try:
  16. req_body = req.get_json()
  17. if  &#39;module&#39;  in  req_body:
  18. module_name = req_body[&#39;module&#39;]
  19. # Try importing the module directly
  20. try:
  21. import_module(module_name)
  22. return  func.HttpResponse(f&quot;Module &#39;{module_name}&#39; already available.&quot;)
  23. except  ImportError:
  24. pass
  25. # Install the module using subprocess
  26. output = install_module(module_name)
  27. return  func.HttpResponse(f&quot;Module &#39;{module_name}&#39; installed successfully.\n{output}&quot;)
  28. else:
  29. return  func.HttpResponse(&quot;Missing &#39;module&#39; key in JSON payload .&quot;,  status_code=400)
  30. except  Exception  as  e:
  31. return  func.HttpResponse(f&quot;An error occurred: {str(e)}&quot;,  status_code=500)
  32. def  install_module(module):
  33. try:
  34. import  subprocess
  35. output = subprocess.run([sys.executable,  &quot;-m&quot;,  &quot;pip&quot;,  &quot;install&quot;,  module],  capture_output=True,  text=True,  check=True)
  36. return  output.stdout
  37. except  Exception  as  ex:
  38. return  f&quot;Failed to execute got {ex}&quot;

v2

  1. import json
  2. import logging
  3. import sys
  4. import subprocess
  5. import azure.functions as func
  6. from importlib import import_module
  8. app = func.FunctionApp()
  10. @app.function_name(&quot;HttpTrigger1&quot;)
  11. @app.route(route=&quot;req&quot;)  # Specify the route here
  12. def main(req: func.HttpRequest) -&gt; func.HttpResponse:
  13.     try:
  14.         req_body = req.get_json()
  15.         if &#39;module&#39; in req_body:
  16.             module_name = req_body[&#39;module&#39;]
  17.             
  18.             # Try importing the module directly
  19.             try:
  20.                 import_module(module_name)
  21.                 return func.HttpResponse(f&quot;Module &#39;{module_name}&#39; already available.&quot;)
  22.             except ImportError:
  23.                 pass
  24.             
  25.             # Install the module using subprocess
  26.             output = install_module(module_name)
  27.             
  28.             return func.HttpResponse(f&quot;Module &#39;{module_name}&#39; installed successfully.\n{output}&quot;)
  29.         else:
  30.             return func.HttpResponse(&quot;Missing &#39;module&#39; key in JSON payload.&quot;, status_code=400)
  31.     except Exception as e:
  32.         return func.HttpResponse(f&quot;An error occurred: {str(e)}&quot;, status_code=500)
  34. def install_module(module):
  35.     try:
  36.         output = subprocess.run([sys.executable, &quot;-m&quot;, &quot;pip&quot;, &quot;install&quot;, module], capture_output=True, text=True, check=True)
  37.         return output.stdout
  38.     except Exception as ex:
  39.         return f&quot;Failed to execute got {ex}&quot;

在Azure Functions中在运行时安装Python模块。
In local:

在Azure Functions中在运行时安装Python模块。

在Azure Functions中在运行时安装Python模块。
In Azure:
在Azure Functions中在运行时安装Python模块。

在Azure Functions中在运行时安装Python模块。

In SSH:
在Azure Functions中在运行时安装Python模块。

在Azure Functions中在运行时安装Python模块。

huangapple
  • 本文由 发表于 2023年8月10日 21:49:54
  • 转载请务必保留本文链接：https://go.coder-hub.com/76876373.html
  • azure-functions
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定