sp_execute_external_script 无法找到由 setuptools 安装的模块。

huangapple go评论85阅读模式
英文:

sp_execute_external_script can't find modules installed by setuptools

问题

我正在积极开发一个Python模块,我想要部署到本地安装的SQL Server 2017中,所以我将模块部署到了c:\Program Files\Microsoft SQL Server\<Instance Name>\PYTHON_SERVICES\Lib\site-packages,使用了setuptools,如下所示:

"c:\Program Files\Microsoft SQL Server\<Instance_Name>\PYTHON_SERVICES\python" setup.py develop

这会在我的项目根目录中产生一个.egg-info目录,并在上述site-packages目录中产生一个.egg-link文件。.egg-link文件正确指向了我的项目根目录中的.egg-info目录,所以似乎setuptools正在正确工作。

以下是我的setup.py供参考:

from setuptools import setup

setup(
    setup_requires=['pbr'],
    pbr=True,
)

这是相应的setup.cfg文件:

[metadata]
name = <module_name>
description = <Module Description>
description-file = README.md
description-content-type = text/markdown

[files]
package_root = py/src

因为我只是尝试让它工作,我在<project_root>/py/src中有一个名为uploader.py的Python脚本:

#uploader.py
class Upload:
    pass

有了这个部署,我希望可以简单地将我刚刚发布的模块通过.egg-link导入到sp_execute_external_script调用中,如下所示:

execute sp_execute_external_script @language= N'Python', @script= N'from <module_name>.uploader import Upload';

然而,从SSMS执行这个存储过程会产生以下错误消息:

Msg 39004, Level 16, State 20, Line 10
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 10
An external script error occurred: 

Error in execution.  Check the output for more information.
Traceback (most recent call last):
  File "<string>", line 5, in <module>
  File "C:\SQL-MSSQLSERVER-ExtensibilityData-PY\MSSQLSERVER01\C08BB9A7-66B5-4B5E-AAFC-B0248EE64199\sqlindb.py", line 27, in transform
    from <module_name>.uploader import Upload
ImportError: No module named '<module_name>'

SqlSatelliteCall error: Error in execution.  Check the output for more information.
STDOUT message(s) from external script: 
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
  File "C:\Program Files\Microsoft SQL Server\&lt;Instance_Name&gt;\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 587, in rx_sql_satellite_call
    rx_native_call("SqlSatelliteCall", params)
  File "C:\Program Files\Microsoft SQL Server\&lt;Instance_Name&gt;\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 358, in rx_native_call
    ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.

我显然已经从错误消息中删除了module_nameInstance_Name

我尝试使用install命令代替develop,只是为了确保.egg-link文件不是问题。install.egg-info文件安装到了site-packages,但我仍然得到了相同的错误。

我还尝试从混合中删除pbr,但仍然得到相同的错误。

最后,我尝试按照https://stackoverflow.com/questions/53982439/how-can-i-use-an-external-python-module-with-sql-2017-sp-execute-external-script中的建议将&lt;project_root&gt;添加到sys.path,但这也没有帮助。

所以在这一点上,我不知道我可能做错了什么。

Python版本是3.5.2,我认为项目不需要__init__.py才能被视为模块。在py/src中插入一个空的__init__.py也没有帮助。

我的pip版本是19.3.1setuptools版本是44.0.0pbr版本是5.4.4,我已确认所有模块都安装在上述提到的site-packages目录中。

英文:

I am actively developing a Python module that I would like to deploy in SQL Server 2017 installed locally, so I deploy the module in c:\Program Files\Microsoft SQL Server\&lt;Instance Name&gt;\PYTHON_SERVICES\Lib\site-packagesusing setuptoolslike so:

&quot;c:\Program Files\Microsoft SQL Server\&lt;Instance_Name&gt;\PYTHON_SERVICES\python&quot; setup.py develop

This produces an .egg-info directory in my project root, and a .egg-link file in the site-packages directory mentioned above. The .egg-link file correctly points to the .egg-info directory in my project root, so it appears setuptools is working correctly.

Here's my setup.pyfor reference:

from setuptools import setup

setup(
    setup_requires=[&#39;pbr&#39;],
    pbr=True,
)

And here's the corresponding setup.cfg file:

[metadata]
name = &lt;module_name&gt;
description = &lt;Module Description&gt;
description-file = README.md
description-content-type = text/markdown

[files]
package_root = py/src

Since I am just trying to make the plumbing work, I have a single python script called uploader.py in &lt;project_root&gt;/py/src:

#uploader.py
class Upload:
    pass

With this deployment in place, I am hoping to simply import the module I just published through .egg-link into a sp_execute_external_script call like so:

execute sp_execute_external_script @language= N&#39;Python&#39;, @script= N&#39;from &lt;module_name&gt;.uploader import Upload&#39;;

However, executing this stored procedure from SSMS produces the following error message:

Msg 39004, Level 16, State 20, Line 10
A &#39;Python&#39; script error occurred during execution of &#39;sp_execute_external_script&#39; with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 10
An external script error occurred: 

Error in execution.  Check the output for more information.
Traceback (most recent call last):
  File &quot;&lt;string&gt;&quot;, line 5, in &lt;module&gt;
  File &quot;C:\SQL-MSSQLSERVER-ExtensibilityData-PY\MSSQLSERVER01\C08BB9A7-66B5-4B5E-AAFC-B0248EE64199\sqlindb.py&quot;, line 27, in transform
    from &lt;module_name&gt;.uploader import Upload
ImportError: No module named &#39;&lt;module_name&gt;&#39;

SqlSatelliteCall error: Error in execution.  Check the output for more information.
STDOUT message(s) from external script: 
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
  File &quot;C:\Program Files\Microsoft SQL Server\&lt;Instance_Name&gt;\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py&quot;, line 587, in rx_sql_satellite_call
    rx_native_call(&quot;SqlSatelliteCall&quot;, params)
  File &quot;C:\Program Files\Microsoft SQL Server\&lt;Instance_Name&gt;\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py&quot;, line 358, in rx_native_call
    ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.

I have obviously redacted module_name and Instance_Name from the error message.

I tried using install command instead of develop just to make sure the .egg-link file is not a problem. install installs the .egg-info file in site-packages but I get the same error.

I also tried removing pbr from the mix, but got the same error.

Lastly, I tried adding my &lt;project_root&gt; to sys.path as suggested by https://stackoverflow.com/questions/53982439/how-can-i-use-an-external-python-module-with-sql-2017-sp-execute-external-script, but that didn't help either.

So at this point, I don't have a clue what I might be doing wrong.

The python version is 3.5.2 and I don't think an __init__.py is needed in the project for it to qualify as a module. Inserting a blank __init__.py in py/src doesn't help either.

My pip version is 19.3.1 and setuptools version is 44.0.0 and pbr version is 5.4.4 and I have confirmed all modules are installed in the site-packages directory mentioned above.

答案1

得分: 0

根据我的广泛实验,似乎 sp_execute_external_script 不会跟随符号链接(即通过.egg-link文件)。因此,无论您使用 setuptoolspippbr 还是其他任何工具,开发模式的安装都不起作用。

我甚至尝试将 &lt;package_name&gt; 文件夹创建为操作系统的符号链接。由于我使用的是Windows,我使用命令提示符上的 mklink /D 命令将 /py/src/&lt;package_name&gt; 创建为 site-packages 内的符号链接。尽管命令正确执行,我可以在文件资源管理器中看到符号链接的文件夹,但 sp_execute_external_script 无法检测到该包。这告诉我 sp_execute_external_script 的代码中可能有一些内容会避免遍历符号链接。

我想知道是否有办法让它遍历符号链接。

唯一可行的解决方案是在包的自己目录下开发代码,就像在我的情况下是 /py/src/&lt;package_name&gt;。然后,在运行 exec sp_execute_external_script @language=N&#39;python&#39;, @script=N&#39;...&#39; 之前,将 &lt;package_name&gt; 文件夹复制到 site-packages 目录下。

这在某种程度上相当于 setup.py install,但绕过了中间文件和目录的创建。因此,我将坚持这种简单但繁琐的方法。

我希望有更多知识的人能够提供更好的解决方法。

英文:

Based on my extensive experimentation, it appears that sp_execute_external_script doesn't follow symlinks (i.e. through the.egg-link file). Therefore, development mode installations will not work, whether you use setuptools, pip, pbr or anything else.

I even tried symlinking &lt;package_name&gt; folder as an OS symlink. Since I am on Windows, I used mklink /D command on Command Prompt to symlink /py/src/&lt;package_name&gt; inside site-packages. While the command goes through correctly, and I can see the symlinked folder in File Explorer, sp_execute_external_script fails to detect the package. Which tells me that there is probably something in sp_execute_external_script code that avoids traversing symbolic links.

I wonder if there is a way to make it traverse symbolic links.

The only workable solution is to develop a package's code under its own directory, so, in my case /py/src/&lt;package_name&gt;. Then, before running exec sp_execute_external_script @language=N&#39;python&#39;, @script=N&#39;...&#39; copy the &lt;package_name&gt; folder to the site-packages directory.

This is, sort of, equivalent to setup.py install, but bypasses the creation of intermediate files and directories. So I am going to stick with this simple--though odious--approach.

I am hoping somebody more knowledgeable would offer a better way to solve this problem.

huangapple
  • 本文由 发表于 2020年1月7日 01:35:21
  • 转载请务必保留本文链接:https://go.coder-hub.com/59616530.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定