安装测试文件使用pyproject.toml和setuptools

huangapple go评论67阅读模式
英文:

Installing test files with pyproject.toml and setuptools

问题

我正在将一个旧的Python项目迁移到基于新的pyproject.toml系统,但在安装测试所需的文件方面遇到了问题。在pyproject.toml中,我有以下设置:

[tool.setuptools]
package-data = {"my_pkg_name" = ["tests/*.sdf", "tests/*.urdf", "tests/*.xml", "tests/meshes/*.obj"]}

[build-system]
requires = ["setuptools>=43.0.0", "wheel"]
build-backend = "setuptools.build_meta"

使用pytest运行的测试需要package-data中描述的文件。但在构建和安装后,测试文件并不存在。如何才能安装这些文件?这个问题可能与https://stackoverflow.com/questions/7522250/how-to-include-package-data-with-setuptools-distutils有关,但事情已经改变,而且我宁愿不创建清单文件。

项目结构类似于:

.
├── LICENSE.txt
├── pyproject.toml
├── README.md
├── src
│   ├── my_pkg_name
│   │   ├── __init__.py
└── tests
    ├── ant.xml
    ├── humanoid.xml
    ├── __init__.py
    ├── kuka_iiwa.urdf
    ├── meshes
    │   ├── link_0.obj
    │   ├── link_1.obj
    │   ├── link_2.obj
    │   ├── link_3.obj
    │   ├── link_4.obj
    │   ├── link_5.obj
    │   ├── link_6.obj
    │   └── link_7.obj
    └── test_transform.py

pyproject.toml没有特定的包发现相关设置。

英文:

I'm migrating an old python project to the new pyproject.toml based system and am having trouble with getting files that are required by tests to install. Inside the pyproject.toml I have:

[tool.setuptools]
package-data = {"my_pkg_name" = ["tests/*.sdf", "tests/*.urdf", "tests/*.xml", "tests/meshes/*.obj"]}

[build-system]
requires = ["setuptools>=43.0.0", "wheel"]
build-backend = "setuptools.build_meta"

The tests that are run with pytest require the files described under package-data. After I build and install the build, the test files are not there. How do I get those files to be installed? https://stackoverflow.com/questions/7522250/how-to-include-package-data-with-setuptools-distutils may be related, but things have changed, and I would rather not have to create a manifest file.

The project structure looks something like:

.
├── LICENSE.txt
├── pyproject.toml
├── README.md
├── src
│   ├── my_pkg_name
│   │   ├── __init__.py
└── tests
    ├── ant.xml
    ├── humanoid.xml
    ├── __init__.py
    ├── kuka_iiwa.urdf
    ├── meshes
    │   ├── link_0.obj
    │   ├── link_1.obj
    │   ├── link_2.obj
    │   ├── link_3.obj
    │   ├── link_4.obj
    │   ├── link_5.obj
    │   ├── link_6.obj
    │   └── link_7.obj
    └── test_transform.py

The pyproject.toml has no specific package discovery related settings.

答案1

得分: 1

以下是翻译好的部分:

  • 我的问题是,这些文件不是默认找到的包的一部分,这些包只是位于“src”下的包。也不清楚是否应该安装测试文件 - 许多项目明确排除了测试文件的安装。

  • 回答如何安装测试文件的原始问题,解决方法是将“tests”作为一个包(就像我已经做过的那样),然后在设置工具中指定查找它的方式,如下所示:

    [tool.setuptools.packages.find]
    where = ["src", "tests"]
    
    [tool.setuptools.package-data]
    "*" = ["*.sdf", "*.urdf", "*.xml", "*.obj"]
    

    我相信package-data需要setuptools >= 61,你可以像这样指定:

    [build-system]
    requires = ["setuptools>=61.0.0", "wheel"]
    build-backend = "setuptools.build_meta"
    
  • 进一步回答由于无法找到测试文件而导致安装的测试失败的更深层次问题,我在GitHub操作工作流程中的解决方法是以可编辑的方式安装。具体来说:

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install -e .
        python -m pip install flake8 pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    

    这是因为在我的测试中,我根据模块的路径引用资源文件:

    open(os.path.join(cfg.TEST_DIR, "simple_arm.sdf"))
    

    cfg模块中:

    import os
    
    ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../'))
    TEST_DIR = os.path.join(ROOT_DIR, 'tests')
    

    编辑:更好的解决方法是基于测试脚本路径而不是包的模块路径引用测试文件。这允许我们仍然测试非可编辑安装的项目。为此,我改为在每个测试脚本中使用:

    TEST_DIR = os.path.dirname(__file__)
    

    并替换对cfg的引用。

英文:

Looking more into this, specifically: https://setuptools.pypa.io/en/stable/userguide/datafiles.html#non-package-data-files
my problem is that these files are not part of the default found packages which are just the ones under src. It is also not clear whether test files should be installed or not - many projects explicitly exclude test files from installation.

To answer the original question of how to install the test files,
the solution is to make tests a package (as I already have above), then specify to find it with setup tools like so:

[tool.setuptools.packages.find]
where = ["src", "tests"]

[tool.setuptools.package-data]
"*" = ["*.sdf", "*.urdf", "*.xml", "*.obj"]

I believe package-data requires setuptools >= 61 which you can specify like:

[build-system]
requires = ["setuptools>=61.0.0", "wheel"]
build-backend = "setuptools.build_meta"

Further answering the deeper issue of installed tests failing due to not being able to find test files, the solution in my github actions workflow for doing automated testing on commits was to install in editable mode. Specifically:

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install -e .
        python -m pip install flake8 pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

This is because in my tests, I refer to resource files based on a module's path:

open(os.path.join(cfg.TEST_DIR, "simple_arm.sdf"))

in the cfg module:

import os

ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../'))
TEST_DIR = os.path.join(ROOT_DIR, 'tests')

EDIT: A better solution is to reference the test files based on the test script path rather than the package's module's path. This allows us to still test the non-editable installed project. To do this, I have instead

TEST_DIR = os.path.dirname(__file__)

in each test script, and replace references to cfg.

huangapple
  • 本文由 发表于 2023年2月10日 12:46:38
  • 转载请务必保留本文链接:https://go.coder-hub.com/75407052.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定