如何在同一个项目中组织Python模块时进行分开打包?

huangapple go评论157阅读模式
英文:

How to do in order to package python modules separately when organized in the same project?

问题

我在Java和Maven方面有更丰富的经验,但在Python方面还是一个新手,所以我对于可能做什么以及什么不可能做没有真正的想法。

在Java中,使用 Maven,我们可以在一个文件(一个pom parent / 聚合模块)中管理依赖版本,然后在子模块的 pom.xml 中重新声明所需的依赖关系,同时省略由pom parent管理的版本。通过这种方式,我们可以 将子模块在物理上分开分布(例如:每个模块在一个机器上),每个模块只打包其所需的依赖项,这些依赖项在其自己的pom文件中定义。

现在回到 Python,我目前使用一个 setup.py 文件,据我理解,应该在根模块中。但是这样会将所有模块打包成一个库。

是否有一种更好的方法来管理模块及其依赖关系,并能够将每个(选择的)模块作为独立的软件包单独部署,带有其自己所需的库?我希望能够选择要单独打包的模块,我不希望每个包含 __init__.py 的文件夹都被单独打包。

目前我使用Pipenv来管理我的依赖关系,但如果它不能满足我上面所述的设计,我就准备放弃它。

谢谢

英文:

I am more experienced in Java and Maven and a real beginner in Python, so I don't really have an idea about what possible to do and what not.

In Java, using Maven we can manage dependencies versions in a file (a pom parent / the aggregator module), then re-declare in pom.xml of the sub-modules the necessary dependencies while omitting their version being managed by the pom parent. This way, we can distribute the sub-modules physically separately (Ex: every module in a machine) and every module would package only its required dependencies, the libraries that are defined in its own pom file.

Now back to Python, currently I use a setup.py file which, to my understanding, should be in the root module. But then it packages the whole modules as one library.

Is there a way to manage better the modules and their dependencies and be able to deploy every (chosen) module separately as an independent package with its own needed libraires? I want to be able to choose the modules to package separately, I don't want that every folder containing __init__.py to be packaged seprately.

Currently I use Pipenv to manage my dependencies, but I am ready to drop it if it doesn't satisfy the design I have explained above.

Thank you

答案1

得分: 5

# requirements.txt

在Java中,正如您所说,我们有我们的pom.xml。

在Python中,您有:类似以下内容的requirements.txt:

    # 无版本规范的要求 #
    nose
    nose-cov
    beautifulsoup4
    
    # 带有版本规范的要求 #
    docopt == 0.6.1             # 版本匹配。必须是版本0.6.1
    keyring >= 4.1.1            # 最低版本4.1.1
    coverage != 3.5             # 版本排除。除了版本3.5之外的任何版本
    Mopidy-Dirble ~= 1.1        # 兼容的发布。与>= 1.1,== 1.*相同

要安装,请运行以下命令:

    pip install -r requirements.txt

---
# Java/Maven中的依赖关系

使用Maven,我们有很好的依赖管理:

    org.acme.demo.springboot:acme-api:jar:1.0.0
    +- mysql:mysql-connector-java:jar:8.0.13:compile
    +- io.jsonwebtoken-jjwt:jar:0.9.1:compile
    |  \- com.fasterxml.jackson.core:jackson-databind:jar:2.9.7:compile
    |     +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.0:compile
    |     \- com.fasterxml.jackson.core:jackson-core:jar:2.9.7:compile
    +- com.jayway.jsonpath:json-path:jar:2.4.0:compile
    |  +- net.minidev:json-smart:jar:2.3:compile
    |  |  \- net.minidev:accessors-smart:jar:1.2:compile
    |  |     \- org.ow2.asm:asm:jar:5.0.4:compile
    |  \- org.slf4j:slf4j-api:jar:1.7.25:compile

例如:我的应用程序是**acme-api**,并且在其pom.xml中具有以下依赖项:

- **mysql:mysql-connector-java**,它不依赖任何内容
- **io.jsonwebtoken-jjwt** 依赖于jackson-databind、jackson-annotations和**jackson-core**

如果您查看**jackson-core**的源代码,您将找到另一个仅包含**jackson-core**所需依赖项的pom.xml。

因此,在Java/Maven中,任何源代码存储库(应用程序或库)**必须具有pom.xml**,我们可以在其中查找或下载所需的库。

---
# Python中的依赖关系

先前在Java/Maven中与任何源代码相关的pom.xml策略,在Python中不适用。

我查阅了一些公共库,发现其中没有**requirements.txt** o_O

- https://github.com/jupyter/jupyter_client
- https://github.com/requests/requests-oauthlib(这有一个requirements.txt,但未使用)

只有应用程序(如django)使用**requirements.txt**。

库使用**setup.py**而不是**requirements.txt**,所需的库在**setup.py**中硬编码:

    install_requires=["oauthlib>=3.0.0", "requests>=2.0.0"],
    extras_require={"rsa": ["oauthlib[signedtoken]>=3.0.0"]},

---
# 将requirements.txt视为pom.xml

为了标准化,您可以在任何库或应用程序中使用**requirements.txt**,并修改**setup.py**以从**requirements.txt**中读取值,而不是硬编码的**install_requires**。

    install_requires=["oauthlib>=3.0.0", "requests>=2.0.0"],

通过这样做,您将在任何Python源代码中找到一个**requirements.txt**,并且将更接近我们在Java中使用的Maven。
英文:

requirements.txt

In java, as you said, we have our pom.xml

In python you have: requirements.txt with content like this:

# Requirements without Version Specifiers #`
nose
nose-cov
beautifulsoup4

# Requirements with Version Specifiers #`
docopt == 0.6.1             # Version Matching. Must be version 0.6.1
keyring >= 4.1.1            # Minimum version 4.1.1
coverage != 3.5             # Version Exclusion. Anything except version 3.5
Mopidy-Dirble ~= 1.1        # Compatible release. Same as >= 1.1, == 1.*

To install, run this:

pip install -r requirements.txt

dependencies in java/maven

With maven we have a very good dependency management:

org.acme.demo.springboot:acme-api:jar:1.0.0
+- mysql:mysql-connector-java:jar:8.0.13:compile
+- io.jsonwebtoken-jjwt:jar:0.9.1:compile
|  \- com.fasterxml.jackson.core:jackson-databind:jar:2.9.7:compile
|     +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.0:compile
|     \- com.fasterxml.jackson.core:jackson-core:jar:2.9.7:compile
+- com.jayway.jsonpath:json-path:jar:2.4.0:compile
|  +- net.minidev:json-smart:jar:2.3:compile
|  |  \- net.minidev:accessors-smart:jar:1.2:compile
|  |     \- org.ow2.asm:asm:jar:5.0.4:compile
|  \- org.slf4j:slf4j-api:jar:1.7.25:compile

For example: My app is acme-api and has these dependencies in its pom.xml:

  • mysql:mysql-connector-java which depends of nothing
  • io.jsonwebtoken-jjwt wich depends of jackson-databind, jackson-annotations and jackson-core

If you look the jackson-core source code, you will find another pom.xml with just the dependencies required by jackson-core

So, in java/maven any source code repository must have the pom.xml (app or library) in which we can find or download the required libraries


dependencies in python

The previous strategy in java/maven related to pom.xml in any source code, is not used in python.

I reviewed a couple of public libraries and I don't find the requirements.txt inside of them o_O

Just apps use the requirements.txt like django.

libraries use the setup.py instead requirements.txt and the required libraries are harcoded inside setup.py:

install_requires=["oauthlib>=3.0.0", "requests>=2.0.0"],
extras_require={"rsa": ["oauthlib[signedtoken]>=3.0.0"]},

requirements.txt as pom.xml

In order to standardize, you could use the requirements.txt in any of your libraries or apps, modifying the setup.py to read values from requirements.txt instead of hardcoded install_requires

install_requires=["oauthlib>=3.0.0", "requests>=2.0.0"],

With this, you will find a requirements.txt in any python source code and you will be a little close to our Maven in Java

答案2

得分: 1

希望这个链接对你有所帮助。讨论了一种类似于Java中看到的模式。通常在Python中,我们会有单独的包彼此使用,并且它们彼此之间存在依赖关系(或者有一个伪主包,其中有一个将它们绑在一起的init文件)。

对于Java中的模式,一种简单的方法是在一个源代码控制仓库下拥有多个包。这篇文章详细解释了这一点。

英文:

Hopefully this should be helpful. Talks about a pattern similar to one seen in java. Usually in python we would have separate packages that use each other and say they are dependent on each other (or have one pseudo-master package with an init that ties it together).

For the java pattern, an easy way is to have multiple packages under one source control repo. The article explains it in detail.

答案3

得分: 1

没有任何东西能够完全按照你的要求实现(至少据我所知)。在Python中,我们管理项目或库的依赖的方式是使用requirements.txt和虚拟环境。这两者都可以管理项目作为整体的依赖关系。

要将项目中的单独模块分离出来,你可以将它们视为单独的项目,然后按照每个项目的需求进行操作。将依赖项放入requirements.txt中,或者如果你需要使用setup.py进行安装,则将其放入其中。

因此,对于每个你想要“分离”的“子库”,都需要创建一个不同的requirements.txt或setup.py。

有一个叫做pipreqs(可能还有其他类似的工具)的库,它可以扫描文件夹并自动生成requirements文件。我总是建议在安装任何库后立即手动编写requirements文件。

英文:

There is nothing that does exactly what you want (as far as I know, at least). In python the way we manage dependencies of a project or a library is using requirements.txt and virtual environments. Both of which manages the dependencies of the project as a whole.

To separate the individual modules out of the project you'd treat them as separate projects and do what every project would need. Make the requirements.txt or if you need to install it using setup.py put them inside that.

So, for every 'sub-library' you want to 'separate' make a different requirements.txt or setup.py.

There's this library called pipreqs (and possibly others) which scans a folder and generates the requirements file automatically. I'd always recommend hand-crafting the requirements file right after you install any library.

答案4

得分: 0

如果您想要打包您的Python代码,您可以查看poetry,尽管类似于Maven的多项目构建是无法实现的。

英文:

If Your looking to package your python code, you can look at poetry, although multi-project builds like maven are not achievable.

huangapple
  • 本文由 发表于 2020年5月29日 16:39:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/62081927.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定