如何创建带有子模块的Python C++扩展，可以被导入。

huangapple
go评论81阅读模式
英文:

How to create python C++ extension with submodule that can be imported

问题

I will provide the translation of the code and text you provided. Here it is:

我正在为Python创建一个C++扩展。它创建了一个名为`parent`的模块，其中包含一个子模块`child`。`child`有一个方法`hello()`。如果我像这样调用它，它可以正常工作：

```python
import parent
parent.child.hello()
> 'Hi, World!'

但是，如果我尝试导入我的函数，它会失败：

import parent
from parent.child import hello
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ModuleNotFoundError: No module named 'parent.child'; 'parent' is not a package

parent.child
> <module 'child'>

这是我的代码：

setup.py

from setuptools import Extension, setup
  
# 定义扩展模块
extension_mod = Extension('parent',
                          sources=['custom.cc'])

# 定义设置参数
setup(name='parent',
      version='1.0',
      description='Python的C++扩展模块。',
      ext_modules=[extension_mod],
      )

和我的 custom.cc

#include <Python.h>
#include <string>

std::string hello() {
    return "Hi, World!";
}

static PyObject* hello_world(PyObject* self, PyObject* args) {
    return PyUnicode_FromString(hello().c_str());
}

static PyMethodDef ParentMethods[] = {
    {nullptr, nullptr, 0, nullptr}
};

static PyMethodDef ChildMethods[] = {
    {"hello", hello_world, METH_NOARGS, ""},
    {nullptr, nullptr, 0, nullptr}
};

static PyModuleDef ChildModule = {
    PyModuleDef_HEAD_INIT,
    "child",
    "父模块的子模块。",
    -1,
    ChildMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr

};

static PyModuleDef ParentModule = {
    PyModuleDef_HEAD_INIT,
    "parent",
    "Python的C++扩展模块。",
    -1,
    ParentMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr
};

PyMODINIT_FUNC PyInit_parent(void) {
    PyObject* parent_module = PyModule_Create(&ParentModule);
    if (!parent_module) {
        return nullptr;
    }
    PyObject* child_module = PyModule_Create(&ChildModule);
    if (!child_module) {
        Py_DECREF(parent_module);
        return nullptr;
    }

    PyModule_AddObject(parent_module, "child", child_module);

    return parent_module;
}

我使用 python setup.py build install 进行安装和构建。

那么，如何确保我的 parent 是一个包？

虽然我的代码只是一个示例，但实际上我希望在C++级别定义这两个模块。我不想将它们拆分为多个模块，因为它们共享一些C++代码。

我希望有类似于这个答案的方法：Python extension with multiple modules。

英文:

I'm creating a C++ extension for python. It creates a module parent that contains a sub-module child. The child has one method hello(). It works fine if I call it as

import parent
parent.child.hello()
&gt; &#39;Hi, World!&#39;

If I try to import my function it fails

import parent
from parent.child import hello
&gt; Traceback (most recent call last):
&gt; File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
&gt; ModuleNotFoundError: No module named &#39;parent.child&#39;; &#39;parent&#39; is not a package

parent.child
&gt; &lt;module &#39;child&#39;&gt;

here is my code
setup.py

from setuptools import Extension, setup
  
# Define the extension module
extension_mod = Extension(&#39;parent&#39;,
                          sources=[&#39;custom.cc&#39;])

# Define the setup parameters
setup(name=&#39;parent&#39;,
      version=&#39;1.0&#39;,
      description=&#39;A C++ extension module for Python.&#39;,
      ext_modules=[extension_mod],
      )

and my custom.cc

#include &lt;Python.h&gt;
#include &lt;string&gt;

std::string hello() {
    return &quot;Hi, World!&quot;;
}

static PyObject* hello_world(PyObject* self, PyObject* args) {
    return PyUnicode_FromString(hello().c_str());
}

static PyMethodDef ParentMethods[] = {
    {nullptr, nullptr, 0, nullptr}
};

static PyMethodDef ChildMethods[] = {
    {&quot;hello&quot;, hello_world, METH_NOARGS, &quot;&quot;},
    {nullptr, nullptr, 0, nullptr}
};

static PyModuleDef ChildModule = {
    PyModuleDef_HEAD_INIT,
    &quot;child&quot;,
    &quot;A submodule of the parent module.&quot;,
    -1,
    ChildMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr

};

static PyModuleDef ParentModule = {
    PyModuleDef_HEAD_INIT,
    &quot;parent&quot;,
    &quot;A C++ extension module for Python.&quot;,
    -1,
    ParentMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr
};

PyMODINIT_FUNC PyInit_parent(void) {
    PyObject* parent_module = PyModule_Create(&amp;ParentModule);
    if (!parent_module) {
        return nullptr;
    }
    PyObject* child_module = PyModule_Create(&amp;ChildModule);
    if (!child_module) {
        Py_DECREF(parent_module);
        return nullptr;
    }

    PyModule_AddObject(parent_module, &quot;child&quot;, child_module);

    return parent_module;
}

I install and build with python setup.py build install.

So, how do I make sure that my parent is a package?

My code is a toy example but I actually want both modules defined on C++ level. I don't want to split them into several modules - since they are sharing some C++ code.

I'm hoping for something similar to approach of this answer Python extension with multiple modules

答案1

得分: 2

以下是翻译好的部分：

  1. "The trick is to use packages option." -> "诀窍是使用 packages 选项。"

  2. "This is the standard way, as many popular C/C++ python packages like tensorflow, pyarrow, meinheld, and greenlet are using it." -> "这是标准方式，因为许多流行的C/C++ Python包，如tensorflow，pyarrow，meinheld和greenlet都在使用它。"

  3. "Also it is recommended by the Python Packaging Authority." -> "这也是Python Packaging Authority推荐的方式。"

  4. "The only important points are adding the packages keyword option to setup and naming your module as &quot;parent.child&quot;:" -> "唯一重要的是将 packages 关键字选项添加到设置中，并将模块命名为 &quot;parent.child&quot;："

  5. "That's it. No need to hack around. Also note that ext_modules is not always required, because you can build the C/C++ assets of sub modules separately." -> "就是这样。不需要绕过。还要注意 ext_modules 不总是必需的，因为您可以单独构建子模块的C/C++资源。"

  6. "It is important to add __init__.py under the "parent" folder because, otherwise, it would not be treated as a package." -> "在"parent"文件夹下添加 __init__.py 很重要，否则它将不被视为包。"

  7. "When you say packages = [&#39;foo&#39;] in your setup script, you are promising that the Distutils will find a file foo/__init__.py relative to the directory where your setup script lives." -> "当您在设置脚本中说 packages = [&#39;foo&#39;] 时，您承诺Distutils会在设置脚本所在目录的相对位置找到 foo/__init__.py 文件。"

  8. "If your parent package is located in another directory, use package_dir = {&#39;parent&#39;: &#39;path/to/your/parent_package&#39;}." -> "如果您的父包位于另一个目录中，请使用 package_dir = {&#39;parent&#39;: &#39;path/to/your/parent_package&#39;}。"

  9. "For huge packages like tensorflow, it is also common to use the helper function find_packages to automatically update the packages' list." -> "对于像tensorflow这样的大型包，通常也会使用助手函数 find_packages 来自动更新包的列表。"

  10. "Later, when your parent package grows bigger, sub-modules can be split into separated projects. New child project can refer to the parent using ext_package=[&#39;parent&#39;]." -> "后来，当您的父包变得更大时，子模块可以拆分为独立的项目。新的子项目可以使用 ext_package=[&#39;parent&#39;] 来引用父项目。"

英文:

The trick is to use packages option.

This is the standard way, as many popular C/C++ python packages like
tensorflow,
pyarrow
meinheld and
greenlet are using it.
Also it is recommended by the Python Packaging Authority.

The only important points are adding the packages keyword option to setup and naming your module as &quot;parent.child&quot;:

# Define the extension module
extension_mod = Extension(&#39;parent.child&#39;, ... )

setup(
  name=&#39;parent&#39;, 
  packages=[&quot;parent&quot;], 
  ext_modules=[extension_mod], ... 
)

That's it. No need to hack around. Also note that ext_modules is not always required, because you can build the C/C++ assets of sub modules separately. See the setup scripts of tensorflow and pyarrow for examples.

It is important to add __init__.py under the "parent" folder, because , otherwise, it would not be treated as a package.

> when you say packages = [&#39;foo&#39;] in your setup script, you are promising that the Distutils will find a file foo/__init__.py relative to the directory where your setup script lives.

If your parent package is located in another directory, use package_dir = {&#39;parent&#39;: &#39;path/to/your/parent_package&#39;}.
For huge packages like tensorflow, it is also common to use the helper function find_packages to automatically update the packages' list.

Later, when your parent package grows bigger, sub-modules can be split into separated projects. New child project can refer to the parent using ext_package=[&#39;parent&#39;].

答案2

得分: 1

选项1

如果你只想将它变成一个包。前往 Path\to\Python\Python311\Lib\site-packages\<package_name>。
根据你的源代码，package_name 看起来可能是这样的：parent-1.0-py3.11-win-amd64.egg。将 <package_name> 重命名为 parent，然后在那里创建一个 __init__.py 文件，就完成了。

结果如下：

└── site-packages\
    └── parent\
        ├── __init__.py
        └── parent.py

但是，这样做会达到你想要的效果吗？我不这么认为

# 下面的行可以工作
from parent.parent import child
child.hello()
# 下面的行仍然无法工作，因为 parent.parent 是一个模块
from parent.parent.child import hello

第二个导入不起作用，因为这个 package 中的 python 文件只有 parent.py。

选项2

要执行 from parent.child import hello，你必须创建一个名为 child 的单独模块，其中包含函数 hello，然后按照 选项1 中解释的制作包的过程进行操作。

因此，在编译后，你的结果看起来像这样：

└── site-packages\
    └── parent\
        ├── __init__.py
        └── child.py

选项3

或者，如果你不想重新组织并且想要更好的导入方式，可以使用：

from parent import child
child.hello()
#或者
hello = child.hello
hello()

选项4（最后一种选择）

>我的代码只是一个玩具示例，但实际上我想要在 C++ 层面定义这两个模块。我不想将它们拆分为多个模块-因为它们共享一些 C++ 代码。

这更像是手动方式
setup.py

from setuptools import Extension, setup
  
# 定义扩展模块
extension_mod = Extension('main',
                          sources=['custom.cc'])

# 定义设置参数
setup(name='parent',
      version='1.0',
      description='A C++ extension module for Python.',
      ext_modules=[extension_mod],
      )

创建如下结构：

└── site-packages\
    └── parent\
        ├── __init__.py
        ├── main.py
        └── child.py

init.py

from . import main

child.py

from . import main
hello = main.child.hello

现在你可以这样做：

>>> from parent.child import hello
>>> hello()
'Hi, World!'

我个人建议如果没有严格的要求，就使用第三个选项，但如果无法重新组织并需要确切的结构，你可以尝试实施最后一个选项。

英文:

Option1

If you just want to make it a package. Go to Path\to\Python\Python311\Lib\site-packages\&lt;package_name&gt;.
By looking at your source code package_name might look like this: parent-1.0-py3.11-win-amd64.egg. Rename the &lt;package_name&gt; to parent and then create a __init__.py file there and it's done.

Results in:
<pre>
└── site-packages
└── parent
├── init.py
└── parent.py
</pre>
But, will it do what you want? I don't think so

#The line below works
from parent.parent import child
child.hello()
#The line below will still not work as parent.parent is a module
from parent.parent.child import hello

The second import doesn't work because the python file in this package is only parent.py.

Option2

To do from parent.child import hello you have to make an individual module child which contains function hello then follow the package making process explained in Option 1.

So your result after compilation looks like this:
<pre>
└── site-packages
└── parent
├── init.py
└── child.py
</pre>

Option 3

Or if don't want to restructure and want a better way to import, use:

from parent import child
child.hello()
#OR
hello = child.hello
hello()

Option 4 (Last Option)

>My code is a toy example but I actually want both modules defined on C++ level. I don't want to split them into several modules - since they are sharing some C++ code.

This is more of a manual way<br>
setup.py

from setuptools import Extension, setup
  
# Define the extension module
extension_mod = Extension(&#39;main&#39;,
                          sources=[&#39;custom.cc&#39;])

# Define the setup parameters
setup(name=&#39;parent&#39;,
      version=&#39;1.0&#39;,
      description=&#39;A C++ extension module for Python.&#39;,
      ext_modules=[extension_mod],
      )

Create a structure like this:
<pre>
└── site-packages
└── parent
├── init.py
├── main.py
└── child.py
</pre>
init.py

from . import main

child.py

from . import main
hello = main.child.hello

Now you can do this:

&gt;&gt;&gt; from parent.child import hello
&gt;&gt;&gt; hello()
&#39;Hi, World!&#39;

I personally recommend the third option if there are no strict requirements but if you can't restructure and need the exact structure you may try implementing the last one.

答案3

得分: 1

I'm not sure if this is the standard way or not but works the same way you want. please correct me if it's not correct.
我不确定这是否是标准方法，但它与您想要的方式相同。如果不正确，请纠正我。

I deleted the parts related to parent in your cpp code and made child.cpp. This is the structure of the package dir. For setup.py I got the idea from this link
我删除了您的cpp代码中与parent相关的部分，并创建了child.cpp。这是包目录的结构。关于setup.py，我从这个链接中得到了灵感。

This is child.cpp:
这是child.cpp：

#include <Python.h>
#include <string>

std::string hello() {
    return "Hi, World!";
}

static PyObject* hello_world(PyObject* self, PyObject* args) {
    return PyUnicode_FromString(hello().c_str());
}

static PyMethodDef ChildMethods[] = {
    {"hello", hello_world, METH_NOARGS, ""},
    {nullptr, nullptr, 0, nullptr}
};

static PyModuleDef ChildModule = {
    PyModuleDef_HEAD_INIT,
    "child",
    "A submodule of the parent module.",
    -1,
    ChildMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr
};

PyMODINIT_FUNC PyInit_child(void) {
    PyObject* child_module = PyModule_Create(&ChildModule);
    if (!child_module) {
        return nullptr;
    }

    return child_module;
}

parent/__init__.py is:
parent/__init__.py是：

from .child import *

And changed setup.py to this:
并将setup.py更改为：

from setuptools import Extension, setup
  
# Define the extension module
extension_mod = Extension('parent.child',
                          sources=['parent/child/child.cpp'])

# Define the setup parameters
setup(name='parent',
      packages=["parent"],
      version='1.0',
      description='A C++ extension module for Python.',
      ext_modules=[extension_mod],
      )

You can also check this link
您还可以查看此链接。

英文:

I'm not sure if this is the standard way or not but works the same way you want. please correct me if it's not correct.

I deleted the parts related to parent in your cpp code and made child.cpp. This is the structure of the package dir. For setup.py I got the idea from this link

|package
|- parent/
|- - __init__.py
|- - child/
|- - - child.cpp
|- setup.py

This is child.cpp:

#include &lt;Python.h&gt;
#include &lt;string&gt;

std::string hello() {
    return &quot;Hi, World!&quot;;
}

static PyObject* hello_world(PyObject* self, PyObject* args) {
    return PyUnicode_FromString(hello().c_str());
}

static PyMethodDef ChildMethods[] = {
    {&quot;hello&quot;, hello_world, METH_NOARGS, &quot;&quot;},
    {nullptr, nullptr, 0, nullptr}
};

static PyModuleDef ChildModule = {
    PyModuleDef_HEAD_INIT,
    &quot;child&quot;,
    &quot;A submodule of the parent module.&quot;,
    -1,
    ChildMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr

};


PyMODINIT_FUNC PyInit_child(void) {
    PyObject* child_module = PyModule_Create(&amp;ChildModule);
    if (!child_module) {
        return nullptr;
    }

    return child_module;
}

parent/__init__.py is:

from .child import *

And changed setup.py to this:

from setuptools import Extension, setup
  
# Define the extension module
extension_mod = Extension(&#39;parent.child&#39;,
                          sources=[&#39;parent/child/child.cpp&#39;])

# Define the setup parameters
setup(name=&#39;parent&#39;,
      packages=[&quot;parent&quot;],
      version=&#39;1.0&#39;,
      description=&#39;A C++ extension module for Python.&#39;,
      ext_modules=[extension_mod],
      )

You can also check this link

答案4

得分: 1

我必须承认，在想到这个方法之前，我尝试了一些事情（包括在setup.py中添加packages=["parent"],（顺便说一下，这是不推荐使用的：[SO]: 'setup.py install is deprecated'警告每次我在VSCode中打开终端时都会出现），但都没有成功：

  1. 从（被引用的）[Python.Docs]: importlib - 直接导入源文件获取行为。

  2. 复制它，至少是相关部分（例如在PyInit_parent中添加模块(s)到[Python.Docs]: sys.modules - 感谢@DavisHerring提供的更短的变体）。

    注意事项：

    • 尽管它可以工作，但我有一种感觉，这可能不是最正统的方法，所以有些人可能会将其视为一种解决方法（gainarie）- 在某些情况下，它可能会表现得出乎意料。

    • @TODOs 不一定是必需的（从功能上讲，没有它们也可以工作），但我将它们放在那里是为了遵循标准（它们是我之前尝试过的东西的一部分）。可能需要其他模块常量

    • 用于计算命名空间名称的函数设计为嵌套级别为1（最大）。如果需要更高级别的（例如parent.child.grandchild），它们必须进行调整（也许可以使用树形结构中的字符串进行工作），无论如何，你已经领会了要领。

我修改了你的代码以得到一个可工作的示例。

custom.cc：

#include <string>
#include <vector>

#include <Python.h>


using std::string;
using std::vector;

typedef vector<string> StrVec;
typedef vector<PyObject*> PPOVec;

static const string dotStr = ".";
static const string parentStr = "parent";
static const string childStr = "child";


static string join(const string &outer, const string &inner, const string &sep = dotStr)
{
    return outer + sep + inner;
}


static StrVec join(const string &outer, const StrVec &inners, const string &sep = dotStr)
{
    StrVec ret;
    for (const string &inner : inners) {
        ret.emplace_back(join(outer, inner, sep));
    }
    return ret;
}


static int tweakModules(const string &pkg, const StrVec &names, const PPOVec &mods)
{
    PyObject *dict = PyImport_GetModuleDict();
    if (!dict) {
        return -1;
    }
    int idx = 0;
    for (StrVec::const_iterator it = names.cbegin(); it != names.cend(); ++it, ++idx) {
        if (PyDict_SetItemString(dict, join(pkg, *it).c_str(), mods[idx])) {
            return -2;
        }
    }
    return 0;
}


static void decrefMods(const PPOVec &mods)
{
    for (const PyObject *mod : mods) {
        Py_XDECREF(mod);
    }
}


static std::string hello()
{
    return "Hi, World!";
}

static PyObject* py_hello(PyObject *self, PyObject *args)
{
    return PyUnicode_FromString(hello().c_str());
}


static PyMethodDef parentMethods[] = {
    {nullptr, nullptr, 0, nullptr}
};

static PyMethodDef childMethods[] = {
    {"hello", py_hello, METH_NOARGS, ""},
    {nullptr, nullptr, 0, nullptr}
};


static string childFQ = join(parentStr, childStr);

static PyModuleDef childDef = {
    PyModuleDef_HEAD_INIT,
    //childStr.c_str(),
    childFQ.c_str(),  // @TODO - cfati: Module FQName (rigorousity's sake)
    "A submodule of the parent module.",
    -1,
    childMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr
};


static PyModuleDef parentDef = {
    PyModuleDef_HEAD_INIT,
    parentStr.c_str(),
    "A C++ extension module for Python.",
    -1,
    parentMethods,
    nullptr,
    nullptr,
    nullptr,
    nullptr
};


PyMODINIT_FUNC PyInit_parent()
{
    PyObject *parentMod = PyModule_Create(&parentDef);
    if (!parentMod) {
        return nullptr;
    }
    PyObject *childMod = PyModule_Create(&childDef);
    if (!childMod) {
        Py_XDECREF(parentMod);
        return nullptr;
    }

    // @TODO - cfati: Add modules and their names here (to be handled automatically)
    StrVec childrenStrs = {childStr};
    PPOVec mods = {childMod};

    if (tweakModules(parentStr, childrenStrs, mods) < 0) {
        decrefMods(mods);
        Py_XDECREF(parentMod);
        return nullptr;
    }

    int idx = 0;
    for (StrVec::const_iterator it = childrenStrs.cbegin(); it != childrenStrs.cend(); ++it, ++idx) {
        if (PyModule_AddObject(parentMod, it->c_str(), mods[idx]) < 0) {  // @TODO - cfati: Add modules under parent
            decrefMods(mods);
            Py_XDECREF(parentMod);
            return nullptr;
        }
    }

    mods.push_back(parentMod);
    for (PyObject *mod : mods) {
        if (PyModule_AddStringConstant(mod, "__package__", parentStr.c_str()) < 0) {  // @TODO - cfati: Set __package__ for each module
            decrefMods(mods);
            return nullptr;
        }
    }
    return parentMod;
}

输出：

( py_pc064_03.10_test0 ) [cfati@cfati-5510-0:/mnt/e/Work/Dev/StackExchange/StackOverflow/q076222409]> ~/sopr.sh
### 设置较短的提示以便在粘贴到StackOverflow（或其他）页面时更好地适应###
[064位提示]>
[064位提示]> ls
custom.cc setup.py
[064位提示]>
[064位提示]> python setup.py build
运行构建
运行构建扩展
正在构建 'parent' 扩展
创建build
创建build/temp.linux
<details>
<summary>英文:</summary>
I&#39;ll have to admit that I (unsuccessfully) tried a few things (including adding `packages=[&quot;parent&quot;],` in *setup.py* (which is deprecated *BTW*: [\[SO\]: &#39;setup.py install is deprecated&#39; warning shows up every time I open a terminal in VSCode](https://stackoverflow.com/q/73257839/4788546))) before thinking of this:
1. Take the behavior from (referenced) [\[Python.Docs\]: importlib - Importing a source file directly](https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly)
2. Replicate it, at least the relevant parts (like adding the module(s) in [\[Python.Docs\]: sys.modules](https://docs.python.org/3/library/sys.html#sys.modules) - thanks @DavisHerring for the shorter variant) in *PyInit\_parent*
**Notes**:
- Although it works, I have a feeling that this is not the most orthodox way, so some might see it as a workaround (*gainarie*) - there is a chance that in some cases it might behave unexpectedly
- The *@TODO*s are not necessarily required (functionally, it works without them), but I placed them there in order to be &quot;by the book&quot; (and they are part of the things I tried earlier). There&#39;s the possibility that some other module constants will be required
- Functions computing namespace names are designed for a nesting level of 1 (maximum). If you need higher ones (*e.g.* *parent.child.grandchild*) they have to be adapted (maybe to work with strings organized in tree structures), anyway you&#39;ve got the gist
I modified your code to a working example.
*custom.cc*:
```cpp
#include &lt;string&gt;
#include &lt;vector&gt;
#include &lt;Python.h&gt;
using std::string;
using std::vector;
typedef vector&lt;string&gt; StrVec;
typedef vector&lt;PyObject*&gt; PPOVec;
static const string dotStr = &quot;.&quot;;
static const string parentStr = &quot;parent&quot;;
static const string childStr = &quot;child&quot;;
static string join(const string &amp;outer, const string &amp;inner, const string &amp;sep = dotStr)
{
return outer + sep + inner;
}
static StrVec join(const string &amp;outer, const StrVec &amp;inners, const string &amp;sep = dotStr)
{
StrVec ret;
for (const string &amp;inner : inners) {
ret.emplace_back(join(outer, inner, sep));
}
return ret;
}
static int tweakModules(const string &amp;pkg, const StrVec &amp;names, const PPOVec &amp;mods)
{
PyObject *dict = PyImport_GetModuleDict();
if (!dict) {
return -1;
}
int idx = 0;
for (StrVec::const_iterator it = names.cbegin(); it != names.cend(); ++it, ++idx) {
if (PyDict_SetItemString(dict, join(pkg, *it).c_str(), mods[idx])) {
return -2;
}
}
return 0;
}
static void decrefMods(const PPOVec &amp;mods)
{
for (const PyObject *mod : mods) {
Py_XDECREF(mod);
}
}
static std::string hello()
{
return &quot;Hi, World!&quot;;
}
static PyObject* py_hello(PyObject *self, PyObject *args)
{
return PyUnicode_FromString(hello().c_str());
}
static PyMethodDef parentMethods[] = {
{nullptr, nullptr, 0, nullptr}
};
static PyMethodDef childMethods[] = {
{&quot;hello&quot;, py_hello, METH_NOARGS, &quot;&quot;},
{nullptr, nullptr, 0, nullptr}
};
static string childFQ = join(parentStr, childStr);
static PyModuleDef childDef = {
PyModuleDef_HEAD_INIT,
//childStr.c_str(),
childFQ.c_str(),  // @TODO - cfati: Module FQName (rigorousity&#39;s sake)
&quot;A submodule of the parent module.&quot;,
-1,
childMethods,
nullptr,
nullptr,
nullptr,
nullptr
};
static PyModuleDef parentDef = {
PyModuleDef_HEAD_INIT,
parentStr.c_str(),
&quot;A C++ extension module for Python.&quot;,
-1,
parentMethods,
nullptr,
nullptr,
nullptr,
nullptr
};
PyMODINIT_FUNC PyInit_parent()
{
PyObject *parentMod = PyModule_Create(&amp;parentDef);
if (!parentMod) {
return nullptr;
}
PyObject *childMod = PyModule_Create(&amp;childDef);
if (!childMod) {
Py_XDECREF(parentMod);
return nullptr;
}
// @TODO - cfati: Add modules and their names here (to be handled automatically)
StrVec childrenStrs = {childStr};
PPOVec mods = {childMod};
if (tweakModules(parentStr, childrenStrs, mods) &lt; 0) {
decrefMods(mods);
Py_XDECREF(parentMod);
return nullptr;
}
int idx = 0;
for (StrVec::const_iterator it = childrenStrs.cbegin(); it != childrenStrs.cend(); ++it, ++idx) {
if (PyModule_AddObject(parentMod, it-&gt;c_str(), mods[idx]) &lt; 0) {  // @TODO - cfati: Add modules under parent
decrefMods(mods);
Py_XDECREF(parentMod);
return nullptr;
}
}
mods.push_back(parentMod);
for (PyObject *mod : mods) {
if (PyModule_AddStringConstant(mod, &quot;__package__&quot;, parentStr.c_str()) &lt; 0) {  // @TODO - cfati: Set __package__ for each module
decrefMods(mods);
return nullptr;
}
}
return parentMod;
}

Output:

> lang-bash
&gt; (py_pc064_03.10_test0) [cfati@cfati-5510-0:/mnt/e/Work/Dev/StackExchange/StackOverflow/q076222409]&gt; ~/sopr.sh
&gt; ### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
&gt;
&gt; [064bit prompt]&gt;
&gt; [064bit prompt]&gt; ls
&gt; custom.cc setup.py
&gt; [064bit prompt]&gt;
&gt; [064bit prompt]&gt; python setup.py build
&gt; running build
&gt; running build_ext
&gt; building &#39;parent&#39; extension
&gt; creating build
&gt; creating build/temp.linux-x86_64-cpython-310
&gt; x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/cfati/Work/Dev/VEnvs/py_pc064_03.10_test0/include -I/usr/include/python3.10 -c custom.cc -o build/temp.linux-x86_64-cpython-310/custom.o
&gt; custom.cc:24:15: warning: ‘StrVec join(const string&amp;, const StrVec&amp;, const string&amp;)’ defined but not used [-Wunused-function]
&gt; 25 | static StrVec join(const string &amp;outer, const StrVec &amp;inners, const string &amp;sep = dotStr)
&gt; | ^~~~
&gt; creating build/lib.linux-x86_64-cpython-310
&gt; x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-x86_64-cpython-310/custom.o -L/usr/lib/x86_64-linux-gnu -o build/lib.linux-x86_64-cpython-310/parent.cpython-310-x86_64-linux-gnu.so
&gt; [064bit prompt]&gt;
&gt; [064bit prompt]&gt; ls
&gt; build custom.cc setup.py
&gt; [064bit prompt]&gt; ls build/lib.linux-x86_64-cpython-310/
&gt; parent.cpython-310-x86_64-linux-gnu.so
&gt; [064bit prompt]&gt;
&gt; [064bit prompt]&gt; PYTHONPATH=${PYTHONPATH}:build/lib.linux-x86_64-cpython-310 python -c &quot;from parent.child import hello;print(hello());print(\&quot;Done.\n\&quot;)&quot;
&gt; Hi, World!
&gt; Done.
&gt;

References (that might be useful):

答案5

得分: 1

以下是翻译好的部分：

  1. 在扩展内部执行此操作只是模拟导入系统将其识别为包的模块的行为的“简单”问题。（根据上下文，也许更好的方式是提供一个从外部执行相同操作的导入钩子。）只需进行一些更改：

  2. 使子模块的名称为 "parent.child"。

  3. 将 "parent" 设为包：

    PyModule_AddObject(parent_module, "__path__", PyList_New(0));
PyModule_AddStringConstant(parent_module, "__package__", "parent");

  4. 将 "child" 设为该包的成员：

    PyModule_AddStringConstant(child_module, "__package__", "parent");

  5. 更新 sys.modules，就像 Python 执行导入操作一样：

    PyDict_SetItemString(PyImport_GetModuleDict(), "parent.child", child_module);

当然，这些调用中的一些可能会失败；请注意，如果 PyModule_AddObject 失败，您必须放弃对正在添加的对象的引用（出于明智起见，这里我没有这样做）。

英文:

Doing this from within the extension is a "simple" matter of emulating the behavior for modules that the import system recognizes as packages. (Depending on context, it might be nicer to provide an import hook that did the same thing from the outside.) Just a few changes are needed:

  1. Make the name of the child &quot;parent.child&quot;.
  2. Make parent a package: 
    PyModule_AddObject(parent_module, &quot;__path__&quot;, PyList_New(0));
PyModule_AddStringConstant(parent_module, &quot;__package__&quot;, &quot;parent&quot;);
  3. Make child a member of that package: 
    PyModule_AddStringConstant(child_module, &quot;__package__&quot;, &quot;parent&quot;);
  4. Update sys.modules as Python would if it had performed the import: 
    PyDict_SetItemString(PyImport_GetModuleDict(), &quot;parent.child&quot;, child_module);

Of course, several of these calls can fail; note that if PyModule_AddObject fails you have to drop the reference to the object being added (which I very much did not do here for clarity).

huangapple
  • 本文由 发表于 2023年5月11日 04:41:22
  • 转载请务必保留本文链接：https://go.coder-hub.com/76222409.html
  • c++
  • cpython
  • python
  • python-extensions
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定