从一个Cython生成的.so文件中在C++代码中调用一个函数。

huangapple go评论71阅读模式
英文:

Call a function from a cython generated .so file inside a c++ code

问题

我的目标是从C++中调用Python函数。这些Python函数必须使用Cython编译为.so文件。.so文件必须是与C++程序通信的那个。

在开始之前:

我使用Ubuntu,使用miniconda和Python 3.9进行工作。
我在一个像这样的文件夹中(~/exp):

  • exp
    • exp
      • init.py
      • main.py
  • setup.py
  • run.cpp
  • run.py

我将main.py翻译为main.pyx,该文件包含以下代码:

def add(a, b):
    return a + b

def entry_point():
    print(add(21, 21))

我使用Cython编译了这个脚本,并得到了一个.so文件,使用以下setup.py脚本:

from setuptools import setup
from Cython.Build import cythonize

setup(
   name='exp',
   ext_modules=cythonize("exp/main.pyx"),
   libraries=[('python3.9', {'include_dirs': ["~/miniconda3/include/python3.9"]})],
   library_dirs=['~/miniconda3/lib']
)

以及以下命令:

python3 setup.py build_ext --inplace

现在我在~/exp/exp目录中有一个main.cpython-39-x86_64-linux-gnu.so文件。

当我运行run.py时,它包含以下内容:

from exp.main import entry_point

if __name__ == "__main__":
    entry_point()

我得到了正常的行为:它返回42。

现在,出现了一些问题:

我编译了我的run.cpp源代码,其中包含以下内容:

#include <iostream>
#include <dlfcn.h>
#include "Python.h"

int main(int argc, char *argv[]) {
   setenv("PYTHONPATH",".",1);
   Py_Initialize();
   PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
   // Initialize the Python Interpreter
   Py_Initialize();

   // Build the name object
   pName = PyUnicode_FromString((char*)"exp/main");

   // Load the module object
   pModule = PyImport_Import(pName);

   // pDict is a borrowed reference 
   pDict = PyModule_GetDict(pModule);

   // pFunc is also a borrowed reference 
   pFunc = PyDict_GetItemString(pDict, (char*)"entry_point");

   Py_DECREF(pValue);

   // Clean up
   Py_DECREF(pModule);
   Py_DECREF(pName);
   Py_Finalize();
}

使用以下命令:

g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run.o -ldl

然后执行:./run.o

最终出现了以下错误:

Segmentation fault (core dumped)

我尝试过使用dlopen,但也没有成功。也许我漏掉了一些东西,任何帮助都将不胜感激。

谢谢 从一个Cython生成的.so文件中在C++代码中调用一个函数。

英文:

My goal is to call python functions from C++. These python function must be compiled with cython in a .so file. The .so file must be the one that communicate with the c++ program.

Before all:

I am on Ubuntu, I am working with miniconda with python3.9.
I am in a folder (~/exp) made like this:

  • exp
    • exp
      • __ init __.py
      • main.py
  • setup.py
  • run.cpp
  • run.py

I translate the main.py to main.pyx, the file contains this code:

def add(a, b):
    return a+b

def entry_point():
    print(add(21,21))

I compiled this script with cython and obtained a .so file, with this setup.py script:

from setuptools import setup
from Cython.Build import cythonize

setup(
   name=&#39;exp&#39;,
   ext_modules=cythonize(&quot;exp/main.pyx&quot;),
   libraries=[(&#39;python3.9&#39;, {&#39;include_dirs&#39;: [&quot;~/miniconda3/include/python3.9&quot;]})],
   library_dirs=[&#39;~/miniconda3/lib&#39;]
)

And this command:

python3 setup.py build_ext --inplace

Now I have a main.cpython-39-x86_64-linux-gnu.so file in ~/exp/exp.

When I launch (run.py) which contains:

from exp.main import entry_point

if __name__ == &quot;__main__&quot;:
    entry_point()

I have a normal behavior : It returns 42.

Now, here come the problems

I compile my run.cpp source, which contains :

#include &lt;iostream&gt;
#include &lt;dlfcn.h&gt;
#include &quot;Python.h&quot;


int main(int argc, char *argv[]) {
   setenv(&quot;PYTHONPATH&quot;,&quot;.&quot;,1);
   Py_Initialize();
   PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
   // Initialize the Python Interpreter
   Py_Initialize();

   // Build the name object
   pName = PyUnicode_FromString((char*)&quot;exp/main&quot;);

   // Load the module object
   pModule = PyImport_Import(pName);


   // pDict is a borrowed reference 
   pDict = PyModule_GetDict(pModule);


   // pFunc is also a borrowed reference 
   pFunc = PyDict_GetItemString(pDict, (char*)&quot;entry_point&quot;);

   Py_DECREF(pValue);

   // Clean up
   Py_DECREF(pModule);
   Py_DECREF(pName);
   Py_Finalize();
}

with the command :

g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run.o -ldl

And then execute : ./run.o
To end with a beautiful :

Segmentation fault (core dumped)

I tried with dlopen without success either.
Maybe I miss something, any help would be welcome.

Thank you 从一个Cython生成的.so文件中在C++代码中调用一个函数。

答案1

得分: 1

首先,感谢Craig EsteyDavidW的评论。

所以,最终我成功了,有两个问题:

  1. pValue 没有被使用,所以 Py_DECREF 引发了错误。
  2. 模块路径 "exp/main" 确实无效,但 "exp.main" 是有效的。

最后一件事,我忽略了PyObject_CallObject,它允许调用我的 PyObject pFunc

我最终得到了我的 '42' 答案。

以下是最终的代码:

#include <iostream>
#include <dlfcn.h>
#include "Python.h"

int main(int argc, char *argv[]) {
   setenv("PYTHONPATH",".",1);
   Py_Initialize();

   PyObject *pName, *pModule, *pDict, *pFunc, *presult;
   // 初始化 Python 解释器

   // 创建名称对象
   pName = PyUnicode_FromString((char*)"exp.main");

   // 加载模块对象
   pModule = PyImport_Import(pName);

   // pDict 是借用的引用 
   pDict = PyModule_GetDict(pModule);

   // pFunc 也是借用的引用 
   pFunc = PyDict_GetItemString(pDict, (char*)"entry_point");

   presult = PyObject_CallObject(pFunc, NULL);

   // Py_DECREF(pValue);

   // 清理
   Py_DECREF(pModule);
   Py_DECREF(pName);
   Py_Finalize();
}

(Craig 指出可执行文件可能不以 '.o' 结尾,了解更多: 什么是 *.o 文件
因此,新的编译命令是:

g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run
英文:

Firstly, thank you to Craig Estey and DavidW for their comments.

So I finally was able to make it work, two things was wrong:

  1. pValue was not used, so the Py_DECREF raised an Error
  2. the module path "exp/main" was indeed not valid, but "exp.main" was valid.

A very last thing. Something I omitted was the PyObject_CallObject that allows to call my PyObject pFunc.

I've finally got my '42' answer.

Here the final code:

#include &lt;iostream&gt;
#include &lt;dlfcn.h&gt;
#include &quot;Python.h&quot;


int main(int argc, char *argv[]) {
   setenv(&quot;PYTHONPATH&quot;,&quot;.&quot;,1);
   Py_Initialize();

   PyObject *pName, *pModule, *pDict, *pFunc, *presult;
   // Initialize the Python Interpreter

   // Build the name object
   pName = PyUnicode_FromString((char*)&quot;exp.main&quot;);

   // Load the module object
   pModule = PyImport_Import(pName);


   // pDict is a borrowed reference 
   pDict = PyModule_GetDict(pModule);


   // pFunc is also a borrowed reference 
   pFunc = PyDict_GetItemString(pDict, (char*)&quot;entry_point&quot;);

   presult = PyObject_CallObject(pFunc, NULL);

   // Py_DECREF(pValue);

   // Clean up
   Py_DECREF(pModule);
   Py_DECREF(pName);
   Py_Finalize();
}

(Craig pointed out that executable file might not finish by '.o', learn more: What is *.o file)
So, the new compile command is:

g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run

huangapple
  • 本文由 发表于 2023年2月9日 00:59:01
  • 转载请务必保留本文链接:https://go.coder-hub.com/75389182.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定