英文:
Call a function from a cython generated .so file inside a c++ code
问题
我的目标是从C++中调用Python函数。这些Python函数必须使用Cython编译为.so文件。.so文件必须是与C++程序通信的那个。
在开始之前:
我使用Ubuntu,使用miniconda和Python 3.9进行工作。
我在一个像这样的文件夹中(~/exp):
- exp
- exp
- init.py
- main.py
- exp
- setup.py
- run.cpp
- run.py
我将main.py翻译为main.pyx,该文件包含以下代码:
def add(a, b):
return a + b
def entry_point():
print(add(21, 21))
我使用Cython编译了这个脚本,并得到了一个.so文件,使用以下setup.py脚本:
from setuptools import setup
from Cython.Build import cythonize
setup(
name='exp',
ext_modules=cythonize("exp/main.pyx"),
libraries=[('python3.9', {'include_dirs': ["~/miniconda3/include/python3.9"]})],
library_dirs=['~/miniconda3/lib']
)
以及以下命令:
python3 setup.py build_ext --inplace
现在我在~/exp/exp目录中有一个main.cpython-39-x86_64-linux-gnu.so文件。
当我运行run.py时,它包含以下内容:
from exp.main import entry_point
if __name__ == "__main__":
entry_point()
我得到了正常的行为:它返回42。
现在,出现了一些问题:
我编译了我的run.cpp源代码,其中包含以下内容:
#include <iostream>
#include <dlfcn.h>
#include "Python.h"
int main(int argc, char *argv[]) {
setenv("PYTHONPATH",".",1);
Py_Initialize();
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
// Initialize the Python Interpreter
Py_Initialize();
// Build the name object
pName = PyUnicode_FromString((char*)"exp/main");
// Load the module object
pModule = PyImport_Import(pName);
// pDict is a borrowed reference
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference
pFunc = PyDict_GetItemString(pDict, (char*)"entry_point");
Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
}
使用以下命令:
g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run.o -ldl
然后执行:./run.o
最终出现了以下错误:
Segmentation fault (core dumped)
我尝试过使用dlopen,但也没有成功。也许我漏掉了一些东西,任何帮助都将不胜感激。
谢谢
英文:
My goal is to call python functions from C++. These python function must be compiled with cython in a .so file. The .so file must be the one that communicate with the c++ program.
Before all:
I am on Ubuntu, I am working with miniconda with python3.9.
I am in a folder (~/exp) made like this:
- exp
- exp
- __ init __.py
- main.py
- exp
- setup.py
- run.cpp
- run.py
I translate the main.py to main.pyx, the file contains this code:
def add(a, b):
return a+b
def entry_point():
print(add(21,21))
I compiled this script with cython and obtained a .so file, with this setup.py script:
from setuptools import setup
from Cython.Build import cythonize
setup(
name='exp',
ext_modules=cythonize("exp/main.pyx"),
libraries=[('python3.9', {'include_dirs': ["~/miniconda3/include/python3.9"]})],
library_dirs=['~/miniconda3/lib']
)
And this command:
python3 setup.py build_ext --inplace
Now I have a main.cpython-39-x86_64-linux-gnu.so file in ~/exp/exp.
When I launch (run.py) which contains:
from exp.main import entry_point
if __name__ == "__main__":
entry_point()
I have a normal behavior : It returns 42.
Now, here come the problems
I compile my run.cpp source, which contains :
#include <iostream>
#include <dlfcn.h>
#include "Python.h"
int main(int argc, char *argv[]) {
setenv("PYTHONPATH",".",1);
Py_Initialize();
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
// Initialize the Python Interpreter
Py_Initialize();
// Build the name object
pName = PyUnicode_FromString((char*)"exp/main");
// Load the module object
pModule = PyImport_Import(pName);
// pDict is a borrowed reference
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference
pFunc = PyDict_GetItemString(pDict, (char*)"entry_point");
Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
}
with the command :
g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run.o -ldl
And then execute : ./run.o
To end with a beautiful :
Segmentation fault (core dumped)
I tried with dlopen without success either.
Maybe I miss something, any help would be welcome.
Thank you
答案1
得分: 1
首先,感谢Craig Estey和DavidW的评论。
所以,最终我成功了,有两个问题:
- pValue 没有被使用,所以 Py_DECREF 引发了错误。
- 模块路径 "exp/main" 确实无效,但 "exp.main" 是有效的。
最后一件事,我忽略了PyObject_CallObject
,它允许调用我的 PyObject pFunc
。
我最终得到了我的 '42' 答案。
以下是最终的代码:
#include <iostream>
#include <dlfcn.h>
#include "Python.h"
int main(int argc, char *argv[]) {
setenv("PYTHONPATH",".",1);
Py_Initialize();
PyObject *pName, *pModule, *pDict, *pFunc, *presult;
// 初始化 Python 解释器
// 创建名称对象
pName = PyUnicode_FromString((char*)"exp.main");
// 加载模块对象
pModule = PyImport_Import(pName);
// pDict 是借用的引用
pDict = PyModule_GetDict(pModule);
// pFunc 也是借用的引用
pFunc = PyDict_GetItemString(pDict, (char*)"entry_point");
presult = PyObject_CallObject(pFunc, NULL);
// Py_DECREF(pValue);
// 清理
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
}
(Craig 指出可执行文件可能不以 '.o' 结尾,了解更多: 什么是 *.o 文件)
因此,新的编译命令是:
g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run
英文:
Firstly, thank you to Craig Estey and DavidW for their comments.
So I finally was able to make it work, two things was wrong:
- pValue was not used, so the Py_DECREF raised an Error
- the module path "exp/main" was indeed not valid, but "exp.main" was valid.
A very last thing. Something I omitted was the PyObject_CallObject
that allows to call my PyObject pFunc
.
I've finally got my '42' answer.
Here the final code:
#include <iostream>
#include <dlfcn.h>
#include "Python.h"
int main(int argc, char *argv[]) {
setenv("PYTHONPATH",".",1);
Py_Initialize();
PyObject *pName, *pModule, *pDict, *pFunc, *presult;
// Initialize the Python Interpreter
// Build the name object
pName = PyUnicode_FromString((char*)"exp.main");
// Load the module object
pModule = PyImport_Import(pName);
// pDict is a borrowed reference
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference
pFunc = PyDict_GetItemString(pDict, (char*)"entry_point");
presult = PyObject_CallObject(pFunc, NULL);
// Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
}
(Craig pointed out that executable file might not finish by '.o', learn more: What is *.o file)
So, the new compile command is:
g++ -Wall -I~/miniconda3/include/python3.9 run.cpp -L~/miniconda3/lib -lpython3.9 -o run
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论