英文:
Numba: Cannot determine the type of nested functions
问题
我有一个名为'MySource.py'的numba脚本,我正在尝试编译:
from numpy import empty_like, inf, minimum, max
from numba import pycc
cc = pycc.CC('base_model')
cc.verbose = True
@cc.export('cumMin','float64[:](float64[:])')
def cumMin(A):
r = empty_like(A)
t = inf
for i in range(len(A)):
t = minimum(t, A[i])
r[i] = t
return r
@cc.export('da_comp','List(float64)(float64[:, :])')
def da_comp(rolled_ndarray):
this_series_p = rolled_ndarray[:, 0]
this_series_v = rolled_ndarray[:, 1]
return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]
if __name__ == "__main__":
cc.compile()
在bash中,我执行以下命令:
python3 MySource.py
但是我得到了一个编译错误:
raise TypingError(msg, loc=inst.loc)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name 'cumMin': Cannot determine Numba type of <class 'function'>
File "MySource.py", line 24:
def da_comp(rolled_ndarray):
<source elided>
return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]
我使用的是numba 0.57.1版本。如何解决这个问题?
#编辑:
我尝试按照Eric的答案建议将函数拆分为两个文件,但是这并没有解决问题(我得到了相同的错误)。
英文:
I have this numba script (in a file called 'MySource.py') i'm trying to compile:
from numpy import empty_like, inf, minimum, max
from numba import pycc
cc = pycc.CC('base_model')
cc.verbose = True
@cc.export('cumMin','float64[:](float64[:])')
def cumMin(A):
r = empty_like(A)
t = inf
for i in range(len(A)):
t = minimum(t, A[i])
r[i] = t
return r
# nm -C base_model.so > bingo.txt
@cc.export('da_comp','List(float64)(float64[:, :])')
def da_comp(rolled_ndarray):
this_series_p = rolled_ndarray[:, 0]
this_series_v = rolled_ndarray[:, 1]
return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]
if __name__ == "__main__":
cc.compile()
In bash, I do:
python3 MySource.py
but I get a compilation error:
raise TypingError(msg, loc=inst.loc)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name 'cumMin': Cannot determine Numba type of <class 'function'>
File "MySource.py", line 24:
def da_comp(rolled_ndarray):
<source elided>
return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]
I use numba 0.57.1 How to fix this?
#Edit:
I tried splitting the functions into two files as suggested in the answer by Eric below but this doesn't solve the issue (I get the same error).
答案1
得分: 1
看起来问题是cumMin
没有被Numba识别为正确的类型。当使用@cc.export
编译时,Numba不会在全局范围内查找依赖项,因此在编译da_comp
时找不到cumMin
。一个解决方法是分别编译这两个方法,然后在编译第二个方法之前加载模块。
以下是可能的解决方案:
- 分别编译
cumMin
。
from numba.pycc import CC
from numba import njit
@njit('float64[:](float64[:])')
def cumMin(A):
r = empty_like(A)
t = inf
for i in range(len(A)):
t = minimum(t, A[i])
r[i] = t
return r
# numba.pycc.CC()返回numba.pycc.CompilerBase的子类的实例
cc = CC('cumMin_module')
# 导出cumMin函数
cc.export('cumMin', 'float64[:](float64[:])')(cumMin)
cc.compile()
- 加载生成的模块并使用它:
from cumMin_module import cumMin
cc = CC('base_model')
cc.verbose = True
@cc.export('da_comp','float64[:](float64[:, :])')
def da_comp(rolled_ndarray):
this_series_p = rolled_ndarray[:, 0]
this_series_v = rolled_ndarray[:, 1]
return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]
cc.compile()
- 使用bash运行代码:
python3 MySource.py
这样,cumMin
首先被编译,然后在da_comp
中可用。
英文:
It looks like the problem is that cumMin
is not recognized by numba as a proper type. Numba, when asked to compile with the @cc.export
, doesn't look for dependencies in global scope, so cumMin
is not found when compiling da_comp
. A solution is to compile both methods seperately and then load the module before compiling the second method.
Here is a possible solution:
- compile
cumMin
separately.
from numba.pycc import CC
from numba import njit
@njit('float64[:](float64[:])')
def cumMin(A):
r = empty_like(A)
t = inf
for i in range(len(A)):
t = minimum(t, A[i])
r[i] = t
return r
# numba.pycc.CC() returns an instance of a subclass of numba.pycc.CompilerBase
cc = CC('cumMin_module')
# Export cumMin function
cc.export('cumMin', 'float64[:](float64[:])')(cumMin)
cc.compile()
- load resulting module and use it:
from cumMin_module import cumMin
cc = CC('base_model')
cc.verbose = True
@cc.export('da_comp','float64[:](float64[:, :])')
def da_comp(rolled_ndarray):
this_series_p = rolled_ndarray[:, 0]
this_series_v = rolled_ndarray[:, 1]
return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]
cc.compile()
- run code with bash:
python3 MySource.py
This way cumMin
is compiled first and then available in da_comp
.
答案2
得分: 0
所以我通过替换
@cc.export('cumMin','float64[:](float64[:])')
为:
@numba.jit(nopython = True)
解决了这个问题。
import numpy
import numba
import numba.pycc
cc = numba.pycc.CC('base_model')
cc.verbose = True
@numba.jit(nopython = True)
def cumMin(A):
r = numpy.empty(len(A))
t = numpy.inf
for i in range(len(A)):
t = numpy.minimum(t, A[i])
r[i] = t
return r
@cc.export('da_comp','List(float64)(float64[:, :])')
def da_comp(rolled_ndarray):
this_series_p = rolled_ndarray[:, 0]
this_series_v = rolled_ndarray[:, 1]
t1 = cumMin(this_series_p)
t2 = cumMin(this_series_v)
return [numpy.max(t1), numpy.max(t2)]
if __name__ == "main":
cc.compile()
#python3 MySource.py
英文:
so I solved the problem by replacing
@cc.export('cumMin','float64[:](float64[:])')
by:
@numba.jit(nopython = True)
import numpy
import numba
import numba.pycc
cc = numba.pycc.CC('base_model')
cc.verbose = True
@numba.jit(nopython = True)
def cumMin(A):
r = numpy.empty(len(A))
t = numpy.inf
for i in range(len(A)):
t = numpy.minimum(t, A[i])
r[i] = t
return r
@cc.export('da_comp','List(float64)(float64[:, :])')
def da_comp(rolled_ndarray):
this_series_p = rolled_ndarray[:, 0]
this_series_v = rolled_ndarray[:, 1]
t1 = cumMin(this_series_p)
t2 = cumMin(this_series_v)
return [numpy.max(t1), numpy.max(t2)]
if __name__ == "__main__":
cc.compile()
#python3 MySource.py
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论