Numba:无法确定嵌套函数的类型

huangapple go评论61阅读模式
英文:

Numba: Cannot determine the type of nested functions

问题

我有一个名为'MySource.py'的numba脚本,我正在尝试编译:

from numpy import empty_like, inf, minimum, max
from numba import pycc

cc = pycc.CC('base_model')
cc.verbose = True

@cc.export('cumMin','float64[:](float64[:])')
def cumMin(A):
    r = empty_like(A)
    t = inf
    for i in range(len(A)):
        t = minimum(t, A[i])
        r[i] = t
    return r

@cc.export('da_comp','List(float64)(float64[:, :])')
def da_comp(rolled_ndarray):
    this_series_p = rolled_ndarray[:, 0]
    this_series_v = rolled_ndarray[:, 1]    
    return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]

if __name__ == "__main__":
    cc.compile()

在bash中,我执行以下命令:

python3 MySource.py

但是我得到了一个编译错误:

raise TypingError(msg, loc=inst.loc)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name 'cumMin': Cannot determine Numba type of <class 'function'>

File "MySource.py", line 24:
def da_comp(rolled_ndarray):
    <source elided>

    return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]

我使用的是numba 0.57.1版本。如何解决这个问题?

#编辑:
我尝试按照Eric的答案建议将函数拆分为两个文件,但是这并没有解决问题(我得到了相同的错误)。

英文:

I have this numba script (in a file called 'MySource.py') i'm trying to compile:

from numpy import empty_like, inf, minimum, max
from numba import pycc

cc = pycc.CC(&#39;base_model&#39;)
cc.verbose = True


@cc.export(&#39;cumMin&#39;,&#39;float64[:](float64[:])&#39;)  
def cumMin(A):
    r = empty_like(A)
    t = inf
    for i in range(len(A)):
        t = minimum(t, A[i])
        r[i] = t
    return r

# nm -C base_model.so &gt; bingo.txt
@cc.export(&#39;da_comp&#39;,&#39;List(float64)(float64[:, :])&#39;)  
def da_comp(rolled_ndarray):

    this_series_p = rolled_ndarray[:, 0]
    this_series_v = rolled_ndarray[:, 1]    

    return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]

    
if __name__ == &quot;__main__&quot;:
    cc.compile()

In bash, I do:

 python3 MySource.py

but I get a compilation error:

    raise TypingError(msg, loc=inst.loc)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name &#39;cumMin&#39;: Cannot determine Numba type of &lt;class &#39;function&#39;&gt;

File &quot;MySource.py&quot;, line 24:
def da_comp(rolled_ndarray):
    &lt;source elided&gt;

    return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]

I use numba 0.57.1 How to fix this?

#Edit:
I tried splitting the functions into two files as suggested in the answer by Eric below but this doesn't solve the issue (I get the same error).

答案1

得分: 1

看起来问题是cumMin没有被Numba识别为正确的类型。当使用@cc.export编译时,Numba不会在全局范围内查找依赖项,因此在编译da_comp时找不到cumMin。一个解决方法是分别编译这两个方法,然后在编译第二个方法之前加载模块。

以下是可能的解决方案:

  1. 分别编译cumMin
from numba.pycc import CC
from numba import njit

@njit('float64[:](float64[:])')
def cumMin(A):
    r = empty_like(A)
    t = inf
    for i in range(len(A)):
        t = minimum(t, A[i])
        r[i] = t
    return r

# numba.pycc.CC()返回numba.pycc.CompilerBase的子类的实例
cc = CC('cumMin_module')

# 导出cumMin函数
cc.export('cumMin', 'float64[:](float64[:])')(cumMin)
cc.compile()
  1. 加载生成的模块并使用它:
from cumMin_module import cumMin

cc = CC('base_model')
cc.verbose = True

@cc.export('da_comp','float64[:](float64[:, :])')  
def da_comp(rolled_ndarray):
    this_series_p = rolled_ndarray[:, 0]
    this_series_v = rolled_ndarray[:, 1]    
    
    return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]

cc.compile()
  1. 使用bash运行代码:
python3 MySource.py

这样,cumMin首先被编译,然后在da_comp中可用。

英文:

It looks like the problem is that cumMin is not recognized by numba as a proper type. Numba, when asked to compile with the @cc.export, doesn't look for dependencies in global scope, so cumMin is not found when compiling da_comp. A solution is to compile both methods seperately and then load the module before compiling the second method.

Here is a possible solution:

  1. compile cumMin separately.
from numba.pycc import CC
from numba import njit

@njit(&#39;float64[:](float64[:])&#39;)
def cumMin(A):
    r = empty_like(A)
    t = inf
    for i in range(len(A)):
        t = minimum(t, A[i])
        r[i] = t
    return r

# numba.pycc.CC() returns an instance of a subclass of numba.pycc.CompilerBase
cc = CC(&#39;cumMin_module&#39;)

# Export cumMin function
cc.export(&#39;cumMin&#39;, &#39;float64[:](float64[:])&#39;)(cumMin)
cc.compile()
  1. load resulting module and use it:
from cumMin_module import cumMin

cc = CC(&#39;base_model&#39;)
cc.verbose = True

@cc.export(&#39;da_comp&#39;,&#39;float64[:](float64[:, :])&#39;)  
def da_comp(rolled_ndarray):
    this_series_p = rolled_ndarray[:, 0]
    this_series_v = rolled_ndarray[:, 1]    
   
    return [max(cumMin(this_series_p)), max(cumMin(this_series_v))]

cc.compile()
  1. run code with bash:
python3 MySource.py

This way cumMin is compiled first and then available in da_comp.

答案2

得分: 0

所以我通过替换

@cc.export('cumMin','float64[:](float64[:])') 

为:

@numba.jit(nopython = True)

解决了这个问题。

import numpy
import numba
import numba.pycc

cc = numba.pycc.CC('base_model')
cc.verbose = True


@numba.jit(nopython = True)
def cumMin(A):
    r = numpy.empty(len(A))
    t = numpy.inf
    for i in range(len(A)):
        t = numpy.minimum(t, A[i])
        r[i] = t
    return r

@cc.export('da_comp','List(float64)(float64[:, :])')  
def da_comp(rolled_ndarray):

    this_series_p = rolled_ndarray[:, 0]
    this_series_v = rolled_ndarray[:, 1]    

    t1 = cumMin(this_series_p)
    t2 = cumMin(this_series_v)


    return [numpy.max(t1), numpy.max(t2)]

    
if __name__ == "main":
    cc.compile()

#python3 MySource.py
英文:

so I solved the problem by replacing

@cc.export(&#39;cumMin&#39;,&#39;float64[:](float64[:])&#39;) 

by:

@numba.jit(nopython = True)
import numpy
import numba
import numba.pycc

cc = numba.pycc.CC(&#39;base_model&#39;)
cc.verbose = True


@numba.jit(nopython = True)
def cumMin(A):
    r = numpy.empty(len(A))
    t = numpy.inf
    for i in range(len(A)):
        t = numpy.minimum(t, A[i])
        r[i] = t
    return r

@cc.export(&#39;da_comp&#39;,&#39;List(float64)(float64[:, :])&#39;)  
def da_comp(rolled_ndarray):

    this_series_p = rolled_ndarray[:, 0]
    this_series_v = rolled_ndarray[:, 1]    

    t1 = cumMin(this_series_p)
    t2 = cumMin(this_series_v)


    return [numpy.max(t1), numpy.max(t2)]

    
if __name__ == &quot;__main__&quot;:
    cc.compile()

#python3 MySource.py

huangapple
  • 本文由 发表于 2023年7月27日 17:36:26
  • 转载请务必保留本文链接:https://go.coder-hub.com/76778403.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定