如何在Cython中正确取消引用指针的所有地址?

huangapple go评论59阅读模式
英文:

How to properly dereference all addresses of a pointer in Cython?

问题

  1. c_arr3[:] = [1.0, 2.0] works because [:] is used to copy the values from the Python list [1.0, 2.0] to the memory addresses pointed to by c_arr3. It essentially performs a bulk assignment, copying values from the Python list to the C array.

  2. c_arr3[:] and c_arr4[:] are both considered as double * objects in Cython, but the difference lies in what you are trying to assign to them. np_arr.tolist() is considered a Python object because it involves a Python-specific operation to convert a NumPy array to a Python list. On the other hand, [1.0, 2.0] is a pure Python list, and it is considered a Python object as well.

  3. Yes, you can change the values at the addresses of a pointer more efficiently without manually looping through each address. One efficient approach is to use the memcpy function from the libc.string module to copy the values from a Python list to the pointer. Here's an example:

    from libc.string cimport memcpy
    
    # Allocate memory for the pointer
    cdef double *c_arr = <double *>malloc(2 * sizeof(double))
    
    # Python list containing the values you want to copy
    values_to_copy = [1.0, 2.0]
    
    # Use memcpy to copy the values
    memcpy(c_arr, &values_to_copy[0], 2 * sizeof(double))
    

    This method is more efficient and avoids the need for manual looping.

英文:

I'm learning the basics of Cython 3.0 and I'm trying to understand why one of my approaches to dereferencing a pointer (to multiple memory addresses) in order to initialize the values in those addresses doesn't work while the other approaches do work.

Here is my test code

import numpy as np
from libc.stdlib cimport malloc

np_arr = np.array([1.0, 2.0])

cdef double[2] c_arr1 = [1.0, 2.0] # this works

cdef double[2] c_arr2 = np_arr.tolist() # this works

cdef double *c_arr3 = <double *>malloc(2*sizeof(double))
c_arr3[:] = [1.0, 2.0] # this works

cdef double *c_arr4 = <double *>malloc(2*sizeof(double))
c_arr4[:] = np_arr.tolist() # this doesn't work; gives two compile errors

cdef double *c_arr5 = <double *>malloc(2*sizeof(double))
c_arr5[0] = np_arr.tolist()[0]
c_arr5[1] = np_arr.tolist()[1] # this works

The compile errors for c_arr4 are: "Cannot convert Python object to 'double *'" and "Storing unsafe C derivative of temporary Python reference".

It seems that using [:] on a pointer doesn't actually dereference all of the addresses, yet c_arr3 managed to initialize with it. So my questions are:

  1. Why does c_arr3[:] = [1.0, 2.0] work? What is the [:] doing exactly?

  2. If c_arr3[:] and c_arr4[:] are considered as double * objects to Cython, why is np_arr.tolist() considered a Python object but not [1.0, 2.0]?

  3. Is there another (ideally more efficient) way of changing the values at the addresses of a pointer without manually looping through each address like I did for c_arr5?

答案1

得分: 1

  1. 让我们来看看你的Cython代码
cdef double *c_arr3 = <double *>malloc(2*sizeof(double))
c_arr3[:] = [1.0, 2.0]

在这里,Python列表[1.0, 2.0]仅用于初始化堆上指针c_arr3指向的内存块。因此,从性能的角度来看,没有必要从Python列表中创建一个列表。这就是为什么Cython会生成以下C代码的一个原因:

/* 分配内存 */
double* c_arr3 = (double*) malloc(2*sizeof(double));

/* 创建一个C数组(而不是列表)来初始化c_arr3 */
double tmp[2];
tmp[0] = 1.0;
tmp[1] = 2.0;

/* 将tmp的所有值复制到c_arr3中 */
memcpy(&c_arr3[0], &tmp[0], 2*sizeof(double));

a[:] = expr是Cython的切片语法,通常用于类型化内存视图,与NumPy数组的工作方式非常相似。因此,它主要用于将右侧的expr的值复制到a中。

  1. c_arr3[:]c_arr4[:]不是double*。这只是Cython中用于复制值的语法糖(参见上面的C代码),这也意味着它可以用于初始化内存块。但是,在撰写本文时,通过c_arr3[:] = expr进行初始化仅适用于简单的右侧字面值,比如Python列表。

  2. 是的,你可以使用类型化内存视图来代替原始指针:

cdef double[::1] c_mv = <double[:2]>malloc(2*sizeof(double))

# 将所有值设置为1.0
c_mv[:] = 1.0

# 使用np_arr的值来初始化它
# 注意,你不需要将np_arr转换为列表
c_mv[:] = np_arr[:]

# 不要忘记释放内存!
free(&c_mv[0])
英文:
  1. Let's have a look at your Cython code
cdef double *c_arr3 = <double *>malloc(2*sizeof(double))
c_arr3[:] = [1.0, 2.0]

Here, the python list [1.0, 2.0] is only meant to initialise the block of memory on the heap your pointer c_arr3 points to. Therefore, there's no need to create a python list from a performance perspective. That's one of the reasons why Cython will roughly generate the following C code instead:

/* allocate the memory */
double* c_arr3 = (double*) malloc(2*sizeof(double));

/* create a C array (instead of a list) to initialize c_arr3 */
double tmp[2];
tmp[0] = 1.0;
tmp[1] = 2.0;

/* copy all values of tmp into c_arr3 */
memcpy(&c_arr3[0], &tmp[0], 2*sizeof(double));

The a[:] = expr is Cython's slicing syntax that is typically used for typed memoryviews and works pretty similar to numpy arrays. Thus, it's mainly used to copy values of the right-hand-side expr into a.

  1. c_arr3[:]and c_arr4[:] are no double*. It's just syntactic sugar in Cython for copying values (see the above C code), which also means that it can be used to initialize blocks of memory. However, at the time of writing, initialization via c_arr3[:] = expr only works for simple right-hand side literals like python lists.

  2. Yes, you could use a typed memoryview instead of raw pointers:

cdef double[::1] c_mv = <double[:2]>malloc(2*sizeof(double))

# set all values to 1.0
c_mv[:] = 1.0

# initialize it with the values of np_arr
# note that you don't need to convert np_arr to a list
c_mv[:] = np_arr[:]

# don't forget to free the memory!
free(&c_mv[0])

huangapple
  • 本文由 发表于 2023年3月12日 11:53:36
  • 转载请务必保留本文链接:https://go.coder-hub.com/75710964.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定