英文:
How to properly dereference all addresses of a pointer in Cython?
问题
-
c_arr3[:] = [1.0, 2.0]
works because[:]
is used to copy the values from the Python list[1.0, 2.0]
to the memory addresses pointed to byc_arr3
. It essentially performs a bulk assignment, copying values from the Python list to the C array. -
c_arr3[:]
andc_arr4[:]
are both considered asdouble *
objects in Cython, but the difference lies in what you are trying to assign to them.np_arr.tolist()
is considered a Python object because it involves a Python-specific operation to convert a NumPy array to a Python list. On the other hand,[1.0, 2.0]
is a pure Python list, and it is considered a Python object as well. -
Yes, you can change the values at the addresses of a pointer more efficiently without manually looping through each address. One efficient approach is to use the
memcpy
function from thelibc.string
module to copy the values from a Python list to the pointer. Here's an example:from libc.string cimport memcpy # Allocate memory for the pointer cdef double *c_arr = <double *>malloc(2 * sizeof(double)) # Python list containing the values you want to copy values_to_copy = [1.0, 2.0] # Use memcpy to copy the values memcpy(c_arr, &values_to_copy[0], 2 * sizeof(double))
This method is more efficient and avoids the need for manual looping.
英文:
I'm learning the basics of Cython 3.0 and I'm trying to understand why one of my approaches to dereferencing a pointer (to multiple memory addresses) in order to initialize the values in those addresses doesn't work while the other approaches do work.
Here is my test code
import numpy as np
from libc.stdlib cimport malloc
np_arr = np.array([1.0, 2.0])
cdef double[2] c_arr1 = [1.0, 2.0] # this works
cdef double[2] c_arr2 = np_arr.tolist() # this works
cdef double *c_arr3 = <double *>malloc(2*sizeof(double))
c_arr3[:] = [1.0, 2.0] # this works
cdef double *c_arr4 = <double *>malloc(2*sizeof(double))
c_arr4[:] = np_arr.tolist() # this doesn't work; gives two compile errors
cdef double *c_arr5 = <double *>malloc(2*sizeof(double))
c_arr5[0] = np_arr.tolist()[0]
c_arr5[1] = np_arr.tolist()[1] # this works
The compile errors for c_arr4
are: "Cannot convert Python object to 'double *'" and "Storing unsafe C derivative of temporary Python reference".
It seems that using [:]
on a pointer doesn't actually dereference all of the addresses, yet c_arr3
managed to initialize with it. So my questions are:
-
Why does
c_arr3[:] = [1.0, 2.0]
work? What is the[:]
doing exactly? -
If
c_arr3[:]
andc_arr4[:]
are considered asdouble *
objects to Cython, why isnp_arr.tolist()
considered a Python object but not[1.0, 2.0]
? -
Is there another (ideally more efficient) way of changing the values at the addresses of a pointer without manually looping through each address like I did for
c_arr5
?
答案1
得分: 1
- 让我们来看看你的Cython代码
cdef double *c_arr3 = <double *>malloc(2*sizeof(double))
c_arr3[:] = [1.0, 2.0]
在这里,Python列表[1.0, 2.0]
仅用于初始化堆上指针c_arr3
指向的内存块。因此,从性能的角度来看,没有必要从Python列表中创建一个列表。这就是为什么Cython会生成以下C代码的一个原因:
/* 分配内存 */
double* c_arr3 = (double*) malloc(2*sizeof(double));
/* 创建一个C数组(而不是列表)来初始化c_arr3 */
double tmp[2];
tmp[0] = 1.0;
tmp[1] = 2.0;
/* 将tmp的所有值复制到c_arr3中 */
memcpy(&c_arr3[0], &tmp[0], 2*sizeof(double));
a[:] = expr
是Cython的切片语法,通常用于类型化内存视图,与NumPy数组的工作方式非常相似。因此,它主要用于将右侧的expr
的值复制到a
中。
-
c_arr3[:]
和c_arr4[:]
不是double*
。这只是Cython中用于复制值的语法糖(参见上面的C代码),这也意味着它可以用于初始化内存块。但是,在撰写本文时,通过c_arr3[:] = expr
进行初始化仅适用于简单的右侧字面值,比如Python列表。 -
是的,你可以使用类型化内存视图来代替原始指针:
cdef double[::1] c_mv = <double[:2]>malloc(2*sizeof(double))
# 将所有值设置为1.0
c_mv[:] = 1.0
# 使用np_arr的值来初始化它
# 注意,你不需要将np_arr转换为列表
c_mv[:] = np_arr[:]
# 不要忘记释放内存!
free(&c_mv[0])
英文:
- Let's have a look at your Cython code
cdef double *c_arr3 = <double *>malloc(2*sizeof(double))
c_arr3[:] = [1.0, 2.0]
Here, the python list [1.0, 2.0]
is only meant to initialise the block of memory on the heap your pointer c_arr3
points to. Therefore, there's no need to create a python list from a performance perspective. That's one of the reasons why Cython will roughly generate the following C code instead:
/* allocate the memory */
double* c_arr3 = (double*) malloc(2*sizeof(double));
/* create a C array (instead of a list) to initialize c_arr3 */
double tmp[2];
tmp[0] = 1.0;
tmp[1] = 2.0;
/* copy all values of tmp into c_arr3 */
memcpy(&c_arr3[0], &tmp[0], 2*sizeof(double));
The a[:] = expr
is Cython's slicing syntax that is typically used for typed memoryviews and works pretty similar to numpy arrays. Thus, it's mainly used to copy values of the right-hand-side expr
into a
.
-
c_arr3[:]
andc_arr4[:]
are nodouble*
. It's just syntactic sugar in Cython for copying values (see the above C code), which also means that it can be used to initialize blocks of memory. However, at the time of writing, initialization viac_arr3[:] = expr
only works for simple right-hand side literals like python lists. -
Yes, you could use a typed memoryview instead of raw pointers:
cdef double[::1] c_mv = <double[:2]>malloc(2*sizeof(double))
# set all values to 1.0
c_mv[:] = 1.0
# initialize it with the values of np_arr
# note that you don't need to convert np_arr to a list
c_mv[:] = np_arr[:]
# don't forget to free the memory!
free(&c_mv[0])
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论