英文:
How to allocate 16GB of memory in Go?
问题
我正在使用以下简单的Go代码来分配一个大小为1024x1024x1024的3D数组：
grid = make([][][]TColor, 1024)
for x = 0; x < 1024; x++ {
grid[x] = make([][]TColor, 1024)
for y = 0; y < 1024; y++ {
grid[x][y] = make([]TColor, 1024)
}
}
TColor结构体是一个由4个float64向量组成的：
type TColor struct { R, G, B, A float64 }
在分配的过程中（x=477和y=~600ish），最内层的make()调用会出现恐慌，错误信息为... runtime: out of memory: cannot allocate 65536-byte block (17179869184 in use)
这在较低的网格分辨率下（例如256³、128³等）运行良好。现在，由于结构体的大小为4x4字节，整个网格应该需要正好16 GB的内存。我的机器（openSuse 12.1 64位）有32 GB的可寻址物理内存（即非虚拟内存）。为什么Go（weekly.2012-02-22）甚至无法分配其中一半的内存？
英文:
I'm using the following simple Go code to allocate a 3D array of size 1024x1024x1024:
grid = make([][][]TColor, 1024)
for x = 0; x < 1024; x++ {
grid[x] = make([][]TColor, 1024)
for y = 0; y < 1024; y++ {
grid[x][y] = make([]TColor, 1024)
}
}
That TColor struct is a 4-component float64 vector:
type TColor struct { R, G, B, A float64 }
Halfway (x=477 and y=~600ish) through the allocation, the inner-most make() call panics with... runtime: out of memory: cannot allocate 65536-byte block (17179869184 in use)
This works fine with lower grid resolutions, ie 256³, 128³ etc. Now since the size of the struct is 4x4 bytes, that whole grid should require exactly 16 GB of memory. My machine (openSuse 12.1 64bit) has 32 GB of addressable physical (ie not-virtual) memory. Why can Go (weekly.2012-02-22) not allocate even half of this?
答案1
得分: 6
结构体有4个8字节，而不是4个4字节。
英文:
The struct has 4x8 bytes, not 4x4.
答案2
得分: 5
在Go语言的当前实现中，在64位CPU上，Go运行时从操作系统中保留了16GB的虚拟内存。这限制了Go程序使用的总内存为16GB。
如果您计划在需要大量内存的项目中使用Go，您需要编辑文件malloc.goc中的函数runtime·mallocinit，并将变量arena_size的值从16GB增加到更大的值（例如32GB）。编辑完成后，运行以下命令：
cd $GOROOT/src/pkg/runtime
go tool dist install -v
然后重新编译您的项目。
英文:
In the current implementation of the Go language, on 64-bit CPUs the Go runtime reserves 16GB of virtual memory from the operating system. This limits the total memory used by a Go program to 16GB.
If you plan to use Go in projects that require large amounts of memory you will need to edit the function runtime·mallocinit in file malloc.goc and increase the value of variable arena_size from 16GB to a bigger value (such as 32GB). After the edit, run
cd $GOROOT/src/pkg/runtime
go tool dist install -v
and then recompile your project.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论