英文:
Optimising datastructure/word alignment padding in golang
问题
与我在C++中学到的类似,我认为填充是导致这两个结构体实例大小差异的原因。
type Foo struct {
w byte //1字节
x byte //1字节
y uint64 //8字节
}
type Bar struct {
x byte //1字节
y uint64 //8字节
w byte //1字节
}
func main() {
fmt.Println(runtime.GOARCH)
newFoo := new(Foo)
fmt.Println(unsafe.Sizeof(*newFoo))
newBar := new(Bar)
fmt.Println(unsafe.Sizeof(*newBar))
}
输出:
amd64
16
24
- 在定义结构体成员时是否有一条经验法则要遵循?(例如按类型大小的升序/降序排列)
- 是否有编译时优化可以自动处理这个问题?
- 或者我应该完全不用担心这个问题吗?
英文:
Similar to what I've learned in C++, I believe it's the padding that causes a difference in the size of instances of both structs.
type Foo struct {
w byte //1 byte
x byte //1 byte
y uint64 //8 bytes
}
type Bar struct {
x byte //1 byte
y uint64 //8 bytes
w byte// 1 byte
}
func main() {
fmt.Println(runtime.GOARCH)
newFoo := new(Foo)
fmt.Println(unsafe.Sizeof(*newFoo))
newBar := new(Bar)
fmt.Println(unsafe.Sizeof(*newBar))
}
Output:
amd64
16
24
- Is there a rule of thumb to follow when defining struct members? (like ascending/descending order of size of types)
- Is there a compile time optimisation which we can pass, that can automatically take care of this?
- Or shouldn't I be worried about this at all?
答案1
得分: 26
目前没有编译时优化;在x64上,值会被填充为8字节。
你可以手动安排结构体以最优地利用空间;通常是从较大的类型到较小的类型;例如,连续的8字节字段只使用8字节,但一个单独的字节会被填充为8字节对齐,请参考以下示例:https://play.golang.org/p/0qsgpuAHHp
package main
import (
"fmt"
"unsafe"
)
type Compact struct {
a, b uint64
c, d, e, f, g, h, i, j byte
}
// 内存占用比"Compact"大,但字段较少!
type Inefficient struct {
a uint64
b byte
c uint64
d byte
}
func main() {
newCompact := new(Compact)
fmt.Println(unsafe.Sizeof(*newCompact))
newInefficient := new(Inefficient)
fmt.Println(unsafe.Sizeof(*newInefficient))
}
如果你考虑到这一点,你可以优化结构体的内存占用。
英文:
Currently there's no compile-time optimisation; the values are padded to 8 bytes on x64.
You can manually arrange structs to optimally utilise space; typically by going from larger types to smaller; 8 consecutive byte fields for example, will only use 8 bytes, but a single byte would be padded to an 8 byte alignment, consider this: https://play.golang.org/p/0qsgpuAHHp
package main
import (
"fmt"
"unsafe"
)
type Compact struct {
a, b uint64
c, d, e, f, g, h, i, j byte
}
// Larger memory footprint than "Compact" - but less fields!
type Inefficient struct {
a uint64
b byte
c uint64
d byte
}
func main() {
newCompact := new(Compact)
fmt.Println(unsafe.Sizeof(*newCompact))
newInefficient := new(Inefficient)
fmt.Println(unsafe.Sizeof(*newInefficient))
}
If you take this into consideration; you can optimise the memory footprint of your structs.
答案2
得分: 17
>或者我根本不应该担心这个吗?
是的,你应该担心。
这也被称为机械同理心(请参阅这个Go Time播客剧集),所以它还取决于你编译的硬件架构。
以此为例:
- "字节对齐的一天让我吃了苦头"(2014年1月)
- "关于Go切片值的内存对齐"(2016年7月)
>Go切片中的值是16字节对齐的,而不是32字节对齐。
Go指针是按字节对齐的。
英文:
> Or shouldn't I be worried about this at all?
Yes you should.
This is also called mechanical sympathy (see this Go Time podcast episode), so it also depends on the hardware architecture you are compiling for.
See as illustration:
- "The day byte alignment came back to bite me" (January 2014)
- "On the memory alignment of Go slice values" (July 2016)
> The values in Go slices are 16-byte aligned. They are not 32 byte aligned.
Go pointers are byte-aligned.
答案3
得分: 2
这取决于你正在开发的应用程序的类型以及对这些结构的使用方式。如果应用程序需要满足一些内存/性能标准,你肯定应该关注内存对齐和填充,但不仅仅是这样 - 有一篇很好的文章 https://www.usenix.org/legacy/publications/library/proceedings/als00/2000papers/papers/full_papers/sears/sears_html/index.html 强调了最佳CPU缓存使用和结构布局与性能之间的关系。它强调了缓存行对齐、伪共享等问题。
此外,还有一个很好的Go语言工具 https://github.com/1pkg/gopium 可以帮助自动化这些优化,你可以去看看!
英文:
It depends on type of application that you are developing and on usage of those structures. If application needs to meet some memory/performance criteria you definitely should care about memory alignment and paddings, but not only - there is nice article https://www.usenix.org/legacy/publications/library/proceedings/als00/2000papers/papers/full_papers/sears/sears_html/index.html that highlights theme of optimal CPU caches usage and correlation between struct layouts and performance. It highlights cache line alignment, false sharing, etc.
Also there is a nice golang tool https://github.com/1pkg/gopium that helps to automate those optimizations, check it out!
答案4
得分: 2
一些指导原则
> 为了最小化填充字节的数量,我们必须按照从最高分配到最低分配的顺序布置字段。
一个例外是空结构体。
我们知道empty
的大小为零。
type empty struct {
a struct{}
}
按照上述常规规则,我们可以将结构体的字段排列如下:
type E struct {
a int64
b int64
c struct{}
}
然而,E
的大小为24。
当按照以下方式排列结构体的字段时:
type D struct {
b struct{}
a int64
c int64
}
D
的大小为16,参考链接:https://go.dev/play/p/ID_hN1zwIwJ
在我看来,最好使用能够帮助我们自动优化结构体对齐的工具
- aligncheck — https://gitlab.com/opennota/check
- maligned — <del>https://github.com/mdempsky/maligned</del>,原始的maligned已经被弃用。请使用https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment
- golang clint
- 只需在‘golangci-lint’的设置中启用‘maligned’。
例如,从配置文件.golangci.example.yml
中:
- 只需在‘golangci-lint’的设置中启用‘maligned’。
linters-settings:
maligned:
# 是否打印具有更有效内存布局的结构体,默认为false
suggest-new: true
英文:
Some guideline
> To minimize the number of padding bytes, we must lay out the fields from
the highest allocation to lowest allocation.
One exception is an empty structure
As we know the size of empty
is zero
type empty struct {
a struct{}
}
Following the common rule above, we may arrange the fields of structure as below
type E struct {
a int64
b int64
c struct{}
}
However, the size of E
is 24,
When arrange the fields of structure as
type D struct {
b struct{}
a int64
c int64
}
The size of D
is 16, refer to https://go.dev/play/p/ID_hN1zwIwJ
IMO, it is better to use tools that help us to automate structure alignment optimizations
- aligncheck — https://gitlab.com/opennota/check
- maligned — <del>https://github.com/mdempsky/maligned</del>, the original maligned is deprecated. Use https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment
- golang clint
- you just need to enable ‘maligned’ in the ‘golangci-lint’ settings.
Example, from the configuration file.golangci.example.yml
- you just need to enable ‘maligned’ in the ‘golangci-lint’ settings.
linters-settings:
maligned:
# print struct with more effective memory layout or not, false by default
suggest-new: true
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论