优化Golang中的数据结构/单词对齐填充

huangapple go评论147阅读模式
英文:

Optimising datastructure/word alignment padding in golang

问题

与我在C++中学到的类似,我认为填充是导致这两个结构体实例大小差异的原因。

type Foo struct {
    w byte    //1字节
    x byte    //1字节
    y uint64  //8字节
}
type Bar struct {
    x byte    //1字节
    y uint64  //8字节
    w byte    //1字节
}
func main() {
    fmt.Println(runtime.GOARCH)
    newFoo := new(Foo)
    fmt.Println(unsafe.Sizeof(*newFoo))
    newBar := new(Bar)
    fmt.Println(unsafe.Sizeof(*newBar))
}

输出:

amd64
16
24
  • 在定义结构体成员时是否有一条经验法则要遵循?(例如按类型大小的升序/降序排列)
  • 是否有编译时优化可以自动处理这个问题?
  • 或者我应该完全不用担心这个问题吗?
英文:

Similar to what I've learned in C++, I believe it's the padding that causes a difference in the size of instances of both structs.

type Foo struct {
    w byte //1 byte
    x byte //1 byte
    y uint64 //8 bytes
}
type Bar struct {
    x byte //1 byte
    y uint64 //8 bytes
    w byte// 1 byte
}
func main() {
    fmt.Println(runtime.GOARCH)
    newFoo := new(Foo)
    fmt.Println(unsafe.Sizeof(*newFoo))
    newBar := new(Bar)
    fmt.Println(unsafe.Sizeof(*newBar))
}

Output:

amd64
16
24
  • Is there a rule of thumb to follow when defining struct members? (like ascending/descending order of size of types)
  • Is there a compile time optimisation which we can pass, that can automatically take care of this?
  • Or shouldn't I be worried about this at all?

答案1

得分: 26

目前没有编译时优化;在x64上,值会被填充为8字节。

你可以手动安排结构体以最优地利用空间;通常是从较大的类型到较小的类型;例如,连续的8字节字段只使用8字节,但一个单独的字节会被填充为8字节对齐,请参考以下示例:https://play.golang.org/p/0qsgpuAHHp

package main

import (
    "fmt"
    "unsafe"
)

type Compact struct {
    a, b                   uint64
    c, d, e, f, g, h, i, j byte
}

// 内存占用比"Compact"大,但字段较少!
type Inefficient struct {
    a uint64
    b byte
    c uint64
    d byte
}

func main() {
    newCompact := new(Compact)
    fmt.Println(unsafe.Sizeof(*newCompact))
    newInefficient := new(Inefficient)
    fmt.Println(unsafe.Sizeof(*newInefficient))
}

如果你考虑到这一点,你可以优化结构体的内存占用。

英文:

Currently there's no compile-time optimisation; the values are padded to 8 bytes on x64.

You can manually arrange structs to optimally utilise space; typically by going from larger types to smaller; 8 consecutive byte fields for example, will only use 8 bytes, but a single byte would be padded to an 8 byte alignment, consider this: https://play.golang.org/p/0qsgpuAHHp

package main

import (
	"fmt"
	"unsafe"
)

type Compact struct {
	a, b                   uint64
	c, d, e, f, g, h, i, j byte
}

// Larger memory footprint than "Compact" - but less fields!
type Inefficient struct {
	a uint64
	b byte
	c uint64
	d byte
}

func main() {
	newCompact := new(Compact)
	fmt.Println(unsafe.Sizeof(*newCompact))
	newInefficient := new(Inefficient)
	fmt.Println(unsafe.Sizeof(*newInefficient))
}

If you take this into consideration; you can optimise the memory footprint of your structs.

答案2

得分: 17

>或者我根本不应该担心这个吗?

是的,你应该担心。
这也被称为机械同理心(请参阅这个Go Time播客剧集),所以它还取决于你编译的硬件架构。

以此为例:

>Go切片中的值是16字节对齐的,而不是32字节对齐。
Go指针是按字节对齐的。

英文:

> Or shouldn't I be worried about this at all?

Yes you should.
This is also called mechanical sympathy (see this Go Time podcast episode), so it also depends on the hardware architecture you are compiling for.

See as illustration:

> The values in Go slices are 16-byte aligned. They are not 32 byte aligned.
Go pointers are byte-aligned.

答案3

得分: 2

这取决于你正在开发的应用程序的类型以及对这些结构的使用方式。如果应用程序需要满足一些内存/性能标准,你肯定应该关注内存对齐和填充,但不仅仅是这样 - 有一篇很好的文章 https://www.usenix.org/legacy/publications/library/proceedings/als00/2000papers/papers/full_papers/sears/sears_html/index.html 强调了最佳CPU缓存使用和结构布局与性能之间的关系。它强调了缓存行对齐、伪共享等问题。

此外,还有一个很好的Go语言工具 https://github.com/1pkg/gopium 可以帮助自动化这些优化,你可以去看看!

英文:

It depends on type of application that you are developing and on usage of those structures. If application needs to meet some memory/performance criteria you definitely should care about memory alignment and paddings, but not only - there is nice article https://www.usenix.org/legacy/publications/library/proceedings/als00/2000papers/papers/full_papers/sears/sears_html/index.html that highlights theme of optimal CPU caches usage and correlation between struct layouts and performance. It highlights cache line alignment, false sharing, etc.

Also there is a nice golang tool https://github.com/1pkg/gopium that helps to automate those optimizations, check it out!

答案4

得分: 2

一些指导原则

> 为了最小化填充字节的数量,我们必须按照从最高分配到最低分配的顺序布置字段。

一个例外是空结构体。

我们知道empty的大小为零。

type empty struct {
	a struct{}
}

按照上述常规规则,我们可以将结构体的字段排列如下:

type E struct {
	a int64
	b int64
	c struct{}
}

然而,E的大小为24。

当按照以下方式排列结构体的字段时:

type D struct {
    b struct{}
	a int64
	c int64
}

D的大小为16,参考链接:https://go.dev/play/p/ID_hN1zwIwJ


在我看来,最好使用能够帮助我们自动优化结构体对齐的工具

  • aligncheck — https://gitlab.com/opennota/check
  • maligned — <del>https://github.com/mdempsky/maligned</del>,原始的maligned已经被弃用。请使用https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment
  • golang clint
    • 只需在‘golangci-lint’的设置中启用‘maligned’。
      例如,从配置文件.golangci.example.yml中:
linters-settings:
  maligned:
      # 是否打印具有更有效内存布局的结构体,默认为false
      suggest-new: true
英文:

Some guideline

> To minimize the number of padding bytes, we must lay out the fields from
the highest allocation to lowest allocation.

One exception is an empty structure

As we know the size of empty is zero

type empty struct {
	a struct{}
}

Following the common rule above, we may arrange the fields of structure as below

type E struct {
	a int64
	b int64
	c struct{}
}

However, the size of E is 24,

When arrange the fields of structure as

type D struct {
    b struct{}
	a int64
	c int64
}

The size of D is 16, refer to https://go.dev/play/p/ID_hN1zwIwJ


IMO, it is better to use tools that help us to automate structure alignment optimizations

linters-settings:
  maligned:
      # print struct with more effective memory layout or not, false by default
      suggest-new: true

huangapple
  • 本文由 发表于 2016年8月21日 19:10:51
  • 转载请务必保留本文链接:https://go.coder-hub.com/39063530.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定