英文:
Computing the memory footprint (or byte length) of a map
问题
我想将一个地图限制为最大X字节。然而，似乎没有直接计算地图字节长度的方法。
"encoding/binary"包有一个很好的Size函数，但它只适用于切片或"固定值"，而不适用于地图。
我可以尝试从地图中获取所有键/值对，推断它们的类型（如果是map[string]interface{}），然后计算长度 - 但这既麻烦又可能不正确（因为这将排除地图本身的"内部"Go成本 - 管理指向元素的指针等）。
有没有建议的方法来做到这一点？最好有一个代码示例。
英文:
I want to limit a map to be maximum X bytes. It seems there is no straightforward way of computing the byte length of a map though.
"encoding/binary" package has a nice Size function, but it only works for slices or "fixed values", not for maps.
I could try to get all key/value pairs from the map, infer their type (if it's a map[string]interface{}) and compute the length - but that would be both cumbersome and probably incorrect (because that would exclude the "internal" Go cost of the map itself - managing pointers to elements etc).
Any suggested way of doing this? Preferably a code example.
答案1
得分: 11
这是一个地图头的定义：
// 一个 Go 地图的头部。
type hmap struct {
// 注意：Hmap 的格式已经编码在 ../../cmd/gc/reflect.c 和 ../reflect/type.go 中。
// 不要在不修改那些代码的情况下更改此结构！
count int // 存活单元格的数量 == 地图的大小。必须是第一个字段（由 len() 内建函数使用）
flags uint32
hash0 uint32 // 哈希种子
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
buckets unsafe.Pointer // 2^B 个桶的数组。如果 count==0，则可能为 nil。
oldbuckets unsafe.Pointer // 之前的桶数组，大小为当前大小的一半，只有在扩容时非 nil
nevacuate uintptr // 疏散的进度计数器（小于此值的桶已经被疏散）
}
计算其大小非常简单（使用 unsafe.Sizeof）。
这是地图指向的每个单独桶的定义：
// Go 地图的一个桶。
type bmap struct {
tophash [bucketCnt]uint8
// 后面跟着 bucketCnt 个键，然后是 bucketCnt 个值。
// 注意：将所有键放在一起，然后将所有值放在一起，比交替键/值/键/值/... 的代码稍微复杂一些，但它允许我们消除填充，而对于例如 map[int64]int8，需要填充。
// 后面跟着一个溢出指针。
}
bucketCnt 是一个常量，定义如下：
bucketCnt = 1 << bucketCntBits // 等于十进制 8
bucketCntBits = 3
最终的计算公式为：
unsafe.Sizeof(hmap) + (len(theMap) * 8) + (len(theMap) * 8 * unsafe.Sizeof(x)) + (len(theMap) * 8 * unsafe.Sizeof(y))
其中 theMap 是你的地图值，x 是地图的键类型的值，y 是地图的值类型的值。
你需要通过汇编将 hmap 结构与你的包共享，类似于运行时中的 thunk.s。
英文:
This is the definition for a map header:
// A header for a Go map.
type hmap struct {
// Note: the format of the Hmap is encoded in ../../cmd/gc/reflect.c and
// ../reflect/type.go. Don't change this structure without also changing that code!
count int // # live cells == size of map. Must be first (used by len() builtin)
flags uint32
hash0 uint32 // hash seed
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than this have been evacuated)
}
Calculating its size is pretty straightforward (unsafe.Sizeof).
This is the definition for each individual bucket the map points to:
// A bucket for a Go map.
type bmap struct {
tophash [bucketCnt]uint8
// Followed by bucketCnt keys and then bucketCnt values.
// NOTE: packing all the keys together and then all the values together makes the
// code a bit more complicated than alternating key/value/key/value/... but it allows
// us to eliminate padding which would be needed for, e.g., map[int64]int8.
// Followed by an overflow pointer.
}
bucketCnt is a constant defined as:
bucketCnt = 1 << bucketCntBits // equals decimal 8
bucketCntBits = 3
The final calculation would be:
unsafe.Sizeof(hmap) + (len(theMap) * 8) + (len(theMap) * 8 * unsafe.Sizeof(x)) + (len(theMap) * 8 * unsafe.Sizeof(y))
Where theMap is your map value, x is a value of the map's key type and y a value of the map's value type.
You'll have to share the hmap structure with your package via assembly, analogously to thunk.s in the runtime.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论