英文:
C# HashSet allocate memory on each Add even within capacity
问题
我做了一个简单的基准测试来测试 HashSet<T>.Add 方法,对我来说结果看起来很奇怪。
对于基准测试,我使用了在 GitHub 上可用的 BenchmarkDotNet。
以下是基准测试的完整代码:
```C#
// (以下代码部分未做翻译)
我的期望是,由于 HashSet 已经初始化了足够的容量,并且在清理之前我只进行了一次添加操作,因此根本不应该有额外的分配。
但实际上这是一个使用以下命令执行的基准测试结果:
dotnet run -c Release
方法 | ArrayLength | 均值 | 误差 | 标准偏差 | 中位数 | Gen 0 | 已分配内存 |
---|---|---|---|---|---|---|---|
Struct | 10 | 644.2 纳秒 | 12.90 纳秒 | 28.86 纳秒 | 633.8 纳秒 | 0.0572 | 240 字节 |
Struct | 500 | 30,080.1 纳秒 | 151.63 纳秒 | 134.42 纳秒 | 30,096.3 纳秒 | 2.8687 | 12,000 字节 |
Struct | 1000 | 64,610.0 纳秒 | 1,264.29 纳秒 | 1,352.78 纳秒 | 64,436.2 纳秒 | 5.7373 | 24,000 字节 |
分配的字节大小等于迭代次数 * 24。所以每次 'Add' 都会进行新的分配。
一些建议:
- 我尝试过只使用 Clear 进行相同的基准测试 - 没有分配,因此它肯定是由 Add 添加的。
- 改变结构体中字段的数量会改变分配的总字节数。例如,7 个整数字段每次操作使用 48 字节。
更新
问题并不在于结构体复制:
我写了另一个测试用例来检查:
// (以下代码部分未做翻译)
结果中分配 = 0
<details>
<summary>英文:</summary>
I made a simple benchmark to test HashSet<T>.Add method and for me results looks strange.
For the benchmarking I use BenchmarkDotNet available on github.
Here is a full code of a benchmark:
```C#
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Attributes;
using System.Collections.Generic;
public struct Point
{
public int X, Y;
}
[MemoryDiagnoser]
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run<Program>();
}
[Params(10, 500, 1000)]
public int ArrayLength { get; set; }
[GlobalSetup]
public void Setup()
{
hs = new HashSet<Point>(100); // Capacity is 100 to definitely have space for 1 element
p = new Point(); // Even point is struct I initialize it in Setup
hs.Add(p); // Do warm-up run to be sure at least 1 element was there
}
Point p;
HashSet<Point> hs;
[Benchmark]
public void Struct()
{
for (var i = 0; i < ArrayLength; i++)
{ // The test do the same operation multiple times
hs.Clear(); // Clear hashset so there will be 0 elements
hs.Add(p); // Add 1 element back
}
}
}
My expectation is that as HashSet already initialized with enough capacity AND I do this add only once before cleanup - there should be no additional allocations at all.
But in reality here is a benchmark result executed with the command
dotnet run -c Release
Method | ArrayLength | Mean | Error | StdDev | Median | Gen 0 | Allocated |
---|---|---|---|---|---|---|---|
Struct | 10 | 644.2 ns | 12.90 ns | 28.86 ns | 633.8 ns | 0.0572 | 240 B |
Struct | 500 | 30,080.1 ns | 151.63 ns | 134.42 ns | 30,096.3 ns | 2.8687 | 12,000 B |
Struct | 1000 | 64,610.0 ns | 1,264.29 ns | 1,352.78 ns | 64,436.2 ns | 5.7373 | 24,000 B |
The size of allocation bytes equal to number of iterations * 24. So each 'Add' do new allocation.
A few comments:
- I tried the same benchmark with just Clear - 0 allocations, so it is definitely added by Add
- Changing the number of fields in struct DO change the total amount of allocated bytes. For example 7 int fields uses 48 bytes per operation.
UPDATE
The problem is not in struct copy:
I wrote another testcase to check that:
private Point t;
public void Test(Point p){
t = p;
}
[Benchmark]
public void Struct()
{
var test = new Point(1, 2);
Test(test);
}
And in result Allocations = 0
答案1
得分: 1
> 作为确认,我进行了以下测试[...] 结果分配 = 0
您的测试将副本存储在堆栈上(在Struct
中)和始终存在于类字段中(在Test
中),所以当然您不会获得任何增量分配。这两个条目将始终占用内存。
> 据我理解,这个'已分配'列只显示托管内存
结构是托管内存,所有包含它们的集合也是如此。也许更好的测试,以可视化.NET中内存分配的工作方式,是将大量值对象(struct
)添加到列表中,并观察分配的(托管)内存增加。
英文:
> As a confirmation I did the following test [...] And in result Allocation = 0
Your test stores the copy on the stack (in Struct
) and in a class field that's always there (in Test
), so of course you won't get any delta allocations. Both entries will always be there using memory.
> And as I understand this 'Allocated' column shows only managed memory
Structs are managed memory, as are all collections holding them. A better test to perhaps visualize how memory allocation works in .Net would be to add a bunch of value objects (struct
) to a list and watch allocated (managed) memory going up.
答案2
得分: 1
问题已找到:
HashSet 使用 GetHashCode 方法(真是个惊喜 )。但是这个方法是在 Object 级别上定义的。
要运行来自 Object 的方法,.net 需要对结构进行装箱,这会在执行期间分配内存。
最终的测试证明了这一点:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Attributes;
using System.Collections.Generic;
using Test;
public struct Point
{
public int X, Y;
}
[MemoryDiagnoser]
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run<Program>();
}
private Point p;
[GlobalSetup]
public void Setup()
{
p = new Point { X = 1, Y = 2 }; // 即使 point 是结构体,我也在 Setup 中初始化它
}
[Benchmark]
public void Struct()
{
var hc = p.GetHashCode();
}
}
和输出:
方法 | 平均值 | 错误 | 标准偏差 | Gen 0 | 分配的内存 |
---|---|---|---|---|---|
Struct | 40.03 ns | 0.559 ns | 0.523 ns | 0.0057 | 24 B |
解决方案:
要解决这个问题,应该为结构体重写 GetHashCode 方法。
还别忘了重写 bool Equals(object obj) 和 IEquatable<Point>,因为 HashSet.Contains 也使用这些方法。
英文:
Ok, the problem is found:
HashSet is using GetHashCode method (what a surprise ). But this method is defined on Object level.
To run methods from Object, .net requires to do boxing for struct and this is the memory that is allocated during the execution.
Final test that proves it:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Attributes;
using System.Collections.Generic;
using Test;
public struct Point
{
public int X, Y;
}
[MemoryDiagnoser]
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run<Program>();
}
private Point p;
[GlobalSetup]
public void Setup()
{
p = new Point { X = 1, Y = 2 }; // Even point is struct I initialize it in Setup
}
[Benchmark]
public void Struct()
{
var hc = p.GetHashCode();
}
}
And the output:
Method | Mean | Error | StdDev | Gen 0 | Allocated |
---|---|---|---|---|---|
Struct | 40.03 ns | 0.559 ns | 0.523 ns | 0.0057 | 24 B |
SOLUTION
To solve the issue GethashCode should be overriden for the struct.
Also do not forget to override bool Equals(object obj), and IEquatable<Point> as HashSet.Contains also use those methods.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论