英文:
Solving data race in benchmark function that simulates stream
问题
我正在尝试编写一个函数，用于对流式传输的CSV数据进行基准测试，并将其发送到HTTP端点。
为了做到这一点，我想生成数据并将其POST出去。
然而，go的数据竞争检测器显示存在数据竞争，并且基准测试完成得比我认为的合理时间要快，所以我猜测HTTP请求没有被正确处理。
我应该如何组织我的测试代码以避免这个问题？
有没有一种方法可以等待HTTP客户端调用被处理完毕？
func BenchmarkStream(b *testing.B) {
header := "header\n"
buf := bytes.NewBufferString(header)
var wg sync.WaitGroup
wg.Add(1)
go func() {
for i := 0; i < b.N; i++ {
buf.WriteString(fmt.Sprintf("%d\n", i+1))
}
wg.Done()
}() // 这一行在数据竞争检测器中被提到
w := httptest.NewRecorder()
r, _ := http.NewRequest("POST", "/", buf)
h := &MyHandler{}
h.ServeHTTP(w, r)
wg.Wait()
if w.Code != 200 {
b.Errorf("test failed")
}
}
编辑：@Grzegorz Żurs的评论让我对我的方法产生了疑问，我使用了io.Pipe进行了重构：
func BenchmarkStream(b *testing.B) {
pr, pw := io.Pipe()
go func() {
pw.Write([]byte("header\n"))
for i := 0; i < b.N; i++ {
pw.Write([]byte(fmt.Sprintf("%d\n", i+1)))
}
}()
w := httptest.NewRecorder()
r, _ := http.NewRequest("POST", "/", pr)
h := &MyHandler{}
h.ServeHTTP(w, r)
if w.Code != 200 {
b.Errorf("test failed")
}
}
英文:
I'm trying to write a function that basically benchmarks streamed CSV over to a HTTP endpoint.
To do this I want to generate data and POST that data.
However, gos data race detector says that there is a data race and the benchmark finishes faster than I would believe is reasonable, so I guess the HTTP request is not properly processed.
How should I structure my test code to avoid this?
Is there a way to wait until the HTTP client call has been processed?
func BenchmarkStream(b *testing.B) {
header := "header\n"
buf := bytes.NewBufferString(header)
var wg sync.WaitGroup
wg.Add(1)
go func() {
for i := 0; i < b.N; i++ {
buf.WriteString(fmt.Sprintf("%d\n", i+1))
}
wg.Done()
}() <-- this line is mentioned in the data race detector
w := httptest.NewRecorder()
r, _ := http.NewRequest("POST", "/", buf)
h := &MyHandler{}
h.ServeHTTP(w, r)
wg.Wait()
if w.Code != 200 {
b.Errorf("test failed")
}
}
EDIT: @Grzegorz Żurs comment made me question my approach to begin with, I refactored it with an io.Pipe:
func BenchmarkStream(b *testing.B) {
pr, pw := io.Pipe()
go func() {
pw.Write([]byte("header\n"))
for i := 0; i < b.N; i++ {
pw.Write([]byte(fmt.Sprintf("%d\n", i+1)))
}
}()
w := httptest.NewRecorder()
r, _ := http.NewRequest("POST", "/", pr)
h := &MyHandler{}
h.ServeHTTP(w, r)
if w.Code != 200 {
b.Errorf("test failed")
}
}
答案1
得分: 1
你正在两个 goroutine 之间共享 buf。
英文:
You are sharing buf between two goroutines.
答案2
得分: 1
如果你只调用处理程序一次，是不可能得到有用的基准结果的。你需要先构建请求体，然后重复调用处理程序。
buf := &bytes.Buffer{}
buf.WriteString("header\n")
buf.WriteString(strings.Repeat("1\n", 1000))
body := buf.Bytes()
b.ResetTimer()
for i := 0; i < b.N; i++ {
w := httptest.NewRecorder()
r, err := http.NewRequest("POST", "/", bytes.NewReader(body))
if err != nil {
b.Fatal(err)
}
h := &MyHandler{}
h.ServeHTTP(w, r)
if w.Code != 200 {
b.Errorf("test failed")
}
}
希望对你有帮助！
英文:
You're not going to get useful benchmark results if you only invoke the handler once. Build the request body once and then invoke your handler over and over again.
buf := &bytes.Buffer{}
buf.WriteString("header\n")
buf.WriteString(strings.Repeat("1\n", 1000)
body := buf.Bytes()
b.ResetTimer()
for i := 0; i < b.N; i++ {
w := httptest.NewRecorder()
r, err := http.NewRequest("POST", "/", bytes.NewReader(body))
if err != nil {
b.Fatal(err)
}
h := &MyHandler{}
h.ServeHTTP(w, r)
if w.Code != 200 {
b.Errorf("test failed")
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论