Go HTTP服务器测试ab与wrk结果差异如此之大

huangapple go评论83阅读模式
英文:

Go HTTP server testing ab vs wrk so much difference in result

问题

我正在尝试查看我的机器上go HTTP服务器可以处理多少个请求,所以我尝试进行一些测试,但结果差异太大,让我感到困惑。

首先,我尝试使用ab进行基准测试,并运行以下命令:

$ ab -n 100000 -c 1000 http://127.0.0.1/

进行1000个并发请求。

结果如下:

并发级别:1000
测试所用时间:12.055秒
完成的请求数:100000
失败的请求数:0
写入错误数:0
总传输字节数:12800000字节
HTML传输字节数:1100000字节
每秒请求数:8295.15 [#/sec] (平均值)
每个请求的时间:120.552 [ms] (平均值)
每个请求的时间:0.121 [ms] (平均值,所有并发请求)
传输速率:1036.89 [Kbytes/sec](接收)

每秒处理8295个请求,这似乎是合理的。

但是,然后我尝试在wrk上运行它,使用以下命令:

$ wrk -t1 -c1000 -d5s http://127.0.0.1:80/

我得到以下结果:

在 http://127.0.0.1:80/ 上运行5秒测试
  1个线程和1000个连接
  线程统计信息   平均值      标准差     最大值   +/- 标准差
    延迟       18.92毫秒   13.38毫秒 234.65毫秒   94.89%
    每秒请求数  27.03k     1.43k   29.73k    63.27%
  5.10秒内完成136475个请求,读取16.66MB
每秒请求数:26767.50
每秒传输速率:3.27MB

每秒处理26767个请求?我不明白为什么会有这么大的差异。

代码运行的是最简单的Go服务器:

package main

import (
	"net/http"
)

func main() {

	http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
		w.Write([]byte("Hello World"))
	})

	http.ListenAndServe(":80", nil)
}

我的目标是在增加核心数时查看go服务器可以处理多少个请求,但在我甚至开始增加CPU性能之前,差异就太大了。有人知道在增加更多核心时Go服务器如何扩展吗?还有为什么ab和wrk之间存在如此巨大的差异吗?

英文:

I am trying to see how many requests the go HTTP server can handle on my machine so I try to do some test but the difference is so large that I am confused.

First I try to bench with ab and run this command

$ ab -n 100000 -c 1000 http://127.0.0.1/

Doing 1000 concurrent requests.

The result is as follows:

Concurrency Level:      1000
Time taken for tests:   12.055 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      12800000 bytes
HTML transferred:       1100000 bytes
Requests per second:    8295.15 [#/sec] (mean)
Time per request:       120.552 [ms] (mean)
Time per request:       0.121 [ms] (mean, across all concurrent requests)
Transfer rate:          1036.89 [Kbytes/sec] received

8295 requests per second which seems reasonable.

But then I try to run it on wrk with this command:

$ wrk -t1 -c1000 -d5s http://127.0.0.1:80/

And I get these results:

Running 5s test @ http://127.0.0.1:80/
  1 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    18.92ms   13.38ms 234.65ms   94.89%
    Req/Sec    27.03k     1.43k   29.73k    63.27%
  136475 requests in 5.10s, 16.66MB read
Requests/sec:  26767.50
Transfer/sec:      3.27MB

26767 requests per second? I don't understand why there is such a huge difference.

The code run was the simplest Go server

package main

import (
	"net/http"
)

func main() {

	http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
		w.Write([]byte("Hello World"))
	})

	http.ListenAndServe(":80", nil)
}

My goal is to see how many requests the go server can handle as I increase the cores, but this is just too much of a difference before I even start adding more CPU power. Does anyone know how the Go server scales when adding more cores? And also why the huge difference between ab and wrk?

答案1

得分: 33

首先,基准测试往往相当人为。一旦你开始添加数据库调用、模板渲染、会话解析等操作,返回一小部分字节将会导致非常不同的结果(预计相差一个数量级)。

然后还要考虑本地问题 - 开发机器与生产环境的文件/套接字限制、基准测试工具(ab/wrk)与Go服务器之间对这些资源的竞争、本地环回适配器或操作系统TCP堆栈(以及TCP堆栈调优)等等。问题还有很多!

此外:

  • ab 并不受高度评价
  • 它仅支持 HTTP/1.0,因此不支持 keepalive
  • 你的其他指标差异很大 - 例如,看看每个工具报告的平均延迟 - ab 的延迟要高得多
  • 你的 ab 测试运行时间为 12s,而 wrk 测试运行时间为 5s
  • 即使每秒处理 8k 个请求也是巨大的负载 - 每小时达到 2800 万个请求。即使在执行数据库调用、编组 JSON 结构等操作后,每秒处理的请求数降低到 3k/req/s,你仍然能够处理大量负载。不要过于纠结于这种早期的基准测试。

我不知道你使用的是什么样的机器,但我的 iMac(搭载 3.5GHz i7-4771)在单个线程上可以达到超过 64k req/s,响应内容为 w.Write([]byte("Hello World\n"))

简而言之:使用 wrk,并记住基准测试工具存在很大的差异性。

英文:

Firstly: benchmarks are often pretty artificial. Sending back a handful of bytes is going to net you very different results once you start adding database calls, template rendering, session parsing, etc. (expect an order of magnitude difference)

Then tack on local issues - open file/socket limits on your dev machine vs. production, competition between your benchmarking tool (ab/wrk) and your Go server for those resources, the local loopback adapter or OS TCP stacks (and TCP stack tuning), etc. It goes on!

In addition:

  • ab is not highly regarded
  • It is HTTP/1.0 only, and therefore doesn't do keepalives
  • Your other metrics vary wildly - e.g. look at the avg latency reported by each tool - ab has a much higher latency
  • Your ab test also runs for 12s and not the 5s your wrk test does.
  • Even 8k req/s is a huge amount of load - that's 28 million requests an hour. Even if—after making a DB call, marshalling a JSON struct, etc—that went down to 3k/req/s you'd still be able to handle significant amounts of load. Don't get too tied up in these kind of benchmarks this early.

I have no idea what kind of machine you're on, but my iMac with a 3.5GHz i7-4771 can push upwards of 64k req/s on a single thread responding with a w.Write([]byte("Hello World\n"))

Short answer: use wrk and keep in mind that benchmarking tools have a lot of variance.

答案2

得分: 1

主要区别在于,默认情况下ab使用HTTP/1.0,因此在每个请求之后关闭每次传输,而wrk使用HTTP/1.1,因此保持连接并重用它进行下一个请求。

因此,添加-k开关,你会看到类似的数字,我猜测:

$ ab -n 100000 -c 1000 -k http://127.0.0.1/
英文:

The main difference is that by default ab uses HTTP/1.0, so close each transmission after each request, whereas wrk uses HTTP/1.1, so keep alive the connection and reuse it for the next request.

So add the -k switch, and you will see similar numbers, I guess:

$ ab -n 100000 -c 1000 -k http://127.0.0.1/

huangapple
  • 本文由 发表于 2015年7月2日 09:03:23
  • 转载请务必保留本文链接:https://go.coder-hub.com/31174076.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定