如何测量函数运行时间

huangapple go评论85阅读模式
英文:

How to measure function run times

问题

在一个 Golang 的 Web 服务器中,我想要测量一些 HTTP 控制器的执行时间。我在调用控制器函数之前使用 time.Now(),在控制器函数返回后使用 time.Since()。但是,如果控制器中有长时间的远程 I/O 请求(耗时1秒),或者进程被限制,或者控制器使用 goroutine 进行并行处理,那么测量得到的时间可能不是我想要的准确时间。

如果我们假设类似于 Bash 的 time 命令,那么使用这种技术我可以得到 real 时间:

time go build

real    0m5.204s
user    0m12.012s
sys     0m2.043s

我该如何在 Golang 程序中测量函数运行的 usersys 时间(最好是针对一个 goroutine 及其派生的子 goroutine)(最好使用标准库)?

这是我的性能分析器实现。我该如何扩展它以包含每个 goroutine 的 sysuser 时间?

const HeaderCost = "Cost"

// Timed 中间件将在 HTTP 响应的头部设置 Cost 字段
func Timed(h http.Handler) http.HandlerFunc {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		h.ServeHTTP(&responseWriterWithTimer{
			ResponseWriter: w,
			headerWritten:  false,
			startedAt:      time.Now(),
		}, r)
	})
}

type responseWriterWithTimer struct {
	http.ResponseWriter
	headerWritten bool
	startedAt     time.Time
}

func (w *responseWriterWithTimer) WriteHeader(statusCode int) {
	w.Header().Set(
		HeaderCost,
		strconv.FormatFloat(
			time.Since(w.startedAt).Seconds(),
			'g',
			64,
			64,
		),
	)
	w.ResponseWriter.WriteHeader(statusCode)
	w.headerWritten = true
}

func (w *responseWriterWithTimer) Write(b []byte) (int, error) {
	if !w.headerWritten {
		w.WriteHeader(http.StatusOK)
	}
	return w.ResponseWriter.Write(b)
}
英文:

In a golang web server I want to measure times taken by some http controller. I am calling time.Now() before calling controller function, and time.Since() after controller function returns. But if it has long remote io request that takes 1 second, or the process is throttled, or controller is parallelized with goroutines - then that time will be not exactly what I want.

If we assume analogy to bash time command - then I am getting real time with this technique:

time go build

real	0m5,204s
user	0m12,012s
sys	0m2,043s

How can I measure user and sys times for a function run(preferably for a goroutine plus its forked children) in a golang program (preferably with standard packages)?

this is my profiler implementation. How can i extend it with sys and user time per goroutine?

const HeaderCost = "Cost"

// Timed middleware will set Cost header in http response
func Timed(h http.Handler) http.HandlerFunc {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		h.ServeHTTP(&responseWriterWithTimer{
			ResponseWriter: w,
			headerWritten:  false,
			startedAt:      time.Now(),
		}, r)
	})
}

type responseWriterWithTimer struct {
	http.ResponseWriter
	headerWritten bool
	startedAt     time.Time
}

func (w *responseWriterWithTimer) WriteHeader(statusCode int) {
	w.Header().Set(
		HeaderCost,
		strconv.FormatFloat(
			time.Since(w.startedAt).Seconds(),
			'g',
			64,
			64,
		),
	)
	w.ResponseWriter.WriteHeader(statusCode)
	w.headerWritten = true
}

func (w *responseWriterWithTimer) Write(b []byte) (int, error) {
	if !w.headerWritten {
		w.WriteHeader(http.StatusOK)
	}
	return w.ResponseWriter.Write(b)
}

答案1

得分: 4

如果你想在运行时进行基本的仪器化,你可以包装你的处理程序来测量它们的执行时间:

func perfMiddleware(h http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        t := time.Now()
        h.ServeHTTP(w, r)
        log.Printf("handler took %s", time.Since(t))
    })
}

你可以使用expvar更轻松地公开这个功能。除此之外,如果你寻找的话,还有许多可用于Go的仪器化/遥测/APM库,以及度量管理解决方案,如TICK堆栈、Datadog等等。

至于time输出的realusersys数据,这些是posix度量标准,不完全适用于在运行时仪器化Go的HTTP处理程序(或任何其他代码单元),原因如下:

  • goroutine之间没有父/子关系;它们都是平等的同级,因此没有衡量你的处理程序的“子级”所花费的时间的指标。
  • 大部分I/O在stdlib中处理,stdlib没有被仪器化到这个级别(而在这个级别进行仪器化本身会有非常可观的性能影响)。

当然,你可以分别对每个部分进行仪器化,这通常更有用;例如,对你的HTTP处理程序进行仪器化,以及对任何进行自己的外部请求的代码进行仪器化,以测量每个组件的性能。通过这样,你可以分析数据并更清楚地了解哪些部分花费了时间,以解决你发现的任何性能问题。

英文:

If you want to do basic instrumentation at runtime, you can wrap your handlers to measure their execution time:

func perfMiddleware(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		t := time.Now()
		h.ServeHTTP(w, r)
		log.Printf("handler took %s", time.Since(t))
	})
}

You could expose this more easily using expvar. Going beyond this, there are also numerous instrumentation/telemetry/APM libraries available for Go if you look for them, along with metrics management solutions like the TICK stack, Datadog, and so on.

As for the real, user, and sys data output by time, these are posix measures that don't perfectly apply to instrumenting a Go HTTP handler (or any other unit of code at runtime), for a number of reasons:

  • goroutines have no parent/child relationship; all are equal peers, so there is no metric of the time taken by "children" of your handler.
  • most of the I/O is handled within the stdlib, which isn't instrumented to this level (and instrumentation at this level would have a non-negligible performance impact of its own)

You can of course instrument each piece individually, which is often more useful; for example, instrument your HTTP handlers, as well as any code that is making its own external requests, in order to measure the performance of each component. From this you can analyze the data and get a much clearer picture of what is taking time, in order to address any performance issues you find.

答案2

得分: 2

如果你想单独测量某个东西,基准测试可能正是你想要的。

如果你想测量一个 http.Handler,你可以使用 httptest.NewRecorderhttptest.NewRequest 来创建一个新的响应写入器和请求对象,并直接在基准测试中调用处理程序。

func BenchmnarkHttpHandler(b *testing.B) {

  req := httptest.NewRequest("GET", "/foo", nil)

  myHandler := thingtotest.Handler{}

  for n := 0; n < b.N; n++ {
     myHandler.ServeHTTP(httptest.NewRecorder(), req);
  }
}
英文:

If you want to measure something in isolation, benchmarks are probably exactly what you're after.

If you're trying to measure a http.Handler, you can use httptest.NewRecorder and httptest.NewRequest to create a new response writer and request object and just invoke the handler directly inside your benchmark.

func BenchmnarkHttpHandler(b*testing.B) {

  req := httptest.NewRequest(&quot;GET&quot;, &quot;/foo&quot;, nil)

  myHandler := thingtotest.Handler{}

  for n := 0; n &lt; b.N; n++ {
     myHandler.ServeHTTP(httptest.NewRecorder(), req);
  }
}

答案3

得分: 2

你无法测量Go函数的用户时间和系统时间。这种区分对于Go函数来说是不可观测的。

(但说实话:测量它们没有真正的用处,也没有太多意义。这听起来像是一个XY问题。)

英文:

> How can I measure user and sys times for a function run

You cannot. That distinction is not an observable for Go functions.

(But honestly: Measuring them is of no real use and doesn't make much sense. This sound like a XY problem.)

huangapple
  • 本文由 发表于 2021年11月18日 00:04:22
  • 转载请务必保留本文链接:https://go.coder-hub.com/70007779.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定