Count sent and received bytes in Go in an http.Handler ServeHTTP function?

huangapple go评论87阅读模式
英文:

Count sent and received bytes in Go in an http.Handler ServeHTTP function?

问题

如何在Go的ServeHTTP函数中计算发送和接收的字节数?

计数需要相对准确。跳过连接建立不是理想的,但可以接受。但是头部必须包括在内。

它还需要快速。通常迭代的速度太慢。

计数本身不需要在ServeHTTP内部发生,只要给定连接的计数可以在ServeHTTP中使用即可。

这也不能破坏HTTPS或HTTP/2。

我尝试过的方法

通过迭代Request头部可以获得接收字节数的粗略、慢速估计。这种方法太慢了,而且Go标准库会删除和合并头部,所以也不准确。

我尝试编写一个拦截的Listener,它创建了一个内部的tls.Listennet.Listen监听器,其Accept()函数从内部监听器的Accept()获取一个net.Conn,然后将其包装在一个拦截的net.Conn中,其ReadWrite函数调用真正的net.Conn并计算它们的读取和写入。然后可以通过互斥的共享变量将这些计数可用于ServeHTTP函数。

问题是,拦截的Conn会破坏HTTP/2,因为Go的内部库将net.Conn强制转换为*tls.Conn(例如https://golang.org/src/net/http/server.go#L1730),而且似乎不可能在Go中同时包装对象并使该转换成功(如果可以的话,将解决此问题)。

通过计算写入ResponseWriter的内容可以相对准确地计算发送的字节数。通过Request.Body可以实现计算接收的HTTP正文字节数。关键问题在于快速准确地计算请求头字节数。虽然也计算连接建立字节数将是理想的。

这种可能吗?如何实现?

英文:

How can sent and received bytes be counted from within a ServeHTTP function in Go?

The count needs to be relatively accurate. Skipping connection establishment is not ideal, but acceptable. But headers must be included.

It also needs to be fast. Iterating is generally too slow.

The counting itself doesn't need to occur within ServeHTTP, as long the count for a given connection can be made available to ServeHTTP.

This must also not break HTTPS or HTTP/2.

Things I've Tried

It's possible to get a rough, slow estimate of received bytes by iterating over the Request headers. This is far too slow, and the Go standard library removes and combines headers, so it's not accurate either.

I tried writing an intercepting Listener, which created an internal tls.Listen or net.Listen Listener, and whose Accept() function got a net.Conn from the internal Listener's Accept(), and then wrapped that in an intercepting net.Conn whose Read and Write functions call the real net.Conn and count their reads and writes. It's then possible to make those counts available to the ServeHTTP function via mutexed shared variables.

The problem is, the intercepting Conn breaks HTTP/2, because Go's internal libraries cast the net.Conn as a *tls.Conn (e.g. https://golang.org/src/net/http/server.go#L1730), and it doesn't appear possible in Go to wrap the object while still making that cast succeed (if it is, it would solve this problem).

Counting sent bytes can be done relatively accurately by counting what is written to the ResponseWriter. Counting received bytes in the HTTP body is also achievable, via Request.Body. The critical issue here appears to be quickly and accurately counting request header bytes. Though again, also counting connection establishment bytes would be ideal.

Is this possible? How?

答案1

得分: 3

我认为这是可能的,但我不能说我已经做过。然而,根据浏览HTTP服务器和TLS监听器的stdlib实现,我不明白为什么这不可能;关键是在TLS之前而不是之后包装连接。这样还可以获得更准确的字节计数,而不是解密后的字节计数。

你已经有一个拦截的Listener,你只需要将它插入到正确的位置。而不是将你的Listener传递给http.Serve(或者你插入的任何地方),你需要首先将它传递给tls.NewListener,它会将其包装在TLS处理程序中,然后将结果传递给HTTP服务器,结果将是一个TLS监听器(使Go的HTTP/2支持正常工作)。

当然,如果你想要解密后的字节计数而不是传输字节计数,可能会有问题-仅仅包装net.Conn是不够的。你可能需要尽力计算头部和正文的字节数。

英文:

I think it is possible, but I can't say I've done it. However, based on browsing the stdlib implementation of the HTTP server and TLS listener, I don't see why it shouldn't be possible; the key is wrapping the connection before TLS instead of after. This also gets you a more accurate count of bytes on the wire, rather than a count of decrypted bytes.

You've already got an intercepting Listener, you just need to insert it in the right spot. Rather than passing your Listener to http.Serve (or wherever you're inserting it), you want to pass it to tls.NewListener first, which wraps it in the TLS handler, and then pass the result, which will be a TLS listener (making Go's HTTP/2 support happy) into the HTTP server.

Of course, if you want a count of decrypted bytes rather than wire bytes, you may be SOL - wrapping the net.Conn just won't get you there. You'll likely have to do the best you can with counting headers & body.

huangapple
  • 本文由 发表于 2017年5月13日 02:29:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/43944378.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定