将gzip压缩应用于http.ResponseWriter。

huangapple
go评论131阅读模式
英文:

gzip compression to http responseWriter

问题

我是你的中文翻译助手，以下是翻译好的内容：

我刚开始学习Go语言，但正在使用REST API进行实践。我在两个函数中使用json.Marshal和json.Encoder时无法获得相同的行为。

我想使用以下函数来对响应进行gzip压缩：

  1. func gzipFast(a *[]byte) []byte {
  2.     var b bytes.Buffer
  3.     gz := gzip.NewWriter(&b)
  4.     defer gz.Close()
  5.     if _, err := gz.Write(*a); err != nil {
  6.         panic(err)
  7.     }
  8.     return b.Bytes()
  9. }

但是这个函数返回的结果是：

  1. curl http://localhost:8081/compressedget --compressed --verbose
  2. *   Trying 127.0.0.1...
  3. * Connected to localhost (127.0.0.1) port 8081 (#0)
  4. > GET /compressedget HTTP/1.1
  5. > Host: localhost:8081
  6. > User-Agent: curl/7.47.0
  7. > Accept: */*
  8. > Accept-Encoding: deflate, gzip
  9. > 
  10. < HTTP/1.1 200 OK
  11. < Content-Encoding: gzip
  12. < Content-Type: application/json
  13. < Date: Wed, 24 Aug 2016 00:59:38 GMT
  14. < Content-Length: 30
  15. < 
  16. * Connection #0 to host localhost left intact

以下是Go函数的代码：

  1. func CompressedGet(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
  3.     box := Box{Width: 10, Height: 20, Color: "green", Open: false}
  4.     box.ars = make([]int, 100)
  5.     for i := range box.ars {
  6.         box.ars[i] = i
  7.     }
  8.     //fmt.Println(r.Header.Get("Content-Encoding"))
  10.     w.Header().Set("Content-Type", "application/json")
  11.     w.Header().Set("Content-Encoding", "gzip")
  12.     b, _ := json.Marshal(box)
  13.     //fmt.Println(len(b))
  14.     //fmt.Println(len(gzipFast(&b)))
  16.     fmt.Fprint(w, gzipFast(&b))
  17.     //fmt.Println(len(gzipSlow(b)))
  18.     //gz := gzip.NewWriter(w)
  19.     //defer gz.Close()
  20.     //json.NewEncoder(gz).Encode(box)
  21.     r.Body.Close()
  23. }

但是当我取消以下代码的注释时：

  1. //gz := gzip.NewWriter(w)
  2. //defer gz.Close()
  3. //json.NewEncoder(gz).Encode(box)

它可以正常工作。

英文:

I'm new to Go. But am playing with a REST Api. I cant get the same behavior out of json.Marshal as json.Encoder in two functions i have

I wanted to use this function to gzip my responses:

  1. func gzipFast(a *[]byte) []byte {
  2. 	var b bytes.Buffer
  3. 	gz := gzip.NewWriter(&amp;b)
  4. 	defer gz.Close()
  5. 	if _, err := gz.Write(*a); err != nil {
  6. 		panic(err)
  7. 	}
  8. 	return b.Bytes()
  9. }

But this function returns this:

  1. curl http://localhost:8081/compressedget --compressed --verbose
  2. *   Trying 127.0.0.1...
  3. * Connected to localhost (127.0.0.1) port 8081 (#0)
  4. &gt; GET /compressedget HTTP/1.1
  5. &gt; Host: localhost:8081
  6. &gt; User-Agent: curl/7.47.0
  7. &gt; Accept: */*
  8. &gt; Accept-Encoding: deflate, gzip
  9. &gt; 
  10. &lt; HTTP/1.1 200 OK
  11. &lt; Content-Encoding: gzip
  12. &lt; Content-Type: application/json
  13. &lt; Date: Wed, 24 Aug 2016 00:59:38 GMT
  14. &lt; Content-Length: 30
  15. &lt; 
  16. * Connection #0 to host localhost left intact

Here is the go function:

  1. func CompressedGet(w http.ResponseWriter, r *http.Request, ps  httprouter.Params) {
  3. 	box := Box{Width: 10, Height: 20, Color: &quot;gree&quot;, Open: false}
  4. 	box.ars = make([]int, 100)
  5. 	for i := range box.ars {
  6. 		box.ars[i] = i
  7. 	}
  8. 	//fmt.Println(r.Header.Get(&quot;Content-Encoding&quot;))
  10. 	w.Header().Set(&quot;Content-Type&quot;, &quot;application/json&quot;)
  11. 	w.Header().Set(&quot;Content-Encoding&quot;, &quot;gzip&quot;)
  12. 	b, _ := json.Marshal(box)
  13. 	//fmt.Println(len(b))
  14. 	//fmt.Println(len(gzipFast(&amp;b)))
  16. 	fmt.Fprint(w, gzipFast(&amp;b))
  17. 	//fmt.Println(len(gzipSlow(b)))
  18. 	//gz := gzip.NewWriter(w)
  19. 	//defer gz.Close()
  20. 	//json.NewEncoder(gz).Encode(box)
  21. 	r.Body.Close()
  23. }

But when i uncomment:

  1. //gz := gzip.NewWriter(w)
  2. //defer gz.Close()
  3. //json.NewEncoder(gz).Encode(box)

it works fine.

答案1

得分: 2

我会避免手动对[]byte进行gzip压缩。你可以轻松地使用标准库中已经存在的写入器。此外，你可以看一下compress/flate，我认为它应该用于替代gzip。

  1. package main
  3. import (
  4. 	"net/http"
  5. 	"encoding/json"
  6. 	"compress/gzip"
  7. 	"log"
  8. )
  10. type Box struct {
  11. 	Width int `json:"width"`
  12. }
  14. func writeJsonResponseCompressed(w http.ResponseWriter, r *http.Request) {
  16. 	box := &Box{Width: 10}
  18. 	w.Header().Set("Content-Type", "application/json")
  19. 	w.Header().Set("Content-Encoding", "gzip")
  21. 	body, err := json.Marshal(box)
  22. 	if err != nil {
  23. 		// 处理错误
  24. 		return
  25. 	}
  27. 	writer, err := gzip.NewWriterLevel(w, gzip.BestCompression)
  28. 	if err != nil {
  29. 		// 处理错误
  30. 		return
  31. 	}
  33. 	defer writer.Close()
  35. 	writer.Write(body)
  36. }
  38. func main() {
  39. 	http.HandleFunc("/compressedget", writeJsonResponseCompressed)
  40. 	log.Fatal(http.ListenAndServe(":8081", nil))
  41. }

希望对你有帮助！

英文:

I would avoid gzip-ing []byte manually. You can easily use already existing writers from standard library. Additionally, take a look on compress/flate which I think should be use instead of gzip.

  1. package main
  3. import (
  4. 	&quot;net/http&quot;
  5. 	&quot;encoding/json&quot;
  6. 	&quot;compress/gzip&quot;
  7. 	&quot;log&quot;
  8. )
  10. type Box struct {
  11. 	Width int `json:&quot;width&quot;`
  12. }
  14. func writeJsonResponseCompressed(w http.ResponseWriter, r *http.Request) {
  16. 	box := &amp;Box{Width: 10}
  18. 	w.Header().Set(&quot;Content-Type&quot;, &quot;application/json&quot;)
  19. 	w.Header().Set(&quot;Content-Encoding&quot;, &quot;gzip&quot;)
  21. 	body, err := json.Marshal(box)
  22. 	if err != nil {
  23. 		// Your error handling
  24. 		return
  25. 	}
  27. 	writer, err := gzip.NewWriterLevel(w, gzip.BestCompression)
  28. 	if err != nil {
  29. 		// Your error handling
  30. 		return
  31. 	}
  33. 	defer writer.Close()
  35. 	writer.Write(body)
  36. }
  38. func main() {
  39. 	http.HandleFunc(&quot;/compressedget&quot;, writeJsonResponseCompressed)
  40. 	log.Fatal(http.ListenAndServe(&quot;:8081&quot;, nil))
  41. }

答案2

得分: 1

我认为问题出在fmt.Fprint(w, gzipFast(&b))的使用上。

如果你查看gzipFast的定义，它返回一个[]byte。你将这个切片放入了打印函数中，该函数会将所有内容“打印”到w中。

如果你查看io.Writer的定义：

  1. type Writer interface {
  2.     Write(p []byte) (n int, err error)
  3. }

你会发现写入器可以处理[]byte作为输入。

所以，你应该使用w.Write(gzipFast(&b))而不是fmt.Fprint(w, gzipFast(&b))。然后你就不需要取消注释以下代码：

  1. //gz := gzip.NewWriter(w)
  2. //defer gz.Close()
  3. //json.NewEncoder(gz).Encode(box)

这里有一个小例子，展示了你的代码中发生了什么：

https://play.golang.org/p/6rzqLWTGiI

英文:

I think the problem is the use of fmt.Fprint(w, gzipFast(&amp;b)).

If you look to the definition of gzipFast it returns a []byte. You are putting this slice into the print function, which is "printing" everything into w.

If you look at the definition of the io.Writer:

  1. type Writer interface {
  2.         Write(p []byte) (n int, err error) }

You see that the writer can handle []byte as input.

Instead of fmt.Fprint(w, gzipFast(&amp;b)) you should use w.Write(gzipFast(&amp;b)). Then you don't need to uncomment:

  1. //gz := gzip.NewWriter(w)
  2. //defer gz.Close()
  3. //json.NewEncoder(gz).Encode(box)

Everything as a small example, which shows what is happening in your code:

https://play.golang.org/p/6rzqLWTGiI

答案3

得分: 1

在访问底层字节之前，你需要刷新或关闭gzip写入器，例如：

  1. func gzipFast(a *[]byte) []byte {
  2.     var b bytes.Buffer
  3.     gz := gzip.NewWriter(&b)
  4.     if _, err := gz.Write(*a); err != nil {
  5.         gz.Close()
  6.         panic(err)
  7.     }
  8.     gz.Close()
  9.     return b.Bytes()
  10. }

否则，已经在gzip写入器中缓冲但尚未写入最终流的内容将不会被收集起来。

英文:

You need to flush or close your gzip writer before accessing the underlying bytes, e.g.

  1. func gzipFast(a *[]byte) []byte {
  2.     var b bytes.Buffer
  3.     gz := gzip.NewWriter(&amp;b)
  4.     if _, err := gz.Write(*a); err != nil {
  5.         gz.Close()
  6.         panic(err)
  7.     }
  8.     gz.Close()
  9.     return b.Bytes()
  10. }

Otherwise what's been buffer in the gzip writer, but not yet written out to the final stream isn't getting collected up.

huangapple
  • 本文由 发表于 2016年8月24日 09:06:59
  • 转载请务必保留本文链接：https://go.coder-hub.com/39112912.html
  • go
  • gzip
  • http
  • json
  • rest
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定