英文:
How to measure both before and after byte size and time to compress
问题
我想压缩一个字符串(实际上是一个 JSON 响应)
var b bytes.Buffer
gz := gzip.NewWriter(&b)
if _, err := gz.Write([]byte("YourDataHere")); err != nil {
panic(err)
}
我该如何轻松地输出压缩前后的字节大小,更重要的是,我该如何计算压缩和解压缩回字符串所需的时间?
英文:
I want to gzip a string (it is actually a JSON response)
var b bytes.Buffer
gz := gzip.NewWriter(&b)
if _, err := gz.Write([]byte("YourDataHere")); err != nil {
panic(err)
}
How can I easily output the size of bytes before and after compression and more importantly how can I time it takes to compress and decompress back to a string?
答案1
得分: 1
你可以按照Nipun Talukdar的评论来计算大小。
len([]byte("YourDataHere")) b.Len()
使用time.Now()
和time.Since()
来获取所花费的时间。
var b bytes.Buffer
input := []byte("YourDataHere")
fmt.Println("Input size : ", len(input))
gz := gzip.NewWriter(&b)
start := time.Now()
gz.Write(input)
if _, err := gz.Flush(); err != nil {
panic(err)
}
totalTime := time.Since(start)
fmt.Println("Compressed size : ", b.Len(), "\nTime taken : ", totalTime)
gz.Close()
相同的方法也可以用于解压缩。你还可以创建一个支持函数来计时。
func timer(startTime time.Time) {
totalTime := time.Since(startTime)
log.Println("Time taken : ", totalTime)
}
用法:在函数的开头使用defer timer(time.Now())
。
英文:
You can calculate the size as per Nipun Talukdar's comment.
len([]byte("YourDataHere"))
b.Len()
And use time.Now()
and time.Since()
to get the time taken.
var b bytes.Buffer
input := []byte("YourDataHere")
fmt.Println("Input size : ", len(input))
gz := gzip.NewWriter(&b)
start := time.Now()
gz.Write(input)
if _, err := gz.Flush(); err != nil {
panic(err)
}
totalTime := time.Since(start)
fmt.Println("Compressed size : ", b.Len(), "\nTime taken : ", totalTime)
gz.Close()
Same method can be applied with unzipping.
You can also create a support function that can do the timing.
func timer(startTime time.Time) {
totalTime := time.Since(startTime)
log.Println("Time taken : ",totalTime)
}
Usage : defer timer(time.Now())
at the start of the function.
答案2
得分: 0
这里有一个在Go语言中如何实现的示例:
https://golang.org/test/bench/go1/gzip_test.go
幸运的是,它是根据BSD许可证发布的...
// 版权所有 2011 年 Go 作者。保留所有权利。
// 使用本源代码受 BSD 风格的许可证管辖,
// 可在 LICENSE 文件中找到该许可证。
// 此基准测试 gzip 和 gunzip 的性能。
package go1
import (
"bytes"
gz "compress/gzip"
"io"
"io/ioutil"
"testing"
)
var (
jsongunz = bytes.Repeat(jsonbytes, 10)
jsongz []byte
)
func init() {
var buf bytes.Buffer
c := gz.NewWriter(&buf)
c.Write(jsongunz)
c.Close()
jsongz = buf.Bytes()
}
func gzip() {
c := gz.NewWriter(ioutil.Discard)
if _, err := c.Write(jsongunz); err != nil {
panic(err)
}
if err := c.Close(); err != nil {
panic(err)
}
}
func gunzip() {
r, err := gz.NewReader(bytes.NewBuffer(jsongz))
if err != nil {
panic(err)
}
if _, err := io.Copy(ioutil.Discard, r); err != nil {
panic(err)
}
r.Close()
}
func BenchmarkGzip(b *testing.B) {
b.SetBytes(int64(len(jsongunz)))
for i := 0; i < b.N; i++ {
gzip()
}
}
func BenchmarkGunzip(b *testing.B) {
b.SetBytes(int64(len(jsongunz)))
for i := 0; i < b.N; i++ {
gunzip()
}
}
希望对你有帮助!
英文:
There are examples of how to do this in Go here
https://golang.org/test/bench/go1/gzip_test.go
Thankfully it's BSD licensed...
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This benchmark tests gzip and gunzip performance.
package go1
import (
"bytes"
gz "compress/gzip"
"io"
"io/ioutil"
"testing"
)
var (
jsongunz = bytes.Repeat(jsonbytes, 10)
jsongz []byte
)
func init() {
var buf bytes.Buffer
c := gz.NewWriter(&buf)
c.Write(jsongunz)
c.Close()
jsongz = buf.Bytes()
}
func gzip() {
c := gz.NewWriter(ioutil.Discard)
if _, err := c.Write(jsongunz); err != nil {
panic(err)
}
if err := c.Close(); err != nil {
panic(err)
}
}
func gunzip() {
r, err := gz.NewReader(bytes.NewBuffer(jsongz))
if err != nil {
panic(err)
}
if _, err := io.Copy(ioutil.Discard, r); err != nil {
panic(err)
}
r.Close()
}
func BenchmarkGzip(b *testing.B) {
b.SetBytes(int64(len(jsongunz)))
for i := 0; i < b.N; i++ {
gzip()
}
}
func BenchmarkGunzip(b *testing.B) {
b.SetBytes(int64(len(jsongunz)))
for i := 0; i < b.N; i++ {
gunzip()
}
}
答案3
得分: 0
很遗憾,gzip.Writer
无法报告写入底层流的压缩字节数。当底层流不在内存中时,情况会变得更加复杂。
为了解决这个问题,我编写了一个“计数io.Writer
”,将其放置在gzip.Writer
和底层流之间,以便我可以计算和提取已写入的压缩字节数。
在Go Playground中尝试以下代码。
package main
import (
"compress/gzip"
"fmt"
"io"
"os"
)
// countingWriter是一个计算写入总字节数的io.Writer。
type countingWriter struct {
w io.Writer
Count int
}
var _ io.Writer = &countingWriter{}
func newCountingWriter(w io.Writer) *countingWriter {
return &countingWriter{w: w}
}
func (cw *countingWriter) Write(p []byte) (int, error) {
n, err := cw.w.Write(p)
cw.Count += n
return n, err
}
func ExampleUse(w io.Writer) (int, error) {
cw := newCountingWriter(w)
zw, err := gzip.NewWriterLevel(cw, gzip.BestCompression)
if err != nil {
return 0, err
}
if _, err := zw.Write([]byte("hello world")); err != nil {
return cw.Count, err
}
err = zw.Close()
return cw.Count, err
}
func main() {
n, err := ExampleUse(os.Stderr)
if err != nil {
panic(err)
}
fmt.Printf("写入了%d个字节\n", n)
}
英文:
It's unfortunate that gzip.Writer
can't report the number of compressed bytes written to the underlying stream. It gets more complicated when that underlying stream is not in-memory.
To solve this, I wrote a "counting io.Writer
" that I place in between gzip.Writer
and the underlying stream, so I can count and extract the number of compressed bytes written.
Try out the following code in the Go Playground.
package main
import (
"compress/gzip"
"fmt"
"io"
"os"
)
// countingWriter is an io.Writer that counts the total bytes written to it.
type countingWriter struct {
w io.Writer
Count int
}
var _ io.Writer = &countingWriter{}
func newCountingWriter(w io.Writer) *countingWriter {
return &countingWriter{w: w}
}
func (cw *countingWriter) Write(p []byte) (int, error) {
n, err := cw.w.Write(p)
cw.Count += n
return n, err
}
func ExampleUse(w io.Writer) (int, error) {
cw := newCountingWriter(w)
zw, err := gzip.NewWriterLevel(cw, gzip.BestCompression)
if err != nil {
return 0, err
}
if _, err := zw.Write([]byte("hello world")); err != nil {
return cw.Count, err
}
err = zw.Close()
return cw.Count, err
}
func main() {
n, err := ExampleUse(os.Stderr)
if err != nil {
panic(err)
}
fmt.Printf("wrote %d bytes\n", n)
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论