堆叠写入器和zlib写入器的校验和错误。

huangapple
go评论163阅读模式
英文:

Wrong checksum on stacked writer & zlib.writer

问题

我有两个对象写入器，第一个封装了io.Writer，在写入时还计算内容的SHA1校验和，第二个封装了第一个写入器，并写入了zlib压缩的数据。

我的单元测试用例对于第一个写入器的一些测试数据通过了，但对于第二个写入器失败了。

我在哪里做错了？

英文:

I have two object writers, first one encapsulates io.Writer and along writes it also coputes SHA1 checksum of content, and second encapsulates first and writes zlib compressed data.

My unit test case with some test data passes for first writer, but fails with second.

Where and what I'm doing wrong?

code snippet:

  1. import (
  2.     "compress/zlib"
  3.     "crypto/sha1"
  4.     "hash"
  5.     "io"
  6. )
  8. type ObjectWriter interface {
  9.     io.Writer
  10.     Commit() ([]byte, error)
  11. }
  13. type oWriter struct {
  14.     writer io.Writer
  15.     sum    hash.Hash
  16. }
  18. func (ow *oWriter) Write(b []byte) (int, error) {
  19.     ow.sum.Write(b)  // Hash writer never returs error -> hash.Hash docs
  20.     return ow.writer.Write(b)
  21. }
  23. func (ow *oWriter) Commit() ([]byte, error) {
  24.     return ow.sum.Sum(nil), nil
  25. }
  27. func NewWriter(w io.Writer) ObjectWriter {
  28.     return &oWriter{w, sha1.New()}
  29. }
  31. type compressedWriter struct {
  32.     oWriter ObjectWriter
  33.     zWriter io.WriteCloser
  34. }
  36. func (ow *compressedWriter) Write(b []byte) (int, error) {
  37.     return ow.zWriter.Write(b)
  38. }
  40. func (ow *compressedWriter) Commit() ([]byte, error) {
  41.     if err := ow.zWriter.Close(); err != nil {
  42.         return nil, err
  43.     }
  45.     return ow.oWriter.Commit()
  46. }
  48. func NewCompressedWriter(w io.Writer) ObjectWriter {
  49.     ow := NewWriter(w)
  50.     zw := zlib.NewWriter(ow)
  52.     return &compressedWriter{ow, zw}
  53. }

passing test case:

  1. func TestObjectWriter(t *testing.T) {
  2.     var buf bytes.Buffer
  4.     ow := NewWriter(&buf)
  5.     ow.Write([]byte("test content"))
  6.     sum, err := ow.Commit()
  8.     if err != nil {
  9.         t.Errorf("Commit error: %s", err)
  10.     }
  12.     expected := "1eebdf4fdc9fc7bf283031b93f9aef3338de9052"
  13.     given := fmt.Sprintf("%x", sum)
  15.     if  expected != given {
  16.         t.Errorf("Invalid SHA1 sum: <%s> != <%s>", expected, given)
  17.     }
  18. }

failing test case:

  1. func TestCompressedObjectWriter(t *testing.T) {
  2.     var buf bytes.Buffer
  4.     ow := NewCompressedWriter(&buf)
  5.     ow.Write([]byte("test content"))
  6.     sum, err := ow.Commit()
  8.     if err != nil {
  9.         t.Errorf("Commit error: %s", err)
  10.     }
  12.     expected := "7efbe4015afd95478282c3774f47b4195031d27e"
  13.     given := fmt.Sprintf("%x", sum)
  15.     if  expected != given {
  16.         t.Errorf("Invalid SHA1 sum: <%s> != <%s>", expected, given)
  17.     }
  18. }

答案1

得分: 0

你的测试输出是什么？

你是如何计算你的期望值的？

例如，

  1. package main
  3. import (
  4. 	"bytes"
  5. 	"compress/zlib"
  6. 	"crypto/sha1"
  7. 	"fmt"
  8. )
  10. func ObjectHash(s string) (string, error) {
  11. 	var buf bytes.Buffer
  12. 	_, err := buf.Write([]byte(s))
  13. 	if err != nil {
  14. 		return "", err
  15. 	}
  16. 	hash := sha1.New()
  17. 	hash.Write(buf.Bytes())
  18. 	sum := fmt.Sprintf("%x", hash.Sum(nil))
  19. 	return sum, nil
  20. }
  22. func CompressedHash(s string) (string, error) {
  23. 	var buf bytes.Buffer
  24. 	zw := zlib.NewWriter(&buf)
  25. 	_, err := zw.Write([]byte(s))
  26. 	if err != nil {
  27. 		return "", err
  28. 	}
  29. 	zw.Close()
  30. 	hash := sha1.New()
  31. 	hash.Write(buf.Bytes())
  32. 	sum := fmt.Sprintf("%x", hash.Sum(nil))
  33. 	return sum, nil
  34. }
  36. func main() {
  37. 	s := "test content"
  38. 	sum, err := ObjectHash(s)
  39. 	if err == nil {
  40. 		fmt.Println("Object:    ", sum)
  41. 	}
  42. 	sum, err = CompressedHash(s)
  43. 	if err == nil {
  44. 		fmt.Println("Compressed:", sum)
  45. 	}
  46. }

输出：

  1. Object:     1eebdf4fdc9fc7bf283031b93f9aef3338de9052
  2. Compressed: 8138de8ff9d0cd30126f7e26191996dce781d652

如果你比较苹果和橙子，SHA1算法可以区分不同类型的水果。

苹果（Go）：

  1. 00000000  78 9c 2a 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 04  |x.*I-.QH..+I.+..|
  2. 00000010  00 00 ff ff 1f 29 04 dc                           |.....)..|
  4. $ sha1sum apples
  5. 8138de8ff9d0cd30126f7e26191996dce781d652  apples

橙子（Python）：

  1. 00000000  78 9c 2b 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 00  |x.+I-.QH..+I.+..|
  2. 00000010  1f 29 04 dc                                       |.)..|
  4. $ sha1sum oranges
  5. 7efbe4015afd95478282c3774f47b4195031d27e  oranges
英文:

What was your test output?

How did you calculate your expected values?

For example,

  1. package main
  3. import (
  4. 	"bytes"
  5. 	"compress/zlib"
  6. 	"crypto/sha1"
  7. 	"fmt"
  8. )
  10. func ObjectHash(s string) (string, error) {
  11. 	var buf bytes.Buffer
  12. 	_, err := buf.Write([]byte(s))
  13. 	if err != nil {
  14. 		return "", err
  15. 	}
  16. 	hash := sha1.New()
  17. 	hash.Write(buf.Bytes())
  18. 	sum := fmt.Sprintf("%x", hash.Sum(nil))
  19. 	return sum, nil
  20. }
  22. func CompressedHash(s string) (string, error) {
  23. 	var buf bytes.Buffer
  24. 	zw := zlib.NewWriter(&buf)
  25. 	_, err := zw.Write([]byte(s))
  26. 	if err != nil {
  27. 		return "", err
  28. 	}
  29. 	zw.Close()
  30. 	hash := sha1.New()
  31. 	hash.Write(buf.Bytes())
  32. 	sum := fmt.Sprintf("%x", hash.Sum(nil))
  33. 	return sum, nil
  34. }
  36. func main() {
  37. 	s := "test content"
  38. 	sum, err := ObjectHash(s)
  39. 	if err == nil {
  40. 		fmt.Println("Object:    ", sum)
  41. 	}
  42. 	sum, err = CompressedHash(s)
  43. 	if err == nil {
  44. 		fmt.Println("Compressed:", sum)
  45. 	}
  46. }

Output:

  1. Object:     1eebdf4fdc9fc7bf283031b93f9aef3338de9052
  2. Compressed: 8138de8ff9d0cd30126f7e26191996dce781d652

If you compare apples and oranges, SHA1, by design, can tell the difference between different types of fruit.

  1. Apples (Go):
  3. 00000000  78 9c 2a 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 04  |x.*I-.QH..+I.+..|
  4. 00000010  00 00 ff ff 1f 29 04 dc                           |.....)..|
  6. $ sha1sum apples
  7. 8138de8ff9d0cd30126f7e26191996dce781d652  apples
  9. Oranges (Python):
  11. 00000000  78 9c 2b 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 00  |x.+I-.QH..+I.+..|
  12. 00000010  1f 29 04 dc                                       |.)..|
  14. $ sha1sum oranges
  15. 7efbe4015afd95478282c3774f47b4195031d27e  oranges

huangapple
  • 本文由 发表于 2013年5月23日 05:06:20
  • 转载请务必保留本文链接：https://go.coder-hub.com/16701567.html
  • go
  • sha1
  • zlib
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定