堆叠写入器和zlib写入器的校验和错误。

huangapple go评论127阅读模式
英文:

Wrong checksum on stacked writer & zlib.writer

问题

我有两个对象写入器,第一个封装了io.Writer,在写入时还计算内容的SHA1校验和,第二个封装了第一个写入器,并写入了zlib压缩的数据。

我的单元测试用例对于第一个写入器的一些测试数据通过了,但对于第二个写入器失败了。

我在哪里做错了?

英文:

I have two object writers, first one encapsulates io.Writer and along writes it also coputes SHA1 checksum of content, and second encapsulates first and writes zlib compressed data.

My unit test case with some test data passes for first writer, but fails with second.

Where and what I'm doing wrong?

code snippet:

import (
    "compress/zlib"
    "crypto/sha1"
    "hash"
    "io"
)

type ObjectWriter interface {
    io.Writer
    Commit() ([]byte, error)
}

type oWriter struct {
    writer io.Writer
    sum    hash.Hash
}

func (ow *oWriter) Write(b []byte) (int, error) {
    ow.sum.Write(b)  // Hash writer never returs error -> hash.Hash docs
    return ow.writer.Write(b)
}

func (ow *oWriter) Commit() ([]byte, error) {
    return ow.sum.Sum(nil), nil
}

func NewWriter(w io.Writer) ObjectWriter {
    return &oWriter{w, sha1.New()}
}

type compressedWriter struct {
    oWriter ObjectWriter
    zWriter io.WriteCloser
}

func (ow *compressedWriter) Write(b []byte) (int, error) {
    return ow.zWriter.Write(b)
}

func (ow *compressedWriter) Commit() ([]byte, error) {
    if err := ow.zWriter.Close(); err != nil {
        return nil, err
    }

    return ow.oWriter.Commit()
}

func NewCompressedWriter(w io.Writer) ObjectWriter {
    ow := NewWriter(w)
    zw := zlib.NewWriter(ow)

    return &compressedWriter{ow, zw}
}

passing test case:

func TestObjectWriter(t *testing.T) {
    var buf bytes.Buffer

    ow := NewWriter(&buf)
    ow.Write([]byte("test content"))
    sum, err := ow.Commit()

    if err != nil {
        t.Errorf("Commit error: %s", err)
    }

    expected := "1eebdf4fdc9fc7bf283031b93f9aef3338de9052"
    given := fmt.Sprintf("%x", sum)

    if  expected != given {
        t.Errorf("Invalid SHA1 sum: <%s> != <%s>", expected, given)
    }
}

failing test case:

func TestCompressedObjectWriter(t *testing.T) {
    var buf bytes.Buffer

    ow := NewCompressedWriter(&buf)
    ow.Write([]byte("test content"))
    sum, err := ow.Commit()

    if err != nil {
        t.Errorf("Commit error: %s", err)
    }

    expected := "7efbe4015afd95478282c3774f47b4195031d27e"
    given := fmt.Sprintf("%x", sum)

    if  expected != given {
        t.Errorf("Invalid SHA1 sum: <%s> != <%s>", expected, given)
    }
}

答案1

得分: 0

你的测试输出是什么?

你是如何计算你的期望值的?

例如,

package main

import (
	"bytes"
	"compress/zlib"
	"crypto/sha1"
	"fmt"
)

func ObjectHash(s string) (string, error) {
	var buf bytes.Buffer
	_, err := buf.Write([]byte(s))
	if err != nil {
		return "", err
	}
	hash := sha1.New()
	hash.Write(buf.Bytes())
	sum := fmt.Sprintf("%x", hash.Sum(nil))
	return sum, nil
}

func CompressedHash(s string) (string, error) {
	var buf bytes.Buffer
	zw := zlib.NewWriter(&buf)
	_, err := zw.Write([]byte(s))
	if err != nil {
		return "", err
	}
	zw.Close()
	hash := sha1.New()
	hash.Write(buf.Bytes())
	sum := fmt.Sprintf("%x", hash.Sum(nil))
	return sum, nil
}

func main() {
	s := "test content"
	sum, err := ObjectHash(s)
	if err == nil {
		fmt.Println("Object:    ", sum)
	}
	sum, err = CompressedHash(s)
	if err == nil {
		fmt.Println("Compressed:", sum)
	}
}

输出:

Object:     1eebdf4fdc9fc7bf283031b93f9aef3338de9052
Compressed: 8138de8ff9d0cd30126f7e26191996dce781d652

如果你比较苹果和橙子,SHA1算法可以区分不同类型的水果。

苹果(Go):

00000000  78 9c 2a 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 04  |x.*I-.QH..+I.+..|
00000010  00 00 ff ff 1f 29 04 dc                           |.....)..|

$ sha1sum apples
8138de8ff9d0cd30126f7e26191996dce781d652  apples

橙子(Python):

00000000  78 9c 2b 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 00  |x.+I-.QH..+I.+..|
00000010  1f 29 04 dc                                       |.)..|

$ sha1sum oranges
7efbe4015afd95478282c3774f47b4195031d27e  oranges
英文:

What was your test output?

How did you calculate your expected values?

For example,

package main

import (
	"bytes"
	"compress/zlib"
	"crypto/sha1"
	"fmt"
)

func ObjectHash(s string) (string, error) {
	var buf bytes.Buffer
	_, err := buf.Write([]byte(s))
	if err != nil {
		return "", err
	}
	hash := sha1.New()
	hash.Write(buf.Bytes())
	sum := fmt.Sprintf("%x", hash.Sum(nil))
	return sum, nil
}

func CompressedHash(s string) (string, error) {
	var buf bytes.Buffer
	zw := zlib.NewWriter(&buf)
	_, err := zw.Write([]byte(s))
	if err != nil {
		return "", err
	}
	zw.Close()
	hash := sha1.New()
	hash.Write(buf.Bytes())
	sum := fmt.Sprintf("%x", hash.Sum(nil))
	return sum, nil
}

func main() {
	s := "test content"
	sum, err := ObjectHash(s)
	if err == nil {
		fmt.Println("Object:    ", sum)
	}
	sum, err = CompressedHash(s)
	if err == nil {
		fmt.Println("Compressed:", sum)
	}
}

Output:

Object:     1eebdf4fdc9fc7bf283031b93f9aef3338de9052
Compressed: 8138de8ff9d0cd30126f7e26191996dce781d652

If you compare apples and oranges, SHA1, by design, can tell the difference between different types of fruit.

Apples (Go):

00000000  78 9c 2a 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 04  |x.*I-.QH..+I.+..|
00000010  00 00 ff ff 1f 29 04 dc                           |.....)..|

$ sha1sum apples
8138de8ff9d0cd30126f7e26191996dce781d652  apples

Oranges (Python):

00000000  78 9c 2b 49 2d 2e 51 48  ce cf 2b 49 cd 2b 01 00  |x.+I-.QH..+I.+..|
00000010  1f 29 04 dc                                       |.)..|

$ sha1sum oranges
7efbe4015afd95478282c3774f47b4195031d27e  oranges

huangapple
  • 本文由 发表于 2013年5月23日 05:06:20
  • 转载请务必保留本文链接:https://go.coder-hub.com/16701567.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定