GOLANG Base64编码和解码对于大文件大小不匹配

huangapple go评论84阅读模式
英文:

GOLANG Base64 Encode and Decode size mismatched for large files

问题

当我尝试使用golang对大文件进行base64编码和解码时,我发现原始文件和解码后的文件之间的字节长度不匹配。

在我的测试中,文本文件的不匹配是因为多了1个字节的换行符,而二进制文件的不匹配是因为多了2个字节。

是什么原因导致了这些字节的丢失?

以下是您提供的代码:

package main

import (
	"encoding/base64"
	"io"
	"os"
	"log"
)

func Encode(infile, outfile string) error {
	input, err := os.Open(infile)
	if err != nil {
		return err
	}
	// Close input file
	defer input.Close()

	// Open output file
	output, err := os.Create(outfile)
	if err != nil {
		return err
	}
	// Close output file
	defer output.Close()

	encoder := base64.NewEncoder(base64.StdEncoding, output)
	l, err := io.Copy(encoder, input)
	if err!=nil {
		log.Printf("Failed to encode file:%v",err)
		return err
	} else {
		log.Printf("Wrote %v bytes",l)
	}

	return nil
}

func Decode(infile, outfile string) error {
	input, err := os.Open(infile)
	if err != nil {
		return err
	}
	// Close input file
	defer input.Close()

	// Open output file
	output, err := os.Create(outfile)
	if err != nil {
		return err
	}
	// Close output file
	defer output.Close()

	decoder := base64.NewDecoder(base64.StdEncoding, input)
	l, err := io.Copy(output, decoder)
	if err!=nil {
		log.Printf("Failed to encode file:%v",err)
		return err
	} else {
		log.Printf("Wrote %v bytes",l)
	}

	return nil
}

希望能对您有所帮助!

英文:

When I am trying to do base64 encode and decode for large files using golang, I am getting byte length missmatched between original and decoded file.

During my testing text file mismatched(1 byte) new line, and with binary file mismatched (2 bytes).

What could be cause missing of these bytes?

<!-- begin snippet: golang hide: false console: true babel: false -->

<!-- language: lang-golang -->

package main
import (
&quot;encoding/base64&quot;
&quot;io&quot;
&quot;os&quot;
&quot;log&quot;
)
func Encode(infile, outfile string) error {
input, err := os.Open(infile)
if err != nil {
return err
}
// Close input file
defer input.Close()
// Open output file
output, err := os.Create(outfile)
if err != nil {
return err
}
// Close output file
defer output.Close()
encoder := base64.NewEncoder(base64.StdEncoding, output)
l, err := io.Copy(encoder, input)
if err!=nil {
log.Printf(&quot;Failed to encode file:%v&quot;,err)
return err
} else {
log.Printf(&quot;Wrote %v bytes&quot;,l)
}
return nil
}
func Decode(infile, outfile string) error {
input, err := os.Open(infile)
if err != nil {
return err
}
// Close input file
defer input.Close()
// Open output file
output, err := os.Create(outfile)
if err != nil {
return err
}
// Close output file
defer output.Close()
decoder := base64.NewDecoder(base64.StdEncoding, input)
l, err := io.Copy(output, decoder)
if err!=nil {
log.Printf(&quot;Failed to encode file:%v&quot;,err)
return err
} else {
log.Printf(&quot;Wrote %v bytes&quot;,l)
}
return nil
}

<!-- end snippet -->

答案1

得分: 3

你没有调用Close()方法关闭encoder,所以它没有刷新所有的数据。根据文档(我强调):

func NewEncoder(enc *Encoding, w io.Writer) io.WriteCloser

NewEncoder返回一个新的base64流编码器。写入返回的写入器的数据将使用enc进行编码,然后写入w。Base64编码以4字节块操作;在写入完成后,调用者必须关闭返回的编码器以刷新任何部分写入的块

我还引用了文档中的示例,其中有一个很好的注释:

package main

import (
	"encoding/base64"
	"os"
)

func main() {
	input := []byte("foo\x00bar")
	encoder := base64.NewEncoder(base64.StdEncoding, os.Stdout)
	encoder.Write(input)
	// 在完成后必须关闭编码器以刷新任何部分块。
	// 如果你注释掉下面的这行,最后一个部分块“r”将不会被编码。
	encoder.Close()
}
英文:

you don't Close() the encoder so it doesn't flush all the data. from the docs (emphasis mine):

> func NewEncoder(enc *Encoding, w io.Writer) io.WriteCloser
>
> NewEncoder returns a new base64 stream encoder. Data written to the
> returned writer will be encoded using enc and then written to w.
> Base64 encodings operate in 4-byte blocks; when finished writing, the
> caller must Close the returned encoder to flush any partially written
> blocks.

I also quote the example from the docs which has a nice comment:

package main
import (
&quot;encoding/base64&quot;
&quot;os&quot;
)
func main() {
input := []byte(&quot;foo\x00bar&quot;)
encoder := base64.NewEncoder(base64.StdEncoding, os.Stdout)
encoder.Write(input)
// Must close the encoder when finished to flush any partial blocks.
// If you comment out the following line, the last partial block &quot;r&quot;
// won&#39;t be encoded.
encoder.Close()
}

huangapple
  • 本文由 发表于 2021年12月5日 22:33:12
  • 转载请务必保留本文链接:https://go.coder-hub.com/70235347.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定