Go:解压缩字节切片的 zlib

huangapple go评论76阅读模式
英文:

Go: zlib uncompressing a slice of bytes

问题

我正在尝试解析一个由许多单独压缩的片段组成的文件。我已经逐个解析这些片段,并将它们存储为字节切片,现在我想边解压边读取它们。

以下是我当前的解压缩代码,但它不起作用。fromto 只是一个示例,在实际情况下它们是由代码设置的。data 是包含整个文件的字节数组。由于文件位于另一台服务器上,我不想在它还在磁盘上时进行定位,所以实际上我只能先将整个文件加载到 []byte 中,然后再解析它。

from, to := 0, 1000;
b := bytes.NewReader(data[from:from+to])
z, err := zlib.NewReader(b)
CheckErr(err)
defer z.Close()
p := make([]byte,0,1024)
z.Read(p)
fmt.Println(string(p))

所以,解压缩一个字节切片为什么如此困难呢?不管怎样...

问题似乎出在读取部分。在代码中的 z.Read 那一行似乎没有起作用。

如何一次性将整个内容读取到一个字节切片中呢?

英文:

I am trying to parse a file that annoying consists of many separately zipped segments. I have parsed these segments one at a time into a slice of bytes and I want to uncompress them as I go.

Here is my current code that does the decompressing, which doesn't work. from and to are just set at the top as an example, in reality they are set by the code. data is the byte array containing the entire file. I don't want to seek it while it's on disk because its location on another server, so it's only realistic for me to load the entire file to []byte first and then parse it.

from, to := 0, 1000;
b := bytes.NewReader(data[from:from+to])
z, err := zlib.NewReader(b)
CheckErr(err)
defer z.Close()
p := make([]byte,0,1024)
z.Read(p)
fmt.Println(string(p))

So how is it so massively difficult just to unzip a slice of bytes? Anyway...

The problem appears to with how I am reading it out. Where it says z.Read, that doesn't seem to do anything.

How can I read the entire thing in one go into a slice of bytes?

答案1

得分: 5

这是给你的大纲。注意:在Go语言中,检查错误!

package main

import (
	"bytes"
	"compress/zlib"
	"fmt"
	"io/ioutil"
)

func readSegment(data []byte, from, to int) ([]byte, error) {
	b := bytes.NewReader(data[from : from+to])
	z, err := zlib.NewReader(b)
	if err != nil {
		return nil, err
	}
	defer z.Close()
	p, err := ioutil.ReadAll(z)
	if err != nil {
		return nil, err
	}
	return p, nil
}

func main() {
	from, to := 0, 1000
	data := make([]byte, from+to)
	// ** 将输入段解析到数据中 **
	p, err := readSegment(data, from, to)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(string(p))
}
英文:

Here's an outline for you. Note: In Go, CHECK FOR ERRORS!

package main

import (
	"bytes"
	"compress/zlib"
	"fmt"
	"io/ioutil"
)

func readSegment(data []byte, from, to int) ([]byte, error) {
	b := bytes.NewReader(data[from : from+to])
	z, err := zlib.NewReader(b)
	if err != nil {
		return nil, err
	}
	defer z.Close()
	p, err := ioutil.ReadAll(z)
	if err != nil {
		return nil, err
	}
	return p, nil
}

func main() {
	from, to := 0, 1000
	data := make([]byte, from+to)
	// ** parse input segments into data **
	p, err := readSegment(data, from, to)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(string(p))
}

答案2

得分: 2

使用io/ioutil包中的ReadAll(r io.Reader) ([]byte, error)函数。

p, err := ioutil.ReadAll(b)

fmt.Println(string(p))

Read函数只会读取给定切片的长度(在你的情况下是1024字节)。

要以1024字节的块进行读取:

p := make([]byte, 1024)

for {
  numBytes, err := l.Read(p)
  if err == io.EOF {
    // 你已经完成了,numBytes可能小于len(p)
    break
  }
  // 对p进行处理
}

如果你从web服务器获取数据,你甚至可以这样做:

import (
  "net/http"
  "io/ioutil"
)
...

resp, errGet := http.Get("http://example.com/somefile")
// 进行错误处理
z, errZ := zlib.NewReader(resp.Body)
// 进行错误处理
resp.Body.Close()
p, err := ioutil.ReadAll(b)
// 进行错误处理

因为resp.Body恰好是一个io.Reader,大多数与io相关的类型都是如此。

英文:

Use <code>ReadAll(r io.Reader) ([]byte, error)</code> from the <code>io/ioutil</code> package.

p, err := ioutil.ReadAll(b)

fmt.Println(string(p))

Read only reads up to the length of the given slice (1024 bytes in your case).

To read in chunks of 1024 bytes:

 p := make([]byte,1024)
 
 for {
   numBytes, err := l.Read(p)
   if err == io.EOF {
      // you are done, numBytes might be less than len(p)
      break
   }
   // do what you want with p
 }

If you are getting the data from a webserver, you might even do

 import (
   &quot;net/http&quot;
   &quot;io/ioutil&quot;
 )
 ...
 resp, errGet := http.Get(&quot;http://example.com/somefile&quot;)
 // do error handling
 z, errZ := zlib.NewReader(resp.Body)
 // do error handling
 resp.Body.Close()
 p, err := ioutil.ReadAll(b)
 // do error handling

since resp.Body happens to be an io.Reader as most io related types.

huangapple
  • 本文由 发表于 2014年7月26日 23:01:08
  • 转载请务必保留本文链接:https://go.coder-hub.com/24972334.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定