英文:
Go: zlib uncompressing a slice of bytes
问题
我正在尝试解析一个由许多单独压缩的片段组成的文件。我已经逐个解析这些片段,并将它们存储为字节切片,现在我想边解压边读取它们。
以下是我当前的解压缩代码,但它不起作用。from
和 to
只是一个示例,在实际情况下它们是由代码设置的。data
是包含整个文件的字节数组。由于文件位于另一台服务器上,我不想在它还在磁盘上时进行定位,所以实际上我只能先将整个文件加载到 []byte 中,然后再解析它。
from, to := 0, 1000;
b := bytes.NewReader(data[from:from+to])
z, err := zlib.NewReader(b)
CheckErr(err)
defer z.Close()
p := make([]byte,0,1024)
z.Read(p)
fmt.Println(string(p))
所以,解压缩一个字节切片为什么如此困难呢?不管怎样...
问题似乎出在读取部分。在代码中的 z.Read 那一行似乎没有起作用。
如何一次性将整个内容读取到一个字节切片中呢?
英文:
I am trying to parse a file that annoying consists of many separately zipped segments. I have parsed these segments one at a time into a slice of bytes and I want to uncompress them as I go.
Here is my current code that does the decompressing, which doesn't work. from
and to
are just set at the top as an example, in reality they are set by the code. data
is the byte array containing the entire file. I don't want to seek it while it's on disk because its location on another server, so it's only realistic for me to load the entire file to []byte first and then parse it.
from, to := 0, 1000;
b := bytes.NewReader(data[from:from+to])
z, err := zlib.NewReader(b)
CheckErr(err)
defer z.Close()
p := make([]byte,0,1024)
z.Read(p)
fmt.Println(string(p))
So how is it so massively difficult just to unzip a slice of bytes? Anyway...
The problem appears to with how I am reading it out. Where it says z.Read, that doesn't seem to do anything.
How can I read the entire thing in one go into a slice of bytes?
答案1
得分: 5
这是给你的大纲。注意:在Go语言中,检查错误!
package main
import (
"bytes"
"compress/zlib"
"fmt"
"io/ioutil"
)
func readSegment(data []byte, from, to int) ([]byte, error) {
b := bytes.NewReader(data[from : from+to])
z, err := zlib.NewReader(b)
if err != nil {
return nil, err
}
defer z.Close()
p, err := ioutil.ReadAll(z)
if err != nil {
return nil, err
}
return p, nil
}
func main() {
from, to := 0, 1000
data := make([]byte, from+to)
// ** 将输入段解析到数据中 **
p, err := readSegment(data, from, to)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(p))
}
英文:
Here's an outline for you. Note: In Go, CHECK FOR ERRORS!
package main
import (
"bytes"
"compress/zlib"
"fmt"
"io/ioutil"
)
func readSegment(data []byte, from, to int) ([]byte, error) {
b := bytes.NewReader(data[from : from+to])
z, err := zlib.NewReader(b)
if err != nil {
return nil, err
}
defer z.Close()
p, err := ioutil.ReadAll(z)
if err != nil {
return nil, err
}
return p, nil
}
func main() {
from, to := 0, 1000
data := make([]byte, from+to)
// ** parse input segments into data **
p, err := readSegment(data, from, to)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(p))
}
答案2
得分: 2
使用io/ioutil
包中的ReadAll(r io.Reader) ([]byte, error)
函数。
p, err := ioutil.ReadAll(b)
fmt.Println(string(p))
Read函数只会读取给定切片的长度(在你的情况下是1024字节)。
要以1024字节的块进行读取:
p := make([]byte, 1024)
for {
numBytes, err := l.Read(p)
if err == io.EOF {
// 你已经完成了,numBytes可能小于len(p)
break
}
// 对p进行处理
}
如果你从web服务器获取数据,你甚至可以这样做:
import (
"net/http"
"io/ioutil"
)
...
resp, errGet := http.Get("http://example.com/somefile")
// 进行错误处理
z, errZ := zlib.NewReader(resp.Body)
// 进行错误处理
resp.Body.Close()
p, err := ioutil.ReadAll(b)
// 进行错误处理
因为resp.Body
恰好是一个io.Reader
,大多数与io相关的类型都是如此。
英文:
Use <code>ReadAll(r io.Reader) ([]byte, error)</code> from the <code>io/ioutil</code> package.
p, err := ioutil.ReadAll(b)
fmt.Println(string(p))
Read only reads up to the length of the given slice (1024 bytes in your case).
To read in chunks of 1024 bytes:
p := make([]byte,1024)
for {
numBytes, err := l.Read(p)
if err == io.EOF {
// you are done, numBytes might be less than len(p)
break
}
// do what you want with p
}
If you are getting the data from a webserver, you might even do
import (
"net/http"
"io/ioutil"
)
...
resp, errGet := http.Get("http://example.com/somefile")
// do error handling
z, errZ := zlib.NewReader(resp.Body)
// do error handling
resp.Body.Close()
p, err := ioutil.ReadAll(b)
// do error handling
since resp.Body happens to be an io.Reader as most io related types.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论