如何使用”compress/gzip”包来压缩一个文件?

huangapple go评论75阅读模式
英文:

How can I use the "compress/gzip" package to gzip a file?

问题

我是Go的新手,不知道如何利用compress/gzip包。基本上,我只想将一些内容写入文件,对其进行gzip压缩,并通过另一个脚本直接从压缩格式中读取。如果有人能给我一个示例,我将非常感激。

英文:

I'm new to Go, and can't figure out how to use the compress/gzip package to my advantage. Basically, I just want to write something to a file, gzip it and read it directly from the zipped format through another script. I would really appreciate if someone could give me an example on how to do this.

答案1

得分: 61

所有的压缩包都实现了相同的接口。你可以像这样使用它来进行压缩:

var b bytes.Buffer
w := gzip.NewWriter(&b)
w.Write([]byte("hello, world\n"))
w.Close()

而这样来解压缩:

r, err := gzip.NewReader(&b)
io.Copy(os.Stdout, r)
r.Close()
英文:

All the compress packages implement the same interface. You would use something like this to compress:

var b bytes.Buffer
w := gzip.NewWriter(&b)
w.Write([]byte("hello, world\n"))
w.Close()

And this to unpack:

r, err := gzip.NewReader(&b)
io.Copy(os.Stdout, r)
r.Close()

答案2

得分: 11

import (
"bytes"
"compress/gzip"
"io/ioutil"
)
// ...
var b bytes.Buffer
w := gzip.NewWriter(&b)
w.Write([]byte("hello, world\n"))
w.Close() // 您必须先关闭它以将字节刷新到缓冲区。
err := ioutil.WriteFile("hello_world.txt.gz", b.Bytes(), 0666)

英文:

Pretty much the same answer as Laurent, but with the file io:

import (
  "bytes"
  "compress/gzip"
  "io/ioutil"
)
// ...
var b bytes.Buffer
w := gzip.NewWriter(&b)
w.Write([]byte("hello, world\n"))
w.Close() // You must close this first to flush the bytes to the buffer.
err := ioutil.WriteFile("hello_world.txt.gz", b.Bytes(), 0666)

答案3

得分: 9

对于读取部分,类似于对**.gz**文件有用的ioutil.ReadFile的代码可能是:

func ReadGzFile(filename string) ([]byte, error) {
    fi, err := os.Open(filename)
    if err != nil {
        return nil, err
    }
    defer fi.Close()

    fz, err := gzip.NewReader(fi)
    if err != nil {
        return nil, err
    }
    defer fz.Close()

    s, err := ioutil.ReadAll(fz)
    if err != nil {
        return nil, err
    }
    return s, nil    
}
英文:

For the Read part, something like the useful ioutil.ReadFile for .gz files could be :

func ReadGzFile(filename string) ([]byte, error) {
	fi, err := os.Open(filename)
	if err != nil {
		return nil, err
	}
	defer fi.Close()

	fz, err := gzip.NewReader(fi)
	if err != nil {
		return nil, err
	}
	defer fz.Close()

	s, err := ioutil.ReadAll(fz)
	if err != nil {
		return nil, err
	}
	return s, nil	
}

答案4

得分: 7

这里是将gzip文件解压到目标文件的函数:

func UnpackGzipFile(gzFilePath, dstFilePath string) (int64, error) {
    gzFile, err := os.Open(gzFilePath)
    if err != nil {
        return 0, fmt.Errorf("打开文件 %q 进行解压失败:%w", gzFilePath, err)
    }
    dstFile, err := os.OpenFile(dstFilePath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0660)
    if err != nil {
        return 0, fmt.Errorf("创建目标文件 %q 进行解压失败:%w", dstFilePath, err)
    }
    defer dstFile.Close()

    ioReader, ioWriter := io.Pipe()
    defer ioReader.Close()

    go func() { // 可能会有goroutine泄漏
        gzReader, _ := gzip.NewReader(gzFile)
        // 关闭写入器或从管道的另一端读取非常重要,否则io.copy()将永远无法完成
        defer func(){
            gzFile.Close()
            gzReader.Close()
            ioWriter.Close()
        }()

        io.Copy(ioWriter, gzReader)
    }()

    written, err := io.Copy(dstFile, ioReader)
    if err != nil {
        return 0, err // 可能会有goroutine泄漏
    }

    return written, nil
}
英文:

Here the func for unpack gzip file to destination file:

func UnpackGzipFile(gzFilePath, dstFilePath string) (int64, error) {
    gzFile, err := os.Open(gzFilePath)
    if err != nil {
        return 0, fmt.Errorf("open file %q to unpack: %w", gzFilePath, err)
    }
    dstFile, err := os.OpenFile(dstFilePath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0660)
    if err != nil {
        return 0, fmt.Errorf("create destination file %q to unpack: %w", dstFilePath, err)
    }
    defer dstFile.Close()

    ioReader, ioWriter := io.Pipe()
    defer ioReader.Close()

    go func() { // goroutine leak is possible here
        gzReader, _ := gzip.NewReader(gzFile)
        // it is important to close the writer or reading from the other end of the
        // pipe or io.copy() will never finish
        defer func(){
            gzFile.Close()
            gzReader.Close()
            ioWriter.Close()
        }()

        io.Copy(ioWriter, gzReader)
    }()

    written, err := io.Copy(dstFile, ioReader)
    if err != nil {
        return 0, err // goroutine leak is possible here
    }

    return written, nil
}

答案5

得分: 2

我决定结合其他答案的想法,提供一个完整的示例程序。显然,有很多不同的方法可以做同样的事情。这只是一种方法:

package main

import (
    "compress/gzip"
    "fmt"
    "io/ioutil"
    "os"
)

var zipFile = "zipfile.gz"

func main() {
    writeZip()
    readZip()
}

func writeZip() {
    handle, err := openFile(zipFile)
    if err != nil {
        fmt.Println("[ERROR] 打开文件:", err)
    }

    zipWriter, err := gzip.NewWriterLevel(handle, 9)
    if err != nil {
        fmt.Println("[ERROR] 新建gzip写入器:", err)
    }
    numberOfBytesWritten, err := zipWriter.Write([]byte("Hello, World!\n"))
    if err != nil {
        fmt.Println("[ERROR] 写入:", err)
    }
    err = zipWriter.Close()
    if err != nil {
        fmt.Println("[ERROR] 关闭zip写入器:", err)
    }
    fmt.Println("[INFO] 写入的字节数:", numberOfBytesWritten)

    closeFile(handle)
}

func readZip() {
    handle, err := openFile(zipFile)
    if err != nil {
        fmt.Println("[ERROR] 打开文件:", err)
    }

    zipReader, err := gzip.NewReader(handle)
    if err != nil {
        fmt.Println("[ERROR] 新建gzip读取器:", err)
    }
    defer zipReader.Close()

    fileContents, err := ioutil.ReadAll(zipReader)
    if err != nil {
        fmt.Println("[ERROR] 读取全部:", err)
    }

    fmt.Printf("[INFO] 解压后的内容:%s\n", fileContents)

    // ** 另一种读取文件的方法 **
    //
    // fileInfo, _ := handle.Stat()
    // fileContents := make([]byte, fileInfo.Size())
    // bytesRead, err := zipReader.Read(fileContents)
    // if err != nil {
    //     fmt.Println("[ERROR] 读取gzip文件:", err)
    // }
    // fmt.Println("[INFO] 从文件中读取的字节数:", bytesRead)

    closeFile(handle)
}

func openFile(fileToOpen string) (*os.File, error) {
    return os.OpenFile(fileToOpen, openFileOptions, openFilePermissions)
}

func closeFile(handle *os.File) {
    if handle == nil {
        return
    }

    err := handle.Close()
    if err != nil {
        fmt.Println("[ERROR] 关闭文件:", err)
    }
}

const openFileOptions int = os.O_CREATE | os.O_RDWR
const openFilePermissions os.FileMode = 0660

像这样有一个完整的示例对于以后的参考应该是有帮助的。

英文:

I decided to combine ideas from others answers and just provide a full example program. Obviously there are many different ways to do the same thing. This is just one way:

package main

import (
    "compress/gzip"
    "fmt"
    "io/ioutil"
    "os"
)

var zipFile = "zipfile.gz"

func main() {
    writeZip()
    readZip()
}

func writeZip() {
    handle, err := openFile(zipFile)
    if err != nil {
        fmt.Println("[ERROR] Opening file:", err)
    }

    zipWriter, err := gzip.NewWriterLevel(handle, 9)
    if err != nil {
        fmt.Println("[ERROR] New gzip writer:", err)
    }
    numberOfBytesWritten, err := zipWriter.Write([]byte("Hello, World!\n"))
    if err != nil {
        fmt.Println("[ERROR] Writing:", err)
    }
    err = zipWriter.Close()
    if err != nil {
        fmt.Println("[ERROR] Closing zip writer:", err)
    }
    fmt.Println("[INFO] Number of bytes written:", numberOfBytesWritten)

    closeFile(handle)
}

func readZip() {
    handle, err := openFile(zipFile)
    if err != nil {
        fmt.Println("[ERROR] Opening file:", err)
    }

    zipReader, err := gzip.NewReader(handle)
    if err != nil {
        fmt.Println("[ERROR] New gzip reader:", err)
    }
    defer zipReader.Close()

    fileContents, err := ioutil.ReadAll(zipReader)
    if err != nil {
        fmt.Println("[ERROR] ReadAll:", err)
    }

    fmt.Printf("[INFO] Uncompressed contents: %s\n", fileContents)

    // ** Another way of reading the file **
    //
    // fileInfo, _ := handle.Stat()
    // fileContents := make([]byte, fileInfo.Size())
    // bytesRead, err := zipReader.Read(fileContents)
    // if err != nil {
    //     fmt.Println("[ERROR] Reading gzip file:", err)
    // }
    // fmt.Println("[INFO] Number of bytes read from the file:", bytesRead)

    closeFile(handle)
}

func openFile(fileToOpen string) (*os.File, error) {
    return os.OpenFile(fileToOpen, openFileOptions, openFilePermissions)
}

func closeFile(handle *os.File) {
    if handle == nil {
        return
    }

    err := handle.Close()
    if err != nil {
        fmt.Println("[ERROR] Closing file:", err)
    }
}

const openFileOptions int = os.O_CREATE | os.O_RDWR
const openFilePermissions os.FileMode = 0660

Having a full example like this should be helpful for future reference.

答案6

得分: 1

压缩任何接口类型的Go对象作为输入

func compress(obj interface{}) ([]byte, error) {
    var b bytes.Buffer
    objBytes, err := json.Marshal(obj)
    if err != nil {
        return nil, err
    }
    gz := gzip.NewWriter(&b)
    defer gz.Close() //不够,不要延迟关闭写入器对象
    if _, err := gz.Write(objBytes); err != nil {
        return nil, err
    }
    // 需要显式关闭
    if err := gz.Close(); err != nil {
        return nil, err
    }
    return b.Bytes(), nil
}

解压相同的对象

func decompress(obj []byte) ([]byte, error) {
    r, err := gzip.NewReader(bytes.NewReader(obj))
    if err != nil {
        return nil, err
    }
    defer r.Close()
    res, err := ioutil.ReadAll(r)
    if err != nil {
        return nil, err
    }
    return res, nil
}

注意,如果在写入后不关闭写入器对象,ioutil.ReadAll(r)会返回io.EOFio.ErrUnexpectedEOF。我以为在Close()上使用defer会正确关闭对象,但实际上不会。不要延迟关闭写入器对象

英文:

To compress any Go object of interface type as input

func compress(obj interface{}) ([]byte, error) {
	var b bytes.Buffer
	objBytes, err := json.Marshal(obj)
	if err != nil {
		return nil, err
	}
	gz := gzip.NewWriter(&b)
	defer gz.Close() //NOT SUFFICIENT, DON'T DEFER WRITER OBJECTS
	if _, err := gz.Write(objBytes); err != nil {
		return nil, err
	}
    // NEED TO CLOSE EXPLICITLY
	if err := gz.Close(); err != nil {
		return nil, err
	}
	return b.Bytes(), nil
}

To decompress the same,

func decompress(obj []byte) ([]byte, error) {
	r, err := gzip.NewReader(bytes.NewReader(obj))
	if err != nil {
		return nil, err
	}
	defer r.Close()
	res, err := ioutil.ReadAll(r)
	if err != nil {
		return nil, err
	}
	return res, nil
}

Note, ioutil.ReadAll(r) returns io.EOF or io.ErrUnexpectedEOF if you do not close the Writer object after writing. I assumed defer on Close() would close the object properly, but it won't. Don't defer writer objects.

huangapple
  • 本文由 发表于 2013年6月3日 13:46:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/16890648.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定