如何获取压缩文件夹的大小

huangapple go评论87阅读模式
英文:

How to get the size of zipped folder

问题

使用情况是:
我想要压缩一个文件夹,其中可能包含不同大小的n个文件。我想要添加一个限制条件,如果压缩文件的大小大于x(MB|GB),则不创建压缩文件并抛出错误。我尝试在迭代文件夹后创建压缩文件后获取压缩文件的大小,但无法做到。在这里请求帮助。我正在使用io.writer来创建压缩文件,而不是缓冲区。示例代码如下:

**注意:**当使用"buf := new(bytes.Buffer)"时,我可以计算大小,但我想要使用下面所示的io.writer进行计算

filepath - 文件夹的路径
downloadLimit - 我想要设置的下载压缩文件的限制。例如,压缩文件的大小不应超过100 MB

func GetZipFileSize(filepath string, downloadLimit int, w io.Writer) error {
	
	zipWriter := zip.NewWriter(w)

	// 列出文件
	var files []FileAndPath
	var err error
	
    // 我们有一个用于获取文件的函数,存储在变量"files"中,下面会使用到

	// 压缩文件
	for _, file := range files {
		
		// 确定ZIP中的文件名
		zipFilename := file.Path
		
		filename := fp.Join(SOME_PATH, file.Path)

		// 创建ZIP条目
		fileToZip, err := os.Open(filename)
		if err != nil {
			return SOME_ERROR
		}
		info, err := fileToZip.Stat()
		if err != nil {
			return err
		}
		header, err := zip.FileInfoHeader(info)
		if err != nil {
			return err
		}
		header.Name = zipFilename
		header.Method = zip.Deflate
		writer, err := zipWriter.CreateHeader(header)
		if err != nil {
			return err
		}

		// 设置ZIP条目的内容
		if _, err = io.Copy(writer, fileToZip); err != nil {
			return err
		}
		if err = fileToZip.Close(); err != nil {
			// 无操作
		}
	}

	// 结束
	if err := zipWriter.Close(); err != nil {
		return err
	}

	return nil
}

上述函数创建了压缩文件,但我希望在达到限制时立即停止创建并退出,或者至少获取所创建的压缩文件的大小。

英文:

Use case is:
I want to zip a folder which could contain n number of files with difference sizes. I want to add limit like if the zip size > x (MB|GB) then do not create the zip and throw error. I tried to get the size of zip file after the zip is created after iterating the folder but not able to do so. Request help here. I am using io.writer to create zip file instead buffer. Sample code is

Note: I am able to calculate the size when using "buf := new(bytes.Buffer)" but I want to calculate using io.writer as seen below

filepath - Path to the folder
downloadLimit - the limit I want to set for downloaded zip file. Like zip file size should not be more then 100 MB

func GetZipFileSize(filepath string, downloadLimit int, w io.Writer) error {
	
	zipWriter := zip.NewWriter(w)

	// List files
	var files []FileAndPath
	var err error
	
    // We have function to fetch files in variable "files" which is used below

	// ZIP files
	for _, file := range files {
		
		// Determine filename in ZIP
		zipFilename := file.Path
		
		filename := fp.Join(SOME_PATH, file.Path)

		// Create ZIP entry
		fileToZip, err := os.Open(filename)
		if err != nil {
			return SOME_ERROR
		}
		info, err := fileToZip.Stat()
		if err != nil {
			return err
		}
		header, err := zip.FileInfoHeader(info)
		if err != nil {
			return err
		}
		header.Name = zipFilename
		header.Method = zip.Deflate
		writer, err := zipWriter.CreateHeader(header)
		if err != nil {
			return err
		}

		// Set content for ZIP entry
		if _, err = io.Copy(writer, fileToZip); err != nil {
			return err
		}
		if err = fileToZip.Close(); err != nil {
			// Nothing
		}
	}

	// Wrap-up
	if err := zipWriter.Close(); err != nil {
		return err
	}

	return nil
}

Above function create zip file, but I want it to stop creation as soon the limit is reached and exit OR at least get the size of the zip file created

答案1

得分: 2

标准API没有提供一个实现io.Writer来完成这个任务,据我所知。虽然有一个LimitReader,但对于你的情况几乎没有用处,而且会导致编写相当复杂的代码。

相反,你可以编写自己的io.Writer包装器实现,以确保如果写入的总字节数超过某个长度,或者即将超过某个长度,就返回一个错误。

package main

import (
	"errors"
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	bigBuf := strings.NewReader(strings.Repeat("borng string", 10000))

	dst := &LimitWriter{Writer: os.Stdout, N: 20}

	n, err := io.Copy(dst, bigBuf)
	fmt.Println("n=", n)
	fmt.Println("err=", err)
}

var ErrWriteOverflow = errors.New("write overflow, data is too large")

type LimitWriter struct {
	io.Writer
	N       int64
	written int64
}

func (l *LimitWriter) Write(p []byte) (n int, err error) {
	if l.written+int64(len(p)) >= l.N {
		return 0, ErrWriteOverflow
	}
	n, err = l.Writer.Write(p)
	l.written += int64(n)
	return
}

https://play.golang.org/p/ktgF-pOzFjm

英文:

The standard API does not provide an io.Writer implementaion to do that, afaik. It exists a LimitReader but it is of little use for your case and would led us to write rather complex code.

Instead, you can write your own io.Writer wrapper implementation to ensure that if the total amount of bytes written onto it exceeds some length, or is about to, return an error.

package main

import (
	"errors"
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	bigBuf := strings.NewReader(strings.Repeat("borng string", 10000))

	dst := &LimitWriter{Writer: os.Stdout, N: 20}

	n, err := io.Copy(dst, bigBuf)
	fmt.Println("n=", n)
	fmt.Println("err=", err)
}

var ErrWriteOverflow = errors.New("write overflow, data is too large")

type LimitWriter struct {
	io.Writer
	N       int64
	written int64
}

func (l *LimitWriter) Write(p []byte) (n int, err error) {
	if l.written+int64(len(p)) >= l.N {
		return 0, ErrWriteOverflow
	}
	n, err = l.Writer.Write(p)
	l.written += int64(n)
	return
}

https://play.golang.org/p/ktgF-pOzFjm

答案2

得分: 0

这是一个关于如何解决这个问题的建议。代码中有解释。

代码的第一行假设你已经有了要压缩的文件的某种字节表示。

func main() {    	
    
    // 创建一个缓冲区,用于将我们的归档写入其中(内存中)
    buf := new(bytes.Buffer)
    
    // 创建一个新的 zip 归档写入器,将其写入我们的内存缓冲区
    archiveWriter := zip.NewWriter(buf)

    // 确保稍后关闭你的 zip 写入器
    defer archiveWriter.Close()

    // 你可能想要在循环中执行此操作,并为要添加到 zip 归档的每个文件重复此操作
    // 这是你要压缩的文件的内容,你也可以使用 ioutil.ReadFile() 或其他方法从磁盘上读取它
    dummyBytes := []byte("这里是你源文件的内容")
    fileWriter, _ := archiveWriter.Create("在压缩文件中的源文件名")
    _, _ = fileWriter.Write(dummyBytes)
        
    // 验证内存中的 zip 文件表示不超过你的限制
    if buf.Len() > 1000 {
        // todo 抛出错误
        return
    }
    	
    // 将 zip 文件写入磁盘,你也可以使用 io.writer
    _ = ioutil.WriteFile("目标压缩文件.zip", buf.Bytes(), 0644)
}

请注意,我没有实现任何错误处理 - 你可能想要添加错误处理。

英文:

Here's a suggestion how you could solve this. Explaination in the code.
I think there is no (easy) way to calculate/estimate the size of a future zip file without executing the actual compression.

Line 1 of the code assumes you already have some kind of byte representation of the file(s) you want to zip.

func main() {    	
// Create a buffer to write our archive to (in memory)
buf := new(bytes.Buffer)
// Create a new zip archive writer, that writes to our in memory buffer
archiveWriter := zip.NewWriter(buf)
// ensure to close your zip writer later
defer archiveWriter.Close()
// you probably want to do this in a loop and repeat for each file you want add to the zip archive
// this is the content of a file you want to zip / instead read it from disk with ioutil.ReadFile() or some other approach
dummyBytes := []byte("ThisCouldBeTheContentOfYourSourceFile")
fileWriter, _ := archiveWriter.Create("fileNameOfSourceFileInZip")
_, _ = fileWriter.Write(dummyBytes)
// validate the in memory representation of the zip file does not exceed your limit
if buf.Len() > 1000 {
// todo raise error
return
}
// write the zip file to the disk, you could ofc also use io.writer
_ = ioutil.WriteFile("TargetZipFile.zip", buf.Bytes(), 0644)
}

Please note that I did not implement any error handling - you probably want to add that.

huangapple
  • 本文由 发表于 2021年7月23日 15:32:30
  • 转载请务必保留本文链接:https://go.coder-hub.com/68495559.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定