如何追加文件(io.Reader)?

huangapple go评论87阅读模式
英文:

How to append files (io.Reader)?

问题

func SimpleUploader(r *http.Request, w http.ResponseWriter) {
// 临时文件夹路径
chunkDirPath := "./creatives/.uploads/" + userUUID
// 创建文件夹
err = os.MkdirAll(chunkDirPath, 02750)

// 从多部分请求中获取文件句柄
var file io.Reader
mr, err := r.MultipartReader()

var fileName string
// 读取多部分主体直到 "file" 部分
for {
    part, err := mr.NextPart()
    if err == io.EOF {
        break
    }
    if part.FormName() == "file" {
        file = part
        fileName = part.FileName()
        fmt.Println(fileName)
        break
    }
}

// 创建文件
tempFile := chunkDirPath + "/" + fileName
dst, err := os.Create(tempFile)

defer dst.Close()

buf := make([]byte, 1024*1024)
file.Read(buf)
// 将缓冲区写入磁盘
ioutil.WriteFile(tempFile, buf, os.ModeAppend)
if http.DetectContentType(buf) != "video/mp4" {
    response, _ := json.Marshal(&Response{"文件上传已取消"})
    settings.WriteResponse(w, http.StatusInternalServerError, response)
    return
}

// joinedFile := io.MultiReader(bytes.NewReader(buf), file)
_, err = io.Copy(dst, file)
if err != nil {
    settings.LogError(err, methodName, "复制文件时出错")
}

response, _ := json.Marshal(&Response{"文件上传成功"})
settings.WriteResponse(w, http.StatusInternalServerError, response)

}

我正在上传一个视频文件。
在上传整个文件之前,我想进行一些检查,所以我将前1MB保存到一个文件中:

buf := make([]byte, 1024*1024)
file.Read(buf)
// 将缓冲区写入磁盘
ioutil.WriteFile(tempFile, buf, os.ModeAppend)

然后,如果检查通过,我想上传文件的剩余部分dst是用来保存第一个1MB的相同文件,所以基本上我试图追加到文件中:

_, err = io.Copy(dst, file)

上传的文件大小是正确的,但文件损坏(无法播放视频)。

**我还尝试了什么?:**将两个读取器连接起来并保存到一个新文件中。但是使用这种方法,文件大小增加了1MB,并且损坏了。

joinedFile := io.MultiReader(bytes.NewReader(buf), file)
_, err = io.Copy(newDst, joinedFile)

请帮忙解决。

英文:
func SimpleUploader(r *http.Request, w http.ResponseWriter) {
	// temp folder path
	chunkDirPath := "./creatives/.uploads/" + userUUID
	// create folder
	err = os.MkdirAll(chunkDirPath, 02750)

	// Get file handle from multipart request
	var file io.Reader
	mr, err := r.MultipartReader()
	    
	var fileName string
	// Read multipart body until the "file" part
	for {
		part, err := mr.NextPart()
		if err == io.EOF {
			break
		}
		if part.FormName() == "file" {
			file = part
			fileName = part.FileName()
			fmt.Println(fileName)
			break
		}
	}

	// Create files
	tempFile := chunkDirPath + "/" + fileName
	dst, err := os.Create(tempFile)

	defer dst.Close()

	buf := make([]byte, 1024*1024)
	file.Read(buf)
	// write/save buffer to disk
    ioutil.WriteFile(tempFile, buf, os.ModeAppend)
	if http.DetectContentType(buf) != "video/mp4" {
		response, _ := json.Marshal(&Response{"File upload cancelled"})
		settings.WriteResponse(w, http.StatusInternalServerError, response)
		return
	}

	// joinedFile := io.MultiReader(bytes.NewReader(buf), file)
    _, err = io.Copy(dst, file)
	if err != nil {
		settings.LogError(err, methodName, "Error copying file")
	}

	response, _ := json.Marshal(&Response{"File uploaded successfully"})
	settings.WriteResponse(w, http.StatusInternalServerError, response)
}

I am uploading a Video file.
Before uploading the entire file I want to do some checks so I save the first 1mb to a file :

buf := make([]byte, 1024*1024)
file.Read(buf)
// write/save buffer to disk
ioutil.WriteFile(tempFile, buf, os.ModeAppend)

Then if the checks pass I want to upload the rest of the file dst is the same file used to save the 1st 1 mb so basically i am trying to append to the file :

_, err = io.Copy(dst, file)

The uploaded file size is correct but the file is corrupted(can't play the video).

What else have I tried? : Joining both the readers and saving to a new file. But with this approach the file size increases by 1 mb and is corrupted.

joinedFile := io.MultiReader(bytes.NewReader(buf), file)
_, err = io.Copy(newDst, joinedFile)

Kindly help.

答案1

得分: 1

你实际上通过执行os.Create和ioutil.WriteFile两次打开了文件。

问题在于os.Create的返回值(dst)就像是指向该文件开头的指针。WriteFile不会改变dst指向的位置。

你实际上是在WriteFile之后,又在第一组WriteFile写入的字节之上执行了io.Copy。

尝试先使用WriteFile(带有Create标志),然后使用Append标志使用os.OpenFile(而不是os.Create)打开同一个文件,将剩余的字节追加到末尾。

此外,允许客户端提供文件名非常危险,因为它可能是"../../.bashrc"(例如),这样你就会用用户决定上传的内容覆盖你的shell初始化文件。

如果你自己计算文件名,会更安全。如果需要记住用户选择的文件名,可以将其存储在数据库中,或者甚至是一个metadata.json类型的文件中,以后加载时使用。

英文:

You've basically opened the file twice by doing os.Create and ioutil.WriteFile

the issue being is that os.Create's return value (dst) is like a pointer to the beginning of that file. WriteFile doesn't move where dst points to.

You are basically doing WriteFile, then io.Copy on top of the first set of bytes WriteFile wrote.

Try doing WriteFile first (with Create flag), and then os.OpenFile (instead of os.Create) that same file with Append flag to append the remaining bytes to the end.

Also, it's extremely risky to allow a client to give you the filename as it could be ../../.bashrc (for example), to which you'd overwrite your shell init with whatever the user decided to upload.

It would be much safer if you computed a filename yourself, and if you need to remember the user's selected filename, store that in your database or even a metadata.json type file that you load later.

huangapple
  • 本文由 发表于 2016年4月28日 00:31:03
  • 转载请务必保留本文链接:https://go.coder-hub.com/36895714.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定