英文:
How to download only the beginning of a large file with go http.Client (and app engine urlfetch)
问题
我正在尝试使用Go语言中的http.Client只下载大文件的前1KB，但似乎response.Body总是完全缓冲的。是否有控制缓冲区大小的方法？
如果有的话，如何在App Engine的urlfetch服务中使用？
以下是在Python中使用App Engine的urlfetch正常工作的代码，我正在尝试将其移植到Go语言中：
import (
"io"
"net/http"
)
func main() {
client := &http.Client{}
req, err := http.NewRequest("GET", url, nil)
if err != nil {
// 处理错误
}
req.Header.Set("Range", "bytes=0-1023") // 设置Range头部，指定只下载前1KB
resp, err := client.Do(req)
if err != nil {
// 处理错误
}
defer resp.Body.Close()
buf := make([]byte, 1024)
_, err = io.ReadFull(resp.Body, buf) // 读取前1KB的内容
if err != nil && err != io.ErrUnexpectedEOF {
// 处理错误
}
// 使用buf中的数据
}
希望对你有帮助！
英文:
I'm trying to download only the first 1kb of large files using http.Client in go, but it seems like response.Body is always fully buffered. Is there a control over how much to buffer?
If so, how can this be used with app engine urlfetch service?
The following works fine with app engine urlfetch in python, and I'm trying to port this to go:
from urllib2 import urlopen
req = Request(url)
urlopen(req).read(1024) # Read the first 1kb.
答案1
得分: 10
将"Range"请求头属性设置为"bytes=0-1023"应该可以工作。
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {
req, _ := http.NewRequest("GET", "http://example.com", nil)
req.Header.Add("Range", "bytes=0-1023")
fmt.Println(req)
var client http.Client
resp, _ := client.Do(req)
fmt.Println(resp)
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(len(body))
}
同样的方法也适用于在应用引擎中提供的http.Client。
英文:
Setting "Range" Request Header attribute to "bytes=0-1023" should work.
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {
req, _ := http.NewRequest("GET", "http://example.com", nil)
req.Header.Add("Range", "bytes=0-1023")
fmt.Println(req)
var client http.Client
resp, _ := client.Do(req)
fmt.Println(resp)
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(len(body))
}
Same thing should work for http.Client provieded in app engine.
答案2
得分: 3
在io包中还有io.LimitReader()函数，可以创建一个有限制的读取器。
package main
import (
"io"
"io/ioutil"
"net/http"
)
func main() {
var client http.Client
responce, _ := client.Get("http://example.com")
body := responce.Body
chunk := io.LimitReader(body, 1024)
stuff, _ := ioutil.ReadAll(chunk)
// 对stuff进行你想要的操作
}
以上是要翻译的内容。
英文:
There is also io.LimitReader() in io pkg, which can make limited reader
package main
import (
"io"
"io/ioutil"
"net/http"
)
func main() {
var client http.Client
responce, _:=client.Get("http://example.com")
body:=responce.Body
chunk:=io.LimitReader(body, 1024)
stuff, _:=ioutil.ReadAll(chunk)
//Do what you want with stuff
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论