Golang的http get请求在某些URL上出现问题,但不是所有的URL都有问题。

huangapple go评论79阅读模式
英文:

Golang http get request breaks on some but not all urls

问题

现在我正在从indiegogo获取URL,作为一个边项目,使用基本的get请求模板[这里][1]。然后,我使用以下代码将字节数据转换为字符串:

responseText, err:= ioutil.ReadAll(response.Body)
trueText:= string(responseText)

在需要的地方进行适当的错误处理。

对于重复尝试获取和其他长度不同的URL(至少与上一个URL一样大,有些比下一个URL更长),它都可以正常工作。

奇怪的是,当我尝试获取 时,它会出现运行时错误:

panic: runtime error: index out of range

并以2的状态退出。我想知道问题可能是什么。

我知道这不是indiegogo对我每分钟一次的请求感到愤怒并切断我的连接,因为我可以连续请求20分钟而没有问题。给它一点时间停顿,它仍然完全中断在

感谢您的帮助

编辑,似乎是一些页面中的格式错误的html导致了一个基于内容运行的循环出错,这个错误只在一些URL上中断了go的运行。谢谢帮助

[1]:

英文:

Right now I'm fetching urls from indiegogo as part of a side project using the basic get request template found [here][1]. I then translate the byte data into a string using

responseText, err:= ioutil.ReadAll(response.Body)
trueText:= string(responseText)

with appropriate error handling where needed

It works fine for repeated attempts at getting and some other urls of varying length(at least as large as the previous url and some longer than the next).

Strangely, when I attempt to get it breaks and throws a runtime error of

panic: runtime error: index out of range

and exits with a status of 2. I'm curious as to what the issue could be.

I know it isn't indiegogo getting angry about my once a minute requests and cutting my connection because I can request continiously for 20 minutes at with no issue. Give it a bit of downtime and it still completely breaks on

Thanks for the assistance

EDIT, it appears as though it was a malformed bit of html in some of the pages that messed with a loop I was running based on the content that managed to break go in the runtime on only some urls. Thanks for the help

[1]:

答案1

得分: 0

从URL获取并将body转换为Go string类型时没有错误。例如,

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
)

func main() {
	url := "http://www.indiegogo.com/projects/culcharge-smallest-usb-charge-and-data-cable-for-iphone-and-android"
	res, err := http.Get(url)
	if err != nil {
		log.Fatal(err)
	}
	body, err := ioutil.ReadAll(res.Body)
	res.Body.Close()
	if err != nil {
		log.Fatal(err)
	}
	text := string(body)
	fmt.Println(len(body), len(text))
}

输出:

66363 66363

您没有提供一个能够编译、运行并以您描述的方式失败的小代码片段。这让我们都在猜测。

英文:

There is no error when getting from the url and converting the body to the Go string type. For example,

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
)

func main() {
	url := "http://www.indiegogo.com/projects/culcharge-smallest-usb-charge-and-data-cable-for-iphone-and-android"
	res, err := http.Get(url)
	if err != nil {
		log.Fatal(err)
	}
	body, err := ioutil.ReadAll(res.Body)
	res.Body.Close()
	if err != nil {
		log.Fatal(err)
	}
	text := string(body)
	fmt.Println(len(body), len(text))
}

Output:

66363 66363

You didn't provide us with a small fragment of code which compiles, runs, and fails in the manner you describe. That leaves us all guessing.

huangapple
  • 本文由 发表于 2013年7月24日 06:34:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/17822161.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定