如何处理Golang中的HTTP超时错误和访问状态码

huangapple go评论85阅读模式
英文:

How to handle HTTP timeout errors and accessing status codes in golang

问题

我有一些用Go编写的代码(见下文),它应该可以“扇出”HTTP请求,并将详细信息汇总/聚合回来。

我是Go语言的新手,所以请理解我可能是个菜鸟,我的知识有限。

目前程序的输出类似于:

{
    "Status":"success",
    "Components":[
        {"Id":"foo","Status":200,"Body":"..."},
        {"Id":"bar","Status":200,"Body":"..."},
        {"Id":"baz","Status":404,"Body":"..."},
        ...
    ]
}

有一个本地服务器正在运行,故意运行缓慢(休眠5秒,然后返回响应)。但是我还列出了其他一些站点(见下面的代码),有时也会触发错误(如果它们出错,那就没关系)。

我目前遇到的问题是如何最好地处理这些错误,特别是与“超时”相关的错误;我不确定如何识别失败是否是超时或其他错误?

目前我一直得到一个总是相同的错误:

Get http://localhost:8080/pugs: read tcp 127.0.0.1:8080: use of closed network connection

其中http://localhost:8080/pugs通常是失败的URL(希望是超时!)。但是从代码中可以看出,我不知道如何确定错误代码是否与超时相关,也不知道如何访问响应的状态代码(我目前只是将其统一设置为404,但显然这是不正确的——如果服务器出错,我期望得到类似于500的状态代码,并且我希望在发送回的聚合响应中反映出来)。

完整的代码如下所示。任何帮助都将不胜感激。

package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
    "sync"
    "time"
)

type Component struct {
    Id  string `json:"id"`
    Url string `json:"url"`
}

type ComponentsList struct {
    Components []Component `json:"components"`
}

type ComponentResponse struct {
    Id     string
    Status int
    Body   string
}

type Result struct {
    Status     string
    Components []ComponentResponse
}

var overallStatus string = "success"

func main() {
    var cr []ComponentResponse
    var c ComponentsList

    b := []byte(`{"components":[{"id":"local","url":"http://localhost:8080/pugs"},{"id":"google","url":"http://google.com/"},{"id":"integralist","url":"http://integralist.co.uk/"},{"id":"sloooow","url":"http://stevesouders.com/cuzillion/?c0=hj1hfff30_5_f&t=1439194716962"}]}`)

    json.Unmarshal(b, &c)

    var wg sync.WaitGroup

    timeout := time.Duration(1 * time.Second)
    client := http.Client{
        Timeout: timeout,
    }

    for i, v := range c.Components {
        wg.Add(1)

        go func(i int, v Component) {
            defer wg.Done()

            resp, err := client.Get(v.Url)

            if err != nil {
                fmt.Printf("Problem getting the response: %s\n", err)

                cr = append(cr, ComponentResponse{
                    v.Id,
                    404,
                    err.Error(),
                })
            } else {
                defer resp.Body.Close()
                contents, err := ioutil.ReadAll(resp.Body)
                if err != nil {
                    fmt.Printf("Problem reading the body: %s\n", err)
                }

                cr = append(cr, ComponentResponse{
                    v.Id,
                    resp.StatusCode,
                    string(contents),
                })
            }
        }(i, v)
    }
    wg.Wait()

    j, err := json.Marshal(Result{overallStatus, cr})
    if err != nil {
        fmt.Printf("Problem converting to JSON: %s\n", err)
        return
    }

    fmt.Println(string(j))
}
英文:

I have some code (see below) written in Go which is supposed to "fan-out" HTTP requests, and collate/aggregate the details back.

> I'm new to golang and so expect me to be a nOOb and my knowledge to be limited

The output of the program is currently something like:

{
"Status":"success",
"Components":[
{"Id":"foo","Status":200,"Body":"..."},
{"Id":"bar","Status":200,"Body":"..."}, 
{"Id":"baz","Status":404,"Body":"..."}, 
...
]
}

There is a local server running that is purposely slow (sleeps for 5 seconds and then returns a response). But I have other sites listed (see code below) that sometime trigger an error as well (if they error, then that's fine).

The problem I have at the moment is how best to handle these errors, and specifically the "timeout" related errors; in that I'm not sure how to recognise if a failure is a timeout or some other error?

At the moment I get a blanket error back all the time:

Get http://localhost:8080/pugs: read tcp 127.0.0.1:8080: use of closed network connection

Where http://localhost:8080/pugs will generally be the url that failed (hopefully by timeout!). But as you can see from the code (below), I'm not sure how to determine the error code is related to a timeout nor how to access the status code of the response (I'm currently just blanket setting it to 404 but obviously that's not right - if the server was to error I'd expect something like a 500 status code and obviously I'd like to reflect that in the aggregated response I send back).

The full code can be seen below. Any help appreciated.

	package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"sync"
"time"
)
type Component struct {
Id  string `json:"id"`
Url string `json:"url"`
}
type ComponentsList struct {
Components []Component `json:"components"`
}
type ComponentResponse struct {
Id     string
Status int
Body   string
}
type Result struct {
Status     string
Components []ComponentResponse
}
var overallStatus string = "success"
func main() {
var cr []ComponentResponse
var c ComponentsList
b := []byte(`{"components":[{"id":"local","url":"http://localhost:8080/pugs"},{"id":"google","url":"http://google.com/"},{"id":"integralist","url":"http://integralist.co.uk/"},{"id":"sloooow","url":"http://stevesouders.com/cuzillion/?c0=hj1hfff30_5_f&t=1439194716962"}]}`)
json.Unmarshal(b, &c)
var wg sync.WaitGroup
timeout := time.Duration(1 * time.Second)
client := http.Client{
Timeout: timeout,
}
for i, v := range c.Components {
wg.Add(1)
go func(i int, v Component) {
defer wg.Done()
resp, err := client.Get(v.Url)
if err != nil {
fmt.Printf("Problem getting the response: %s\n", err)
cr = append(cr, ComponentResponse{
v.Id,
404,
err.Error(),
})
} else {
defer resp.Body.Close()
contents, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Printf("Problem reading the body: %s\n", err)
}
cr = append(cr, ComponentResponse{
v.Id,
resp.StatusCode,
string(contents),
})
}
}(i, v)
}
wg.Wait()
j, err := json.Marshal(Result{overallStatus, cr})
if err != nil {
fmt.Printf("Problem converting to JSON: %s\n", err)
return
}
fmt.Println(string(j))
}

答案1

得分: 3

我正在为您翻译以下内容:

我在接受答案的评论中添加了这个完整的部分,因为Dave C提供了正确的答案。

我们可以尝试将错误转换为net.Error并检查是否为超时错误。

resp, err := client.Get(url)
if err != nil {
	// 如果有错误,检查是否为超时错误
	if e, ok := err.(net.Error); ok && e.Timeout() {
		// 处理超时
           return
	} 
	// 否则处理其他类型的错误
}
英文:

I am adding this for completes, as the correct answer was provided by Dave C in the comments of the accepted answer.

We can try to cast the error to a net.Error and check if it is a timeout.

resp, err := client.Get(url)
if err != nil {
	// if there is an error check if its a timeout error
	if e, ok := err.(net.Error); ok && e.Timeout() {
		// handle timeout
           return
	} 
	// otherwise handle other types of error
}

答案2

得分: 2

如果你想要展开然后聚合结果,并且你想要特定的超时行为,而net/http包无法提供给你,那么你可能想要使用goroutines和channels。

我刚刚今天看了这个视频,它将通过Go语言的并发特性,为你详细介绍这些场景。此外,演讲者Rob Pike是相当权威的人物,他的解释比我能做得更好。

https://www.youtube.com/watch?v=f6kdp27TYZs

英文:

If you want to fan out then aggregate results and you want specific timeout behavior the net/http package isn't giving you, then you may want to use goroutines and channels.

I just watched this video today and it will walk you through exactly those scenarios using the concurrency features of Go. Plus, the speaker Rob Pike is quite the authority -- he explains it much better than I could.

https://www.youtube.com/watch?v=f6kdp27TYZs

答案3

得分: -3

Go 1.5版本通过更具体地处理错误类型来解决了这个问题。

所以,如果你看一下这个例子https://github.com/Integralist/Go-Requester/blob/master/requester.go#L38,你会发现我能够对错误消息应用正则表达式模式,以判断错误是否确实是超时错误。

status := checkError(err.Error())
func checkError(msg string) int {
timeout, _ := regexp.MatchString("Timeout", msg)
if timeout {
return 408
}
return 500
}
英文:

The Go 1.5 release solved this issue by being more specific about the type of error it has handled.

So if you see this example https://github.com/Integralist/Go-Requester/blob/master/requester.go#L38 you'll see that I'm able to apply a regex pattern to the error message to decipher if the error was indeed a timeout or not

status := checkError(err.Error())
func checkError(msg string) int {
timeout, _ := regexp.MatchString("Timeout", msg)
if timeout {
return 408
}
return 500
}

huangapple
  • 本文由 发表于 2015年8月17日 02:22:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/32038440.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定