恐慌:在GO(golang)中进行持久化HTTP调用期间关闭了已关闭的通道。

huangapple go评论82阅读模式
英文:

panic: close of closed channel during persistent http call in GO(golang)

问题

我们正在使用这个函数进行HTTP调用,其中maxIdleConnsPerHost=100,我们每分钟最多收到20000个请求,从第三方服务器获取结果。

函数:

func (hr *Client) Do(req *http.Request) (rsp *http.Response, err error) {
rspCh := make(chan *http.Response)
errCh := make(chan error)

var timeoutCh <-chan time.Time
if hr.Timeout > 0 {
    timeoutCh = time.After(hr.Timeout)
}

go func(req *http.Request) {
    rsp, err := hr.Client.Do(req)
    if err != nil {
        errCh <- err
        rspCh <- rsp
        logger.SetLog(" error in http_client_Do : ", logs.LevelError, err)
        if nil != rsp {
            fmt.Println(rsp.Header)
        } else {
            fmt.Println("error with nil rsp")
        }
    } else {
        rspCh <- rsp
        //errCh <- nil
    }
}(req)

var now time.Time

select {
case err = <-errCh:
    rsp = <-rspCh
case now = <-timeoutCh:
    go hr.Transport.CancelRequest(req)
    err = fmt.Errorf("error requesting %s: read timed out at %s after waiting %s",
        req.URL, now.Format(time.RFC3339), hr.Timeout)
case rsp = <-rspCh:
    err = nil
}
return

}

我不明白为什么在某些情况下会出错(长时间运行正常)。

错误信息:

panic: close of closed channel

goroutine 3822098 [running]:
net/http.func·018()
/usr/local/go/src/net/http/transport.go:517 +0x2a
net/http.(*Transport).CancelRequest(0xc2080b02d0, 0xc20880de10)
/usr/local/go/src/net/http/transport.go:284 +0x97
created by Mediation/pkg/services/restApi.(*Client).Do
/usr/local/gopath/src/ ***.go:179 +0x3d2

goroutine 1 [chan receive, 308 minutes]:
github.com/astaxie/beego.(*App).Run(0xc20800a4b0)
/usr/local/gopath/src/github.com/astaxie/beego/app.go:171 +0x363
github.com/astaxie/beego.Run(0x0, 0x0, 0x0)
/usr/local/gopath/src/github.com/astaxie/beego/beego.go:277 +0x17c
main.main()
/usr/local/gopath/src/ ***/main.go:33 +0x423

goroutine 5 [syscall, 308 minutes]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 2163451 [select, 146 minutes]:
net/http.(*persistConn).readLoop(0xc208501ce0)
/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 17 [chan receive, 8 minutes]:
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc20810a230)
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:340 +0xe3
github.com/Shopify/sarama.*Broker.(github.com/Shopify/sarama.responseReceiver)·fm()
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x27
github.com/Shopify/sarama.withRecover(0xc20800a6b0)
/usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.func·008
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x610

goroutine 8 [select, 8 minutes]:
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc2080b6100)
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:553 +0x2f3
github.com/Shopify/sarama.*client.(github.com/Shopify/sarama.backgroundMetadataUpdater)·fm()
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x27
github.com/Shopify/sarama.withRecover(0xc2080ba730)
/usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.NewClient
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x8ce

英文:

We are using this function for HTTP call with maxIdleConnsPerHost=100, where we are getting upto 20000/min request to fetch result from a third party server.

function :

func (hr *Client) Do(req *http.Request) (rsp *http.Response, err error) {
rspCh := make(chan *http.Response)
errCh := make(chan error)

var timeoutCh &lt;-chan time.Time
if hr.Timeout &gt; 0 {
	timeoutCh = time.After(hr.Timeout)
}

go func(req *http.Request) {
	rsp, err := hr.Client.Do(req)
	if err != nil {
		errCh &lt;- err
		rspCh &lt;- rsp
		logger.SetLog(&quot; error in http_client_Do : &quot;, logs.LevelError, err)
		if nil!=rsp{
			fmt.Println(rsp.Header)
		}else{
			fmt.Println(&quot;error with nil rsp&quot;)
		}		
	} else {
		rspCh &lt;- rsp
		//errCh &lt;- nil
	}
}(req)

var now time.Time

select {
case err = &lt;-errCh:
	rsp = &lt;-rspCh
case now = &lt;-timeoutCh:
	go hr.Transport.CancelRequest(req)
	err = fmt.Errorf(&quot;error requesting %s: read timed out at %s after waiting %s&quot;,
		req.URL, now.Format(time.RFC3339), hr.Timeout)
case rsp = &lt;-rspCh:
	err = nil
}
return
}

I am not getting why this is breaking in some case. (it was running smooth for a long time ).

Error Message :

panic: close of closed channel

goroutine 3822098 [running]:
net/http.func&#183;018()
        /usr/local/go/src/net/http/transport.go:517 +0x2a
net/http.(*Transport).CancelRequest(0xc2080b02d0, 0xc20880de10)
        /usr/local/go/src/net/http/transport.go:284 +0x97
created by Mediation/pkg/services/restApi.(*Client).Do
        /usr/local/gopath/src/ ***.go:179 +0x3d2

goroutine 1 [chan receive, 308 minutes]:
github.com/astaxie/beego.(*App).Run(0xc20800a4b0)
        /usr/local/gopath/src/github.com/astaxie/beego/app.go:171 +0x363
github.com/astaxie/beego.Run(0x0, 0x0, 0x0)
        /usr/local/gopath/src/github.com/astaxie/beego/beego.go:277 +0x17c
main.main()
        /usr/local/gopath/src/ ***/main.go:33 +0x423

goroutine 5 [syscall, 308 minutes]:
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init&#183;1
        /usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 2163451 [select, 146 minutes]:
net/http.(*persistConn).readLoop(0xc208501ce0)
        /usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
        /usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 17 [chan receive, 8 minutes]:
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc20810a230)
        /usr/local/gopath/src/github.com/Shopify/sarama/broker.go:340 +0xe3
github.com/Shopify/sarama.*Broker.(github.com/Shopify/sarama.responseReceiver)&#183;fm()
        /usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x27
github.com/Shopify/sarama.withRecover(0xc20800a6b0)
        /usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.func&#183;008
        /usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x610

goroutine 8 [select, 8 minutes]:
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc2080b6100)
        /usr/local/gopath/src/github.com/Shopify/sarama/client.go:553 +0x2f3
github.com/Shopify/sarama.*client.(github.com/Shopify/sarama.backgroundMetadataUpdater)&#183;fm()
        /usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x27
github.com/Shopify/sarama.withRecover(0xc2080ba730)
        /usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.NewClient
        /usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x8ce

答案1

得分: 2

transport.CancelRequest 在你的代码中被多次调用。

你可能正在做以下一项或两项操作:

  • 重复使用 *http.Request,导致查找可能返回错误的取消函数。

  • 设置了 http.Client.Timeout,它也使用了 Transport.CancelRequest 机制,导致请求被取消的竞争。

英文:

transport.CancelRequest is being called multiple times from your code somehow.

You are likely doing one or both of the following:

  • you are re-using the *http.Request, causing the lookup to possibly return the cancel function from the wrong round-trip.

  • You have set http.Client.Timeout, which also uses the Transport.CancelRequest mechanism, causing a race to cancel the request

huangapple
  • 本文由 发表于 2015年7月29日 13:21:24
  • 转载请务必保留本文链接:https://go.coder-hub.com/31691756.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定