英文:
panic: close of closed channel during persistent http call in GO(golang)
问题
我们正在使用这个函数进行HTTP调用,其中maxIdleConnsPerHost=100,我们每分钟最多收到20000个请求,从第三方服务器获取结果。
函数:
func (hr *Client) Do(req *http.Request) (rsp *http.Response, err error) {
rspCh := make(chan *http.Response)
errCh := make(chan error)
var timeoutCh <-chan time.Time
if hr.Timeout > 0 {
timeoutCh = time.After(hr.Timeout)
}
go func(req *http.Request) {
rsp, err := hr.Client.Do(req)
if err != nil {
errCh <- err
rspCh <- rsp
logger.SetLog(" error in http_client_Do : ", logs.LevelError, err)
if nil != rsp {
fmt.Println(rsp.Header)
} else {
fmt.Println("error with nil rsp")
}
} else {
rspCh <- rsp
//errCh <- nil
}
}(req)
var now time.Time
select {
case err = <-errCh:
rsp = <-rspCh
case now = <-timeoutCh:
go hr.Transport.CancelRequest(req)
err = fmt.Errorf("error requesting %s: read timed out at %s after waiting %s",
req.URL, now.Format(time.RFC3339), hr.Timeout)
case rsp = <-rspCh:
err = nil
}
return
}
我不明白为什么在某些情况下会出错(长时间运行正常)。
错误信息:
panic: close of closed channel
goroutine 3822098 [running]:
net/http.func·018()
/usr/local/go/src/net/http/transport.go:517 +0x2a
net/http.(*Transport).CancelRequest(0xc2080b02d0, 0xc20880de10)
/usr/local/go/src/net/http/transport.go:284 +0x97
created by Mediation/pkg/services/restApi.(*Client).Do
/usr/local/gopath/src/ ***.go:179 +0x3d2
goroutine 1 [chan receive, 308 minutes]:
github.com/astaxie/beego.(*App).Run(0xc20800a4b0)
/usr/local/gopath/src/github.com/astaxie/beego/app.go:171 +0x363
github.com/astaxie/beego.Run(0x0, 0x0, 0x0)
/usr/local/gopath/src/github.com/astaxie/beego/beego.go:277 +0x17c
main.main()
/usr/local/gopath/src/ ***/main.go:33 +0x423
goroutine 5 [syscall, 308 minutes]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/go/src/os/signal/signal_unix.go:27 +0x35
goroutine 2163451 [select, 146 minutes]:
net/http.(*persistConn).readLoop(0xc208501ce0)
/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:660 +0xc9f
goroutine 17 [chan receive, 8 minutes]:
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc20810a230)
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:340 +0xe3
github.com/Shopify/sarama.*Broker.(github.com/Shopify/sarama.responseReceiver)·fm()
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x27
github.com/Shopify/sarama.withRecover(0xc20800a6b0)
/usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.func·008
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x610
goroutine 8 [select, 8 minutes]:
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc2080b6100)
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:553 +0x2f3
github.com/Shopify/sarama.*client.(github.com/Shopify/sarama.backgroundMetadataUpdater)·fm()
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x27
github.com/Shopify/sarama.withRecover(0xc2080ba730)
/usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.NewClient
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x8ce
英文:
We are using this function for HTTP call with maxIdleConnsPerHost=100, where we are getting upto 20000/min request to fetch result from a third party server.
function :
func (hr *Client) Do(req *http.Request) (rsp *http.Response, err error) {
rspCh := make(chan *http.Response)
errCh := make(chan error)
var timeoutCh <-chan time.Time
if hr.Timeout > 0 {
timeoutCh = time.After(hr.Timeout)
}
go func(req *http.Request) {
rsp, err := hr.Client.Do(req)
if err != nil {
errCh <- err
rspCh <- rsp
logger.SetLog(" error in http_client_Do : ", logs.LevelError, err)
if nil!=rsp{
fmt.Println(rsp.Header)
}else{
fmt.Println("error with nil rsp")
}
} else {
rspCh <- rsp
//errCh <- nil
}
}(req)
var now time.Time
select {
case err = <-errCh:
rsp = <-rspCh
case now = <-timeoutCh:
go hr.Transport.CancelRequest(req)
err = fmt.Errorf("error requesting %s: read timed out at %s after waiting %s",
req.URL, now.Format(time.RFC3339), hr.Timeout)
case rsp = <-rspCh:
err = nil
}
return
}
I am not getting why this is breaking in some case. (it was running smooth for a long time ).
Error Message :
panic: close of closed channel
goroutine 3822098 [running]:
net/http.func·018()
/usr/local/go/src/net/http/transport.go:517 +0x2a
net/http.(*Transport).CancelRequest(0xc2080b02d0, 0xc20880de10)
/usr/local/go/src/net/http/transport.go:284 +0x97
created by Mediation/pkg/services/restApi.(*Client).Do
/usr/local/gopath/src/ ***.go:179 +0x3d2
goroutine 1 [chan receive, 308 minutes]:
github.com/astaxie/beego.(*App).Run(0xc20800a4b0)
/usr/local/gopath/src/github.com/astaxie/beego/app.go:171 +0x363
github.com/astaxie/beego.Run(0x0, 0x0, 0x0)
/usr/local/gopath/src/github.com/astaxie/beego/beego.go:277 +0x17c
main.main()
/usr/local/gopath/src/ ***/main.go:33 +0x423
goroutine 5 [syscall, 308 minutes]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
/usr/local/go/src/os/signal/signal_unix.go:27 +0x35
goroutine 2163451 [select, 146 minutes]:
net/http.(*persistConn).readLoop(0xc208501ce0)
/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:660 +0xc9f
goroutine 17 [chan receive, 8 minutes]:
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc20810a230)
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:340 +0xe3
github.com/Shopify/sarama.*Broker.(github.com/Shopify/sarama.responseReceiver)·fm()
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x27
github.com/Shopify/sarama.withRecover(0xc20800a6b0)
/usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.func·008
/usr/local/gopath/src/github.com/Shopify/sarama/broker.go:93 +0x610
goroutine 8 [select, 8 minutes]:
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc2080b6100)
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:553 +0x2f3
github.com/Shopify/sarama.*client.(github.com/Shopify/sarama.backgroundMetadataUpdater)·fm()
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x27
github.com/Shopify/sarama.withRecover(0xc2080ba730)
/usr/local/gopath/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.NewClient
/usr/local/gopath/src/github.com/Shopify/sarama/client.go:142 +0x8ce
答案1
得分: 2
transport.CancelRequest
在你的代码中被多次调用。
你可能正在做以下一项或两项操作:
-
重复使用
*http.Request
,导致查找可能返回错误的取消函数。 -
设置了
http.Client.Timeout
,它也使用了Transport.CancelRequest
机制,导致请求被取消的竞争。
英文:
transport.CancelRequest
is being called multiple times from your code somehow.
You are likely doing one or both of the following:
-
you are re-using the
*http.Request
, causing the lookup to possibly return the cancel function from the wrong round-trip. -
You have set
http.Client.Timeout
, which also uses theTransport.CancelRequest
mechanism, causing a race to cancel the request
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论