英文:
io.Copy write: broken pipe for big file
问题
我有一个文件服务器和一个代理服务器,通过代理服务器可以通过http访问文件服务器。
简单的文件服务器:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, "tmp/1.mp4")
})
if err := http.ListenAndServe("localhost:3000", nil); err != http.ErrServerClosed {
log.Fatalf("HTTP server ListenAndServe: %v", err)
}
和代理服务器:
type Server struct {
conn net.Conn
}
func (s *Server) Handle(w http.ResponseWriter, r *http.Request) error {
if err := r.Write(s.conn); err != nil {
log.Fatal("req.Write ", err)
}
resp, err := http.ReadResponse(bufio.NewReader(s.conn), r)
if err != nil {
return fmt.Errorf("http.ReadResponse: %s", err)
}
defer func() {
if resp.Body != nil {
if err := resp.Body.Close(); err != nil && err != io.ErrUnexpectedEOF {
fmt.Printf("resp.Body Close error: %s", err)
}
}
}()
copyHeader(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
if _, err := io.Copy(w, resp.Body); err != nil {
if err == io.ErrUnexpectedEOF {
fmt.Println("Client closed the connection, couldn't copy response")
} else {
fmt.Printf("copy err: %s\n", err)
}
}
return nil
}
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if err := s.Handle(w, r); err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
}
}
func copyHeader(dst, src http.Header) {
for k, v := range src {
vv := make([]string, len(v))
copy(vv, v)
dst[k] = vv
}
}
func main() {
conn, err := net.Dial("tcp", "localhost:3000")
if err != nil {
fmt.Errorf("net.Dial(tcp, localhost:3000) %s", err)
}
server := &Server{conn}
log.Fatal(http.ListenAndServe("localhost:8000", server))
}
如果文件服务器分发一个小图片,那么一切正常工作。
如果文件服务器提供一个大的视频文件(1.mp4),那么我会得到以下错误:
copy err: readfrom tcp 127.0.0.1:8000->127.0.0.1:58964: write tcp 127.0.0.1:8000->127.0.0.1:58964: write: broken pipe
可能的问题是什么?
英文:
I have a file server and a proxy server through which I can access the file server via http
Simple file server :
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, "tmp/1.mp4")
})
if err := http.ListenAndServe("localhost:3000", nil); err != http.ErrServerClosed {
log.Fatalf("HTTP server ListenAndServe: %v", err)
}
and proxy server :
type Server struct {
conn net.Conn
}
func (s *Server) Handle(w http.ResponseWriter, r *http.Request) error {
if err := r.Write(s.conn); err != nil {
log.Fatal("req.Write ", err)
}
resp, err := http.ReadResponse(bufio.NewReader(s.conn), r)
if err != nil {
return fmt.Errorf("http.ReadResponse: %s", err)
}
defer func() {
if resp.Body != nil {
if err := resp.Body.Close(); err != nil && err != io.ErrUnexpectedEOF {
fmt.Printf("resp.Body Close error: %s", err)
}
}
}()
copyHeader(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
if _, err := io.Copy(w, resp.Body); err != nil {
if err == io.ErrUnexpectedEOF {
fmt.Println("Client closed the connection, couldn't copy response")
} else {
fmt.Printf("copy err: %s\n", err)
}
}
return nil
}
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if err := s.Handle(w, r); err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
}
}
func copyHeader(dst, src http.Header) {
for k, v := range src {
vv := make([]string, len(v))
copy(vv, v)
dst[k] = vv
}
}
func main() {
conn, err := net.Dial("tcp", "localhost:3000")
if err != nil {
fmt.Errorf("net.Dial(tcp, localhost:3000) %s", err)
}
server := &Server{conn}
log.Fatal(http.ListenAndServe("localhost:8000", server))
}
if the file server distributes a small picture, then everything works correctly
if the file server is serving a large video file (1.mp4) then I get the error:
copy err: readfrom tcp 127.0.0.1:8000->127.0.0.1:58964: write tcp 127.0.0.1:8000->127.0.0.1:58964: write: broken pipe
What could be the problem ?
答案1
得分: 2
broken pipe
是与错误码32 EPIPE
相关联的Golang消息。
在处理TCP连接时,当你的进程在另一端已经关闭连接时向其写入数据时,会触发该错误。
根据你对“客户端关闭连接”的理解,如果你的意图是寻找一个指示连接到你的代理的外部客户端提前关闭连接的指标,那么你应该查找EPIPE
错误,而不是io.ErrUnexpectedEOF
。
可以使用以下代码:
func isEPIPE(err error) bool {
var syscallErr syscall.Errno
b := errors.As(err, &syscallErr) && syscallErr == syscall.EPIPE
return b
}
...
if isEPIPE(err) {
fmt.Println("客户端关闭了连接,无法复制响应")
} else {
fmt.Printf("复制错误:%s\n", err)
}
io.ErrUnexpectedEOF
可能在某些情况下发生,但它是由与文件服务器的连接引发的,而不是由客户端发起的连接。
英文:
broken pipe
is the golang message linked to errno 32 EPIPE
.
When dealing with a TCP connection, it is triggered when your process writes to a connection when the other end has already closed it.
Depening on what you mean by "Client closed the connection"
, if your intention is to look for an indicator that the outside client that connected to your proxy closed its connection early, then you should be looking for EPIPE
errors, not for io.ErrUnexpectedEOF
.
Something like :
func isEPIPE(err error) bool {
var syscallErr syscall.Errno
b := errors.As(err, &syscallErr) && syscallErr == syscall.EPIPE
return b
}
...
if isEPIPE(err) {
fmt.Println("Client closed the connection, couldn't copy response")
} else {
fmt.Printf("copy err: %s\n", err)
}
io.ErrUnexpectedEOF
could probably occur in some situation, but it would be triggered from the connection with your file server, not from the connection initiated by the client.
答案2
得分: 0
当浏览器接收到一个mp4文件时,它会切换到流媒体模式。也就是说,它开始发送带有范围头的HTTP请求,并期望得到一个状态码为206的响应。
我的代理服务器总是以状态码200作为响应。
copyHeader(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
这会强制浏览器断开连接。
英文:
The browser, when receiving an mp4 file, switches to streaming mode
That is, it starts sending http requests with the range header and expects a response with a status of 206
My proxy server always responds with the status 200
copyHeader(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
this forces the browser to disconnect the connection
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论