英文:
how browser knows end of reading data
问题
我正在尝试使用Golang创建一个HTTPS代理。我知道首先,浏览器会发送带有\r\n
结尾的头部,并且套接字会在读取这些字符之前阻塞read()
。但是当它是加密的(ssl/tls)和HTTP 1.1(保持连接活动)时:
- 浏览器如何知道读取数据的结束?
- 他们是逐字节读取并在末尾有一些特殊字符吗(对于大数据来说,这是一个好的方式吗)?
- 或者他们首先发送数据的大小(如TCP套接字的主题中建议的那样)?
- 作为代理,当流式传输或加载简单的HTML页面时,我如何理解数据的结束?
这只是代码的一部分,在本地网络中运行时它可以工作,但在服务器(VPS)上会在读取时阻塞直到连接关闭。完整的代码在这里
func write(client_to_proxy net.Conn, browser_to_client net.Conn) {
defer client_to_proxy.Close()
buffer := make([]byte, 1024)
reader := bufio.NewReader(browser_to_client)
for {
length, err := reader.Read(buffer)
if length > 0 {
fmt.Println(time.Now().Format(time.Stamp) + " 从客户端读取到浏览器:" + strconv.Itoa(length))
//fmt.Println(string(buffer[:readLeng]))
writeLength, err := client_to_proxy.Write(buffer[:length])
if writeLength > 0 {
fmt.Println(time.Now().Format(time.Stamp) + " 从客户端写入到浏览器:" + strconv.Itoa(writeLength))
}
if err != nil {
fmt.Println("错误6:", err)
return
}
}
if err != nil {
fmt.Println("错误5:", err)
return
}
}
}
func read(client_to_proxy net.Conn, browser_to_client net.Conn) {
defer browser_to_client.Close()
buffer := make([]byte, 1024)
reader := bufio.NewReader(client_to_proxy)
length, err := reader.Read(buffer)
fmt.Println(time.Now().Format(time.Stamp) + " 从代理读取到客户端:" + strconv.Itoa(length))
fmt.Println(string(buffer))
if length > 0 {
writeLength, err := browser_to_client.Write(buffer[:length])
fmt.Println(time.Now().Format(time.Stamp) + " 从客户端写入到浏览器:" + strconv.Itoa(writeLength))
if err != nil {
fmt.Println("错误7:", err)
return
}
}
if err != nil {
return
}
go write(client_to_proxy, browser_to_client)
for {
length, err := reader.Read(buffer)
fmt.Println(time.Now().Format(time.Stamp) + " 从代理读取到客户端:" + strconv.Itoa(length))
//fmt.Println(string(buffer[:length]))
if length > 0 {
writeLength, err := browser_to_client.Write(buffer[:length])
fmt.Println(time.Now().Format(time.Stamp) + " 从客户端写入到浏览器:" + strconv.Itoa(writeLength))
if err != nil {
fmt.Println("错误8:", err)
return
}
}
if err != nil {
return
}
}
}
编辑1:
我使用一个类似于浏览器->客户端->代理->so.com然后so.com->代理->客户端->浏览器的客户端和服务器go应用程序。
我不想要加密的数据!我的问题在于'client'应用程序,我不知道应该读取多少字节才能解除read()
的阻塞!
英文:
I am trying to make a https proxy with Golang. I know at first, browser sends header with ending to \r\n
and socket blocks read()
until reading those character. But when it is encrypted(ssl/tls) and HTTP 1.1(keep connection alive)
- how Browser knows end of reading data?
- Do they read byte by byte and some special character at end(Is this a good way for large data at all?)?
- Or they send size of data first(as suggested in topics for tcp socket)?
- As proxy, how can I understand end of data when streaming or loading simple html page?
This is only part of code, it works when i run it in local network, but in server(vps) will block on read until connection closed. complete code here
func write(client_to_proxy net.Conn, browser_to_client net.Conn) {
defer client_to_proxy.Close()
buffer := make([]byte, 1024)
reader := bufio.NewReader(browser_to_client)
for {
length, err := reader.Read(buffer)
if length > 0 {
fmt.Println(time.Now().Format(time.Stamp) + " READ from client to browser: " + strconv.Itoa(length))
//fmt.Println(string(buffer[:readLeng]))
writeLength, err := client_to_proxy.Write(buffer[:length])
if writeLength > 0 {
fmt.Println(time.Now().Format(time.Stamp) + " WRITE from client to browser: " + strconv.Itoa(writeLength))
}
if err != nil {
fmt.Println("ERR6 ", err)
return
}
}
if err != nil {
fmt.Println("ERR5 ", err)
return
}
}
}
func read(client_to_proxy net.Conn, browser_to_client net.Conn) {
defer browser_to_client.Close()
buffer := make([]byte, 1024)
reader := bufio.NewReader(client_to_proxy)
length, err := reader.Read(buffer)
fmt.Println(time.Now().Format(time.Stamp) + " READ from proxy to client: " + strconv.Itoa(length))
fmt.Println(string(buffer))
if length > 0 {
writeLength, err := browser_to_client.Write(buffer[:length])
fmt.Println(time.Now().Format(time.Stamp) + " WRITE from client to browser: " + strconv.Itoa(writeLength))
if err != nil {
fmt.Println("ERR7 ", err)
return
}
}
if err != nil {
return
}
go write(client_to_proxy, browser_to_client)
for {
length, err := reader.Read(buffer)
fmt.Println(time.Now().Format(time.Stamp) + " READ from proxy to client: " + strconv.Itoa(length))
//fmt.Println(string(buffer[:length]))
if length > 0 {
writeLength, err := browser_to_client.Write(buffer[:length])
fmt.Println(time.Now().Format(time.Stamp) + " WRITE from client to browser: " + strconv.Itoa(writeLength))
if err != nil {
fmt.Println("ERR8 ", err)
return
}
}
if err != nil {
return
}
}
}
EDIT 1:
I use a client and server go app like this browser->client->proxy->so.com then so.com->prxoy->client->browser
I don't want encrypted data! my problem is at 'client' app, I don't know know how much should read bytes to unblock read()
!
答案1
得分: 3
浏览器如何知道数据读取结束?
通过实现标准。
对于HTTP/1.1,请参考RFC 7230第3.3节消息主体长度了解详细信息,了解如何确定主体长度。提示:基于HTTP头部的信息,如Content-length和Transfer-Encoding。
作为代理,当流式传输或加载简单的HTML页面时,我如何理解数据的结束?
代理在处理普通的HTTP消息时表现相同。
HTTPS流量通过代理在客户端和服务器之间进行端到端加密传输,使用隧道进行传输。这个隧道是通过CONNECT请求创建的,只有在TCP连接关闭时才会结束。代理无法了解连接的内部情况,因此也无法确定加密流量中各个请求和响应的HTTP消息头和主体的位置。
英文:
> How Browser knows end of reading data?
By implementing the standard.
For HTTP/1.1 see RFC 7230 section 3.3 Message Body Length for the details, how the length of the body is determined. Hint: based on information on the HTTP header, like Content-length and Transfer-Encoding.
> As proxy, how can I understand end of data when streaming or loading simple html page?
A proxy behaves the same regarding plain HTTP messages.
HTTPS traffic is passed through nd-to-end encrypted between client and server over the proxy using a tunnel. This tunnel gets created with a CONNECT request, which ends only with TCP connection close. The proxy has no insight into the connection and can thus also not determine where HTTP message header and body of the various requests and responses inside the encrypted traffic are.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论