服务器在被快速洪水攻击时无法解析数据包。

huangapple
go评论112阅读模式
英文:

Server failing to parse packets when flooded too fast

问题

当我用每秒发送太多数据包来淹没服务器时，我遇到了以下错误：

  1. 2014/11/28 12:52:49 main.go:59: 加载插件: print
  2. 2014/11/28 12:52:49 main.go:86: 在 0.0.0.0:8080 上启动服务器
  3. 2014/11/28 12:52:59 server.go:15: 客户端已连接: 127.0.0.1:59146
  4. 2014/11/28 12:52:59 server.go:43: 从客户端 127.0.0.1:59146 接收到数据: &main.Observation{SensorId:"1", Timestamp:1416492023}
  5. 2014/11/28 12:52:59 server.go:29: 从 127.0.0.1:59146 读取错误: zlib: 无效的头部
  6. 2014/11/28 12:52:59 server.go:18: 关闭与 127.0.0.1:59146 的连接

它能够解码一个数据包（有时候可能是2或3个），然后出现错误。以下是执行淹没操作的代码：

  1. import socket
  2. import struct
  3. import json
  4. import zlib
  5. import time
  7. def serialize(data):
  8.     data = json.dumps(data)
  9.     data = zlib.compress(data)
  11.     packet  = struct.pack('!I', len(data))
  12.     packet += data
  14.     return len(data), packet
  16. message = {
  17.     'sensor_id': '1',
  18.     'timestamp': 1416492023,
  19. }
  21. length, buffer = serialize([message])
  23. client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  25. client.connect(('127.0.0.1', 8080))
  27. while True:
  28.     client.send(buffer)
  30.     #time.sleep(0.0005)

当我取消注释time.sleep()调用时，服务器正常工作。似乎每秒发送太多数据包会导致服务器崩溃。为什么会这样？

以下是相关的Go代码。连接处理程序：

  1. func (self *Server) handleConnection(connection net.Conn) {
  2.     for {
  3.         connection.SetReadDeadline(time.Now().Add(30 * time.Second))
  5.         observations, err := self.Protocol.Unserialize(connection)
  7.         if err != nil {
  8.             log.Printf("从 %s 读取错误: %s\n", connection.RemoteAddr(), err)
  10.             return
  11.         }
  12. }

以下是反序列化器：

  1. // Length Value protocol to read zlib compressed, JSON encoded packets.
  2. type ProtocolV2 struct{}
  4. func (self *ProtocolV2) Unserialize(packet io.Reader) ([]*Observation, error) {
  5.     var length uint32
  7.     if err := binary.Read(packet, binary.BigEndian, &length); err != nil {
  8.         return nil, err
  9.     }
  11.     buffer := make([]byte, length)
  12.     rawreader := bufio.NewReader(packet)
  14.     if _, err := rawreader.Read(buffer); err != nil {
  15.         return nil, err
  16.     }
  18.     bytereader := bytes.NewReader(buffer)
  19.     zreader, err := zlib.NewReader(bytereader)
  21.     if err != nil {
  22.         return nil, err
  23.     }
  25.     defer zreader.Close()
  27.     var observations []*Observation
  29.     decoder := json.NewDecoder(zreader)
  31.     if err := decoder.Decode(&observations); err != nil {
  32.         return nil, err
  33.     }
  35.     return observations, nil
  36. }
英文:

Here's the error I get when I flood the server with too many packets per second:

  1. 2014/11/28 12:52:49 main.go:59: loading plugin: print
  2. 2014/11/28 12:52:49 main.go:86: starting server on 0.0.0.0:8080
  3. 2014/11/28 12:52:59 server.go:15: client has connected: 127.0.0.1:59146
  4. 2014/11/28 12:52:59 server.go:43: received data from client 127.0.0.1:59146: &main.Observation{SensorId:"1", Timestamp:1416492023}
  5. 2014/11/28 12:52:59 server.go:29: read error from 127.0.0.1:59146: zlib: invalid header
  6. 2014/11/28 12:52:59 server.go:18: closing connection to: 127.0.0.1:59146

It manages to decode one packet (sometimes, maybe 2 or 3) then errors out. Here's the code doing the flooding:

  1. import socket
  2. import struct
  3. import json
  4. import zlib
  5. import time
  7. def serialize(data):
  8.     data = json.dumps(data)
  9.     data = zlib.compress(data)
  11.     packet  = struct.pack('!I', len(data))
  12.     packet += data
  14.     return len(data), packet
  16. message = {
  17.     'sensor_id': '1',
  18.     'timestamp': 1416492023,
  19. }
  21. length, buffer = serialize([message])
  23. client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  25. client.connect(('127.0.0.1', 8080))
  27. while True:
  28.     client.send(buffer)
  30.     #time.sleep(0.0005)

When I uncomment the time.sleep() call, the server works fine. It seems too many packets/per second is killing the server. Why?

Here's the relevent Go code. The connection handler:

  1. func (self *Server) handleConnection(connection net.Conn) {
  2.     for {
  3.         connection.SetReadDeadline(time.Now().Add(30 * time.Second))
  5.         observations, err := self.Protocol.Unserialize(connection)
  7.         if err != nil {
  8.             log.Printf("read error from %s: %s\n", connection.RemoteAddr(), err)
  10.             return
  11.         }
  12. }

And here's the unserializer:

  1. // Length Value protocol to read zlib compressed, JSON encoded packets.
  2. type ProtocolV2 struct{}
  4. func (self *ProtocolV2) Unserialize(packet io.Reader) ([]*Observation, error) {
  5.     var length uint32
  7.     if err := binary.Read(packet, binary.BigEndian, &length); err != nil {
  8.         return nil, err
  9.     }
  11.     buffer := make([]byte, length)
  12.     rawreader := bufio.NewReader(packet)
  14.     if _, err := rawreader.Read(buffer); err != nil {
  15.         return nil, err
  16.     }
  18.     bytereader := bytes.NewReader(buffer)
  19.     zreader, err := zlib.NewReader(bytereader)
  21.     if err != nil {
  22.         return nil, err
  23.     }
  25.     defer zreader.Close()
  27.     var observations []*Observation
  29.     decoder := json.NewDecoder(zreader)
  31.     if err := decoder.Decode(&observations); err != nil {
  32.         return nil, err
  33.     }
  35.     return observations, nil
  36. }

答案1

得分: 0

在Python脚本的客户端部分似乎出现了错误。

没有检查client.send的返回值，所以脚本没有正确处理部分写入的情况。基本上，当套接字缓冲区已满时，只会写入部分消息，导致服务器无法解码消息。

这段代码有问题，但添加等待使其工作，因为它防止套接字缓冲区满。

您可以使用client.sendall来确保写操作完成。

有关更多信息，请参阅Python文档：

https://docs.python.org/2/library/socket.html

https://docs.python.org/2/howto/sockets.html#using-a-socket

现在在Go服务器中，也存在类似的问题。文档中提到：

> Read将数据读入p中。它返回读入p的字节数。它在底层Reader上最多调用一次Read，因此n可能小于len(p)。在EOF时，计数将为零，err将为io.EOF。

rawreader.Read调用可能返回比您期望的字节数少。您可能希望使用io包的ReadFull()函数来确保完整读取消息。

英文:

It seems there is an error on client side in the Python script.

The return of client.send is not checked, so the script does not handle partial writes in a correct way. Basically, when the socket buffer is full, only part of the message will be written, resulting in the server unable to decode the message.

This code is broken, but adding the wait makes it work because it prevents the socket buffer to be full.

You can use client.sendall instead to ensure the write operations are complete.

More information in the Python documentation:

https://docs.python.org/2/library/socket.html

https://docs.python.org/2/howto/sockets.html#using-a-socket

Now in the Go server, there is also a similar problem. The documentation says:

> Read reads data into p. It returns the number of bytes read into p. It calls Read at most once on the underlying Reader, hence n may be less than len(p). At EOF, the count will be zero and err will be io.EOF.

The rawreader.Read call may return less bytes than you expect. You may want to use the ReadFull() function of the io package to ensure the full message is read.

huangapple
  • 本文由 发表于 2014年11月29日 19:16:10
  • 转载请务必保留本文链接：https://go.coder-hub.com/27201621.html
  • go
  • server
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定