Working with raw bytes from a network in go

huangapple go评论73阅读模式
英文:

Working with raw bytes from a network in go

问题

(抱歉,问题很长!)最近,我在一个副业项目中尝试使用Go而不是C++作为游戏服务器仿真器的实现语言,我开始怀疑自己是否以合理的Go方式实现了它。正如你所预期的那样,服务器通过发送符合特定协议规范的原始数据包(TCP)与一个或多个游戏客户端进行通信。相关部分的实现大致如下:

接收头部 -> 解密头部 -> 接收字节直到达到头部长度 -> 解密剩余数据包 -> 分发到处理程序 -> 解码数据包 -> 根据需要处理 -> 发送响应

该协议以小端字节顺序定义,因此在我的C++实现中,数据包头部如下所示(我知道,它只适用于小端机器):

struct pkt_header {
    uint16_t length;
    uint16_t type;
    uint32_t flags;
};

在接收和解密该头部后,我会提取字段:

// client->recv_buffer 的类型为 u_char[1024]
header = (pkt_header*) client->recv_buffer;

if (client->recv_size < header->length) {
    // 接收更多数据
}
// 解密等操作

在处理程序中,我可以将上述头部结构嵌套在其他数据包结构定义中,并将其强制转换为 byte[] 缓冲区数组,以直接访问字段。根据我所了解的,Go中的结构体对齐(不出所料)是困难的/不可能的,并且强烈不建议使用。

不知道该怎么办,我编写了以下函数,用于将任意结构体转换为 []byte:

// 将结构体的字段按照声明的顺序序列化为字节数组。如果 data 不是结构体或结构体指针,则调用 panic()。
func StructToBytes(data interface{}) []byte {
    val := reflect.ValueOf(data)
    valKind := val.Kind()
    if valKind == reflect.Ptr {
        val = reflect.ValueOf(data).Elem()
        valKind = val.Kind()
    }

    if valKind != reflect.Struct {
        panic("data must of type struct or struct ptr, got: " + valKind.String())
    }

    bytes := new(bytes.Buffer)
    for i := 0; i < val.NumField(); i++ {
        field := val.Field(i)

        switch kind := field.Kind(); kind {
        case reflect.Struct:
            binary.Write(bytes, binary.LittleEndian, StructToBytes(field.Interface()))
        case reflect.Array, reflect.Slice:
            binary.Write(bytes, binary.LittleEndian, field.Interface())
        case reflect.Uint8:
            binary.Write(bytes, binary.LittleEndian, uint8(field.Uint()))
        case reflect.Uint16:
            binary.Write(bytes, binary.LittleEndian, uint16(field.Uint()))
        // 你明白了
        }
    }
    return bytes.Bytes()
}

然后在处理程序中可以这样使用:

type Header struct {
    length uint16
    size uint16
    flags uint32
}
newHeader := new(Header)
// 初始化等操作
client.Conn.Write(StructToBytes(newHeader)) // 例如 [C8 00 03 00 00 00 01 00]

作为一个Go新手,我非常欢迎关于如何更高效地实现这个功能的反馈。到目前为止,它运行良好,但现在我面临的挑战是如何做相反的操作:从 []byte 转换为结构体(例如,[C8 00 03 00 00 01 00 00] 转换为 Header { length = C8, size = 03, flags = 0100 })。

我是否只需要实现这个功能的反向操作,还是有更好的方法可以从字节数组转换为任意结构体(或反之亦然,而不是使用我的函数)?如果需要更多的清晰度,请告诉我。

英文:

(Sorry, long question!) I've recently been trying out Go as opposed to C++ for a game server emulator I'm working on as a side project and am questioning whether I'm implementing it in sensible Go terms. As you might expect, the server communicates with one or more game clients by sending raw packets (TCP) that adhere to a particular protocol specification. The relevant part goes something like this:

receive a header -> decrypt it -> recv bytes until the header length is reached -> decrypt the rest of the packet -> dispatch to a handler -> decode the packet -> handle as necessary -> send a response

The protocol is defined in terms of bytes in little endian order, so in my C++ implementation the packet header looks like this (I know, it only works on LE machines):

struct pkt_header {
    uint16_t length;
    uint16_t type;
    uint32_t flags;
};

Upon recv()'ing and decrypting this header, I'll extract the fields:

// client-&gt;recv_buffer is of type u_char[1024]
header = (pkt_header*) client-&gt;recv_buffer;

if (client-&gt;recv_size &lt; header-&gt;length) {
    // Recv some more
}
// Decrypt and so on

In the handlers themselves I can nest the above header struct in other packet structure definitions and cast those onto the byte[] buffer arrays in order to directly access the fields. From what I've read, struct alignment (unsurprisingly) is difficult/impossible and highly discouraged in Go.

Not knowing what else to do, I wrote this function for going from an arbitrary Struct -> []byte:

// Serializes the fields of a struct to an array of bytes in the order in which the fields are
// declared. Calls panic() if data is not a struct or pointer to struct.
func StructToBytes(data interface{}) []byte {
	val := reflect.ValueOf(data)
	valKind := val.Kind()
	if valKind == reflect.Ptr {
		val = reflect.ValueOf(data).Elem()
		valKind = val.Kind()
	}

	if valKind != reflect.Struct {
		panic(&quot;data must of type struct or struct ptr, got: &quot; + valKind.String())
	}

	bytes := new(bytes.Buffer)
	for i := 0; i &lt; val.NumField(); i++ {
		field := val.Field(i)

		switch kind := field.Kind(); kind {
		case reflect.Struct:
			binary.Write(bytes, binary.LittleEndian, StructToBytes(field.Interface()))
		case reflect.Array, reflect.Slice:
			binary.Write(bytes, binary.LittleEndian, field.Interface())
		case reflect.Uint8:
			binary.Write(bytes, binary.LittleEndian, uint8(field.Uint()))
		case reflect.Uint16:
			binary.Write(bytes, binary.LittleEndian, uint16(field.Uint()))
        // You get the idea
        }
    }
    return bytes.Bytes()
}

And would do this in a handler:

type Header struct {
    length uint16
    size uint16
    flags uint32
}
newHeader := new(Header)
// Initialization, etc
client.Conn.Write(StructToBytes(newHeader)) // ex. [C8 00 03 00 00 00 01 00]  

As a Go newbie, feedback on how I might implement this more efficiently is more than welcome. So far it's worked well, but now I'm faced with the challenge of how to do the opposite: go from []byte->Struct (for example, [C8 00 03 00 00 01 00 00] to a Header { length = C8, size = 03, flags = 0100 }

Do I need to just implement the reverse of this or is there a better method of going from an array of bytes to an arbitrary struct (or vice versa, as opposed to my function)? Please let me know if any additional clarity would be helpful.

答案1

得分: 7

使用encoding/binary包是Go语言中处理二进制数据的常见方式。你可以按照以下代码示例来使用它:

package main

import (
	"bytes"
	"encoding/binary"
	"fmt"
	"log"
)

type Header struct {
	Length uint16
	Size   uint16
	Flags  uint32
}

func main() {
	header := &Header{Length: 0xC8, Size: 3, Flags: 0x100}
	fmt.Printf("in = %#v\n", header)
	buf := new(bytes.Buffer)

	err := binary.Write(buf, binary.LittleEndian, header)
	if err != nil {
		log.Fatalf("binary.Write failed: %v", err)
	}
	b := buf.Bytes()
	fmt.Printf("wire = % x\n", b)

	var header2 Header
	buf2 := bytes.NewReader(b)
	err = binary.Read(buf2, binary.LittleEndian, &header2)
	if err != nil {
		log.Fatalf("binary.Read failed: %v", err)
	}
	fmt.Printf("out = %#v\n", header2)
}

这段代码会输出以下结果:

in = &main.Header{Length:0xc8, Size:0x3, Flags:0x100}
wire = c8 00 03 00 00 01 00 00
out = main.Header{Length:0xc8, Size:0x3, Flags:0x100}

你可以在这里查看encoding/binary包的更多信息。

英文:

The go way would be to use encoding/binary which internally does pretty much what you've written above.

(playground)

package main

import (
	&quot;bytes&quot;
	&quot;encoding/binary&quot;
	&quot;fmt&quot;
	&quot;log&quot;
)

type Header struct {
	Length uint16
	Size   uint16
	Flags  uint32
}

func main() {
	header := &amp;Header{Length: 0xC8, Size: 3, Flags: 0x100}
	fmt.Printf(&quot;in = %#v\n&quot;, header)
	buf := new(bytes.Buffer)

	err := binary.Write(buf, binary.LittleEndian, header)
	if err != nil {
		log.Fatalf(&quot;binary.Write failed: %v&quot;, err)
	}
	b := buf.Bytes()
	fmt.Printf(&quot;wire = % x\n&quot;, b)

	var header2 Header
	buf2 := bytes.NewReader(b)
	err = binary.Read(buf2, binary.LittleEndian, &amp;header2)
	if err != nil {
		log.Fatalf(&quot;binary.Read failed: %v&quot;, err)
	}
	fmt.Printf(&quot;out = %#v\n&quot;, header2)
}

Which prints

in = &amp;main.Header{Length:0xc8, Size:0x3, Flags:0x100}
wire = c8 00 03 00 00 01 00 00
out = main.Header{Length:0xc8, Size:0x3, Flags:0x100}

huangapple
  • 本文由 发表于 2015年1月7日 15:27:51
  • 转载请务必保留本文链接:https://go.coder-hub.com/27814408.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定