What's a good way to ensure delivery of an entire message when using raw tcp?

huangapple go评论102阅读模式
英文:

What's a good way to ensure delivery of an entire message when using raw tcp?

问题

假设我有一个应用程序,在后端我想使用原始的TCP,以便在不同的服务之间进行双向通信。在这个应用程序中,我想发送一个由JSON对象组成的有效载荷,当JSON数据被发送时,每隔几条消息,它会被截断,剩余的部分会被合并到下一个响应中。我不想使用像WebSocket这样的东西,因为它需要升级从HTTP。有没有一种好的(最好的)方法来确保JSON对象从一个节点发送并在另一个节点上作为完整的JSON对象被读取?

我知道发送和接收固定大小的缓冲区以及发送心跳消息是一种经验法则,但我能看到一个例子吗?最好是用JavaScript(Node.js的net标准库)或Golang(它的net标准库)的例子,因为这些是我最熟悉的语言,但最终我并不在乎用什么语言实现。

(我知道有一些类似的问题关于如何确保使用TCP传递消息,但我没有找到一个提供示例的问题)

我知道TCP是一个流。我只是想知道如何确保在将特定的JSON对象写入此流时,如何确保在另一端获得相同的JSON对象,即“从节点A发送JSON对象X,确保节点B接收到相同的对象X”。

英文:

Say I have an application that on the backend I want to use raw tcp so that I can have bi-directional communication between different services. In this application I want to send over a payload consisting of a json object, when the json data is sent, every few messages, it gets cut off and the remainder is then clumped to the next response. I don't want to use something like say websockets due to the time used to upgrade from http. What is a good (and hopefully best) way to ensure that that json object will go from one node and be read from another node as the whole json object?

I know sending and receiving buffers of a set size and a message for a heartbeet are the rule of thumb, but can I see an example? Preferably in Javascript (node's net stdlib) or Golang (it's net stdlib) because those are my most proficient languages, though I don't really care what language it is done in ultimately.

(I know there's a few questions out there asking similar things regarding ensuring delivery of message with tcp, but none asked for an example that I found)

I know tcp is a stream. I'm just asking of a way to ensure that when writing a specific json object to this stream, how do I ensure I get the same json object on the other end, as in "send json object X from node a, ok node b received that same object X"

答案1

得分: 4

你不需要心跳或固定大小的消息来确认传递。如果你需要确保传递,你需要应用层的确认。如果你需要确保传递的是正确的消息,你需要包含一个唯一的消息ID来进行确认。如果你需要确保消息没有被篡改,你需要包含一个校验和或MAC。

在这里,听起来你在处理消息的帧格式方面遇到了问题。虽然有很多方法可以对消息进行帧格式化(简单的长度前缀、类型-长度-值、HTTP/1.1等),一个简单的解决方案是使用内置的json.Encoderjson.Decoder

以下是一个每秒发送一个“PING”消息的示例客户端代码:

type Message struct {
    Payload string
}

func sendMessages(c net.Conn) {
    message := Message{}

    encoder := json.NewEncoder(c)

    for i := 0; ; i++ {
        message.Payload = fmt.Sprintf("PING %d", i)
        err := encoder.Encode(message)
        if err != nil {
            log.Fatal(err)
        }
        time.Sleep(time.Second)
    }
}

以下是一个示例服务器代码:

type Message struct {
    Payload string
}

func receiveMessages(c net.Conn) {
    m := Message{}

    decoder := json.NewDecoder(c)
    for {
        err := decoder.Decode(&m)
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("Received: %#v\n", m)
    }
}
英文:

You don't need a heartbeat, or fixed size messages for delivery confirmation. If you need to ensure delivery, you need an application level acknowledgement. If you need to ensure delivery of the correct message, you'll need to include a unique message ID to acknowledge. If you need to ensure that the message is unaltered, you'll need to include a checksum or MAC.

Here it sounds like you're having trouble with message framing. While there are many ways to frame your messages (simple length-prefix, type-length-value, HTTP/1.1, etc.), a simple solution is to use the built-in json.Encoder and json.Decoder.

Example client, which sends a "PING" message every second:

type Message struct {
	Payload string
}

func sendMessages(c net.Conn) {
	message := Message{}

	encoder := json.NewEncoder(c)

	for i := 0; ; i++ {
		message.Payload = fmt.Sprintf("PING %d", i)
		err := encoder.Encode(message)
		if err != nil {
			log.Fatal(err)
		}
		time.Sleep(time.Second)
	}
}

Example Server:

type Message struct {
	Payload string
}

func receiveMessages(c net.Conn) {
	m := Message{}

	decoder := json.NewDecoder(c)
	for {
		err := decoder.Decode(&m)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("Received: %#v\n", m)
	}
}

huangapple
  • 本文由 发表于 2015年9月4日 21:58:24
  • 转载请务必保留本文链接:https://go.coder-hub.com/32399966.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定