英文:
Golang: Parsing benchmarking between message pack and JSON
问题
我们正在开发一个基于TCP的服务器,可以通过TCP接收简单的文本命令(类似于Redis)。
我们正在考虑使用原始文本命令、JSON或者消息打包(http://msgpack.org/)。
一个命令的示例可以是:
文本命令:LOCK some_random_key 1000
JSON命令:{"command":"LOCK","key":"some_random_key","timeout":1000}
消息打包:\x83\xA7command\xA4LOCK\xA3key\xAFsome_random_key\xA7timeout\xCD\x03\xE8
问题:
编辑:我已经解决了自己的问题,即解析JSON和MsgPack的速度比较。请参见我的答案中的结果。
英文:
We are working on a TCP server which takes simple textbased commands over TCP (similar to redis)
We are tossing up between using raw text command, JSON or message pack (http://msgpack.org/)
An example of a command could be:
text command: LOCK some_random_key 1000
JSON command: {"command":"LOCK","key":"some_random_key","timeout":1000}
messagePack: \x83\xA7command\xA4LOCK\xA3key\xAFsome_random_key\xA7timeout\xCD\x03\xE8
Question:
EDIT: I have figured out my own question which is the speed comparison between parsing JSON and MsgPack. Please see results in my answer
答案1
得分: 3
解析速度比较:
BenchmarkJSON 100000 17888 ns/op
BenchmarkMsgPack 200000 10432 ns/op
我的基准测试代码:
package benchmark
import (
"encoding/json"
"github.com/vmihailenco/msgpack"
"testing"
)
var in = map[string]interface{}{"c": "LOCK", "k": "31uEbMgunupShBVTewXjtqbBv5MndwfXhb", "T/O": 1000, "max": 200}
func BenchmarkJSON(b *testing.B) {
for i := 0; i < b.N; i++ {
jsonB := EncodeJSON(in)
DecodeJSON(jsonB)
}
}
func BenchmarkMsgPack(b *testing.B) {
for i := 0; i < b.N; i++ {
b := EncodeMsgPack(in)
DecodeMsgPack(b)
}
}
func EncodeMsgPack(message map[string]interface{}) []byte {
b, _ := msgpack.Marshal(message)
return b
}
func DecodeMsgPack(b []byte) (out map[string]interface{}) {
_ = msgpack.Unmarshal(b, &out)
return
}
func EncodeJSON(message map[string]interface{}) []byte {
b, _ := json.Marshal(message)
return b
}
func DecodeJSON(b []byte) (out map[string]interface{}) {
_ = json.Unmarshal(b, &out)
return
}
英文:
Parsing Speed Comparison:
BenchmarkJSON 100000 17888 ns/op
BenchmarkMsgPack 200000 10432 ns/op
My benchmarking code:
package benchmark
import (
"encoding/json"
"github.com/vmihailenco/msgpack"
"testing"
)
var in = map[string]interface{}{"c": "LOCK", "k": "31uEbMgunupShBVTewXjtqbBv5MndwfXhb", "T/O": 1000, "max": 200}
func BenchmarkJSON(b *testing.B) {
for i := 0; i < b.N; i++ {
jsonB := EncodeJSON(in)
DecodeJSON(jsonB)
}
}
func BenchmarkMsgPack(b *testing.B) {
for i := 0; i < b.N; i++ {
b := EncodeMsgPack(in)
DecodeMsgPack(b)
}
}
func EncodeMsgPack(message map[string]interface{}) []byte {
b, _ := msgpack.Marshal(message)
return b
}
func DecodeMsgPack(b []byte) (out map[string]interface{}) {
_ = msgpack.Unmarshal(b, &out)
return
}
func EncodeJSON(message map[string]interface{}) []byte {
b, _ := json.Marshal(message)
return b
}
func DecodeJSON(b []byte) (out map[string]interface{}) {
_ = json.Unmarshal(b, &out)
return
}
答案2
得分: 1
我建议对机器之间将要交流的数据进行一些基准测试。
我建议尝试使用Protocol Buffers(编码)+ Snappy(压缩)。
英文:
I would suggest to do some benchmarks on the kind of data that the machines will be talking to each other.
I would suggest to try Protocol Buffers (Encoding) + Snappy (compression)
答案3
得分: 1
msgpack只承诺比json更短,而不是解析速度更快。在这两种情况下,您的测试字符串如此短且简单,以至于您的基准测试可能只是在测试特定实现的成熟度,而不是底层算法。
如果您的所有消息确实都这么短,解析速度可能是您最不用担心的问题。我建议您设计服务器,使解析部分易于替换,并实际对代码进行性能分析。
Donald Knuth在优化方面提出了以下观点:
"我们应该忘记小的效率问题,大约有97%的时间:过早的优化是万恶之源"
最后,如果您真的想知道发生了什么,您需要对代码进行性能分析。请参阅以下链接以了解如何使用Go对代码进行性能分析的示例:
http://blog.golang.org/profiling-go-programs
英文:
msgpack only promises to be shorter than json, not faster to parse. In both cases, your test string is so short and simple that your benchmarking may simply be testing the maturity of the particular implementation, rather than the underlying algorithms.
If all your messages really are this short, parsing speed may be the least of your problems. I'd suggest designing your server such that the parsing bits are easily replaceable and actually profiling the code in action.
Donald Knuth made the following statement on optimization:
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil"
Lastly if you really want to know what is going on, you need to profile the code. See
http://blog.golang.org/profiling-go-programs
for an example of how to profile code with go.
答案4
得分: 0
另外,你的测试用例是颠倒的。
BenchmarkJSON实际上调用了MsgPack,
而BenchmarkMsgPack调用了Json。
这可能与此有关吗?
英文:
Also, your test cases are reversed
BenchmarkJSON actually calls MsgPack
and
BenchmarkMsgPack calls Json
could that have something to do with it?
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论