英文:
Best way to store golang map[string]interface into data store
问题
场景：我有一个从MongoDB读取的任意大小的JSON数据，范围从300 KB到6.5 MB。由于数据是任意/动态的，我无法在golang中定义结构类型，所以我使用map[string]interface{}类型。并且使用encoding/json的Unmarshal方法解析JSON数据，类似于Generic JSON with interface{}中所述的方式。
问题：但问题是将字符串JSON解析为map[string]interface{}需要更长的时间（大约30毫秒到180毫秒）。与使用json_encode/decode、igbinary/msgpack解析JSON的php相比。
问题：有没有办法预处理并存储在缓存中？
我的意思是将字符串解析为map[string]interface{}，并将其序列化并存储到某个缓存中，然后在检索时不需要花费太多时间进行反序列化并继续执行。
注意：我是golang的新手，非常感谢任何建议。
更新：已经尝试过使用Gob、binary内置包和Golang的Msgpack实现包进行序列化。没有运气，反序列化的时间没有改善。
英文:
Scenario: I am having a arbitrary JSON size ranging from 300 KB - 6.5 MB read from MongoDB. Since it is very much arbitrary/dynamic data I cannot have struct type defined in golang, So I am using map[sting]interface{} type. And string of JSON data is parsed by using encoding/json's Unmarshal method. Some what similar to what is mentioned in Generic JSON with interface{}.
Issue: But the problem is its taking more time (around 30ms to 180ms) to parse the string json into map[string]interface{}. (Comparing to php parsing json using json_encode/decode / igbinary/ msgpack)
Question: Is there any way to pre-process it and store in cache?
I mean parse string into map[string]interface{} and serialize it and store to some cache, then when we retrieve it should not take much time to unserialization and proceed with execution.
Note: I am Newbie for the golang any suggestion are highly appriciated. Thanks
Updates: Serialization using Gob, binary built-in package & Msgpack implementation for Golang package are already tried. No luck, No improvement in the time to unserialization.
答案1
得分: 2
JSON的标准库包在性能上是出了名的慢。这是有一个很好的原因的：它使用RTTI提供了一个非常灵活但简单的接口。因此，与PHP相比，解组的速度较慢...
幸运的是，有一种替代方法，就是在你想要使用的类型上实现json.Unmarshaller接口。如果实现了这个接口，包将使用它而不是标准方法，从而可以获得巨大的性能提升。
为了帮助你，出现了一小组工具，其中包括：
（从记忆中列出了主要的工具，可能还有其他的）
这些工具将为你请求的类型生成定制的json.Unmarshaller接口实现。而且通过使用go:generate，你甚至可以将它们无缝地集成到构建步骤中。
英文:
The standard library package for JSON is notoriously slow. There is a good reason for that: it use RTTI to provide a really flexible interface that is really simple. Hence the unmarshalling being slower than PHP's…
Fortunately, there is an alternative way, which is to implement the json.Unmarshaller interface on the types you want to use. If this interface is implemented, the package will use it instead of its standard method so you can see huge performance boosts.
And to help you, there is a small group of tools that appeared, among which:
(listing here the main players from memory, there must be others)
These tools will generate tailored implementations of the json.Unmarshaller interface for the types you requested. And with go:generate, you can even integrate them seamlessly to your build step.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论