英文:
Streaming BSON->JSON HTTP reply in Go?
问题
目前我正在一个HTTP处理程序中运行以下代码：
err := mongoCollection.Find(bson.M{"name": vars["name"]}).One(&result)
data, err := json.Marshal(result)
w.Write(data)
在完整的BSON数据到达之前，我该如何开始提供结果？
编辑：答案需要超出mgo扩展，并涉及到bson。据我所见，mgo只会提供完整的文档，如果我没有弄错的话。正如我的代码示例所清楚显示的，我有一个（可能很大）的文档。
英文:
Currently I am running this in a http handler:
err := mongoCollection.Find(bson.M{"name": vars["name"]}).One(&result)
data, err := json.Marshal(result)
w.Write(data)
How can I begin serving the result before the full BSON data is in?
Edit: the answer needs to go beyond the mgo extension and go into bson. mgo as far as I can see will only serve full documents if I not mistaken. I have one - possibly large - document as my code example clearly shows.
答案1
得分: 3
为了实现这个目标，你需要以下几点：
- 访问用于处理传入 bson 流的
Reader。 - 一个用于处理通用文档部分的数据类型。
- 一个从
Reader中读取并生成文档部分的 bson 流解码器。 - 一个消费文档部分并将其写入
Writer的 json 流编码器。
mgo 不提供第一点。encoding/json 不提供第二点和第四点。mgo/bson 不提供第三点。在 Go 语言中，虽然有其他语言中的流式 json 解析器（参见 https://stackoverflow.com/questions/444380/is-there-a-streaming-api-for-json 的答案），但是通过一些搜索并没有找到相关的帮助。
你的需求是合理的，但是目前还没有相应的支持。幸运的是，json 和 bson 都相对简单，并且你使用的所有组件都是开源的，所以理论上你可以编写所需的工具。
英文:
In order for this to be possible, you would need these things:
- Access to a
Readerfor the incoming bson stream - A datatype for generic document parts
- A streaming decoder for bson which reads from the
Readerand produces document parts - A streaming encoder for json that consumes document parts and writes to a
Writer
mgo does not provide number 1. encoding/json does not provide number 2 or 4. mgo/bson does not provide number 3. A bit of googling doesn't turn up any help for any of those points in Go, though there are streaming json parsers in other languages (see answers for https://stackoverflow.com/questions/444380/is-there-a-streaming-api-for-json).
Your desire to do this is reasonable, but the support just doesn't exist yet. Fortunately, json and bson are simple enough and all the components you're using are open source, so in theory you could write the tools you need.
答案2
得分: 1
我认为除非对mgo进行修改，否则无法避免完全解组装BSON（因此在BSON完全传递之前不提供结果）。mgo的API只处理完整的、未解组装的文档，无法访问任何可能逐个元素进行bson解码然后json编码的BSON编码的[]byte或Reader。
英文:
I don't think there's anything you can do to avoid unmarshalling the whole BSON (and therefore not serving the result until the BSON has been fully delivered by mgo), short of hacking on mgo. Its API only deals in whole, unmarshalled documents, with no access to any BSON-encoded []byte or Reader that you could potentially elementwise bsondecode-then-jsonencode as data comes in.
答案3
得分: 0
请看chanson，你可以轻松地构建和流式传输JSON数据。
这里有一个例子，从通道中读取数据并将元素添加到列表中。
你可能可以做类似的事情。
英文:
Take a look at chanson, you can easily construct and stream json.
There's an example that reads data from channels for adding elements to a list.
You could probably do something similar
答案4
得分: -1
请看json.Encoder。它将JSON对象写入输出流。json.Marshal一次性生成一个[]byte，不提供流式处理。
在MongoDB方面，请看mgo.Iter。如果你的结果中有大量的文档，你可以分批序列化它们，使你的应用程序更加内存高效。
使用json.Encode的示例：
data := map[string]int{"apple": 5, "lettuce": 7}
enc := json.NewEncoder(w)
enc.Encode(data)
英文:
Take a look at json.Encoder. It writes JSON objects to an output stream. json.Marshal produces a []byte in one shot and does not provide a stream.
On the MongoDB side take a look at mgo.Iter. In case you have a large number of documents in your result you can serialize them in batches and make your application more memory efficient.
Sample of using json.Encode:
data := map[string]int{"apple": 5, "lettuce": 7}
enc := json.NewEncoder(w)
enc.Encode(data)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论