英文:
Best method to stream results from web service
问题
我目前正在针对一个返回批处理结果的 XML 服务进行编写。我目前有以下代码:
type QueryEnvelope struct {
XMLName xml.Name `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
Body *QueryBody `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
}
type QueryBody struct {
QueryResult *QueryResult `xml:"queryResponse>result"`
}
type QueryResult struct {
QueryLocator QueryLocator `xml:"queryLocator"`
Done bool `xml:"done"`
Size int `xml:"size"`
Records interface{} `xml:"records"`
}
type QueryLocator string
func (r *Resource) Query(sql string, r interface{}) error {
// 构建 XML 请求
// 通过包含网络客户端的 Resource 发送 XML 请求
// 将结果解码为 r(QueryResult 中的 records 类型)
}
func (r *Resource) QueryMore(q QueryLocator, r interface{}) error {
// 与上述相同,只是接受 queryLocator 并调用不同的端点以继续检索结果,当 QueryResult.Done == true 时,表示完成获取结果
}
显然,这需要进行重构,因为客户端需要查看 Done 是否为 true,以便可以继续获取。我考虑添加以下内容,并将 Query 和 QueryMore 移动为 Querier 的方法:
type Querier struct {
r *Resource
Done bool
QueryLocator QueryLocator
}
func New(r *Resource) *Querier {}
客户端代码将如下所示:
err := q.Query("sql 语句", r)
if err != nil {
// 处理错误
}
// 使用 r 进行操作
for q.Done == false {
q.QueryMore(r)
// 使用 r 进行操作
}
我想知道在这里最符合惯用方法的方式是什么,以便最好地“流式”处理结果。
英文:
I'm currently writing against an xml service in which batched results are returned. I currently have the following:
type QueryEnvelope struct {
XMLName xml.Name `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
Body *QueryBody `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
}
type QueryBody struct {
QueryResult *QueryResult `xml:"queryResponse>result"`
}
type QueryResult struct {
QueryLocator QueryLocator `xml:"queryLocator"`
Done bool `xml:"done"`
Size int `xml:"size"`
Records interface{} `xml:"records"`
}
type QueryLocator string
func (r *Resource) Query(sql string, r interface{}) error {
// builds xml request
// sends xml request through Resource which contains the net client
// decodes the result into r (type for records in QueryResult)
}
func (r *Resource) QueryMore(q QueryLocator, r interface{}) error {
// same as above except takes queryLocator and makes call to different endpoint in order to continue retrieving results, when QueryResult.Done == true, done fetching results
}
Obviously this needs refactored since the Client needs to see whether Done == true so they can continue to fetch. I was thinking of adding the following the following, and moving Query and QueryMore to be a method of Querier:
type Querier struct {
r *Resource
Done bool
QueryLocator QueryLocator
}
func New(r *Resource) *Querier {}
Client side would then behave as such:
err := q.Query("sql statement", r)
if err != nil {
// handle
}
// do stuff with r
for q.Done == false {
q.QueryMore(r)
// do stuff with r
}
I'm wondering what the idiomatic approach would be here in order to best "stream" the results.
答案1
得分: 1
一种选择是使用标准库sql
包中使用的模式来迭代行。这样,您可以将初始查询与后续调用统一起来。
这里有一个Next()
方法,它将查询结果填充到内部结构中,并在有待处理的结果时返回true
。在您的情况下,您仍然需要一个初始构造函数,比如Query
,但它只是设置数据结构。调用Next()
完成实际工作,调用Scan(r)
(或者您想要调用的读取结果的方法)只是呈现结果。迭代完成后,您可以使用Err()
方法检查任何操作错误。
稍微修改一下sql包的示例:
// 设置查询,但延迟任何操作直到开始迭代。
query, err := NewQuery(queryParameters)
// 当然要检查错误
defer query.Close()
for query.Next() {
err = query.Scan(&r)
...
}
err = query.Err() // 获取迭代过程中遇到的任何错误
您还可以查看其他驱动程序,比如mgo
,以了解此模式的变化。
英文:
One option is to use the pattern the stdlib sql
package uses to iterate over rows. This way you can unify the initial query with subsequent calls.
There's a Next()
method, which populates the internal structure with the query results, and returns true
if there's a result pending. In your case, you would still need an initial constructor, like Query
, but that would just setup the data-structure. Calling Next()
does the real work, and calling Scan(r)
(or whatever you'd like to call reading the result) just presents to result. Once iteration is done, you have an Err()
method to check for any operational errors.
Altering the example from the sql package a bit:
// setup the query, but delay any action until we start iterating.
query, err := NewQuery(queryParameters)
// check err of course
defer query.Close()
for query.Next() {
err = query.Scan(&r)
...
}
err = query.Err() // get any error encountered during iteration
You can also check out other drivers like mgo
for variations on this pattern.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论