从网络服务流式传输结果的最佳方法

huangapple go评论71阅读模式
英文:

Best method to stream results from web service

问题

我目前正在针对一个返回批处理结果的 XML 服务进行编写。我目前有以下代码:

type QueryEnvelope struct {
    XMLName xml.Name   `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
    Body    *QueryBody `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
}

type QueryBody struct {
    QueryResult *QueryResult `xml:"queryResponse>result"`
}

type QueryResult struct {
    QueryLocator QueryLocator `xml:"queryLocator"`
    Done         bool         `xml:"done"`
    Size         int          `xml:"size"`
    Records      interface{}  `xml:"records"`
}

type QueryLocator string

func (r *Resource) Query(sql string, r interface{}) error {
    // 构建 XML 请求
    // 通过包含网络客户端的 Resource 发送 XML 请求
    // 将结果解码为 r(QueryResult 中的 records 类型)
}

func (r *Resource) QueryMore(q QueryLocator, r interface{}) error {
    // 与上述相同,只是接受 queryLocator 并调用不同的端点以继续检索结果,当 QueryResult.Done == true 时,表示完成获取结果
}

显然,这需要进行重构,因为客户端需要查看 Done 是否为 true,以便可以继续获取。我考虑添加以下内容,并将 Query 和 QueryMore 移动为 Querier 的方法:

type Querier struct {
    r           *Resource
    Done        bool
    QueryLocator QueryLocator
}

func New(r *Resource) *Querier {}

客户端代码将如下所示

err := q.Query("sql 语句", r)
if err != nil {
    // 处理错误
}
// 使用 r 进行操作
for q.Done == false {
    q.QueryMore(r)
    // 使用 r 进行操作
}

我想知道在这里最符合惯用方法的方式是什么,以便最好地“流式”处理结果。

英文:

I'm currently writing against an xml service in which batched results are returned. I currently have the following:

type QueryEnvelope struct {
	XMLName xml.Name   `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
	Body    *QueryBody `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
}

type QueryBody struct {
	QueryResult *QueryResult `xml:"queryResponse>result"`
}

type QueryResult struct {
	QueryLocator QueryLocator `xml:"queryLocator"`
	Done         bool         `xml:"done"`
	Size         int          `xml:"size"`
	Records      interface{}  `xml:"records"`
}

type QueryLocator string

func (r *Resource) Query(sql string, r interface{}) error {
     // builds xml request
     // sends xml request through Resource which contains the net client
     // decodes the result into r (type for records in QueryResult)
}

func (r *Resource) QueryMore(q QueryLocator, r interface{}) error {
     // same as above except takes queryLocator and makes call to different endpoint in order to continue retrieving results, when QueryResult.Done == true, done fetching results
}

Obviously this needs refactored since the Client needs to see whether Done == true so they can continue to fetch. I was thinking of adding the following the following, and moving Query and QueryMore to be a method of Querier:

type Querier struct {
r *Resource
Done bool
QueryLocator QueryLocator
}

func New(r *Resource) *Querier {}

Client side would then behave as such:

err := q.Query("sql statement", r)
if err != nil {
     // handle
}
// do stuff with r
for q.Done == false {
     q.QueryMore(r)
     // do stuff with r
}

I'm wondering what the idiomatic approach would be here in order to best "stream" the results.

答案1

得分: 1

一种选择是使用标准库sql包中使用的模式来迭代行。这样,您可以将初始查询与后续调用统一起来。

这里有一个Next()方法,它将查询结果填充到内部结构中,并在有待处理的结果时返回true。在您的情况下,您仍然需要一个初始构造函数,比如Query,但它只是设置数据结构。调用Next()完成实际工作,调用Scan(r)(或者您想要调用的读取结果的方法)只是呈现结果。迭代完成后,您可以使用Err()方法检查任何操作错误。

稍微修改一下sql包的示例:

// 设置查询,但延迟任何操作直到开始迭代。
query, err := NewQuery(queryParameters)
// 当然要检查错误
defer query.Close()

for query.Next() {
    err = query.Scan(&r)
    ...
}
err = query.Err() // 获取迭代过程中遇到的任何错误

您还可以查看其他驱动程序,比如mgo,以了解此模式的变化。

英文:

One option is to use the pattern the stdlib sql package uses to iterate over rows. This way you can unify the initial query with subsequent calls.

There's a Next() method, which populates the internal structure with the query results, and returns true if there's a result pending. In your case, you would still need an initial constructor, like Query, but that would just setup the data-structure. Calling Next() does the real work, and calling Scan(r) (or whatever you'd like to call reading the result) just presents to result. Once iteration is done, you have an Err() method to check for any operational errors.

Altering the example from the sql package a bit:

// setup the query, but delay any action until we start iterating.
query, err := NewQuery(queryParameters)
// check err of course
defer query.Close()

for query.Next() {
    err = query.Scan(&r)
    ...
}
err = query.Err() // get any error encountered during iteration 

You can also check out other drivers like mgo for variations on this pattern.

huangapple
  • 本文由 发表于 2014年8月25日 23:01:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/25488826.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定