使用UseNumber解码GET请求

huangapple go评论84阅读模式
英文:

Decode GET request with UseNumber

问题

我有以下代码,用于向Elasticsearch发出请求:

// 准备请求
tr := &http.Transport{
    TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
req, err := http.NewRequest("GET", url, nil)
req.Header.Add("Authorization", ES_AUTH)

// 发送请求
resp, err := client.Do(req)
if err != nil {
    log.Fatal(err)
}
defer resp.Body.Close()

d := json.NewDecoder(resp.Body)
d.UseNumber()

var x interface{}
if err = d.Decode(&x); err != nil {
    log.Fatal(err)
}

fmt.Printf("decoded to %#v\n", x)

输出结果如下:

decoded to map[string]interface{}{
    "took": "2",
    "timed_out": false,
    "_shards": map[string]interface{}{
        "total": "1",
        "successful": "1",
        "failed": "0",
    },
    "hits": map[string]interface{}{
        "total": "15",
        "max_score": "1.0",
        "hits": []interface{}{
            map[string]interface{}{
                "_index": "reports2",
                "_type": "insiders",
                "_id": "cbaf7513-0046-4929-a740-afe13d6616b4",
                "_score": "1.0",
                "_routing": "524364278561685819",
                "_source": map[string]interface{}{
                    "insider_mean": "420.7546961325966",
                    "friend_total": "3.7859686E7",
                    "sum_of_followers": "3.531951269E10",
                    "follower_total": "50000.0",
                },
            },
        },
    },
}

我尝试使用d.UseNumber()来使输出中的数字尽可能以整数形式表示,而不是使用浮点数。我期望"friend_total""sum_of_followers""follower_total"的值应该是整数形式。

如果我使用Postman发送相同的请求,你可以看到这些值应该是整数:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 15,
    "max_score": 1,
    "hits": [
      {
        "_index": "reports2",
        "_type": "insiders",
        "_id": "cbaf7513-0046-4929-a740-afe13d6616b4",
        "_score": 1,
        "_routing": "524364278561685819",
        "_source": {
          "insider_mean": 420.7546961325966,
          "sum_of_followers": 35319512690,
          "follower_total": 50000,
          "friend_total": 37859686
        }
      }
    ]
  }
}

我无法在这个Go Playground中复现这个错误,所以我认为错误可能是由于请求本身而不是解码造成的。


编辑:

我发现使用curl请求返回的有效载荷,所以现在我确定解码不是问题所在。

curl localhost:9200/524364278561685819/insiders/_search\?size\=1
{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":15,"max_score":1.0,"hits":[{"_index":"reports2","_type":"insiders","_id":"cbaf7513-0046-4929-a740-afe13d6616b4","_score":1.0,"_routing":"524364278561685819","_source":{"insider_mean":420.7546961325966,"sum_of_followers":3.531951269E10,"follower_total":50000.0,"friend_total":3.7859686E7}}]}}
英文:

I have the following code that makes a request to Elasticsearch:

// Prepare request
tr := &http.Transport{
    TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
req, err := http.NewRequest("GET", url, nil)
req.Header.Add("Authorization", ES_AUTH)

// Make request
resp, err := client.Do(req)
if err != nil {
    log.Fatal(err)
}
defer resp.Body.Close()

d := json.NewDecoder(resp.Body)
d.UseNumber()

var x interface{}
if err = d.Decode(&x); err != nil {
    log.Fatal(err)
}

fmt.Printf("decoded to %#v\n", x)

The output is this:

decoded to map[string]interface {}{"took":"2", "timed_out":false, "_shards":map[string]interface {}{"total":"1", "successful":"1", "failed":"0"}, "hits":map[string]interface {}{"total":"15", "max_score":"1.0", "hits":[]interface {}{map[string]interface {}{"_index":"reports2", "_type":"insiders", "_id":"cbaf7513-0046-4929-a740-afe13d6616b4", "_score":"1.0", "_routing":"524364278561685819", "_source":map[string]interface {}{"insider_mean":"420.7546961325966",  "friend_total":"3.7859686E7", "sum_of_followers":"3.531951269E10", "follower_total":"50000.0"}}}}}

I am trying to use d.UseNumber() to get the output to have integer numbers where possible, instead of using floats for everything. I was expecting "friend_total", "sum_of_followers", and "follower_total" to have integer-style values.

If I make the same request with postman you can see that these values should be whole numbers:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 15,
    "max_score": 1,
    "hits": [
      {
        "_index": "reports2",
        "_type": "insiders",
        "_id": "cbaf7513-0046-4929-a740-afe13d6616b4",
        "_score": 1,
        "_routing": "524364278561685819",
        "_source": {
          "insider_mean": 420.7546961325966,
          "sum_of_followers": 35319512690,
          "follower_total": 50000,
          "friend_total": 37859686
        }
      }
    ]
  }
}

I was unable to replicate the error in this Go Playground so I think that the error may be because of the request itself, and not the decoding.


Edit:

I have discovered that doing a curl request returns the payload, so now I am certain that the decoding is not the problem.

curl localhost:9200/524364278561685819/insiders/_search\?size\=1                   
{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":15,"max_score":1.0,"hits":[{"_index":"reports2","_type":"insiders","_id":"cbaf7513-0046-4929-a740-afe13d6616b4","_score":1.0,"_routing":"524364278561685819","_source":{"insider_mean":420.7546961325966,"sum_of_followers":3.531951269E10,"follower_total":50000.0,"friend_total":3.7859686E7}}]}}

答案1

得分: 0

根据curl输出显示,Elasticsearch实际上以浮点数格式提供结果。似乎Postman和jq会为您进行转换,所以我现在的解决方案是只需调用jq的系统调用:

// 发送请求
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()

// 使用jq将浮点数转换为长整型/整型
cmd := exec.Command("jq", ".")
cmd.Stdin = resp.Body
var out bytes.Buffer
cmd.Stdout = &out
err = cmd.Run()
if err != nil {
log.Fatal(err)
}

// 解码jq的输出
d := json.NewDecoder(bytes.NewReader(out.Bytes()))
d.UseNumber()

var x interface{}
if err = d.Decode(&x); err != nil {
log.Fatal(err)
}

fmt.Printf("解码结果为 %#v\n", x)

关于Postman / curl的澄清:

(它们)只是使用不同的格式表示浮点数。JSON中没有整数的概念,所以35319512690只是没有小数点的浮点数,等同于35319512690.0和3.531951269E10 - JimB

英文:

As the curl output indicated, Elasticsearch was actually giving the results in floating point format. It seems the Postman and jq do the conversion for you, so my solution for now is to just make a system call to jq:

// Make request
resp, err := client.Do(req)
if err != nil {
    log.Fatal(err)
}
defer resp.Body.Close()

// Use jq to convert the floating point stuff to Longs / Ints
cmd := exec.Command("jq", ".")
cmd.Stdin = resp.Body
var out bytes.Buffer
cmd.Stdout = &out
err = cmd.Run()
if err != nil {
	log.Fatal(err)
}

// Decode the output from jq
d := json.NewDecoder(bytes.NewReader(out.Bytes())
d.UseNumber()

var x interface{}
if err = d.Decode(&x); err != nil {
    log.Fatal(err)
}

fmt.Printf("decoded to %#v\n", x)

To clarify about Postman / curl:

> (They) are simply using a different format for the float. there is no
> such thing as an integer in JSON, so 35319512690 is just a float
> without the decimal point which equivalent to both 35319512690.0 and
> 3.531951269E10 - JimB

huangapple
  • 本文由 发表于 2016年12月14日 01:07:59
  • 转载请务必保留本文链接:https://go.coder-hub.com/41126762.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定