Proper json unmarshaling in Go with the empty interface

huangapple go评论97阅读模式
英文:

Proper json unmarshaling in Go with the empty interface

问题

我目前正在学习golang,并且(可能和之前的许多人一样)我正在努力理解空接口。

作为练习,我正在阅读由Postman生成的一个大型JSON文件,并尝试访问其中的一个字段(而不是所有可用的字段)。

以下是一个简单的JSON表示,去除了我不想读取的不必要字段(但它们仍然存在):

{
	"results": [
		{
			"times": [
				1,
				2,
				3,
				4
			]
		}
	]
}

由于JSON对象很大,我选择不使用自定义结构进行解组,而是决定使用空接口interface{}

经过一段时间,我设法编写了一些可行的代码,但我相当确定这不是正确的方法。

byteValue, _ := ioutil.ReadAll(jsonFile)

var result map[string]interface{}
err = json.Unmarshal(byteValue, &result)
if err != nil {
	log.Fatalln(err)
}

// 特别丑陋
r := result["results"].([]interface{})
r1 := r[0].(map[string]interface{})
r2 := r1["times"].([]interface{})

times := make([]float64, len(r2))
for i := range r2 {
	times[i] = r2[i].(float64)
}

有没有更好的方法在我的JSON对象中导航,而不必每次深入对象时都实例化新变量?

英文:

I'm currently learning golang and (probably as many others before me) I'm trying to properly understand the empty interface.

As an exercise, I'm reading a big json file produced by Postman and trying to access just one field (out of the many available).

Here is a simple representation of the json without the unnecessary fields I don't want to read (but that are still there):

{
	"results": [
		{
			"times": [
				1,
				2,
				3,
				4
				]
		}
	]
}

Since the json object is big, I opted out of unmarshaling it with a custom struct, and rather decided to use the empty interface interface{}

After some time, I managed to get some working code, but I'm quite sure this isn't the correct way of doing it.

byteValue, _ := ioutil.ReadAll(jsonFile)

var result map[string]interface{}
err = json.Unmarshal(byteValue, &result)
if err != nil {
	log.Fatalln(err)
}

// ESPECIALLY UGLY
r := result["results"].([]interface{})
r1 := r[0].(map[string]interface{})
r2 := r1["times"].([]interface{})

times := make([]float64, len(r2))
for i := range r2 {
	times[i] = r2[i].(float64)
}

Is there a better way to navigate through my json object without having to instantiate new variables every time i move deeper and deeper into the object?

答案1

得分: 4

  1. 即使JSON很大,你只需要定义你真正关心的字段。
  2. 只有在键不是有效的Go标识符时(在这种情况下键是标识符),你才需要使用JSON标签,即使在这种情况下,有时候可以通过使用map[string]something来避免使用JSON标签。
  3. 除非你需要将子结构体用于某些函数或其他用途,否则你不需要定义它们。
  4. 除非你需要重用该类型,否则你甚至不需要定义它,你可以在声明时直接定义结构体。

示例:

package main

import (
   "encoding/json"
   "fmt"
)

const s = `
{
   "results": [
      {
         "times": [1, 2, 3, 4]
      }
   ]
}

func main() {
   var t struct {
      Results []struct {
         Times []int
      }
   }
   json.Unmarshal([]byte(s), &t)
   fmt.Printf("%+v\n", t) // {Results:[{Times:[1 2 3 4]}]}
}
英文:
  1. Even if the JSON is large, you only have to define the fields you actually care about
  2. You only need to use JSON tags if the keys aren't valid Go
    identifiers (keys are idents in this case), even then you can sometimes avoid it by using a map[string]something
  3. Unless you need the sub structs for some function or whatnot, you don't need to define them
  4. Unless you need to reuse the type, you don't even have to define that, you can just define the struct at declaration time

Example:

package main

import (
   "encoding/json"
   "fmt"
)

const s = `
{
   "results": [
      {
         "times": [1, 2, 3, 4]
      }
   ]
}
`

func main() {
   var t struct {
      Results []struct {
         Times []int
      }
   }
   json.Unmarshal([]byte(s), &t)
   fmt.Printf("%+v\n", t) // {Results:[{Times:[1 2 3 4]}]}
}

答案2

得分: 2

> [...]尝试访问一个字段(而不是许多可用的字段)。

对于这个具体的用例,我会使用一个库来查询和访问已知路径中的单个值,比如:

https://github.com/jmespath/go-jmespath

另一方面,如果你正在练习如何访问JSON中的嵌套值,我建议你尝试编写一个递归函数,以相同的方式(但更简单)跟随未知结构中的路径,就像go-jmespath一样。

好的,我接受了挑战,并花了一个小时编写了这个代码。它可以工作。不确定性能或错误,而且它的功能非常有限 Proper json unmarshaling in Go with the empty interface

https://play.golang.org/p/dlIsmG6Lk-p

package main

import (
	"encoding/json"
	"errors"
	"fmt"
	"strings"
)

func main() {

	// 我只是添加了一些更多的数据到结构中,以便能够测试不同的路径
	fileContent := []byte(`
	{"results": [
        {"times": [
                1,
                2,
                3,
                4
	]},
	{"times2": [
                5,
                6,
                7,
                8
	]},
	{"username": "rosadabril"},
	{"age": 42},
	{"location": [41.5933262, 1.8376757]}
	
],
"more_results": {
	"nested_1": {
		"nested_2":{
			"foo": "bar"
		}
	}
}
}`)

	var content map[string]interface{}
	if err := json.Unmarshal(fileContent, &content); err != nil {
		panic(err)
	}

	// 一些要测试的路径
	valuePaths := []string{
		"results.times",
		"results.times2",
		"results.username",
		"results.age",
		"results.doesnotexist",
		"more_results.nested_1.nested_2.foo",
	}

	for _, p := range valuePaths {
		breadcrumbs := strings.Split(p, ".")

		value, err := search(breadcrumbs, content)
		if err != nil {
			fmt.Printf("\n在'%s'中搜索时出错:%s\n", p, err)
			continue
		}

		fmt.Printf("\n在%s中找到一个值\n", p)
		fmt.Printf("类型:%T\n值:%#v\n", value, value)
	}

}

// search是我们神奇的递归函数!这个函数的思路是以一种非常基本的方式在结构中搜索,对于复杂的查询,请使用jmespath
func search(breadcrumbs []string, content map[string]interface{}) (interface{}, error) {
	// 我们永远不应该到达这个点,但是安全起见,我们可能会遇到一个超出范围的错误(检查第82行)
	if len(breadcrumbs) == 0 {
		return nil, errors.New("面包屑用完了 :'(")
	}

	// 表示我们是否到达了旅程的终点,应该返回值而不进行更多的检查
	lastBreadcrumb := len(breadcrumbs) == 1

	// 当前面包屑始终是第一个元素。
	currentBreadcrumb := breadcrumbs[0]

	if value, found := content[currentBreadcrumb]; found {
		if lastBreadcrumb {
			return value, nil
		}
		// 如果值是map[string]interface{},则继续递归下去
		if aMap, isAMap := value.(map[string]interface{}); isAMap {
			// 我们调用自己,弹出第一个面包屑,并传递当前的map
			return search(breadcrumbs[1:], aMap)
		}
		// 如果它是一个接口数组,事情就变得复杂了 :(
		if anArray, isArray := value.([]interface{}); isArray {
			for _, something := range anArray {
				if aMap, isAMap := something.(map[string]interface{}); isAMap && len(breadcrumbs) > 1 {
					if v, err := search(breadcrumbs[1:], aMap); err == nil {
						return v, nil
					}
				}
			}
		}
	}
	return nil, errors.New("糟糕,这里什么都没有")
}

希望这可以帮助到你!

英文:

> [...] trying to access just one field (out of the many available).

For this concrete use case I would use a library to query and access to a single value in a known path like:

https://github.com/jmespath/go-jmespath

In the other hand, if you're practicing how to access nested values in a JSON, I would recommend you to give a try to write a recursive function that follows a path in an unknown structure the same way (but simple) like go-jmespath does.

Ok, I challenged myself and spent an hour writing this. It works. Not sure about performance or bugs and it's really limited Proper json unmarshaling in Go with the empty interface

https://play.golang.org/p/dlIsmG6Lk-p

package main

import (
	"encoding/json"
	"errors"
	"fmt"
	"strings"
)

func main() {

	// I Just added a bit more of data to the structure to be able to test different paths
	fileContent := []byte(`
	{"results": [
        	{"times": [
                	1,
                	2,
                	3,
                	4
		]},
		{"times2": [
                	5,
                	6,
                	7,
                	8
		]},
		{"username": "rosadabril"},
		{"age": 42},
		{"location": [41.5933262, 1.8376757]}
		
	],
	"more_results": {
		"nested_1": {
			"nested_2":{
				"foo": "bar"
			}
		}
	}
	}`)

	var content map[string]interface{}
	if err := json.Unmarshal(fileContent, &content); err != nil {
		panic(err)
	}

	// some paths to test
	valuePaths := []string{
		"results.times",
		"results.times2",
		"results.username",
		"results.age",
		"results.doesnotexist",
		"more_results.nested_1.nested_2.foo",
	}

	for _, p := range valuePaths {
		breadcrumbs := strings.Split(p, ".")

		value, err := search(breadcrumbs, content)
		if err != nil {
			fmt.Printf("\nerror searching '%s': %s\n", p, err)
			continue
		}

		fmt.Printf("\nFOUND A VALUE IN: %s\n", p)
		fmt.Printf("Type: %T\nValue: %#v\n", value, value)
	}

}

// search is our fantastic recursive function! The idea is to search in the structure in a very basic way, for complex querying use jmespath
func search(breadcrumbs []string, content map[string]interface{}) (interface{}, error) {
	// we should never hit this point, but better safe than sorry and we could incurr in an out of range error (check line 82)
	if len(breadcrumbs) == 0 {
		return nil, errors.New("ran out of breadcrumbs :'(")
	}

	// flag that indicates if we are at the end of our trip and whe should return the value without more checks
	lastBreadcrumb := len(breadcrumbs) == 1

	// current breadcrumb is always the first element.
	currentBreadcrumb := breadcrumbs[0]

	if value, found := content[currentBreadcrumb]; found {
		if lastBreadcrumb {
			return value, nil
		}
		// if the value is a map[string]interface{}, go down the rabbit hole, recursion!
		if aMap, isAMap := value.(map[string]interface{}); isAMap {
			// we are calling ourselves popping the first breadcrumb and passing the current map
			return search(breadcrumbs[1:], aMap)
		}
		// if it's an array of interfaces the thing gets complicated :(
		if anArray, isArray := value.([]interface{}); isArray {
			for _, something := range anArray {
				if aMap, isAMap := something.(map[string]interface{}); isAMap && len(breadcrumbs) > 1 {
					if v, err := search(breadcrumbs[1:], aMap); err == nil {
						return v, nil
					}
				}
			}
		}
	}
	return nil, errors.New("woops, nothing here")
}

huangapple
  • 本文由 发表于 2021年7月12日 23:34:24
  • 转载请务必保留本文链接:https://go.coder-hub.com/68350112.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定