goavro和原始的Go数据结构

huangapple go评论78阅读模式
英文:

goavro and original Go data structure

问题

使用goavro库在Golang中如何使用avro重新创建原始数据结构?

使用这个库https://github.com/hamba/avro非常简单。

out := SimpleRecord{}
err = avro.Unmarshal(schema, data, &out)

变量out的类型是SimpleRecord。

假设我有以下结构体和avro模式:

type SimpleRecord struct {
	F1           int      `avro:"f1"`
	F2           string   `avro:"f2"`
	F3           string   `avro:"f3"`
	Dependencies []string `avro:"dependencies"`
}

func main() {
	avro_schema_txt := `{
		"type": "record",
		"name": "AvroData",
		"namespace": "data.avro",
		"doc": "docstring",
		"fields": [
			{
				"name": "f1",
				"type": "int"
			},
			{
				"name": "f2",
				"type": "string"
			},
			{
				"name": "f3",
				"type": "string"
			},
			{
				"name": "dependencies",
				"type": {
					"type": "array",
					"items": "string"
				}
			}
		]
	}`
}

然后使用以下代码:

codec, err := goavro.NewCodec(avro_schema_txt)
if err != nil {
	log.Fatal(err.Error())
}
out, _, err := codec.NativeFromBinary(data)
if err != nil {
	log.Fatal(err.Error())
}
fmt.Println(out)

其中data是使用avro编组的,out的类型是interface{},那么如何将其转换为SimpleRecord类型呢?

英文:

How can I recreate original data structure in Golang serialized with avro using goavro?

With this library https://github.com/hamba/avro it's quite easy.

out := SimpleRecord{}
err = avro.Unmarshal(schema, data, &out)

type of variable out is SimpleRecord.

Let's say I have this struct and avro schema:

type SimpleRecord struct {
	F1           int      `avro:"f1"`
	F2           string   `avro:"f2"`
	F3           string   `avro:"f3"`
	Dependencies []string `avro:"dependencies"`
}

func main() {
	avro_schema_txt := `{
		"type": "record",
		"name": "AvroData",
		"namespace": "data.avro",
		"doc": "docstring",
		"fields": [
			{
				"name": "f1",
				"type": "int"
			},
			{
				"name": "f2",
				"type": "string"
			},
			{
				"name": "f3",
				"type": "string"
			},
			{
				"name": "dependencies",
				"type": {
					"type": "array",
					"items": "string"
				}
			}
		]
	}`
}

and then

codec, err := goavro.NewCodec(avro_schema_txt)
	if err != nil {
		log.Fatal(err.Error())
	}
	out, _, err := codec.NativeFromBinary(data)
	if err != nil {
		log.Fatal(err.Error())
	}
	fmt.Println(out)

where data is marshaled with avro, out is of type interface{}, so how can I "make" it SimpleRecord?

答案1

得分: 2

有两种方法可以实现,一种是将out强制转换为map[string]interface{}并获取值,另一种是使用codec.TextualFromNative。下面分别展示了这两种方法的代码:

方法一

outinterface{}转换为map[string]interface{}并获取值:

...
simpleMap := out.(map[string]interface{})
f1 := simpleMap["f1"]
f2 := simpleMap["f2"]
...

SimpleRecord {
    F1: f1,
    F2: f2,
    ...
}

方法二

使用TextualFromNative,下面的代码展示了编码和解码的过程:

var avro_schema_txt = `{
    "type": "record",
    "name": "AvroData",
    "namespace": "data.avro",
    "doc": "docstring",
    "fields": [
        {
            "name": "f1",
            "type": "int"
        },
        {
            "name": "f2",
            "type": "string"
        },
        {
            "name": "f3",
            "type": "string"
        },
        {
            "name": "dependencies",
            "type": {
                "type": "array",
                "items": "string"
            }
        }
    ]
}`

// 添加json以匹配avro中的字段名
type SimpleRecord struct {
    F1           int      `avro:"f1" json:"f1"`
    F2           string   `avro:"f2" json:"f2"`
    F3           string   `avro:"f3" json:"f3"`
    Dependencies []string `avro:"dependencies" json:"dependencies"`
}

func encodeDecode() {

    data := SimpleRecord{
        F1: 1,
        F2: "tester2",
        F3: "tester3",
        Dependencies: []string { "tester4", "tester5" },
    }

    codec, err := goavro.NewCodec(avro_schema_txt)
    if err != nil {
        log.Fatal(err.Error())
    }

    // 编码

    textualIn, err := json2.Marshal(data)
    if err != nil {
        log.Fatal(err.Error())
    }

    nativeIn, _, err := codec.NativeFromTextual(textualIn)
    if err != nil {
        log.Fatal(err.Error())
    }

    binaryIn, err := codec.BinaryFromNative(nil, nativeIn)
    if err != nil {
        log.Fatal(err.Error())
    }

    // 解码

    nativeOut, _, err := codec.NativeFromBinary(binaryIn)
    if err != nil {
        log.Fatal(err.Error())
    }

    textualOut, err := codec.TextualFromNative(nil, nativeOut)
    if err != nil {
        log.Fatal(err.Error())
    }

    var out = SimpleRecord{}
    err = json2.Unmarshal(textualOut, &out)
    if err != nil {
        log.Fatal(err.Error())
    }

    if !reflect.DeepEqual(data, out) {
        log.Fatal("should be equal")
    }
}
英文:

There are two ways you can do it, either by casting out and doing a 'manual mapping' or by using codec.TextualFromNative. Both approaches are shown below for completeness,

Approach 1

Cast out from interface{} to map[string]interface{} and retrieve the values

...
simpleMap := out.(map[string]interface{})
f1 := simpleMap["f1"]
f2 := simpleMap["f2"] 
...
SimpleRecord {
F1: f1,
F2: f2,
...
} 

Approach 2

Use TextualFromNative, the below code shows both encode decode process

var avro_schema_txt = `{
"type": "record",
"name": "AvroData",
"namespace": "data.avro",
"doc": "docstring",
"fields": [
{
"name": "f1",
"type": "int"
},
{
"name": "f2",
"type": "string"
},
{
"name": "f3",
"type": "string"
},
{
"name": "dependencies",
"type": {
"type": "array",
"items": "string"
}
}
]
}`
// added json to match field names in avro
type SimpleRecord struct {
F1           int      `avro:"f1" json:"f1"`
F2           string   `avro:"f2" json:"f2"`
F3           string   `avro:"f3" json:"f3"`
Dependencies []string `avro:"dependencies" json:"dependencies"`
}
func encodeDecode() {
data := SimpleRecord{
F1: 1,
F2: "tester2",
F3: "tester3",
Dependencies: []string { "tester4", "tester5" },
}
codec, err := goavro.NewCodec(avro_schema_txt)
if err != nil {
log.Fatal(err.Error())
}
// encode
textualIn, err := json2.Marshal(data)
if err != nil {
log.Fatal(err.Error())
}
nativeIn, _, err := codec.NativeFromTextual(textualIn)
if err != nil {
log.Fatal(err.Error())
}
binaryIn, err := codec.BinaryFromNative(nil, nativeIn)
if err != nil {
log.Fatal(err.Error())
}
// decode
nativeOut, _, err := codec.NativeFromBinary(binaryIn)
if err != nil {
log.Fatal(err.Error())
}
textualOut, err := codec.TextualFromNative(nil, nativeOut)
if err != nil {
log.Fatal(err.Error())
}
var out = SimpleRecord{}
err = json2.Unmarshal(textualOut, &out)
if err != nil {
log.Fatal(err.Error())
}
if !reflect.DeepEqual(data, out) {
log.Fatal("should be equal")
}
}

huangapple
  • 本文由 发表于 2021年9月8日 18:49:26
  • 转载请务必保留本文链接:https://go.coder-hub.com/69101669.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定