Protobuf.Any – 从 json.RawMessage 解析

huangapple go评论75阅读模式
英文:

Protobuf.Any - Unmarshal from a json.RawMessage

问题

我有一些以json.RawMessage格式存储在数据库中的数据。具体的列是jsonb类型。

我无法找到一种将数据解组为在proto中定义为protobuf.Any的属性的方法,如下所示:

repeated google.protobuf.Any list = 1;

当我尝试使用json.Unmarshal()从数据库解组数据时,list是空的。文档中提到了类似以下的内容:

foo := &pb.Foo{...}
any, err := anypb.New(foo)
if err != nil {
  ...
}
...
foo := &pb.Foo{}
if err := any.UnmarshalTo(foo); err != nil {
  ...
}

但在这个示例中,foo的类型是proto.Message,而我无法转换为json.RawMessage

有没有办法可以做到这一点?

英文:

I have data from the DB that is in json.RawMessage format. The specific column is jsonb.

I can't really find a way to unmarshal the data to a property that on proto is defined as protobuf.Any like so.

repeated google.protobuf.Any list = 1;

When I try to unmarshal the data from the db using json.Unmarshal() then list is empty. The documentation mention something like this:

foo := &pb.Foo{...}
 any, err := anypb.New(foo)
 if err != nil {
   ...
 }
 ...
 foo := &pb.Foo{}
 if err := any.UnmarshalTo(foo); err != nil {
   ...
 }

But in this example foo is of type proto.Message which I can't convert since I have json.RawMessage.

Is there any way I can do this?

答案1

得分: 1

首先,你应该了解数据库列中存储了什么。json.RawMessage 简单地定义为 type RawMessage []byte(参见文档)。它没有足够的信息来回答你的问题。

我将提供一个演示来展示 google.protobuf.Any 的工作原理,这应该能帮助你更好地理解你的问题。

注意事项:

  1. Any 用于在消息中嵌入其他类型。因此,在演示中我定义了另外两个消息(FooBar)。

    Any 消息类型允许您在没有其 .proto 定义的情况下使用消息作为嵌入类型。Any 包含一个任意序列化的消息字节,以及一个作为全局唯一标识符并解析为该消息类型的 URL。

  2. 实际上,你的问题取决于数据库中存储了什么。请参考 main.go 中的注释。


演示的文件夹结构:

├── go.mod
├── main.go
└── pb
    ├── demo.pb.go
    └── demo.proto

go.mod:

module github.com/ZekeLu/demo

go 1.19

require (
	github.com/golang/protobuf v1.5.2
	google.golang.org/protobuf v1.28.1
)

pb/demo.proto:

syntax = "proto3";
package pb;

import "google/protobuf/any.proto";

option go_package = "github.com/ZekeLu/demo/pb";

message MyMessage {
  repeated google.protobuf.Any list = 1;
}

message Foo {
  int32 v = 1;
}

message Bar {
  string v = 1;
}

main.go:

package main

import (
	"encoding/json"
	"fmt"

	"google.golang.org/protobuf/types/known/anypb"

	"github.com/ZekeLu/demo/pb"
)

func main() {
	// 如果数据库存储了 pb.Foo 的实例,则首先进行解组。
	buf := json.RawMessage([]byte(`{"v":10}`))
	var foo pb.Foo
	err := json.Unmarshal(buf, &foo)
	if err != nil {
		panic(err)
	}

	// 然后将其编组为新的 Any 实例,该实例可用于创建可分配给 pb.MyMessage.List 的切片。
	a1, err := anypb.New(&foo)
	if err != nil {
		panic(err)
	}

	bar := &pb.Bar{V: "10"}
	a2, err := anypb.New(bar)
	if err != nil {
		panic(err)
	}

	// 初始化 List 字段。
	m := pb.MyMessage{List: []*anypb.Any{a1, a2}}

	buf, err = json.Marshal(&m)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s\n", buf)
	// 输出: {"list":[{"type_url":"type.googleapis.com/pb.Foo","value":"CAo="},{"type_url":"type.googleapis.com/pb.Bar","value":"CgIxMA=="}]}

	// 如果数据库存储了上述输出,则可以直接解组。
	var m2 pb.MyMessage
	err = json.Unmarshal(buf, &m2)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%v\n", m2.List)
	// 输出: [[type.googleapis.com/pb.Foo]:{v:10} [type.googleapis.com/pb.Bar]:{v:"10"}]
}

运行演示的步骤:

$ protoc --proto_path=pb --go_out=pb --go_opt=paths=source_relative demo.proto
$ go mod tidy
$ go run main.go
英文:

First of all, you should understand what is stored in the DB column. json.RawMessage is simply defined as type RawMessage []byte (see the doc). And it does not carry enough information to answer your question.

I will provide a demo to show how google.protobuf.Any works, which should help you understand your question better.

Notes:

  1. Any is for embedding other types in a message. So I define two other messages (Foo and Bar) in the demo.
    > The Any message type lets you use messages as embedded types without having their .proto definition. An Any contains an arbitrary serialized message as bytes, along with a URL that acts as a globally unique identifier for and resolves to that message's type.
  2. Actually, your question depends on what is stored in the DB. See the comments in main.go.

Folder structure of the demo:

├── go.mod
├── main.go
└── pb
    ├── demo.pb.go
    └── demo.proto

go.mod:

module github.com/ZekeLu/demo

go 1.19

require (
	github.com/golang/protobuf v1.5.2
	google.golang.org/protobuf v1.28.1
)

pb/demo.proto:

syntax = "proto3";
package pb;

import "google/protobuf/any.proto";

option go_package = "github.com/ZekeLu/demo/pb";

message MyMessage {
  repeated google.protobuf.Any list = 1;
}

message Foo {
  int32 v = 1;
}

message Bar {
  string v = 1;
}

main.go:

package main

import (
	"encoding/json"
	"fmt"

	"google.golang.org/protobuf/types/known/anypb"

	"github.com/ZekeLu/demo/pb"
)

func main() {
	// If the db stores an instance of pb.Foo, then unmarshal it first.
	buf := json.RawMessage([]byte(`{"v":10}`))
	var foo pb.Foo
	err := json.Unmarshal(buf, &foo)
	if err != nil {
		panic(err)
	}

	// And then marshal it into a new Any instance, which can be used to
	// create a slice that can be assigned to pb.MyMessage.List.
	a1, err := anypb.New(&foo)
	if err != nil {
		panic(err)
	}

	bar := &pb.Bar{V: "10"}
	a2, err := anypb.New(bar)
	if err != nil {
		panic(err)
	}

	// Initialize the List field.
	m := pb.MyMessage{List: []*anypb.Any{a1, a2}}

	buf, err = json.Marshal(&m)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s\n", buf)
	// Output: {"list":[{"type_url":"type.googleapis.com/pb.Foo","value":"CAo="},{"type_url":"type.googleapis.com/pb.Bar","value":"CgIxMA=="}]}

	// If the db stores the output above, it can be unmarshal directly
	var m2 pb.MyMessage
	err = json.Unmarshal(buf, &m2)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%v\n", m2.List)
	// Output: [[type.googleapis.com/pb.Foo]:{v:10} [type.googleapis.com/pb.Bar]:{v:"10"}]
}

Steps to run the demo:

$ protoc --proto_path=pb --go_out=pb --go_opt=paths=source_relative demo.proto
$ go mod tidy
$ go run main.go

huangapple
  • 本文由 发表于 2022年11月15日 07:47:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/74439162.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定