protobuf FieldMask在Go中的反序列化

huangapple go评论117阅读模式
英文:

protobuf FieldMask unmarshalling in Go

问题

所以我正在使用Go的kratos框架,并且我的API定义如下:

rpc UpdateMeeting (UpdateMeetingRequest) returns (UpdateMeetingReply) {
    option (google.api.http) = {
      patch: "/v1/meetings/{meeting_id}"
      body: "*"
    };
  };

对应的kratos生成的函数如下:

func _Meeting_UpdateMeeting_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
	in := new(UpdateMeetingRequest)
	if err := dec(in); err != nil {
		return nil, err
	}
	if interceptor == nil {
		return srv.(MeetingServer).UpdateMeeting(ctx, in)
	}
	info := &grpc.UnaryServerInfo{
		Server:     srv,
		FullMethod: Meeting_UpdateMeeting_FullMethodName,
	}
	handler := func(ctx context.Context, req interface{}) (interface{}, error) {
		return srv.(MeetingServer).UpdateMeeting(ctx, req.(*UpdateMeetingRequest))
	}
	return interceptor(ctx, in, info, handler)
}

现在,如果我将以下内容作为消息请求,我不会遇到任何解组问题,并且请求实际上会通过由Kratos生成的方法调用我的UpdateMeeting方法。

message UpdateMeetingRequest {

  string  meeting_id = 1;

  string updated_by = 2;

  MeetingMeta meeting_meta = 3;
  // more fields left out
}

message MeetingMeta {
  string meeting_link = 1;
  // more fields left out
}

但是,当我添加FieldMask时,我实际上需要检查哪个字段存在,哪个字段不存在时,我会遇到以下错误:

message UpdateMeetingRequest {

  string  meeting_id = 1;

  string updated_by = 2;

  MeetingMeta meeting_meta = 3;

  google.protobuf.FieldMask update_mask = 4;
}


Error
{
    "code": 400,
    "reason": "CODEC",
    "message": "body unmarshal proto: (line 5:18): google.protobuf.FieldMask.paths contains invalid path: \"meeting_meta\"",
    "metadata": {}
}

负载

{
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "meeting_meta"
}

对于更新掩码,我尝试了meeting_meta.meeting_link、meeting_meta、meeting_link这三个,都出现了相同或类似的错误。

编辑:
所以根据答案中提到的方法,通过更改update_mask中的meeting_meta为meetingMeta,我成功使其工作。

但是,如果负载中有两个带有下划线的字段,我遇到了相同的问题,即“包含无效路径”。例如:

{
  
  "updated_by": "dsjhvg",
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "updatedBy, meetingMeta"
}

这两个字段单独解组都没有问题。但是当它们组合在一起时,我在update_mask的第二个路径上遇到错误。对于此负载,错误是meeting_meta,如果我重新排列它们,则是updated_by

英文:

So I'm using kratos framework with Go and my API definition is:

rpc UpdateMeeting (UpdateMeetingRequest) returns (UpdateMeetingReply) {
    option (google.api.http) = {
      patch: "/v1/meetings/{meeting_id}"
      body: "*"
    };
  };

And the corresponding kratos generated func

func _Meeting_UpdateMeeting_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
	in := new(UpdateMeetingRequest)
	if err := dec(in); err != nil {
		return nil, err
	}
	if interceptor == nil {
		return srv.(MeetingServer).UpdateMeeting(ctx, in)
	}
	info := &grpc.UnaryServerInfo{
		Server:     srv,
		FullMethod: Meeting_UpdateMeeting_FullMethodName,
	}
	handler := func(ctx context.Context, req interface{}) (interface{}, error) {
		return srv.(MeetingServer).UpdateMeeting(ctx, req.(*UpdateMeetingRequest))
	}
	return interceptor(ctx, in, info, handler)
}

Now if I have the following as the message request I do not have any issues unmarshalling and the request actually lands on my UpdateMeeting method being called from Kratos generated method.

message UpdateMeetingRequest {

  string  meeting_id = 1;

  string updated_by = 2;

  MeetingMeta meeting_meta = 3;
  // more fields left out
}

message MeetingMeta {
  string meeting_link = 1;
  // more fields left out
}

But when I add FieldMask which I actually need to check which field is present and which isn't, I get this error -

message UpdateMeetingRequest {

  string  meeting_id = 1;

  string updated_by = 2;

  MeetingMeta meeting_meta = 3;

  google.protobuf.FieldMask update_mask = 4;
}


Error
{
    "code": 400,
    "reason": "CODEC",
    "message": "body unmarshal proto: (line 5:18): google.protobuf.FieldMask.paths contains invalid path: \"meeting_meta\"",
    "metadata": {}
}

Payload

{
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "meeting_meta"
}

For the update mask tried: meeting_meta.meeting_link, meeting_meta, meeting_link all three with same or similar errors.

Edit:
So with approach mentioned in the answer I was able to make it work by changing the update_mask. meeting_meta -> meetingMeta.

But running into same issue, i.e., "contains invalid path" if there are two fields in the payload with '_' in name. Example:

{
  
  "updated_by": "dsjhvg",
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "updatedBy, meetingMeta"
}

Individually both these fields are being unmarshalled just fine. Combined I'm running into an error for the second path in the update_mask. For this payload it's meeting_meta, if I rearrange them then its updated_by.

答案1

得分: 1

负载应该修改为:

{
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "meetingMeta"
}

("update_mask": "meeting_meta" 被替换为 "update_mask": "meetingMeta")

根据JSON编码的字段掩码,每个路径中的字段名称转换为/从小写驼峰命名约定。

下面的源代码来自google.golang.org/protobuf/encoding/protojson包,显示了字段名称的验证方式(参见第20行,字段名称中的_是不允许的):

 1	func (d decoder) unmarshalFieldMask(m protoreflect.Message) error {
 2		tok, err := d.Read()
 3		if err != nil {
 4			return err
 5		}
 6		if tok.Kind() != json.String {
 7			return d.unexpectedTokenError(tok)
 8		}
 9		str := strings.TrimSpace(tok.ParsedString())
10		if str == "" {
11			return nil
12		}
13		paths := strings.Split(str, ",")
14	
15		fd := m.Descriptor().Fields().ByNumber(genid.FieldMask_Paths_field_number)
16		list := m.Mutable(fd).List()
17	
18		for _, s0 := range paths {
19			s := strs.JSONSnakeCase(s0)
20			if strings.Contains(s0, "_") || !protoreflect.FullName(s).IsValid() {
21				return d.newError(tok.Pos(), "%v contains invalid path: %q", genid.FieldMask_Paths_field_fullname, s0)
22			}
23			list.Append(protoreflect.ValueOfString(s))
24		}
25		return nil
26	}

回答第二个问题。

如果仔细查看错误消息,你会看到类似以下内容:

google.protobuf.FieldMask.paths contains invalid path: " meetingMeta"

你可以看到报告的无效路径是" meetingMeta"。注意前面的空格。这意味着该包不会为你修剪路径。因此,在路径之间不应添加空格。负载应该是:

{
  
  "updated_by": "dsjhvg",
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "updatedBy,meetingMeta"
}
英文:

The payload should be modified to:

{
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "meetingMeta"
}

("update_mask": "meeting_meta" is replaced with "update_mask": "meetingMeta")

According to JSON Encoding of Field Masks, fields name in each path are converted to/from lower-camel naming conventions.

The source code below copied from the google.golang.org/protobuf/encoding/protojson package shows how the fields name is verified (see line 20, _ in the field name is not allowed):

 1	func (d decoder) unmarshalFieldMask(m protoreflect.Message) error {
 2		tok, err := d.Read()
 3		if err != nil {
 4			return err
 5		}
 6		if tok.Kind() != json.String {
 7			return d.unexpectedTokenError(tok)
 8		}
 9		str := strings.TrimSpace(tok.ParsedString())
10		if str == "" {
11			return nil
12		}
13		paths := strings.Split(str, ",")
14	
15		fd := m.Descriptor().Fields().ByNumber(genid.FieldMask_Paths_field_number)
16		list := m.Mutable(fd).List()
17	
18		for _, s0 := range paths {
19			s := strs.JSONSnakeCase(s0)
20			if strings.Contains(s0, "_") || !protoreflect.FullName(s).IsValid() {
21				return d.newError(tok.Pos(), "%v contains invalid path: %q", genid.FieldMask_Paths_field_fullname, s0)
22			}
23			list.Append(protoreflect.ValueOfString(s))
24		}
25		return nil
26	}

Answer to the second question.

If you look at the error message carefully, you will see something like:

google.protobuf.FieldMask.paths contains invalid path: " meetingMeta"

You see that the reported invalid path is " meetingMeta". Pay attention to the leading blank space. It means that the package does not trim the path for you. So you should not add spaces between paths. The payload should be:

{
  
  "updated_by": "dsjhvg",
  "meeting_meta": {
      "meeting_link": "ac"
  },
  "update_mask": "updatedBy,meetingMeta"
}

huangapple
  • 本文由 发表于 2023年7月14日 01:50:37
  • 转载请务必保留本文链接:https://go.coder-hub.com/76682058.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定