一致的跨平台Proto3序列化/反序列化

huangapple go评论90阅读模式
英文:

Consistent Cross Platform Proto3 Serialization / Deserialization

问题

我有一个基于golang的GRPC服务,使用proto3定义。

对于一个简单的响应,只有一个bool字段,如果值为false,整个消息将被序列化为空(零长度)数据负载(作为我的怀疑,将false视为默认值)。

与此同时,使用相同的.proto定义构建的节点的GRPC客户端将该值解释为undefined(而true的情况非常一致)。

调试服务器构建响应的方式,我注意到stream.ContentSubtype()返回一个空字符串,因此在s.getCodec中它会回退到encoding/proto.Name,即proto。然后在这里进一步进行编码。

func marshalAppend(buf []byte, m Message, deterministic bool) ([]byte, error) {
	if m == nil {
		return nil, ErrNil
	}
	mi := MessageV2(m)
	nbuf, err := protoV2.MarshalOptions{
		Deterministic: deterministic,
		AllowPartial:  true,
	}.MarshalAppend(buf, mi)

这看起来确实是protoV2的实现。

所以我不知道真正的问题是什么:

  • stream.ContentSubtype()是否应该在通信层面上进行控制,由谁负责(客户端/服务器)?

  • 在从.proto定义生成客户端/服务器源代码时,是否应该遵循任何其他的控制步骤/操作?.proto本身是否足够泛化元数据以输出一致的源代码?

  • proto3是否应该使用protoV2实现进行成功编码?

  • 还有什么其他原因可能导致这种不一致的值处理?

英文:

I have a golang based GRPC service defined with proto3

syntax = "proto3";

For a simple Response of a single bool filed if the value is false the whole message is serialized just as an empty (zero length) data payload. (as my suspect, treating false as a default one)

Same time a node's GRPC client built from the very same .proto definition interprets the value as an undefined. (while true one is pretty consistent)

Debugging the way server builds the response

data, err := encode(s.getCodec(stream.ContentSubtype()), msg)

I've noticed, stream.ContentSubtype() returns an empty string so that within s.getCodec it is falls back to encoding/proto.Name which is just proto. And it leads further encoding something over here

func marshalAppend(buf []byte, m Message, deterministic bool) ([]byte, error) {
	if m == nil {
		return nil, ErrNil
	}
	mi := MessageV2(m)
	nbuf, err := protoV2.MarshalOptions{
		Deterministic: deterministic,
		AllowPartial:  true,
	}.MarshalAppend(buf, mi)

Which looks like really protoV2 implementation.

So I don't know what is really the right question here

  • Should stream.ContentSubtype() be somehow controlled on a communication level and who's to be responsible for this (Client/Server)

  • Should any additional control steps/actions be followed while generating client/server source codes from the .proto definition. Doesn't .proto it self generalize the meta enough to output consistent sources?

  • Should/Cloud proto3 be really encoded successfully with protoV2 implementation?

  • What else could cause such inconsistent values treatment?

答案1

得分: 0

原来这不是配置错误,而是特定实现的问题。

六天前,在 porto-gen-ts v0.8.5 中修复了这个问题。

感谢 @Brits 的查找。

英文:

Turned out this was nothing with misconfiguration but a matter of specific implementation.

Six days ago it was fixed within porto-gen-ts v0.8.5

Thanks @Brits for a lookup

huangapple
  • 本文由 发表于 2022年7月15日 09:02:20
  • 转载请务必保留本文链接:https://go.coder-hub.com/72988052.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定