Python客户端发送的GRPC请求在Golang服务器端没有正确接收到。

huangapple go评论81阅读模式
英文:

GRPC request sent from Python client not received properly by Golang server

问题

我已经编写了一个Python GRPC客户端,可以连接到多个GRPC Golang服务。我已经成功实现了以下操作:

from alphausblue.connection.conn import grpc_client_connection
from alphausblue.iam.v1.iam_pb2 import WhoAmIRequest
from alphausblue.iam.v1.iam_pb2_grpc import IamStub

async def main():
    conn = grpc_client_connection(svc="blue")
    stub = IamStub(conn)
    resp = await stub.WhoAmI(WhoAmIRequest())
    print(resp)

其中服务名称为blue。然而,如果我尝试连接到不同的服务并请求数据,代码如下:

from alphausblue.connection.conn import grpc_client_connection
from alphausblue.cost.v1.cost_pb2 import ListAccountsRequest
from alphausblue.cost.v1.cost_pb2_grpc import CostStub

async def main():
    conn = grpc_client_connection(svc="cost")
    stub = CostStub(conn)
    account = await stub.GetAccount(GetAccountRequest(vendor='aws', id='731058950257'))
    print(account)

我会收到一个"UNIMPLEMENTED"的响应。如果服务不存在,这是有道理的,但实际上服务是存在的,我的Golang客户端可以正常连接到它。此外,当我检查服务器的日志时,我可以清楚地看到请求已经到达了服务器。经过进一步的研究,我在服务器上发现了以下代码:

type service struct {
    UserInfo *blueinterceptors.UserData

    cost.UnimplementedCostServer
}

func (s *service) GetAccount(ctx context.Context, in *cost.GetAccountRequest) (*api.Account, error) {
    switch in.Vendor {
    case "aws":
        // Do stuff
    default:
        return status.Errorf(codes.Unimplemented, "not implemented")
    }
}

这告诉我函数被调用了,但是反序列化的有效负载缺少了vendor字段。然而,当我调试时,我可以看到这行代码:

> src/core/lib/security/transport/secure_endpoint.cc:296] WRITE 0000018E2C62FB80: 00 00 00 13 0a 03 61 77 73 12 0c 37 33 31 30 35 38 39 35 30 32 35 37 '......aws..731058950257'

因此,数据已经通过GRPC发送到服务器,但是被反序列化为一个缺少字段的对象。那么,这是什么原因呢?

更新

我查看了Python和Golang客户端的GetAccountRequest定义,如@blackgreen建议的那样。

Golang客户端代码:

// Request message for the Cost.GetAccount rpc.
type GetAccountRequest struct {
    state         protoimpl.MessageState
    sizeCache     protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields

    Vendor string `protobuf:"bytes,1,opt,name=vendor,proto3" json:"vendor,omitempty"`
    Id     string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
}

Python客户端代码:

_GETACCOUNTREQUEST = _descriptor.Descriptor(
    name='GetAccountRequest',
    full_name='blueapi.cost.v1.GetAccountRequest',
    filename=None,
    file=DESCRIPTOR,
    containing_type=None,
    create_key=_descriptor._internal_create_key,
    fields=[
        _descriptor.FieldDescriptor(
            name='vendor', full_name='blueapi.cost.v1.GetAccountRequest.vendor', index=0,
            number=1, type=9, cpp_type=9, label=1,
            has_default_value=False, default_value=b"".decode('utf-8'),
            message_type=None, enum_type=None, containing_type=None,
            is_extension=False, extension_scope=None,
            serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
        _descriptor.FieldDescriptor(
            name='id', full_name='blueapi.cost.v1.GetAccountRequest.id', index=1,
            number=2, type=9, cpp_type=9, label=1,
            has_default_value=False, default_value=b"".decode('utf-8'),
            message_type=None, enum_type=None, containing_type=None,
            is_extension=False, extension_scope=None,
            serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
    ],
    extensions=[
    ],
    nested_types=[],
    enum_types=[
    ],
    serialized_options=None,
    is_extendable=False,
    syntax='proto3',
    extension_ranges=[],
    oneofs=[
    ],
    serialized_start=946,
    serialized_end=993,
)

很明显,这里的字段顺序是相同的,所以我认为问题不在于字段顺序,除非GRPC使用的是index而不是number

英文:

I have written a Python GRPC client that can connect to a number of GRPC Golang services. I have been able to make this work like so:

from alphausblue.connection.conn import grpc_client_connection
from alphausblue.iam.v1.iam_pb2 import WhoAmIRequest
from alphausblue.iam.v1.iam_pb2_grpc import IamStub

async def main():
    conn = grpc_client_connection(svc = "blue")
    stub = IamStub(conn)
    resp = await stub.WhoAmI(WhoAmIRequest())
    print(resp)

where the service name is blue. However, if I were to try to connect to a different service to request data like this:

from alphausblue.connection.conn import grpc_client_connection
from alphausblue.cost.v1.cost_pb2 import ListAccountsRequest
from alphausblue.cost.v1.cost_pb2_grpc import CostStub

async def main():
    conn = grpc_client_connection(svc = "cost")
    stub = CostStub(conn)
    account = await stub.GetAccount(GetAccountRequest(vendor = 'aws', id = '731058950257'))
    print(account)

I get an UNIMPLEMENTED response. This would make sense if the service didn't exist but it does and my Golang client connects to it just fine. Moreover, when I check the server's logs I can clearly see that the request reached the server. Doing some more research, I discovered this code on my server:

type service struct {
	UserInfo *blueinterceptors.UserData

	cost.UnimplementedCostServer
}

func (s *service) GetAccount(ctx context.Context, in *cost.GetAccountRequest) (*api.Account, error) {
	switch in.Vendor {
	case "aws":
		// Do stuff
	default:
		return status.Errorf(codes.Unimplemented, "not implemented")
	}
}

What this tells me is that the function is being called but the payload is being deserialized is missing the vendor field. However, when debugging I can see this line:

> src/core/lib/security/transport/secure_endpoint.cc:296] WRITE 0000018E2C62FB80: 00 00 00 13 0a 03 61 77 73 12 0c 37 33 31 30 35 38 39 35 30 32 35 37 '......aws..731058950257'

So, the data is being sent over GRPC to the server but is being deserialized into an object with missing fields. What, then is the cause of this?

Update

I looked at the definitions for GetAccountRequest for both the Python and Golang clients as suggested by @blackgreen.

Golang client code:

// Request message for the Cost.GetAccount rpc.
type GetAccountRequest struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	Vendor string `protobuf:"bytes,1,opt,name=vendor,proto3" json:"vendor,omitempty"`
	Id string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
}

Python client code:

_GETACCOUNTREQUEST = _descriptor.Descriptor(
  name='GetAccountRequest',
  full_name='blueapi.cost.v1.GetAccountRequest',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  create_key=_descriptor._internal_create_key,
  fields=[
    _descriptor.FieldDescriptor(
      name='vendor', full_name='blueapi.cost.v1.GetAccountRequest.vendor', index=0,
      number=1, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=b"".decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
    _descriptor.FieldDescriptor(
      name='id', full_name='blueapi.cost.v1.GetAccountRequest.id', index=1,
      number=2, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=b"".decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      serialized_options=None, file=DESCRIPTOR,  create_key=_descriptor._internal_create_key),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  serialized_options=None,
  is_extendable=False,
  syntax='proto3',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=946,
  serialized_end=993,
)

It's clear that the fields are in the same order here so I don't think that's the issue unless GRPC is using index rather than number.

答案1

得分: 1

在确保你在Python的version字段中设置的字符串不包含不可见字符后,可能是Python客户端导入的生成代码与Go服务器导入的生成代码之间存在版本不匹配。

特别是,如果两个版本确实都有一个version字段,但是Go代码在某种程度上无法“识别”它,可能是由于字段标签号不匹配引起的。

你展示的字节载荷(十六进制0a 03 61 77 73 12 0c 37 33 31 30 35 38 39 35 30 32 35 37,Base64 CgNhd3MSDDczMTA1ODk1MDI1Nw==)确实在proto字段中使用标签号1发送了aws字符串(如何知道 我知道)。

因此,让我们考虑一个假设的例子,Python客户端中使用的GetAccountRequest的proto模式可能是:

message GetAccountRequest {
    string version = 1; // 字段标签号为1
}

而Go服务器中使用的可能是:

message GetAccountRequest {
    string version = 3; // 字段标签号为3
}

在这种情况下,你会看到在传输过程中的消息具有version字段,但是在反序列化为具有不同标签号的Go结构体时,该字段将为空。客户端使用标签1发送它,而服务器期望使用标签3接收它。

你可以通过检查Go的GetAccountRequest结构体来验证这个假设。它应该类似于以下内容:

type GetAccountRequest struct {
	// 未导出的字段

    // 其他字段,按顺序
	Version string `protobuf:"bytes,3,opt,name=version,proto3" json:"version,omitempty"`
}

反引号<code>之间的部分是结构体标签,其中<code>3</code>是标签号。

我不知道Python中的gRPC代码生成是什么样子的,但你应该能够以某种方式比较标签号。如果它们不同,请确保使用具有相同标签号的生成结构体更新客户端或服务器代码。

英文:

After you make sure that the string you are setting in the Python version field doesn't contain invisible characters, there is probably a version mismatch in the generated code imported in the Python client vs. the one imported by the Go server.

In particular, if both versions do actually have a version field, but somehow the Go code fails to "recognize" it, it could be caused by a mismatch in the field tag number.

The byte payload you show (hexa 0a 03 61 77 73 12 0c 37 33 31 30 35 38 39 35 30 32 35 37, base64 CgNhd3MSDDczMTA1ODk1MDI1Nw==) is indeed sending aws string in the proto field with tag number 1 (how do I know)

So let's consider a contrived example, the proto schema of GetAccountRequest used in the Python client may be

message GetAccountRequest {
    string version = 1; // field tag number 1
}

And the one used in the Go server may be:

message GetAccountRequest {
    string version = 3; // field tag number 3
}

In this case you will see the message on the wire having the version field, but upon deserializing it against a Go struct with a different tag number, it will end up empty. The client sends it with tag 1 and the server expects it with tag 3.

You can validate this hypothesis by inspecting the Go GetAccountRequest struct. It should look like the following:

type GetAccountRequest struct {
	// unexported fields

    // other fields, in order
	Version string `protobuf:&quot;bytes,3,opt,name=version,proto3&quot; json:&quot;version,omitempty&quot;`
}

The part between backticks <code>`</code> is the struct tag, where <code>3</code> is the tag number.

I don't know about what gRPC codegen in Python looks like, but you should be able to compare the tag numbers somehow. If they do differ, make sure to update either the client or server code with generated structs that have the same tag numbers.

huangapple
  • 本文由 发表于 2021年10月11日 11:10:58
  • 转载请务必保留本文链接:https://go.coder-hub.com/69520624.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定