跨语言(de)序列化

huangapple go评论94阅读模式
英文:

cross language (de)serialization

问题

我正在尝试在Python中进行序列化,并在Golang中进行反序列化,但是遇到了错误。

错误信息 -- "无法解析无效的线格式数据"。

代码配置 --

Python代码 --

schema_registry_client = SchemaRegistryClient({'url': 'http://localhost:8082'})
protobuf_serializer = ProtobufSerializer(user_attributes_pb2.UserProperties,
                                           schema_registry_client,
                                           {'use.deprecated.format': True})
  producer_conf = {'bootstrap.servers':  'localhost:9092', 'key.serializer': StringSerializer('utf_8'), 'value.serializer': protobuf_serializer}
  producer = SerializingProducer(producer_conf)
  producer.poll(0.0)
  ########## Add an address #########
  PromptForAddress(user_attr)
  producer.produce(topic=topic, key=str(uuid4()), value=user_attr)
  producer.flush()

Golang代码 --

		Brokers: []string{brokerAddress},
		Topic:   topic,
		GroupID: "test-consumer-group",
		Logger: l,
	})
	for {
		msg, err := r.ReadMessage(ctx)
		user := &pb.UserProperties{}
		err = proto.Unmarshal(msg.Value, user)
		// client.Query() NewKey(Namespace, Set, user.Id)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("\n%s\n", proto.Message(user))
		fmt.Printf("\n%v\n", user)
		if err != nil {
			panic("could not read message " + err.Error())
		}
		// after receiving the message, log its value
		fmt.Println("received: ", string(msg.Value))
	}

proto文件在两种语言中是相似的。

英文:

i am trying to serialize in python and unmarshal in golang but i am facing error.

error message -- "cannot parse invalid wire-format data".

code configuration --

python code --

schema_registry_client = SchemaRegistryClient({'url': 'http://localhost:8082'})
protobuf_serializer = ProtobufSerializer(user_attributes_pb2.UserProperties,
                                           schema_registry_client,
                                           {'use.deprecated.format': True})
  producer_conf = {'bootstrap.servers':  'localhost:9092', 'key.serializer': StringSerializer('utf_8'), 'value.serializer': protobuf_serializer}
  producer = SerializingProducer(producer_conf)
  producer.poll(0.0)
  ########## Add an address #########
  PromptForAddress(user_attr)
  producer.produce(topic=topic, key=str(uuid4()), value=user_attr)
  producer.flush()

golang code --

		Brokers: []string{brokerAddress},
		Topic:   topic,
		GroupID: "test-consumer-group",
		Logger: l,
	})
	for {
		msg, err := r.ReadMessage(ctx)
		user := &pb.UserProperties{}
		err = proto.Unmarshal(msg.Value, user)
		// client.Query() NewKey(Namespace, Set, user.Id)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("\n%s\n", proto.Message(user))
		fmt.Printf("\n%v\n", user)
		if err != nil {
			panic("could not read message " + err.Error())
		}
		// after receiving the message, log its value
		fmt.Println("received: ", string(msg.Value))
	}

proto file is similar in both languages.

答案1

得分: 0

由于您正在使用Confluent Python序列化器与其模式注册表,因此您在Go中需要执行相同的操作,而不能只使用普通的Protobuf反序列化。

Confluent还维护了一个Go客户端,您可以使用。Protobuf的示例代码可以在此处找到:https://github.com/confluentinc/confluent-kafka-go/blob/master/examples/protobuf_consumer_example/protobuf_consumer_example.go

我不确定use.deprecated.format在Python中的作用是什么,但Go可能不接受该选项。

英文:

Since you're using Confluent Python serializer with their Schema Registry, you need to do the same in Go, and cannot just use plain Protobuf deserialization.

Confluent also maintains a Go client which you can use. Example for Protobuf - https://github.com/confluentinc/confluent-kafka-go/blob/master/examples/protobuf_consumer_example/protobuf_consumer_example.go

I'm not sure what use.deprecated.format does for Python, but Go might not accept that

huangapple
  • 本文由 发表于 2022年9月7日 15:51:34
  • 转载请务必保留本文链接:https://go.coder-hub.com/73631871.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定