在单个proto文件中导入具有相同消息名称的多个protobuf文件。

huangapple go评论82阅读模式
英文:

Import multiple protobuf files with common message names in a single proto file

问题

我有几个共享一些消息名称的protobuf文件,因此我不能将所有生成的go文件放在一个文件夹中,否则会遇到诸如“..在此块中重新声明”之类的问题。我附上了一个最小化的go项目,以重现此问题。项目的结构如下所示:

protos/myMessages/Folder0/MyMessage0.proto

syntax = "proto2";

package myMessages.Folder0;

option go_package = "myMessages/Folder0";

message Message0 {
    optional uint32 payload = 1;
}

protos/myMessages/Folder1/MyMessage1.proto

syntax = "proto2";

package myMessages.Folder1;

option go_package = "myMessages/Folder1";

message Message0 {
    optional uint32 payload = 1;
}

protos/myMessages/helloworld.proto

syntax = "proto2";

package myMessages;

option go_package = "./;gen";

import "myMessages/Folder0/MyMessage0.proto";
import "myMessages/Folder1/MyMessage1.proto";

service Greeter {
  rpc SayHello1 (myMessages.Folder0.Message0) returns (myMessages.Folder0.Message0) {}
  rpc SayHello2 (myMessages.Folder1.Message0) returns (myMessages.Folder1.Message0) {}
}

main.go

package main

import (
	"context"
	"flag"
	"fmt"
	"log"
	"net"

	"google.golang.org/grpc"
	
	pb0 "main/build/myMessages/Folder0"
	pb1 "main/build/myMessages/Folder1"
	pb "main/build"
)

var (
	port = flag.Int("port", 50051, "The server port")
)

// server is used to implement helloworld.GreeterServer.
type server struct {
	pb.UnimplementedGreeterServer
}

func (s *server) SayHello1(ctx context.Context, in *pb0.Message0) (*pb0.Message0, error) {
	return &pb0.Message0{}, nil
}

func (s *server) SayHello2(ctx context.Context, in *pb1.Message0) (*pb1.Message0, error) {
	return &pb1.Message0{}, nil
}

func main() {
	flag.Parse()
	lis, err := net.Listen("tcp", fmt.Sprintf(":%d", *port))
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterGreeterServer(s, &server{})
	log.Printf("server listening at %v", lis.Addr())
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

Makefile

all: build protos

protos:
	protoc -Iprotos --go_out=build protos/myMessages/Folder0/MyMessage0.proto
	protoc -Iprotos --go_out=build protos/myMessages/Folder1/MyMessage1.proto
	protoc -Iprotos --go_out=build --go-grpc_out=build protos/helloworld.proto
	go build -o build/server main.go

build:
	mkdir -p build

运行make命令后,我得到以下错误:

mkdir -p build
protoc -Iprotos --go_out=build protos/myMessages/Folder0/MyMessage0.proto
protoc -Iprotos --go_out=build protos/myMessages/Folder1/MyMessage1.proto
protoc -Iprotos --go_out=build --go-grpc_out=build protos/helloworld.proto
go build -o build/server main.go
build/helloworld.pb.go:12:2: package myMessages/Folder0 is not in GOROOT (/home/andrei/env/gosdk/go1.17.2/src/myMessages/Folder0)
build/helloworld.pb.go:13:2: package myMessages/Folder1 is not in GOROOT (/home/andrei/env/gosdk/go1.17.2/src/myMessages/Folder1)
make: *** [Makefile:21: protos] Error 1

生成的build/helloworld.pb.gobuild/helloworld_grpc.pb.go文件具有以下导入语句:

import (
	Folder0 "myMessages/Folder0"
	Folder1 "myMessages/Folder1"
)

但是,如果我将其更改为以下内容,则可以正常工作:

import (
	Folder0 "main/build/myMessages/Folder0"
	Folder1 "main/build/myMessages/Folder1"
)

有什么正确的方法来组织这段代码,以避免在生成的文件中手动添加main/build前缀的导入语句?

英文:

I have several protobuf files that share some message names and for this reason I cannot have all the generated go files in one folder as I run into issues like "..re-declared in this block". I attached a minimal go project how to reproduce this issue. The structure of the project is like below:

在单个proto文件中导入具有相同消息名称的多个protobuf文件。

protos/myMessages/Folder0/MyMessage0.proto

syntax = "proto2";

package myMessages.Folder0;

option go_package = "myMessages/Folder0";

message Message0 {
    optional uint32 payload = 1;
}

protos/myMessages/Folder1/MyMessage1.proto

syntax = "proto2";

package myMessages.Folder1;

option go_package = "myMessages/Folder1";

message Message0 {
    optional uint32 payload = 1;
}

protos/myMessages/helloworld.proto

syntax = "proto2";

package myMessages;

option go_package = "./;gen";

import "myMessages/Folder0/MyMessage0.proto";
import "myMessages/Folder1/MyMessage1.proto";

service Greeter {
  rpc SayHello1 (myMessages.Folder0.Message0) returns (myMessages.Folder0.Message0) {}
  rpc SayHello2 (myMessages.Folder1.Message0) returns (myMessages.Folder1.Message0) {}
}

main.go

package main

import (
	"context"
	"flag"
	"fmt"
	"log"
	"net"

	"google.golang.org/grpc"
	
	pb0 "main/build/myMessages/Folder0"
	pb1 "main/build/myMessages/Folder1"
	pb "main/build"
)

var (
	port = flag.Int("port", 50051, "The server port")
)

// server is used to implement helloworld.GreeterServer.
type server struct {
	pb.UnimplementedGreeterServer
}

func (s *server) SayHello1(ctx context.Context, in *pb0.Message0) (*pb0.Message0, error) {
	return &pb0.Message0{}, nil
}

func (s *server) SayHello2(ctx context.Context, in *pb1.Message0) (*pb1.Message0, error) {
	return &pb1.Message0{}, nil
}

func main() {
	flag.Parse()
	lis, err := net.Listen("tcp", fmt.Sprintf(":%d", *port))
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterGreeterServer(s, &server{})
	log.Printf("server listening at %v", lis.Addr())
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

Makefile

all: build protos

protos:
	protoc -Iprotos --go_out=build protos/myMessages/Folder0/MyMessage0.proto
	protoc -Iprotos --go_out=build protos/myMessages/Folder1/MyMessage1.proto
	protoc -Iprotos --go_out=build --go-grpc_out=build protos/helloworld.proto
	go build -o build/server main.go

build:
	mkdir -p build

After I run the make command, I get the error below:

mkdir -p build
protoc -Iprotos --go_out=build protos/myMessages/Folder0/MyMessage0.proto
protoc -Iprotos --go_out=build protos/myMessages/Folder1/MyMessage1.proto
protoc -Iprotos --go_out=build --go-grpc_out=build protos/helloworld.proto
go build -o build/server main.go
build/helloworld.pb.go:12:2: package myMessages/Folder0 is not in GOROOT (/home/andrei/env/gosdk/go1.17.2/src/myMessages/Folder0)
build/helloworld.pb.go:13:2: package myMessages/Folder1 is not in GOROOT (/home/andrei/env/gosdk/go1.17.2/src/myMessages/Folder1)
make: *** [Makefile:21: protos] Error 1

The generated build/helloworld.pb.go and build/helloworld_grpc.pb.go files have the following imports:

import (
	Folder0 "myMessages/Folder0"
	Folder1 "myMessages/Folder1"
)

But if I change these to the following then it works:

import (
	Folder0 "main/build/myMessages/Folder0"
	Folder1 "main/build/myMessages/Folder1"
)

What's the correct way to organise this code to avoid manually adding the main/build prefix to the imports in the generated files?

答案1

得分: 1

我能够通过以下步骤使其工作:

1. 从go.mod中获取你的模块名称。在这个例子中,我假设它是hello-world

module hello-world
...

2. 配置要构建的proto文件作为你的模块的子包

protos/myMessages/Folder0/MyMessage0.proto

...
option go_package = "hello-world/myMessages/Folder0";
...

protos/myMessages/Folder1/MyMessage1.proto

...
option go_package = "hello-world/myMessages/Folder1";
...

protos/myMessages/helloworld.proto

...
option go_package = "hello-world/myMessages";
...

3. 使用source_relative路径在项目目录中生成proto代码

protoc -Iprotos --go_out=. --go_opt=paths=source_relative protos/myMessages/Folder0/MyMessage0.proto
protoc -Iprotos --go_out=. --go_opt=paths=source_relative protos/myMessages/Folder1/MyMessage1.proto
protoc -Iprotos --go_out=. --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative --go-grpc_out=. protos/myMessages/helloworld.proto

你也可以配置这些和go_package路径来使用build目录:在这一步中使用--go_out=build,并将之前和之后的步骤中的go package路径更改为"hello-world/build/myMessages/<something>"

然而,我认为build更适合构建产品(而不是生成的代码),而且这不是官方示例代码中的做法:https://developers.google.com/protocol-buffers/docs/gotutorial

4. 根据需要导入myMessages子模块

main.go

package main

import (
	...

	pb "hello-world/myMessages"
	pb0 "hello-world/myMessages/Folder0"
	pb1 "hello-world/myMessages/Folder1"
)
...

我还建议使用proto3而不是proto2,参考:https://cloud.google.com/apis/design/proto3

为了简化开发者体验和提高运行时效率,gRPC API应该使用Protocol Buffers版本3(proto3)进行API定义。

我还建议使用buf工具和/或使用bazel进行构建,而不是直接调用protoc,以避免在将来遇到这些问题时进行调试。

英文:

I was able to get this to work by doing the following:

1. Get your module name from go.mod. In this instance, I'll assume it's hello-world

module hello-world
...

2. Configure the protos to build as subpackages of your module

protos/myMessages/Folder0/MyMessage0.proto

...
option go_package = &quot;hello-world/myMessages/Folder0&quot;;
...

protos/myMessages/Folder1/MyMessage1.proto

...
option go_package = &quot;hello-world/myMessages/Folder1&quot;;
...

protos/myMessages/helloworld.proto

...
option go_package = &quot;hello-world/myMessages&quot;;
...

3. Generate the proto code in your project directory using source_relative paths

protoc -Iprotos --go_out=. --go_opt=paths=source_relative protos/myMessages/Folder0/MyMessage0.proto
protoc -Iprotos --go_out=. --go_opt=paths=source_relative protos/myMessages/Folder1/MyMessage1.proto
protoc -Iprotos --go_out=. --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative --go-grpc_out=. protos/myMessages/helloworld.proto

You could also configure these and the go_package paths to use the build directory: use --go_out=build in this step, and change the go package paths in the previous and following steps to &quot;hello-world/build/myMessages/&lt;something&gt;&quot;.

However, I think build is better for build products (not generated code), and it isn't how they do it in the official example code: https://developers.google.com/protocol-buffers/docs/gotutorial

4. Import the myMessages submodules as necessary

main.go

package main

import (
	...

	pb &quot;hello-world/myMessages&quot;
	pb0 &quot;hello-world/myMessages/Folder0&quot;
	pb1 &quot;hello-world/myMessages/Folder1&quot;
)
...

I also suggest using proto3 instead of proto2 per: https://cloud.google.com/apis/design/proto3

> To simplify developer experience and improve runtime efficiency, gRPC APIs should use Protocol Buffers version 3 (proto3) for API definition.

I also suggest using buf tools and/or building with bazel instead of calling protoc directly to avoid chasing these problems around in the future.

huangapple
  • 本文由 发表于 2022年9月6日 01:30:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/73612884.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定