英文:
Golang project Protobuf corrupted imports
问题
我不是一个 Golang 开发者,但是目前我必须修复其中的代码,所以如果我不小心没有理解一些基本的 Go 概念,请提前原谅;)我有一个第三方的 protobuf
合约,我必须使用它,但我对它没有影响力。我无法提供合约的实际示例,所以我在 Github 上创建了一个类似的示例项目,它有相同的问题。
简而言之:有一个深层嵌套的 proto
文档结构,其中一些文档引用了其他文档:
syntax = "proto3";
package company.nested1.nested2;
import "company/common.proto";
option go_package = "nested2";
message CompanyMessage {
CompanyEnum compEnum = 1;
}
引用:
syntax = "proto3";
package company;
option go_package = "company";
enum CompanyEnum {
VAR1 = 0;
VAR2 = 1;
}
有一个带有以下代码的 go.mod
文件:
module go-program
go 1.16
require (
github.com/golang/protobuf v1.5.2
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/grpc v1.40.0 // indirect
google.golang.org/protobuf v1.26.0
)
当编译为 Golang 代码时,这个导入会出现问题。生成的 *.pb.go
文件的导入相对于 company
包,而它们在以 go-project
为根的 go 项目中使用,所以这些导入无法解析:
据我了解,以前这个问题是通过手动编辑生成的文件中的导入来解决的。就像下面的屏幕截图一样。但我认为应该有更好的方法来解决这个问题;)
我使用以下命令从 proto 文件生成 go 代码:
protoc --proto_path=proto (find proto -name '*.proto') --go_out=plugins=grpc:.
所以问题是:在 go 中解决这种导入问题的正确方法是什么。似乎生成的文件中的导入应该以我的程序根命名空间开头。但如何实现这一点 - 也许是 protoc
命令中的一些选项,我还没有找到。我还尝试将生成的文件放在 $GOPATH/src
或 vendors
中,并尝试制作和发布单独的模块 - 没有结果。是否有一些适当的解决方案?
英文:
I'm not a golang developer but currently I have to fix code on it, so sorry in advance if I accidentally not understand some basic Go's concept I have a third party protobuf
contract which I have to use and on which I don't have influence. I'm not able to provide actual example of the contract, so I've made similar example project on Github which has the same problem.
In short: there is a deeply nested structure of proto
documents where some of them import others:
syntax = "proto3";
package company.nested1.nested2;
import "company/common.proto";
option go_package = "nested2";
message CompanyMessage {
CompanyEnum compEnum = 1;
}
references
syntax = "proto3";
package company;
option go_package = "company";
enum CompanyEnum {
VAR1 = 0;
VAR2 = 1;
}
There is a go.mod
file with following code:
module go-program
go 1.16
require (
github.com/golang/protobuf v1.5.2
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/grpc v1.40.0 // indirect
google.golang.org/protobuf v1.26.0
)
And when compiling to the Golang code there is a problem in this import. Generated *.pb.go
files have imports relative to company
package while they are used inside of a go project with root in go-project
so these imports can't be resolved:
As I understand, previously this problem has been solved by manual editing of imports in generated files. Like in the screen below. But in my opinion there should be a better way for this
I'm using this command to generate go code from proto files:
protoc --proto_path=proto (find proto -name '*.proto') --go_out=plugins=grpc:.
So the question is: what is a proper way to solve such import problem in go. Seems like imports in generated files should begin with my program root namespace. But how to achieve this - maybe some options in protoc
command which I haven't found. I tried also to put generated files inside of $GOPATH/src
or in vendors
and tried to make and publish separate module - no result. Is there some proper solution for this?
答案1
得分: 3
这更像是一些建议,而不是一个明确的答案。如果我明天有时间,我会重新生成代码,给你一个明确的答案。
这是一个具有挑战性的问题。Protobufs必须为每种语言的包管理找到解决方案,而Go的包管理(虽然我认为已经改进了)在最近添加模块的情况下有些复杂。
我的方法是在代码中为proto文件和生成的代码使用不同的存储库。如果proto文件发生更改,代码将被重新生成。我可以引用生成的模块(!)或自己重新生成。这种模式将proto文件保持为明确的“源”,并具有“缓存”的生成代码,节省时间。但在这种配置中,我是自己的第三方库维护者。
所以:
- 模块在本地作为一个目录树进行管理,根目录包含
go.mod
和go.sum
文件,子目录表示包。 - 当你为第三方proto文件生成代码时,最好将其创建为它们自己的模块,可以作为一个独立的目录或作为一个供应商子目录。
- 这个模块应该以第三方的名称命名,并可以使用
go mod init
命令创建,应包含protoc
的输出。 - 模块通常从代理或仓库中获取,但你可以通过手动添加
replace
到go mod
文件中,将模块路径重定向到本地路径,从而绕过这个问题(因为第三方没有为你提供生成的代码)。 - 根据你的错误信息,这应该解决了缺少模块的问题,并且使用一个子目录(例如
company
)包含生成的代码,应该也能解决包的问题。
英文:
This is more some pointers rather than a definitive answer. If I find some time tomorrow, I'll repro this to give you a definitive answer.
It is challenging. Protobufs has to find a solution for each language's package management and Go's (while IMO improved) is somewhat compounded with recent addition of Modules.
My approach my with code is to have a separate repo for the protos and generated code. If the protos change, the code is regenerated. I can either reference the generated module (!) or regenerate myself. This model keeps the protos as the definitive "source" with the timesaver of "cached" generated sources. But I'm my own 3rd-party library maintainer in this config.
So:
- Modules are manifest locally as a directory tree whose root contains
go.mod
andgo.sum
and subsidirectories representing packages. - When you generate the code for the 3rd-party protos, I think it would be best to create this under their own module, either as a standalone directory or as a vendored subdirectory.
- This module should be named to reference the 3rd party and can be created with
go mod init
and should contain the output ofprotoc
- Modules are usually pulled from a proxy or repo but you can circumvent this (since the 3rd party isn't hosting the generated code for you) by manually adding a
replace
to thego mod
redirecting from the module path to a local path. - From your error, this should solve the missing module issue and with a subdirectory e.g.
company
containing that generated code, should resolve the package too.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论