Golang项目Protobuf导入损坏

huangapple go评论78阅读模式
英文:

Golang project Protobuf corrupted imports

问题

我不是一个 Golang 开发者,但是目前我必须修复其中的代码,所以如果我不小心没有理解一些基本的 Go 概念,请提前原谅;)我有一个第三方的 protobuf 合约,我必须使用它,但我对它没有影响力。我无法提供合约的实际示例,所以我在 Github 上创建了一个类似的示例项目,它有相同的问题。
简而言之:有一个深层嵌套的 proto 文档结构,其中一些文档引用了其他文档:

syntax = "proto3";

package company.nested1.nested2;

import "company/common.proto";

option go_package = "nested2";

message CompanyMessage {
    CompanyEnum compEnum = 1;
}

引用:

syntax = "proto3";

package company;

option go_package = "company";

enum CompanyEnum {
    VAR1 = 0;
    VAR2 = 1;
}

有一个带有以下代码的 go.mod 文件:

module go-program

go 1.16

require (
	github.com/golang/protobuf v1.5.2
	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
	google.golang.org/grpc v1.40.0 // indirect
	google.golang.org/protobuf v1.26.0
)

当编译为 Golang 代码时,这个导入会出现问题。生成的 *.pb.go 文件的导入相对于 company 包,而它们在以 go-project 为根的 go 项目中使用,所以这些导入无法解析:
Golang项目Protobuf导入损坏

据我了解,以前这个问题是通过手动编辑生成的文件中的导入来解决的。就像下面的屏幕截图一样。但我认为应该有更好的方法来解决这个问题;)

Golang项目Protobuf导入损坏

我使用以下命令从 proto 文件生成 go 代码:

protoc --proto_path=proto (find proto -name '*.proto') --go_out=plugins=grpc:. 

所以问题是:在 go 中解决这种导入问题的正确方法是什么。似乎生成的文件中的导入应该以我的程序根命名空间开头。但如何实现这一点 - 也许是 protoc 命令中的一些选项,我还没有找到。我还尝试将生成的文件放在 $GOPATH/srcvendors 中,并尝试制作和发布单独的模块 - 没有结果。是否有一些适当的解决方案?

英文:

I'm not a golang developer but currently I have to fix code on it, so sorry in advance if I accidentally not understand some basic Go's concept Golang项目Protobuf导入损坏 I have a third party protobuf contract which I have to use and on which I don't have influence. I'm not able to provide actual example of the contract, so I've made similar example project on Github which has the same problem.
In short: there is a deeply nested structure of proto documents where some of them import others:

syntax = "proto3";

package company.nested1.nested2;

import "company/common.proto";

option go_package = "nested2";

message CompanyMessage {
    CompanyEnum compEnum = 1;
}

references

syntax = "proto3";

package company;

option go_package = "company";

enum CompanyEnum {
    VAR1 = 0;
    VAR2 = 1;
}

There is a go.mod file with following code:

module go-program

go 1.16

require (
	github.com/golang/protobuf v1.5.2
	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
	google.golang.org/grpc v1.40.0 // indirect
	google.golang.org/protobuf v1.26.0
)

And when compiling to the Golang code there is a problem in this import. Generated *.pb.go files have imports relative to company package while they are used inside of a go project with root in go-project so these imports can't be resolved:
Golang项目Protobuf导入损坏

As I understand, previously this problem has been solved by manual editing of imports in generated files. Like in the screen below. But in my opinion there should be a better way for this Golang项目Protobuf导入损坏

Golang项目Protobuf导入损坏

I'm using this command to generate go code from proto files:

protoc --proto_path=proto (find proto -name '*.proto') --go_out=plugins=grpc:. 

So the question is: what is a proper way to solve such import problem in go. Seems like imports in generated files should begin with my program root namespace. But how to achieve this - maybe some options in protoc command which I haven't found. I tried also to put generated files inside of $GOPATH/src or in vendors and tried to make and publish separate module - no result. Is there some proper solution for this?

答案1

得分: 3

这更像是一些建议,而不是一个明确的答案。如果我明天有时间,我会重新生成代码,给你一个明确的答案。

这是一个具有挑战性的问题。Protobufs必须为每种语言的包管理找到解决方案,而Go的包管理(虽然我认为已经改进了)在最近添加模块的情况下有些复杂。

我的方法是在代码中为proto文件和生成的代码使用不同的存储库。如果proto文件发生更改,代码将被重新生成。我可以引用生成的模块(!)或自己重新生成。这种模式将proto文件保持为明确的“源”,并具有“缓存”的生成代码,节省时间。但在这种配置中,我是自己的第三方库维护者。

所以:

  1. 模块在本地作为一个目录树进行管理,根目录包含go.modgo.sum文件,子目录表示包。
  2. 当你为第三方proto文件生成代码时,最好将其创建为它们自己的模块,可以作为一个独立的目录或作为一个供应商子目录。
  3. 这个模块应该以第三方的名称命名,并可以使用go mod init命令创建,应包含protoc的输出。
  4. 模块通常从代理或仓库中获取,但你可以通过手动添加replacego mod文件中,将模块路径重定向到本地路径,从而绕过这个问题(因为第三方没有为你提供生成的代码)。
  5. 根据你的错误信息,这应该解决了缺少模块的问题,并且使用一个子目录(例如company)包含生成的代码,应该也能解决包的问题。
英文:

This is more some pointers rather than a definitive answer. If I find some time tomorrow, I'll repro this to give you a definitive answer.

It is challenging. Protobufs has to find a solution for each language's package management and Go's (while IMO improved) is somewhat compounded with recent addition of Modules.

My approach my with code is to have a separate repo for the protos and generated code. If the protos change, the code is regenerated. I can either reference the generated module (!) or regenerate myself. This model keeps the protos as the definitive "source" with the timesaver of "cached" generated sources. But I'm my own 3rd-party library maintainer in this config.

So:

  1. Modules are manifest locally as a directory tree whose root contains go.mod and go.sum and subsidirectories representing packages.
  2. When you generate the code for the 3rd-party protos, I think it would be best to create this under their own module, either as a standalone directory or as a vendored subdirectory.
  3. This module should be named to reference the 3rd party and can be created with go mod init and should contain the output of protoc
  4. Modules are usually pulled from a proxy or repo but you can circumvent this (since the 3rd party isn't hosting the generated code for you) by manually adding a replace to the go mod redirecting from the module path to a local path.
  5. From your error, this should solve the missing module issue and with a subdirectory e.g. company containing that generated code, should resolve the package too.

huangapple
  • 本文由 发表于 2021年8月15日 00:15:15
  • 转载请务必保留本文链接:https://go.coder-hub.com/68785069.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定