How to use genrules with go:embed

huangapple go评论116阅读模式
英文:

How to use genrules with go:embed

问题

我需要在二进制文件中包含一些生成的文件(例如来自go-swagger的输出)。如果没有使用bazel,你可以使用go:generatego:embed组合来实现:

package demo
//go:generate swagger generate spec -w . -o ./openapi.json

import "embed";

// Content contains openapi.json file generated via go-swagger from spec
//go:embed openapi.json
var Content embed.FS

我正在尝试使用bazel做同样的事情。作为一个简单的测试,我有一个BUILD.bazel文件:

genrule(
    name = "hellodata",
    srcs = ["hello.go"],
    outs = ["hello.txt"],
    cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $@",
)

go_library(
    name = "hello",
    srcs = ["hello.go"],
    importpath = "wiggy.net/hello",
    visibility = ["//visibility:public"],
    embedsrcs = [":hellodata"],
)

hello.go的内容如下:

package hello

import (
	_ "embed"
	"io"
)

//go:embed hello.txt
var greeting []byte

func Hello(out io.Writer) error {
	_, err := out.Write(greeting)
	return err
}

这里的意图是让Hello输出其自身源代码的rot13。当我尝试编译时,它成功生成了hello.txt(位于bazel-out/darwin_arm64-fastbuild/bin/hello.txt),但编译器找不到它:

❯ bazel build //...                              
INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured).
INFO: Found 5 targets...
ERROR: /Users/wichert/Hack/bzl/BUILD.bazel:14:11: GoCompilePkg hello.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_arm64 -src hello.go -embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt -importpath wiggy.net/hello ... (remaining 12 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: /private/var/tmp/_bazel_wichert/e7573342ee9452df4c3dfa671d399a16/sandbox/darwin-sandbox/76/execroot/__main__/hello.go:8:12: could not embed hello.txt: no matching files found
INFO: Elapsed time: 0,112s, Critical Path: 0,04s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully
英文:

I need to include some generated files (for example output from go-swagger) in a binary. Without bazel you can do that using go:generate in combination with go:embed:

package demo
//go:generate swagger generate spec -w . -o ./openapi.json

import "embed"

// Content contains openapi.json file generated via go-swagger from spec
//go:embed openapi.json
var Content embed.FS

I am trying to do the same thing with bazel. As a simple test I have have this BUILD.bazel file:

genrule(
    name = "hellodata",
    srcs = ["hello.go"],
    outs = ["hello.txt"],
    cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $@",
)

go_library(
    name = "hello",
    srcs = ["hello.go"],
    importpath = "wiggy.net/hello",
    visibility = ["//visibility:public"],
    embedsrcs = [":hellodata"],
)

with hello.go looking like this:

package hello

import (
	_ "embed"
	"io"
)

//go:embed hello.txt
var greeting []byte

func Hello(out io.Writer) error {
	_, err := out.Write(greeting)
	return err
}

The intention here is to have Hello output the rot13 of it's own source. When I try to compile this it successfully generates hello.txt (located in bazel-out/darwin_arm64-fastbuild/bin/hello.txt), but the compiler can not find it:

❯ bazel build //...                              
INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured).
INFO: Found 5 targets...
ERROR: /Users/wichert/Hack/bzl/BUILD.bazel:14:11: GoCompilePkg hello.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_arm64 -src hello.go -embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt -importpath wiggy.net/hello ... (remaining 12 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: /private/var/tmp/_bazel_wichert/e7573342ee9452df4c3dfa671d399a16/sandbox/darwin-sandbox/76/execroot/__main__/hello.go:8:12: could not embed hello.txt: no matching files found
INFO: Elapsed time: 0,112s, Critical Path: 0,04s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

答案1

得分: 0

我注意到你在问题中的命令行中使用了-embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt,所以我猜想在我的机器上尝试了等效的操作:

//go:embed bazel-out/k8-fastbuild/bin/hello.txt
var greeting []byte

这似乎起作用了。

这并不是很好,因为配置信息被嵌入到源文件中(在你的 Mac 机器上是 darwin_arm64-fastbuild,在我的 Linux 机器上是 k8-fastbuild),所以现在你的源代码人为地依赖于平台。

看起来这个特性是相对较新的(https://github.com/bazelbuild/rules_go/issues/2775,https://github.com/bazelbuild/rules_go/issues/2986)。我建议向 rules_go 提交一个问题报告。

还有一个名为 go_embed_data 的功能,可能会有不同的行为:

https://github.com/bazelbuild/rules_go/blob/master/docs/go/extras/extras.md#go_embed_data

英文:

I noticed -embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt in the command line for the failing action in your question, so as a hunch I tried the equivalent on my machine:

//go:embed bazel-out/k8-fastbuild/bin/hello.txt
var greeting []byte

And that seemed to work.

This is not so great because configuration information gets embedded in the source files (indeed, on your mac machine it's darwin_arm64-fastbuild and on my linux machine it's k8-fastbuild), so now your source code is artificially platform dependent.

It looks like this feature is relatively new (https://github.com/bazelbuild/rules_go/issues/2775, https://github.com/bazelbuild/rules_go/issues/2986). I would file an issue with rules_go about this.

There also appears to be go_embed_data which might behave differently:

https://github.com/bazelbuild/rules_go/blob/master/docs/go/extras/extras.md#go_embed_data

答案2

得分: 0

回答我的问题:诀窍是使用genrule生成要嵌入的文件,然后使用go_embed_data将它们嵌入。一个示例的BUILD.bazel文件如下所示:

genrule(
    name = "hellodata",
    srcs = ["hello.go"],
    outs = ["hello.txt"],
    cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $@",
)

go_embed_data(
    name = "hello_embed",
    src = ":hellodata",
    package = "hello",
    var = "greeting",
)

go_library(
    name = "hello",
    srcs = [
        "hello.go",
        ":hello_embed",
    ],
    importpath = "wiggy.net/hello",
    visibility = ["//visibility:public"],
)
英文:

To answer my own question: the trick is to use genrule to generate the file(s) to be embedded, and then use go_embed_data to embed them. A work BUILD.bazel looks like this:

genrule(
    name = "hellodata",
    srcs = ["hello.go"],
    outs = ["hello.txt"],
    cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $@",
)

go_embed_data(
    name = "hello_embed",
    src = ":hellodata",
    package = "hello",
    var = "greeting",
)

go_library(
    name = "hello",
    srcs = [
        "hello.go",
        ":hello_embed",
    ],
    importpath = "wiggy.net/hello",
    visibility = ["//visibility:public"],
)

huangapple
  • 本文由 发表于 2021年12月29日 16:44:38
  • 转载请务必保留本文链接:https://go.coder-hub.com/70517054.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定