构建所有已下载的 Golang 包。

huangapple go评论87阅读模式
英文:

Golang build all downloaded packages

问题

我正在尝试为我的Go应用程序构建一个Docker镜像,并且我希望在编译应用程序的源代码之前单独构建所有的包依赖项。

这里有一个可能有帮助的示例:

FROM ubuntu:focal

# ... 一堆运行命令,安装我需要的Go和其他二进制文件

ENV GOPATH=/root/go
WORKDIR gadic

COPY go.mod .
RUN go mod download
RUN go build /root/go/pkg/... # 这一步失败了!!

COPY src src
RUN go build path/to/main.go # 我希望通过预编译main.go所依赖的包来加快这一步

当我尝试构建这个Dockerfile时,我得到了以下输出:

 > [ 8/19] RUN go build /root/go/pkg/...:
#12 0.955 pattern /root/go/pkg/...: directory prefix /root/go/pkg outside available modules

有没有办法在构建应用程序的main.go之前构建应用程序依赖的包/模块?

英文:

I'm trying to build a docker image for my go application and I'd like to build all of the package dependencies separately from compiling the source code of my application.

Here's an example that might be helpful:

FROM ubuntu:focal

# ... a bunch of run commands that install go and other binaries I need

ENV GOPATH=/root/go
WORKDIR gadic

COPY go.mod .
RUN go mod download
RUN go build /root/go/pkg/... # this fails!!

COPY src src
RUN go build path/to/main.go # I'd like to speed this step up by pre-compiling the packages main.go depends on

When I try to build this dockerfile I get the following output:

 > [ 8/19] RUN go build /root/go/pkg/...:
#12 0.955 pattern /root/go/pkg/...: directory prefix /root/go/pkg outside available modules

Is there a way to build the packages/modules that my application depends on prior to building the main.go of my application?

答案1

得分: 3

你真正需要的是一个构建缓存,参见这里

有两个文件夹需要挂载到构建容器中:

  • GOPATH

    这里我们设置为/root/go,用于存储下载的依赖包。

  • GOCACHE

    默认为/root/.cache/go-build,用于构建缓存。

下面是一个最简示例。

Dockerfile:

# syntax = docker/dockerfile:1.2
FROM golang
ENV GOPATH=/root/go
WORKDIR /root
COPY main.go .
COPY go.mod .
COPY go.sum .

RUN --mount=type=cache,mode=0755,target=/root/.cache/go-build --mount=type=cache,mode=0755,target=/root/go go build -v main.go

main.go:

package main

import (
     _ "github.com/phachon/go-logger"
)

func main() {
}

go.mod:

module trial

go 1.14

require github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea

go.sum:

github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea h1:IkOONr/u7Wy+j2R4r1eMV8PEuN4kmOhZZNaYxDOF+KQ=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea/go.mod h1:WBIWFH/iYYvuApCvPU+/R6hfX6v0Ogu4apwf0UgzVF0=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

执行:

  • 第一次,你可以看到它下载了依赖包,然后构建了不同的包。
$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 3.100 go: downloading github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea
#14 3.330 go: downloading github.com/fatih/color v1.7.0
#14 3.331 go: downloading github.com/mailru/easyjson v0.7.0
#14 3.523 go: downloading github.com/mattn/go-colorable v0.1.4
#14 3.561 go: downloading github.com/mattn/go-isatty v0.0.11
#14 3.731 go: downloading golang.org/x/sys v0.0.0-20191026070338-33540a1f6037
#14 5.058 github.com/mailru/easyjson/buffer
#14 5.063 golang.org/x/sys/unix
#14 5.070 github.com/mailru/easyjson/jlexer
#14 5.077 github.com/mailru/easyjson/jwriter
#14 5.087 github.com/phachon/go-logger/utils
#14 5.174 github.com/mailru/easyjson
#14 5.346 github.com/mattn/go-isatty
#14 5.355 github.com/mattn/go-colorable
#14 5.368 github.com/fatih/color
#14 5.396 github.com/phachon/go-logger
#14 5.451 command-line-arguments
#14 DONE 7.4s
...
  • 第二次,我将main.go更改为以下内容:

main.go:

package main

import (
     _ "github.com/phachon/go-logger"
     "fmt"
)

func main() {
    fmt.Println("helloworld")
}

然后,再次执行构建:

$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 1.469 command-line-arguments
#14 DONE 3.2s
...

你可以看到,它不再下载包(因为在不同的构建过程中重用了/root/go),只构建了command-line-arguments,对应于main.go的更改,没有重新构建与github.com/phachon/go-logger相关的包(因为/root/.cache/go-build作为构建缓存被重用)。

英文:

What you really need is a build cache, see this.

There are two folders needed to be mounted into build container:

  • GOPATH

    Here we set as /root/go, used to store downloaded dependency

  • GOCACHE

    Default is /root/.cache/go-build, used for build cache

A minimal example as next.

Dockerfile:

# syntax = docker/dockerfile:1.2
FROM golang
ENV GOPATH=/root/go
WORKDIR /root
COPY main.go .
COPY go.mod .
COPY go.sum .
RUN --mount=type=cache,mode=0755,target=/root/.cache/go-build --mount=type=cache,mode=0755,target=/root/go go build -v main.go

main.go:

package main
import (
_ "github.com/phachon/go-logger"
)
func main() {
}

go.mod:

module trial
go 1.14
require github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea

go.sum:

github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea h1:IkOONr/u7Wy+j2R4r1eMV8PEuN4kmOhZZNaYxDOF+KQ=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea/go.mod h1:WBIWFH/iYYvuApCvPU+/R6hfX6v0Ogu4apwf0UgzVF0=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

Execution:

  • 1st time, you could see it downloads the dependency package, and then build different packages.
$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 3.100 go: downloading github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea
#14 3.330 go: downloading github.com/fatih/color v1.7.0
#14 3.331 go: downloading github.com/mailru/easyjson v0.7.0
#14 3.523 go: downloading github.com/mattn/go-colorable v0.1.4
#14 3.561 go: downloading github.com/mattn/go-isatty v0.0.11
#14 3.731 go: downloading golang.org/x/sys v0.0.0-20191026070338-33540a1f6037
#14 5.058 github.com/mailru/easyjson/buffer
#14 5.063 golang.org/x/sys/unix
#14 5.070 github.com/mailru/easyjson/jlexer
#14 5.077 github.com/mailru/easyjson/jwriter
#14 5.087 github.com/phachon/go-logger/utils
#14 5.174 github.com/mailru/easyjson
#14 5.346 github.com/mattn/go-isatty
#14 5.355 github.com/mattn/go-colorable
#14 5.368 github.com/fatih/color
#14 5.396 github.com/phachon/go-logger
#14 5.451 command-line-arguments
#14 DONE 7.4s
...
  • 2nd time, I changed the main.go as next:

main.go:

package main
import (
_ "github.com/phachon/go-logger"
"fmt"
)
func main() {
fmt.Println("helloworld")
}

Then, execute build again:

$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 1.469 command-line-arguments
#14 DONE 3.2s
...

You could see from above, it no longer downloads package (because /root/go was reused during different times build). And, it just build command-line-arguments, corresponding to the change of main.go, no packages related to github.com/phachon/go-logger been rebuilt (because /root/.cache/go-build as build cache be reused).

答案2

得分: 1

我认为你应该利用multi-stage builds的优势。在第一阶段,你可以使用已安装Go的Docker镜像。你可以在这里选择其中一个可用的镜像:https://hub.docker.com/_/golang。此时,你需要将源代码复制到容器中,并在容器中运行go build

在下一步中,你可以使用alpine镜像,并将可执行文件(以及其他所需的文件)复制到目标容器中。

以下是一个示例:

FROM golang:alpine AS build-env
RUN apk --no-cache add build-base git bzr mercurial gcc
ADD . /src
RUN cd /src && go build -o goapp

# final stage
FROM alpine
WORKDIR /app
COPY --from=build-env /src/goapp /app/
ENTRYPOINT ./goapp

这样做有几个好处,其中之一就是最终镜像的大小会更小。

英文:

I think you should take the advantage of the multi-stage builds. In the first stage, you use the docker image with Go installed. You can use one of those available here https://hub.docker.com/_/golang. At this point, you have to copy your source code into the container and run go build there.

In the next step, you (for example) use an alpine image and copy the executable (and other required fields) to the target container.

This is an example how it can be done.

FROM golang:alpine AS build-env
RUN apk --no-cache add build-base git bzr mercurial gcc
ADD . /src
RUN cd /src && go build -o goapp
# final stage
FROM alpine
WORKDIR /app
COPY --from=build-env /src/goapp /app/
ENTRYPOINT ./goapp

This has several benefits and the size of the final image is one of them.

huangapple
  • 本文由 发表于 2021年6月18日 04:00:32
  • 转载请务必保留本文链接:https://go.coder-hub.com/68025858.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定