英文:
Golang build all downloaded packages
问题
我正在尝试为我的Go应用程序构建一个Docker镜像,并且我希望在编译应用程序的源代码之前单独构建所有的包依赖项。
这里有一个可能有帮助的示例:
FROM ubuntu:focal
# ... 一堆运行命令,安装我需要的Go和其他二进制文件
ENV GOPATH=/root/go
WORKDIR gadic
COPY go.mod .
RUN go mod download
RUN go build /root/go/pkg/... # 这一步失败了!!
COPY src src
RUN go build path/to/main.go # 我希望通过预编译main.go所依赖的包来加快这一步
当我尝试构建这个Dockerfile时,我得到了以下输出:
> [ 8/19] RUN go build /root/go/pkg/...:
#12 0.955 pattern /root/go/pkg/...: directory prefix /root/go/pkg outside available modules
有没有办法在构建应用程序的main.go
之前构建应用程序依赖的包/模块?
英文:
I'm trying to build a docker image for my go application and I'd like to build all of the package dependencies separately from compiling the source code of my application.
Here's an example that might be helpful:
FROM ubuntu:focal
# ... a bunch of run commands that install go and other binaries I need
ENV GOPATH=/root/go
WORKDIR gadic
COPY go.mod .
RUN go mod download
RUN go build /root/go/pkg/... # this fails!!
COPY src src
RUN go build path/to/main.go # I'd like to speed this step up by pre-compiling the packages main.go depends on
When I try to build this dockerfile I get the following output:
> [ 8/19] RUN go build /root/go/pkg/...:
#12 0.955 pattern /root/go/pkg/...: directory prefix /root/go/pkg outside available modules
Is there a way to build the packages/modules that my application depends on prior to building the main.go
of my application?
答案1
得分: 3
你真正需要的是一个构建缓存,参见这里。
有两个文件夹需要挂载到构建容器中:
-
GOPATH
这里我们设置为
/root/go
,用于存储下载的依赖包。 -
GOCACHE
默认为
/root/.cache/go-build
,用于构建缓存。
下面是一个最简示例。
Dockerfile:
# syntax = docker/dockerfile:1.2
FROM golang
ENV GOPATH=/root/go
WORKDIR /root
COPY main.go .
COPY go.mod .
COPY go.sum .
RUN --mount=type=cache,mode=0755,target=/root/.cache/go-build --mount=type=cache,mode=0755,target=/root/go go build -v main.go
main.go:
package main
import (
_ "github.com/phachon/go-logger"
)
func main() {
}
go.mod:
module trial
go 1.14
require github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea
go.sum:
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea h1:IkOONr/u7Wy+j2R4r1eMV8PEuN4kmOhZZNaYxDOF+KQ=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea/go.mod h1:WBIWFH/iYYvuApCvPU+/R6hfX6v0Ogu4apwf0UgzVF0=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
执行:
- 第一次,你可以看到它下载了依赖包,然后构建了不同的包。
$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 3.100 go: downloading github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea
#14 3.330 go: downloading github.com/fatih/color v1.7.0
#14 3.331 go: downloading github.com/mailru/easyjson v0.7.0
#14 3.523 go: downloading github.com/mattn/go-colorable v0.1.4
#14 3.561 go: downloading github.com/mattn/go-isatty v0.0.11
#14 3.731 go: downloading golang.org/x/sys v0.0.0-20191026070338-33540a1f6037
#14 5.058 github.com/mailru/easyjson/buffer
#14 5.063 golang.org/x/sys/unix
#14 5.070 github.com/mailru/easyjson/jlexer
#14 5.077 github.com/mailru/easyjson/jwriter
#14 5.087 github.com/phachon/go-logger/utils
#14 5.174 github.com/mailru/easyjson
#14 5.346 github.com/mattn/go-isatty
#14 5.355 github.com/mattn/go-colorable
#14 5.368 github.com/fatih/color
#14 5.396 github.com/phachon/go-logger
#14 5.451 command-line-arguments
#14 DONE 7.4s
...
- 第二次,我将
main.go
更改为以下内容:
main.go:
package main
import (
_ "github.com/phachon/go-logger"
"fmt"
)
func main() {
fmt.Println("helloworld")
}
然后,再次执行构建:
$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 1.469 command-line-arguments
#14 DONE 3.2s
...
你可以看到,它不再下载包(因为在不同的构建过程中重用了/root/go
),只构建了command-line-arguments
,对应于main.go
的更改,没有重新构建与github.com/phachon/go-logger
相关的包(因为/root/.cache/go-build
作为构建缓存被重用)。
英文:
What you really need is a build cache, see this.
There are two folders needed to be mounted into build container:
-
GOPATH
Here we set as
/root/go
, used to store downloaded dependency -
GOCACHE
Default is
/root/.cache/go-build
, used for build cache
A minimal example as next.
Dockerfile:
# syntax = docker/dockerfile:1.2
FROM golang
ENV GOPATH=/root/go
WORKDIR /root
COPY main.go .
COPY go.mod .
COPY go.sum .
RUN --mount=type=cache,mode=0755,target=/root/.cache/go-build --mount=type=cache,mode=0755,target=/root/go go build -v main.go
main.go:
package main
import (
_ "github.com/phachon/go-logger"
)
func main() {
}
go.mod:
module trial
go 1.14
require github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea
go.sum:
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea h1:IkOONr/u7Wy+j2R4r1eMV8PEuN4kmOhZZNaYxDOF+KQ=
github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea/go.mod h1:WBIWFH/iYYvuApCvPU+/R6hfX6v0Ogu4apwf0UgzVF0=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Execution:
- 1st time, you could see it downloads the dependency package, and then build different packages.
$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 3.100 go: downloading github.com/phachon/go-logger v0.0.0-20191215032019-86e4227f71ea
#14 3.330 go: downloading github.com/fatih/color v1.7.0
#14 3.331 go: downloading github.com/mailru/easyjson v0.7.0
#14 3.523 go: downloading github.com/mattn/go-colorable v0.1.4
#14 3.561 go: downloading github.com/mattn/go-isatty v0.0.11
#14 3.731 go: downloading golang.org/x/sys v0.0.0-20191026070338-33540a1f6037
#14 5.058 github.com/mailru/easyjson/buffer
#14 5.063 golang.org/x/sys/unix
#14 5.070 github.com/mailru/easyjson/jlexer
#14 5.077 github.com/mailru/easyjson/jwriter
#14 5.087 github.com/phachon/go-logger/utils
#14 5.174 github.com/mailru/easyjson
#14 5.346 github.com/mattn/go-isatty
#14 5.355 github.com/mattn/go-colorable
#14 5.368 github.com/fatih/color
#14 5.396 github.com/phachon/go-logger
#14 5.451 command-line-arguments
#14 DONE 7.4s
...
- 2nd time, I changed the
main.go
as next:
main.go:
package main
import (
_ "github.com/phachon/go-logger"
"fmt"
)
func main() {
fmt.Println("helloworld")
}
Then, execute build again:
$ export DOCKER_BUILDKIT=1
$ docker build --progress=plain -t abc:1 . --no-cache
...
#14 [stage-0 6/6] RUN --mount=type=cache,mode=0755,target=/root/.cache/go-bu...
#14 1.469 command-line-arguments
#14 DONE 3.2s
...
You could see from above, it no longer downloads package (because /root/go was reused during different times build). And, it just build command-line-arguments
, corresponding to the change of main.go
, no packages related to github.com/phachon/go-logger
been rebuilt (because /root/.cache/go-build as build cache be reused).
答案2
得分: 1
我认为你应该利用multi-stage builds的优势。在第一阶段,你可以使用已安装Go的Docker镜像。你可以在这里选择其中一个可用的镜像:https://hub.docker.com/_/golang。此时,你需要将源代码复制到容器中,并在容器中运行go build
。
在下一步中,你可以使用alpine镜像,并将可执行文件(以及其他所需的文件)复制到目标容器中。
以下是一个示例:
FROM golang:alpine AS build-env
RUN apk --no-cache add build-base git bzr mercurial gcc
ADD . /src
RUN cd /src && go build -o goapp
# final stage
FROM alpine
WORKDIR /app
COPY --from=build-env /src/goapp /app/
ENTRYPOINT ./goapp
这样做有几个好处,其中之一就是最终镜像的大小会更小。
英文:
I think you should take the advantage of the multi-stage builds. In the first stage, you use the docker image with Go installed. You can use one of those available here https://hub.docker.com/_/golang. At this point, you have to copy your source code into the container and run go build
there.
In the next step, you (for example) use an alpine image and copy the executable (and other required fields) to the target container.
This is an example how it can be done.
FROM golang:alpine AS build-env
RUN apk --no-cache add build-base git bzr mercurial gcc
ADD . /src
RUN cd /src && go build -o goapp
# final stage
FROM alpine
WORKDIR /app
COPY --from=build-env /src/goapp /app/
ENTRYPOINT ./goapp
This has several benefits and the size of the final image is one of them.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论