云构建将秘密环境传递给Dockerfile。

huangapple go评论82阅读模式
英文:

cloud build pass secret env to dockerfile

问题

我正在使用Google Cloud Build构建Docker镜像并部署到Cloud Run。该模块依赖于私有的Github仓库。在cloudbuild.yaml文件中,我可以访问密钥,例如Github令牌,但我不知道将此令牌传递给Dockerfile的正确和安全的方法。

我按照官方指南进行操作,但它只适用于cloudbuild.yaml范围,而不适用于Dockerfile通过SSH密钥访问GitHub

cloudbuild.yaml

steps:
  - name: gcr.io/cloud-builders/docker
    args: ["build", "-t", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA", "."]

  - name: gcr.io/cloud-builders/docker
    args: [ "push", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA" ]

  - name: gcr.io/google.com/cloudsdktool/cloud-sdk
    entrypoint: gcloud
    args: [
      "run", "deploy", "$REPO_NAME",
      "--image", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA",
      "--platform", "managed",
      "--region", "us-east1",
      "--allow-unauthenticated",
      "--use-http2",
    ]

images:
  - gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA

availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/GITHUB_USER/versions/1
      env: "GITHUB_USER"
    - versionName: projects/$PROJECT_ID/secrets/GITHUB_TOKEN/versions/1
      env: "GITHUB_TOKEN"

Dockerfile

# [START cloudrun_grpc_dockerfile]
# [START run_grpc_dockerfile]
FROM golang:buster as builder

# 创建并切换到应用程序目录。
WORKDIR /app

# 创建/root/.netrc文件以凭据github
RUN echo machine github.com >> /root/.netrc
RUN echo login "GITHUB_USER" >> /root/.netrc
RUN echo password "GITHUB_PASSWORD" >> /root/.netrc

# 配置Github,这将创建文件/root/.gitconfig
RUN git config --global url."ssh://git@github.com/".insteadOf "https://github.com/"

# GOPRIVATE
RUN go env -w GOPRIVATE=github.com/org/repo

# 我需要删除/root/.netrc文件吗?我不希望这些信息被传播和被第三方看到。

# 检索应用程序依赖项。
# 这允许容器构建重用缓存的依赖项。
# 预计会复制go.mod和如果存在go.sum。
COPY go.* ./
RUN go mod download

# 将本地代码复制到容器镜像中。
COPY . ./

# 构建二进制文件。
# RUN go build -mod=readonly -v -o server ./cmd/server
RUN go build -mod=readonly -v -o server

# 使用官方的Debian slim镜像创建精简的生产容器。
# https://hub.docker.com/_/debian
# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM debian:buster-slim
RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    ca-certificates && \
    rm -rf /var/lib/apt/lists/*

# 从构建阶段将二进制文件复制到生产镜像中。
COPY --from=builder /app/server /server

# 在容器启动时运行Web服务。
CMD ["/server"]

# [END run_grpc_dockerfile]
# [END cloudrun_grpc_dockerfile]

尝试了两天后,我没有找到解决方案,我能做的最简单的事情是生成vendor文件夹并将其提交到仓库中,以避免使用go mod download命令。

英文:

I am using google cloud build to build a docker image and deploy in cloud run. The module has dependencies on Github that are private. In the cloudbuild.yaml file I can access secret keys for example the Github token, but I don't know what is the correct and secure way to pass this token to the Dockerfile.

I was following this official guide but it would only work in the cloudbuild.yaml scope and not in the Dockerfile. Accessing GitHub from a build via SSH keys

cloudbuild.yaml

steps:
- name: gcr.io/cloud-builders/docker
args: ["build", "-t", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA", "."]
- name: gcr.io/cloud-builders/docker
args: [ "push", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA" ]
- name: gcr.io/google.com/cloudsdktool/cloud-sdk
entrypoint: gcloud
args: [
"run", "deploy", "$REPO_NAME",
"--image", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA",
"--platform", "managed",
"--region", "us-east1",
"--allow-unauthenticated",
"--use-http2",
]
images:
- gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/GITHUB_USER/versions/1
env: "GITHUB_USER"
- versionName: projects/$PROJECT_ID/secrets/GITHUB_TOKEN/versions/1
env: "GITHUB_TOKEN"

Dockerfile

# [START cloudrun_grpc_dockerfile]
# [START run_grpc_dockerfile]
FROM golang:buster as builder
# Create and change to the app directory.
WORKDIR /app
# Create /root/.netrc cred github
RUN echo machine github.com >> /root/.netrc
RUN echo login "GITHUB_USER" >> /root/.netrc
RUN echo password "GITHUB_PASSWORD" >> /root/.netrc
# Config Github, this create file /root/.gitconfig
RUN git config --global url."ssh://git@github.com/".insteadOf "https://github.com/"
# GOPRIVATE
RUN go env -w GOPRIVATE=github.com/org/repo
# Do I need to remove the /root/.netrc file? I do not want this information to be propagated and seen by third parties.
# Retrieve application dependencies.
# This allows the container build to reuse cached dependencies.
# Expecting to copy go.mod and if present go.sum.
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
# Build the binary.
# RUN go build -mod=readonly -v -o server ./cmd/server
RUN go build -mod=readonly -v -o server
# Use the official Debian slim image for a lean production container.
# https://hub.docker.com/_/debian
# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM debian:buster-slim
RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
ca-certificates && \
rm -rf /var/lib/apt/lists/*
# Copy the binary to the production image from the builder stage.
COPY --from=builder /app/server /server
# Run the web service on container startup.
CMD ["/server"]
# [END run_grpc_dockerfile]
# [END cloudrun_grpc_dockerfile]

After trying for 2 days I have not found a solution, the simplest thing I could do was to generate the vendor folder and commit it to the repository and avoid go mod download.

答案1

得分: 9

你有几种方法可以完成这个任务。

使用Docker时,当你运行构建时,它会在一个隔离的环境中运行(这是隔离的原则)。因此,在构建过程中,你无法访问环境变量。

为了解决这个问题,你可以使用构建参数,并将你的秘密值放在该参数中。

但是,有一个陷阱:你必须使用bash代码,而不是Cloud Build中的内置步骤代码。让我给你展示一下

# 不起作用
  - name: gcr.io/cloud-builders/docker
    secretEnv: ["GITHUB_USER","GITHUB_TOKEN"]
    args: ["build", "-t", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA", "--build-arg=GITHUB_USER=$GITHUB_USER,GITHUB_TOKEN=$GITHUB_TOKEN","."]

# 可行的版本
  - name: gcr.io/cloud-builders/docker
    secretEnv: ["GITHUB_USER","GITHUB_TOKEN"]
    entrypoint: bash
    args: 
     - -c
     - |
        docker build -t gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA --build-arg=GITHUB_USER=$$GITHUB_USER,GITHUB_TOKEN=$$GITHUB_TOKEN .

你也可以在Dockerfile之外执行操作。大致上是相同的原理:加载一个容器,执行操作,加载另一个容器并继续操作。

英文:

You have several way to do things.

With Docker, when you run a build, you run it in an isolated environment (it's the principle of isolation). So, you haven't access to your environment variables from inside the build process.

To solve that, you can use build args and put your secret values in that parameter.

But, there is a trap: you have to use bash code and not built in step code in Cloud Build. Let me show you

# Doesn't work
- name: gcr.io/cloud-builders/docker
secretEnv: ["GITHUB_USER","GITHUB_TOKEN"]
args: ["build", "-t", "gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA", "--build-arg=GITHUB_USER=$GITHUB_USER,GITHUB_TOKEN=$GITHUB_TOKEN","."]
# Working version
- name: gcr.io/cloud-builders/docker
secretEnv: ["GITHUB_USER","GITHUB_TOKEN"]
entrypoint: bash
args: 
- -c
- |
docker build -t gcr.io/$PROJECT_ID/$REPO_NAME:$COMMIT_SHA --build-arg=GITHUB_USER=$$GITHUB_USER,GITHUB_TOKEN=$$GITHUB_TOKEN .

You can also perform the actions outside of the Dockerfile. It's roughly the same thing: load a container, perform operation, load another container and continue.

huangapple
  • 本文由 发表于 2022年4月29日 02:04:47
  • 转载请务必保留本文链接:https://go.coder-hub.com/72048367.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定