Cloud Build,容器注册表,Cloud Run:在不暴露环境变量的情况下运行测试

huangapple go评论71阅读模式
英文:

Cloud Build, Container Registry, Cloud Run: Run tests without exposing env var

问题

以下是已翻译的部分:

Cloud Build 执行以下操作:

  1. 从 Dockerfile 构建镜像(查看下面的 Dockerfile)
  2. 将镜像推送到容器注册表
  3. 在 Cloud Run 中更新服务

我的问题如下:

因为我在构建时运行测试,所以我需要在构建时获取我的 MONGODB_URI 密钥,但我已经阅读到使用 --build-arg 不安全,会暴露密钥。

我可以在 Cloud Build 容器中运行 npm installnpm run test,但这会使构建时间变长,因为我将不得不运行两次 npm 安装。

有没有办法在不暴露密钥的情况下只运行一次 npm 安装?

Dockerfile

FROM node:16
COPY . ./
WORKDIR /
RUN npm install

ARG env
ARG mongodb_uri

ENV ENVIRONMENT=$env
ENV DB_REMOTE_PROD=$mongodb_uri

RUN npm run test
RUN npm run build
CMD ["node", "./build/index.js"]

我的 cloudbuild.yaml 配置:

steps:
  - name: gcr.io/cloud-builders/docker
    args:
      - '-c'
      - >-
        docker build --no-cache -t
        $_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --build-arg ENVIRONMENT=staging --build-arg mongodb_uri=$$MONGODB_URI -f
        Dockerfile .        
    id: Build
    entrypoint: bash
    secretEnv:
      - MONGODB_URI
  - name: gcr.io/cloud-builders/docker
    args:
      - push
      - '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
    id: Push
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk:slim'
    args:
      - '-c'
      - >-
        gcloud run services update $_SERVICE_NAME --platform=managed
        --image=$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --labels=managed-by=gcp-cloud-build-deploy-cloud-run,commit-sha=$COMMIT_SHA,gcb-build-id=$BUILD_ID,gcb-trigger-id=$_TRIGGER_ID
        --region=$_DEPLOY_REGION --update-env-vars=ENVIRONMENT=staging
        --update-env-vars=DB_REMOTE_PROD=$$MONGODB_URI --quiet        
    id: Deploy
    entrypoint: bash
    secretEnv:
      - MONGODB_URI
images:
  - '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
options:
  substitutionOption: ALLOW_LOOSE
  logging: CLOUD_LOGGING_ONLY
substitutions:
  _TRIGGER_ID: 44b16efe-0219-41af-b32b-9b98438728c3
  _GCR_HOSTNAME: eu.gcr.io
  _PLATFORM: managed
  _SERVICE_NAME: app-staging
  _DEPLOY_REGION: europe-southwest1
availableSecrets:
  secretManager:
    - versionName: projects/PROJECT_ID/secrets/mongodb_app_staging/versions/1
      env: MONGODB_URI

使用我的方法,密钥不仅在容器中暴露,因为我将其作为 build-args 传递,而且它还暴露给所有具有访问 Cloud Run 权限的用户。

英文:

Cloud build do the following:

  1. Build image from dockerfile (see dockerfile below)
  2. Push image to container registry
  3. Update service in Cloud Run

My issue is the following:

As I'm running my tests on build time, I need my MONGODB_URI secret on build time, but I've read that using --build-arg is not safe to expose secrets.

I could run npm install and npm run test in Cloud Build container, but it would make build time longer as I'll have to run npm install two times.

Is there a way I can run only once npm install without having to expose secrets ?

Dockerfile

FROM node:16
COPY . ./
WORKDIR /
RUN npm install 

ARG env
ARG mongodb_uri

ENV ENVIRONMENT=$env
ENV DB_REMOTE_PROD=$mongodb_uri

RUN npm run test
RUN npm run build
CMD ["node", "./build/index.js"]

And my cloudbuild.yaml config:

steps:
  - name: gcr.io/cloud-builders/docker
    args:
      - '-c'
      - >-
        docker build --no-cache -t
        $_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --build-arg ENVIRONMENT=staging --build-arg mongodb_uri=$$MONGODB_URI -f
        Dockerfile .
    id: Build
    entrypoint: bash
    secretEnv:
      - MONGODB_URI
  - name: gcr.io/cloud-builders/docker
    args:
      - push
      - '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
    id: Push
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk:slim'
    args:
      - '-c'
      - >-
        gcloud run services update $_SERVICE_NAME --platform=managed
        --image=$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --labels=managed-by=gcp-cloud-build-deploy-cloud-run,commit-sha=$COMMIT_SHA,gcb-build-id=$BUILD_ID,gcb-trigger-id=$_TRIGGER_ID
        --region=$_DEPLOY_REGION --update-env-vars=ENVIRONMENT=staging
        --update-env-vars=DB_REMOTE_PROD=$$MONGODB_URI --quiet
    id: Deploy
    entrypoint: bash
    secretEnv:
      - MONGODB_URI
images:
  - '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
options:
  substitutionOption: ALLOW_LOOSE
  logging: CLOUD_LOGGING_ONLY
substitutions:
  _TRIGGER_ID: 44b16efe-0219-41af-b32b-9b98438728c3
  _GCR_HOSTNAME: eu.gcr.io
  _PLATFORM: managed
  _SERVICE_NAME: app-staging
  _DEPLOY_REGION: europe-southwest1
availableSecrets:
  secretManager:
    - versionName: projects/PROJECT_ID/secrets/mongodb_app_staging/versions/1
      env: MONGODB_URI

With my method, the secret is exposed not only in the container since I pass it as build-args, but it is also exposed to all users with access to cloud run...

答案1

得分: 2

你的容器构建步骤并不糟糕。你将机密信息提供给容器,但这些信息是不可见的。它只是一个引用。

你几乎无法做得更好。一个解决方案是在Dockerfile内部直接执行机密访问,但我不确定这是否值得增加Dockerfile的复杂性(如果你想要更多细节,请告诉我,我可以分享示例)。

关于Cloud Run,你说得对:在环境变量中以明文方式暴露机密信息是完全不可接受的。

为此,你可以使用Cloud Run和Secret Manager集成。而不是像这样做:

gcloud run services update $_SERVICE_NAME --platform=managed
        --image=$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --labels=managed-by=gcp-cloud-build-deploy-cloud-run,commit-sha=$COMMIT_SHA,gcb-build-id=$BUILD_ID,gcb-trigger-id=$_TRIGGER_ID
        --region=$_DEPLOY_REGION --update-env-vars=ENVIRONMENT=staging
        --update-env-vars=DB_REMOTE_PROD=$$MONGODB_URI --quiet

你可以这样做(现在不再需要将机密信息设置为Cloud Build步骤的环境变量):

gcloud run services update $_SERVICE_NAME --platform=managed
        --image=$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --labels=managed-by=gcp-cloud-build-deploy-cloud-run,commit-sha=$COMMIT_SHA,gcb-build-id=$BUILD_ID,gcb-trigger-id=$_TRIGGER_ID
        --region=$_DEPLOY_REGION --update-env-vars=ENVIRONMENT=staging
        --update-secrets=DB_REMOTE_PROD=mongodb_app_staging:1 --quiet
英文:

Your container build step is not so bad. You provide your secrets to your container, but it is not visible. It's only reference.

You can hardly do better. A solution could be to perform the secret access directly inside the Dockerfile, but I'm not sure that the "security gain" worth the Dockerfile increased complexity. (If you want more details, let me know, I will be able to share sample)


About Cloud Run, you are totally true: exposing secret in plain text in environment variable is totally not acceptable.

For that, you can use the Cloud Run - Secret Manager integration. Instead of doing this

gcloud run services update $_SERVICE_NAME --platform=managed
        --image=$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --labels=managed-by=gcp-cloud-build-deploy-cloud-run,commit-sha=$COMMIT_SHA,gcb-build-id=$BUILD_ID,gcb-trigger-id=$_TRIGGER_ID
        --region=$_DEPLOY_REGION --update-env-vars=ENVIRONMENT=staging
        --update-env-vars=DB_REMOTE_PROD=$$MONGODB_URI --quiet

You can do that (and you now longer need to set the secret as env var of your Cloud Build step)

gcloud run services update $_SERVICE_NAME --platform=managed
        --image=$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA
        --labels=managed-by=gcp-cloud-build-deploy-cloud-run,commit-sha=$COMMIT_SHA,gcb-build-id=$BUILD_ID,gcb-trigger-id=$_TRIGGER_ID
        --region=$_DEPLOY_REGION --update-env-vars=ENVIRONMENT=staging
        --update-secrets=DB_REMOTE_PROD=mongodb_app_staging:1 --quiet

huangapple
  • 本文由 发表于 2023年2月14日 02:38:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/75439983.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定