英文:
How do I set docker-credential-ecr-login in my PATH before anything else in GitLab CI
问题
I'm using AWS ECR to host a private Dockerfile image, and I would like to use it in GitLab CI.
Accordingly to the documentation I need to set docker-credential-ecr-login to fetch the private image, but I have no idea how to do that before anything else. That's my .gitlab-ci file:
image: 0222822883.dkr.ecr.us-east-1.amazonaws.com/api-build:latest
tests:
stage: test
before_script:
- echo "before_script"
- apt install amazon-ecr-credential-helper
- apk add --no-cache curl jq python py-pip
- pip install awscli
script:
- echo "script"
- bundle install
- bundle exec rspec
allow_failure: true # for now as we do not have tests
Thank you.
英文:
I'm using AWS ECR to host a private Dockerfile image, and I would like to use it in GitLab CI.
Accordingly to the documentation I need to set docker-credential-ecr-login to fetch the private image, but I have no idea how to do that before anything else. That's my .gitlab-ci file:
image: 0222822883.dkr.ecr.us-east-1.amazonaws.com/api-build:latest
tests:
stage: test
before_script:
- echo "before_script"
- apt install amazon-ecr-credential-helper
- apk add --no-cache curl jq python py-pip
- pip install awscli
script:
- echo "script"
- bundle install
- bundle exec rspec
allow_failure: true # for now as we do not have tests
Thank you.
答案1
得分: 3
我确认所涉及的功能目前在GitLab CI中还不可用;但是我最近看到可以实现一种通用的解决方案,运行一个在私有Docker镜像中取出的容器内的专用CI脚本。
以下是从OP的示例中调整的模板文件.gitlab-ci.yml
,使用我在这个其他SO答案中建议的Docker-in-Docker方法,它本身受到GitLab CI文档 dealing with dind
的启发:
stages:
- test
variables:
IMAGE: "0222822883.dkr.ecr.us-east-1.amazonaws.com/api-build:latest"
REGION: "ap-northeast-1"
tests:
stage: test
image: docker:latest
services:
- docker:dind
variables:
# GIT_STRATEGY: none # uncomment if "git clone" is unneeded for this job
before_script:
- ': before_script'
- apt install amazon-ecr-credential-helper
- apk add --no-cache curl jq python py-pip
- pip install awscli
- $(aws ecr get-login --no-include-email --region "$REGION")
- docker pull "$IMAGE"
script:
- ': script'
- |
docker run --rm -v "$PWD:/build" -w /build "$IMAGE" /bin/bash -c "
export PS4='+ \e[33;1m($CI_JOB_NAME @ line $LINENO) $\e[0m ' # optional
set -ex
## TODO insert your multi-line shell script here ##
echo \"One comment\" # quotes must be escaped here
: A better comment
echo $PWD # interpolated outside the container
echo $PWD # interpolated inside the container
bundle install
bundle exec rspec
## (cont'd) ##
"
- ': done'
allow_failure: true # for now as we do not have tests
此示例假设Docker $IMAGE
包含 /bin/bash
二进制文件,并依赖所谓的YAML块样式。
上述模板已经包含注释,但为了自包含:
- 如果您的Bash命令中包含双引号,则需要对其进行转义,因为整个代码被包围在
docker run … "
和"
中; - 您还需要转义本地Bash变量(参见上面的
$PWD
),否则这些变量将在运行docker run … "$IMAGE" /bin/bash -c "…"
命令之前解析。 - 我用它们更有效的冒号替换了
echo "stuff"
或类似命令:set -x : stuff : note that these three shell commands do nothing : but printing their args thanks to the -x option.
[欢迎提供反馈,因为我不能直接测试这个配置(我不是AWS ECR的用户),但我对OP的示例同时包含了一些apt
和apk
命令感到困惑……]
关于set -e
的一个陷阱的相关备注
请注意,以下脚本有错误:
set -e
command1 && command2
command3
换句话说,应该写成:
set -e
command1 ; command2
command3
或者:
set -e
( command1 && command2 )
command3
要相信这一点,您可以尝试运行:
bash -e -c 'false && true; echo $?; echo this should not be run'
→ 1
→ this should not be run
bash -e -c 'false; true; echo $?; echo this should not be run'
bash -e -c '( false && true ); echo $?; echo this should not be run'
英文:
I confirm the feature at stake is not yet available in GitLab CI; however I've recently seen it is possible to implement a generic workaround to run a dedicated CI script within a container taken from a private Docker image.
The template file .gitlab-ci.yml
below is adapted from the OP's example, using the Docker-in-Docker approach I suggested in this other SO answer, itself inspired by the GitLab CI doc dealing with dind
:
stages:
- test
variables:
IMAGE: "0222822883.dkr.ecr.us-east-1.amazonaws.com/api-build:latest"
REGION: "ap-northeast-1"
tests:
stage: test
image: docker:latest
services:
- docker:dind
variables:
# GIT_STRATEGY: none # uncomment if "git clone" is unneeded for this job
before_script:
- ': before_script'
- apt install amazon-ecr-credential-helper
- apk add --no-cache curl jq python py-pip
- pip install awscli
- $(aws ecr get-login --no-include-email --region "$REGION")
- docker pull "$IMAGE"
script:
- ': script'
- |
docker run --rm -v "$PWD:/build" -w /build "$IMAGE" /bin/bash -c "
export PS4='+ \e[33;1m($CI_JOB_NAME @ line $LINENO) $\e[0m ' # optional
set -ex
## TODO insert your multi-line shell script here ##
echo \"One comment\" # quotes must be escaped here
: A better comment
echo $PWD # interpolated outside the container
echo $PWD # interpolated inside the container
bundle install
bundle exec rspec
## (cont'd) ##
"
- ': done'
allow_failure: true # for now as we do not have tests
This example assumes the Docker $IMAGE
contains the /bin/bash
binary, and relies on the so-called block style of YAML.
The above template already contains comments, but to be self-contained:
- You need to escape double quotes if your Bash commands contain them, because the whole code is surrounded by
docker run … "
and"
; - You also need to escape local Bash variables (cf. the
\$PWD
above), otherwise these variables will be resolved prior running thedocker run … "$IMAGE" /bin/bash -c "…"
command itself. - I replaced the
echo "stuff"
or so commands with their more effective colon counterpart:set -x : stuff : note that these three shell commands do nothing : but printing their args thanks to the -x option.
[Feedback is welcome as I can't directly test this config (I'm not an AWS ECR user), but I'm puzzled by the fact the OP's example contained at the same time some apt
and apk
commands…]
Related remark on a pitfall of set -e
Beware that the following script is buggy:
set -e
command1 && command2
command3
Namely, write instead:
set -e
command1 ; command2
command3
or:
set -e
( command1 && command2 )
command3
To be convinced about this, you can try running:
bash -e -c 'false && true; echo $?; echo this should not be run'
→ 1
→ this should not be run
bash -e -c 'false; true; echo $?; echo this should not be run'
bash -e -c '( false && true ); echo $?; echo this should not be run'
答案2
得分: 0
从GitLab文档中。为了与您的AWS帐户交互,GitLab CI/CD管道需要在您的GitLab设置下定义AWS_ACCESS_KEY_ID和AWS_SECRET_ACCESS_KEY两个变量,位置在Settings > CI/CD > Variables。然后在您的before script中添加以下内容:
image: 0222822883.dkr.ecr.us-east-1.amazonaws.com/api-build:latest
tests:
stage: test
before_script:
- echo "before_script"
- apt install amazon-ecr-credential-helper
- apk add --no-cache curl jq python py-pip
- pip install awscli
- $( aws ecr get-login --no-include-email )
script:
- echo "script"
- bundle install
- bundle exec rspec
allow_failure: true # for now as we do not have tests
另外,请注意您有一个拼写错误,应为awscli
而不是awsclir
。然后按照相应的步骤构建、测试和推送。
英文:
From GitLab documentation. In order to interact with your AWS account, the GitLab CI/CD pipelines require both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to be defined in your GitLab settings under Settings > CI/CD > Variables. Then add to your before script:
image: 0222822883.dkr.ecr.us-east-1.amazonaws.com/api-build:latest
tests:
stage: test
before_script:
- echo "before_script"
- apt install amazon-ecr-credential-helper
- apk add --no-cache curl jq python py-pip
- pip install awscli
- $( aws ecr get-login --no-include-email )
script:
- echo "script"
- bundle install
- bundle exec rspec
allow_failure: true # for now as we do not have tests
Also, you had a typo is awscli
, not awsclir
.Then add the builds, tests and push accordingly.
答案3
得分: 0
我认为在这种情况下,您的代码中存在某种逻辑错误。构建配置中的image
是一个CI脚本运行器镜像,而不是您构建和部署的镜像。
我认为您不必在任何情况下使用它,因为它只是一个具有实用工具和与GitLab CI等的连接的镜像。这个镜像通常不应该有您项目的任何依赖关系。
请查看像这个示例 https://gist.github.com/jlis/4bc528041b9661ae6594c63cd2ef673c 以更清楚地了解如何以正确的方式进行操作。
英文:
I think that you have some sort of logic error in the case. image
in the build configuration is a CI scripts runner image, not image you build and deploy.
I think you don't have to use it in any case since it is just an image which has utilities & connections to the GitLab CI & etc. The image shouldn't have any dependencies of your project normally.
Please check examples like this one https://gist.github.com/jlis/4bc528041b9661ae6594c63cd2ef673c to get it more clear how to do it a correct way.
答案4
得分: 0
我在使用GitLab Runner的Docker执行模式时遇到了相同的问题。
通过SSH登录到EC2实例后,我发现docker-credential-ecr-login
位于/usr/bin/
目录中。为了将它传递到容器中,我不得不将此包挂载到GitLab Runner容器中。
更多信息可以在此线程中找到:https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1583#note_375018948
英文:
I faced the same problem using docker executor mode of gitlab runner.
SSH into the EC2 instance showed that docker-credential-ecr-login
was present in /usr/bin/
. To pass it to the container I had to mount this package to the gitlab runner container.
gitlab-runner register -n \
--url '${gitlab_url}' \
--registration-token '${registration_token}' \
--template-config /tmp/gitlab_runner.template.toml \
--executor docker \
--tag-list '${runner_name}' \
--description 'gitlab runner for ${runner_name}' \
--docker-privileged \
--docker-image "alpine" \
--docker-disable-cache=true \
--docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
--docker-volumes "/cache" \
--docker-volumes "/usr/bin/docker-credential-ecr-login:/usr/bin/docker-credential-ecr-login" \
--docker-volumes "/home/gitlab-runner/.docker:/root/.docker"
More information on this thread as well: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1583#note_375018948
答案5
得分: 0
以下是翻译好的内容:
我们有类似的设置,需要基于托管在 ECR 上的镜像运行 CI 作业。
按照以下步骤进行:
-
遵循此指南>> https://github.com/awslabs/amazon-ecr-credential-helper
-
上述链接的要点是,如果您使用的是 "Amazon Linux 2",则执行以下命令:
sudo amazon-linux-extras enable docker
sudo yum install amazon-ecr-credential-helper
-
在您的 GitLab Runner 上使用 VI 编辑器打开 ~/.docker/config.json
-
在 ~/.docker/config.json 中粘贴以下代码:
{
"credHelpers":
{
"aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login"
}
} -
运行命令 source ~/.bashrc
-
重启 Docker 服务:systemctl restart docker
-
同时,从您的 GitLab>>CI/CD>> 变量中删除任何关于 DOCKER_AUTH_CONFIG 的引用
就是这样。
英文:
We have a similar setup where we need to run CI jobs based off of an Image that is hosted on ECR.
Steps to follow:-
-
follow this guide here>> https://github.com/awslabs/amazon-ecr-credential-helper
-
gist of this above link is if you are on "Amazon Linux 2"
sudo amazon-linux-extras enable docker
sudo yum install amazon-ecr-credential-helper
-
open the ~/.docker/config.json on your gitlab runner in VI editor
-
Paste this code in the ~/.docker/config.json
{
"credHelpers":
{
"aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login"
}
} -
source ~/.bashrc
-
systemctl restart docker
-
also remove any references of DOCKER_AUTH_CONFIG from your GitLab>>CI/CD>> Variables
That's it
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论