Building image and storing in cache, but those stages are still being built after docker load and –cache-from in docker build

huangapple go评论94阅读模式
英文:

Building image and storing in cache, but those stages are still being built after docker load and --cache-from in docker build

问题

重构我们的Azure Pipelines以尝试加快它们的速度。

目前,对于我们的unit-testingintegration-testing阶段,它正在重新构建相同的依赖阶段... 这实际上是非常慢和低效的。

  • 我正在构建带有--target development的镜像,并将其存储在Cache@2中。
  • 在我们的UnitTest阶段,一个任务成功地从缓存中使用docker load -i加载它,并使用docker images确认。
  • 然后它进入到实际运行unit-tests阶段,在这个阶段我使用了--cache-from=--target unit-tests
  • 在管道中,我可以看到它确认了缓存,但它仍然构建了应该已经有的缓存镜像的阶段:
  1. #4 从companyapp-api:pr-api导入缓存清单
  2. #4 sha256:7c6bf1eebafe5af983d68e3fb7d72c271b8a80918f9799979ebd2b2bea604d10
  3. #4 已完成 0.0s
  4. #5 [python-base 1/1] 从docker.io/library/python:3.9-slim导入
  5. #5 sha256:f876c6f14c8c365d299789228d8a0c38ac92e17ea62116c830f5b7c6bc684e47
  6. #5 已完成 0.0s

至于我正在处理的文件...

  1. # ./api/docker/Dockerfile
  2. # 创建具有共享环境变量的Python基础镜像
  3. FROM python:3.9-slim as python-base
  4. ENV PYTHONUNBUFFERED=1 \
  5. PYTHONDONTWRITEBYTECODE=1 \
  6. PIP_NO_CACHE_DIR=off \
  7. PIP_DISABLE_PIP_VERSION_CHECK=on \
  8. PIP_DEFAULT_TIMEOUT=100 \
  9. POETRY_HOME="/opt/poetry" \
  10. POETRY_VIRTUALENVS_IN_PROJECT=true \
  11. POETRY_NO_INTERACTION=1 \
  12. PYSETUP_PATH="/opt/pysetup" \
  13. VENV_PATH="/opt/pysetup/.venv";
  14. ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH";
  15. # builder-base用于构建依赖项
  16. FROM python-base as builder-base
  17. RUN apt-get update \
  18. && apt-get install --no-install-recommends -y \
  19. curl \
  20. build-essential
  21. # 安装Poetry - 遵守$POETRY_VERSION和$POETRY_HOME
  22. ENV POETRY_VERSION=1.4.1 GET_POETRY_IGNORE_DEPRECATION=1
  23. RUN curl -sSL https://install.python-poetry.org | python3 -
  24. # 我们在这里复制Python要求以进行缓存,并且只在运行时使用Poetry安装
  25. WORKDIR $PYSETUP_PATH
  26. COPY ./poetry.lock ./pyproject.toml ./
  27. RUN poetry install --no-dev
  28. # 'development'阶段安装所有开发依赖项,可用于开发代码。
  29. # 例如,使用docker-compose在/app下挂载本地卷
  30. FROM python-base as development
  31. # 复制poetry和venv到镜像中
  32. COPY --from=builder-base $POETRY_HOME $POETRY_HOME
  33. COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH
  34. # 复制我们的入口点
  35. # COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
  36. RUN chmod +x . /opt/pysetup/.venv/bin/activate
  37. # venv已经安装了运行时依赖项,我们可以更快地安装
  38. WORKDIR $PYSETUP_PATH
  39. RUN poetry install
  40. WORKDIR /app
  41. COPY . .
  42. EXPOSE 5000 5672
  43. CMD ["python", "src/manage.py", "runserver", "0.0.0.0:5000"]
  44. # 'unit-tests'阶段使用unittest和coverage运行我们的单元测试。
  45. FROM development AS unit-tests
  46. RUN coverage run --omit='src/manage.py,src/config/*,*/.venv/*,*/*__init__.py,*/tests.py,*/admin.py' src/manage.py test src --tag=ut &&
  47. coverage report
  1. # ./pipelines/pr.yaml
  2. # # 这由PR和分支策略触发
  3. 触发器:
  4. # 读取基本变量模板
  5. 变量:
  6. imageRepository: companyapp
  7. dockerfilePath: $(Build.SourcesDirectory)
  8. vmImageName: ubuntu-latest
  9. # 使用ubuntu-latest镜像
  10. 池:
  11. vmIMage: $(vmImageName)
  12. 阶段:
  13. - 阶段: 构建
  14. 显示名称: 为测试构建镜像...
  15. 工作:
  16. - 任务: BuildingAndCache
  17. 显示名称: 构建和缓存测试镜像...
  18. 步骤:
  19. - 任务: Cache@2
  20. 显示名称: 创建缓存...
  21. 输入:
  22. key: 'docker | "$(Agent.OS)" | cache'
  23. path: $(Pipeline.Workspace)/docker
  24. cacheHitVar: CACHE_RESTORED
  25. - 任务: Docker@2
  26. 显示名称: 构建测试镜像...
  27. 输入:
  28. command: 'build'
  29. repository: $(imageRepository)-$(service)
  30. dockerfile: $(dockerFilePath)/$(service)/docker/Dockerfile
  31. buildContext: $(dockerFilePath)/$(service)
  32. arguments: |
  33. --target development
  34. tags: |
  35. pr-$(service)
  36. env:
  37. DOCKER_BUILDKIT: 1
  38. - bash: |
  39. mkdir -p $(Pipeline.Workspace)/docker
  40. docker save -o $(Pipeline.Workspace)/docker/cache.tar $(imageRepository)-$(service):pr-$(service)
  41. 显示名称: 保存镜像到缓存...
  42. # condition: and(not(canceled()), not(failed()), ne(variables.CACHE_RESTORED, 'true'))
  43. - 阶段: UnitTest
  44. 显示名称: 运行单元测试...
  45. 工作:
  46. - 任务: UnitTesting
  47. 显示名称: 运行单元测试...
  48. 步骤:
  49. - 任务: Cache@2
  50. 显示名称: 检查现有镜像的缓存...
  51. 输入:
  52. key: 'docker | "$(Agent.OS)" | cache'
  53. path: $(Pipeline.Work
  54. <details>
  55. <summary>英文:</summary>
  56. Reworking our Azure Pipelines to try and speed them up.
  57. Currently it is rebuilding the same dependent stages for our `unit-testing` and `integration-testing` stages... which is really slow and inefficient.
  58. - I&#39;m building the image which `--target development`, storing it in cache with `Cache@2`.
  59. - In our `UnitTest` stage, a task successfully loads it from cache using `docker load -i` and confirmed with `docker images`.
  60. - Then it gets to the actual running of the `unit-tests` stage where I use `--cache-from=` and `--target unit-tests`.
  61. - In the pipeline I can see it acknowledging the cache, but it still builds the stages the cached image should already have:

#4 importing cache manifest from companyapp-api:pr-api
#4 sha256:7c6bf1eebafe5af983d68e3fb7d72c271b8a80918f9799979ebd2b2bea604d10
#4 DONE 0.0s

#5 [python-base 1/1] FROM docker.io/library/python:3.9-slim
#5 sha256:f876c6f14c8c365d299789228d8a0c38ac92e17ea62116c830f5b7c6bc684e47
#5 DONE 0.0s

  1. As for the files I&#39;m working with...

./api/docker/Dockerfile

creating a python base with shared environment variables

FROM python:3.9-slim as python-base
ENV PYTHONUNBUFFERED=1
PYTHONDONTWRITEBYTECODE=1
PIP_NO_CACHE_DIR=off
PIP_DISABLE_PIP_VERSION_CHECK=on
PIP_DEFAULT_TIMEOUT=100
POETRY_HOME="/opt/poetry"
POETRY_VIRTUALENVS_IN_PROJECT=true
POETRY_NO_INTERACTION=1
PYSETUP_PATH="/opt/pysetup"
VENV_PATH="/opt/pysetup/.venv"

ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"

builder-base is used to build dependencies

FROM python-base as builder-base
RUN apt-get update
&& apt-get install --no-install-recommends -y
curl
build-essential

Install Poetry - respects $POETRY_VERSION & $POETRY_HOME

ENV POETRY_VERSION=1.4.1 GET_POETRY_IGNORE_DEPRECATION=1
RUN curl -sSL https://install.python-poetry.org | python3 -

We copy our Python requirements here to cache them

and install on ly runtime deps using poetry

WORKDIR $PYSETUP_PATH
COPY ./poetry.lock ./pyproject.toml ./
RUN poetry install --no-dev

'development' stage installs all dev deps and can be used to develop code.

For example using docker-compose to mount local volume under /app

FROM python-base as development

Copying poetry and venv into image

COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

Copying in our entrypoint

COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh

RUN chmod +x . /opt/pysetup/.venv/bin/activate

venv already has runtime deps installed we get a quicker install

WORKDIR $PYSETUP_PATH
RUN poetry install
WORKDIR /app
COPY . .
EXPOSE 5000 5672
CMD [ "python", "src/manage.py", "runserver", "0.0.0.0:5000"]

'unit-tests' stage runs our unit tests with unittest and coverage.

FROM development AS unit-tests
RUN coverage run --omit='src/manage.py,src/config/,/.venv/,/init.py,/tests.py,*/admin.py' src/manage.py test src --tag=ut &&
coverage report

./pipelines/pr.yaml

# This is triggered by the PR and branch policies

trigger: none

Read in the base variable template

variables:
imageRepository: companyapp
dockerfilePath: $(Build.SourcesDirectory)
vmImageName: ubuntu-latest

Use the ubuntu-latest image

pool:
vmIMage: $(vmImageName)

stages:

  • stage: Build
    displayName: Build image for tests...
    jobs:

    • job: BuildingAndCache
      displayName: Building and caching image for tests...
      steps:

      • task: Cache@2
        displayName: Creating cache...
        inputs:
        key: 'docker | "$(Agent.OS)" | cache'
        path: $(Pipeline.Workspace)/docker
        cacheHitVar: CACHE_RESTORED

      • task: Docker@2
        displayName: Building image for tests...
        inputs:
        command: 'build'
        repository: $(imageRepository)-$(service)
        dockerfile: $(dockerFilePath)/$(service)/docker/Dockerfile
        buildContext: $(dockerFilePath)/$(service)
        arguments: |
        --target development
        tags: |
        pr-$(service)
        env:
        DOCKER_BUILDKIT: 1

      • bash: |
        mkdir -p $(Pipeline.Workspace)/docker
        docker save -o $(Pipeline.Workspace)/docker/cache.tar $(imageRepository)-$(service):pr-$(service)
        displayName: Saving image to cache...

        condition: and(not(canceled()), not(failed()), ne(variables.CACHE_RESTORED, 'true'))

  • stage: UnitTest
    displayName: Run unit tests...
    jobs:

    • job: UnitTesting
      displayName: Running unit tests...
      steps:

      • task: Cache@2
        displayName: Checking cache for existing images...
        inputs:
        key: 'docker | "$(Agent.OS)" | cache'
        path: $(Pipeline.Workspace)/docker
        cacheHitVar: CACHE_RESTORED

      • script: |
        docker load -i $(Pipeline.Workspace)/docker/cache.tar
        docker images
        displayName: Loading existing image from cache...
        condition: and(not(canceled()), eq(variables.CACHE_RESTORED, 'true'))

      • task: Docker@2
        displayName: Running unit-tests...
        inputs:
        command: 'build'
        repository: $(imageRepository)-$(service)
        dockerfile: $(dockerFilePath)/$(service)/docker/Dockerfile
        buildContext: $(dockerFilePath)/$(service)
        arguments: |
        --cache-from=$(imageRepository)-$(service):pr-$(service)
        --target unit-tests

        tags: |

        pr-$(service)

        env:
        DOCKER_BUILDKIT: 1

  1. **Any suggestions for what I&#39;m doing wrong and how to resolve it?**
  2. ---
  3. The resources I&#39;ve been consulting:
  4. - https://docs.docker.com/build/building/multi-stage/
  5. - https://github.com/michaeloliverx/python-poetry-docker-example/blob/master/docker/Dockerfile
  6. - https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#docker-images
  7. - https://stackoverflow.com/questions/59266670/how-to-enable-docker-layer-caching-in-azure-devops
  8. - Tried ChatGPT and all it is doing is providing what I&#39;m currently doing as an answer.
  9. </details>
  10. # 答案1
  11. **得分**: 0
  12. 已经找到问题,这更多是一个疏忽:`Dockerfile` 寻找的图像与在 `pr.yaml` 中构建的图像的名称不匹配。
  13. 我需要更新:

...
FROM development AS unit-tests
...

  1. 以匹配此处正在构建的图像的名称:

...
- task: Docker@2
displayName: 为测试构建镜像...
inputs:
command: 'build'
repository: $(imageRepository)-$(service)
dockerfile: $(dockerFilePath)/$(service)/docker/Dockerfile
buildContext: $(dockerFilePath)/$(service)
arguments: |
--target development
tags: |
pr-$(service)
env:
DOCKER_BUILDKIT: 1

  1. - bash: |
  2. mkdir -p $(Pipeline.Workspace)/docker
  3. docker save -o $(Pipeline.Workspace)/docker/cache.tar $(imageRepository)-$(service):pr-$(service)
  4. displayName: 将镜像保存到缓存中...

...

  1. 正在构建的图像的名称是:

companyapp-api:pr-api

  1. 因此,`Dockerfile` 应该是:

...
FROM companyapp-api:pr-api AS unit-tests
...

  1. 一旦我这样做了,`unit-tests` 状态只需要 10 秒,而不是 1 20 秒。
  2. <details>
  3. <summary>英文:</summary>
  4. Figured it out and was more of an oversight: the image the `Dockerfile` was looking for didn&#39;t match the name of the image that was built in the `pr.yaml`.
  5. I needed to update:

...
FROM development AS unit-tests
...

  1. To match the name of the image being built here:

...
- task: Docker@2
displayName: Building image for tests...
inputs:
command: 'build'
repository: $(imageRepository)-$(service)
dockerfile: $(dockerFilePath)/$(service)/docker/Dockerfile
buildContext: $(dockerFilePath)/$(service)
arguments: |
--target development
tags: |
pr-$(service)
env:
DOCKER_BUILDKIT: 1

  1. - bash: |
  2. mkdir -p $(Pipeline.Workspace)/docker
  3. docker save -o $(Pipeline.Workspace)/docker/cache.tar $(imageRepository)-$(service):pr-$(service)
  4. displayName: Saving image to cache...

...

  1. The name of the image being built was:

companyapp-api:pr-api

  1. So the `Dockerfile` should have been:

...
FROM companyapp-api:pr-api AS unit-tests
...

  1. Once I did that, the `unit-tests` state took 10s instead of 1m20s.
  2. </details>

huangapple
  • 本文由 发表于 2023年6月9日 07:35:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/76436331.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定