超慢的Docker构建

huangapple go评论99阅读模式
英文:

Super Slow Docker Build

问题

Here is the translated portion:

我觉得我快要发疯了。我已经在Stack、GitHub和互联网的其他远程地方到处搜索,似乎找不到一个有效的解决方案。

在这个特定项目上,运行 docker-compose build 需要很长时间。以前不是这样的,在使用 Docker 的其他项目上,这根本不是问题。我指的是大约需要 10-15 分钟的构建时间,而以前只需要不到 2 分钟。我让两个不同的同事下载了相同的存储库(一个在 Ubuntu 18 上,另一个在 macOS 14.x 上)。当他们运行 build 命令时,整个过程只花了约 2 分钟。这两个人以前都没有构建过这个项目,所以他们是从头开始的。

我已经卸载/重新安装了 Docker,运行了完整的 docker system prune -a,通过WiFi连接,通过以太网连接,尝试了不同的WiFi网络,调整了我的compose文件,调整了我的docker文件——什么都没有用。

我的机器是一台2018年的MacBook Pro,配备了四核2.7GHz的i7处理器,运行macOS 10.14.6,安装了16GB的内存,使用的是Docker Desktop 2.1.0.5。

我已经允许Docker Desktop使用高达12GB的内存。在构建过程中,我的机器的CPU使用率从110%上升到270%,运行 com.docker.hyperkit 进程。

要明确一下,它卡在“Building php”(或“Building web”)状态消息上,几乎什么都还没有开始。之后,实际的构建过程运行得很顺利和快速。

这是我的 docker-compose.yaml 文件:

version: '3.1'

services:
  db:
    container_name: clientsname.db
    hostname: db
    image: mariadb:10.4.1-bionic
    volumes:
      - ./db-data:/var/lib/mysql:delegated
    ports:
      - 3307:3306
    environment:
      MYSQL_DATABASE: my_database
      MYSQL_USER: my_user
      MYSQL_PASSWORD: my_pass
      MYSQL_ROOT_PASSWORD: my_pass
      
  php:
    container_name: clientsname.php
    hostname: php
    build:
      dockerfile: php/php.dockerfile
      context: ./
    environment:
      XDEBUG_CONFIG: remote_host=${REMOTE_HOST}

    volumes:
      - ../web:/var/www/web
      - ../moodle:/var/www/moodle
      - ../moodledata:/var/www/moodledata
      - ./php/custom.ini:/usr/local/etc/php/conf.d/zzz-custom.ini
      - ./php/z-errors.ini:/usr/local/etc/php/conf.d/z-errors.ini:delegated
      - ./php/z-upload.ini:/usr/local/etc/php/conf.d/z-upload.ini:delegated
      - ./php/z-xdebug.ini:/usr/local/etc/php/conf.d/z-xdebug.ini:delegated
    depends_on:
      - db
  
  web:
    container_name: clientsname.web
    hostname: web
    build:
      dockerfile: nginx/nginx.dockerfile
      context: ./
    volumes:
      - ../web:/var/www/web
      - ../moodle:/var/www/moodle
      - ../moodledata:/var/www/moodledata
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf
      - ./nginx/ssl:/etc/nginx/ssl
      - ./logs:/var/log/nginx
    ports:
      - 80:80
      - 443:443
    depends_on:
      - php
      - db

这是引用的 php.dockerfile 文件:

FROM php:7.2.26-fpm
LABEL maintainer="My Clients Name"

# Environment variables
ENV DEBIAN_FRONTEND=noninteractive
ENV COMPOSER_ALLOW_SUPERUSER=1
ENV COMPOSER_NO_INTERACTION=1
ENV COMPOSER_HOME=/usr/local/share/composer

# Working Directory
WORKDIR /var/www/web
WORKDIR /var/www/moodle
WORKDIR /var/www/moodledata

RUN rm /etc/apt/preferences.d/no-debian-php && apt-get update && apt-get install -y --no-install-recommends apt-utils \
    build-essential        \
    php-soap            \
    libzip-dev          \
    libmagickcore-dev   \
    libmagickwand-dev   \
    libmagic-dev        \
    libpng-dev          \
    libfreetype6-dev    \
    libjpeg62-turbo-dev \
    libmcrypt-dev       \
    libmemcached-dev    \
    zlib1g-dev          \
    nano                \
    sudo                \
    gnupg               \
    curl                \
    unzip &&            \
docker-php-ext-install soap pdo_mysql mysqli && \
docker-php-ext-install -j$(nproc) gd iconv && \
docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ && \
pecl install zip-1.15.2 imagick memcached-3.0.4 xdebug && \
docker-php-ext-enable memcached imagick zip xdebug

# Install Composer, Node, Gulp, and SASS
RUN curl -s https://getcomposer.org/installer | php && mv composer.phar /usr/local/bin/composer

RUN curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - && apt-get install -y nodejs && npm install npm@latest -g && npm install --global gulp-cli && npm config set unsafe-perm=true

# Export composer vendor path
RUN echo "" >> ~/.bashrc && echo 'export PATH="$HOME/.composer/vendor/bin:$PATH"' >> ~/.bashrc

和引用的 nginx.dockerfile

FROM nginx:stable-alpine

RUN apk add --update bash && rm -rf /var/cache/apk/*

WORKDIR /var/www/web

这让我很疯狂... 我到底做错了什么?如果有任何我遗漏的信息您想了解的,请告诉我,我会

英文:

I think I'm going to go crazy. I've searched all over and can't seem to find a working solution both here on Stack, GitHub and other far reaches of the interwebs.

On this particular project, running docker-compose build is taking FOREVER. It didn't use to be this way, and on other projects that use Docker, it's not an issue at all. And by forever... I'm talking around 10-15 minute build times when it used to only take around 2 minutes tops. I had two separate coworkers DL the same repo (one on Ubuntu 18, and the other on macOS 14.x). When they ran the build command, the entire process took ~2 minutes. Both of these people had never built this project before, so they were starting from complete scratch.

I've uninstalled/reinstalled Docker, ran a complete docker system prune -a, connected via wifi, connected via Ethernet, tried a different wifi network, tweaked my compose file, tweaked my docker file -- nothing.

My machine is a 2018 MacBook Pro with a quad-core 2.7GHz i7, running macOS 10.14.6 with 16gb of installed RAM with Docker Desktop 2.1.0.5.

I've allowed Docker Desktop to have up to 12gb or RAM. During the build process, my machine cpu usage spikes on average from 110% up to 270% running the com.docker.hyperkit process.

To be clear, it's hanging on the "Building php" (or "Building web") status message(s) before anything really even starts. After that, the actual build process runs smoothly and quick.

Here is my docker-compose.yaml file:

version: '3.1'

services:
  db:
    container_name: clientsname.db
    hostname: db
    image: mariadb:10.4.1-bionic
    volumes:
      - ./db-data:/var/lib/mysql:delegated
    ports:
      - 3307:3306
    environment:
      MYSQL_DATABASE: my_database
      MYSQL_USER: my_user
      MYSQL_PASSWORD: my_pass
      MYSQL_ROOT_PASSWORD: my_pass
      
  php:
    container_name: clientsname.php
    hostname: php
    build:
      dockerfile: php/php.dockerfile
      context: ./
    environment:
      XDEBUG_CONFIG: remote_host=${REMOTE_HOST}

    volumes:
      - ../web:/var/www/web
      - ../moodle:/var/www/moodle
      - ../moodledata:/var/www/moodledata
      - ./php/custom.ini:/usr/local/etc/php/conf.d/zzz-custom.ini
      - ./php/z-errors.ini:/usr/local/etc/php/conf.d/z-errors.ini:delegated
      - ./php/z-upload.ini:/usr/local/etc/php/conf.d/z-upload.ini:delegated
      - ./php/z-xdebug.ini:/usr/local/etc/php/conf.d/z-xdebug.ini:delegated
    depends_on:
      - db
  
  web:
    container_name: clientsname.web
    hostname: web
    build:
      dockerfile: nginx/nginx.dockerfile
      context: ./
    volumes:
      - ../web:/var/www/web
      - ../moodle:/var/www/moodle
      - ../moodledata:/var/www/moodledata
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf
      - ./nginx/ssl:/etc/nginx/ssl
      - ./logs:/var/log/nginx
    ports:
      - 80:80
      - 443:443
    depends_on:
      - php
      - db

Here is the referenced php.dockerfile file:

FROM php:7.2.26-fpm
LABEL maintainer="My Clients Name"

# Environment variables
ENV DEBIAN_FRONTEND=noninteractive
ENV COMPOSER_ALLOW_SUPERUSER=1
ENV COMPOSER_NO_INTERACTION=1
ENV COMPOSER_HOME=/usr/local/share/composer

# Working Directory
WORKDIR /var/www/web
WORKDIR /var/www/moodle
WORKDIR /var/www/moodledata


RUN rm /etc/apt/preferences.d/no-debian-php && apt-get update && apt-get install -y --no-install-recommends apt-utils \
		build-essential		\
		php-soap			\
		libzip-dev			\
		libmagickcore-dev	\
		libmagickwand-dev	\
		libmagic-dev		\
		libpng-dev			\
		libfreetype6-dev	\
		libjpeg62-turbo-dev	\
		libmcrypt-dev		\
		libmemcached-dev	\
		zlib1g-dev			\
		nano				\
		sudo				\
		gnupg				\
		curl				\
		unzip &&			\
	docker-php-ext-install soap pdo_mysql mysqli && \
	docker-php-ext-install -j$(nproc) gd iconv && \
	docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ && \
	pecl install zip-1.15.2 imagick memcached-3.0.4 xdebug && \
	docker-php-ext-enable memcached imagick zip xdebug

# Install Composer, Node, Gulp, and SASS
RUN curl -s https://getcomposer.org/installer | php && mv composer.phar /usr/local/bin/composer

RUN curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - && apt-get install -y nodejs && npm install npm@latest -g && npm install --global gulp-cli && npm config set unsafe-perm=true


# Export composer vendor path
RUN echo "" >> ~/.bashrc && echo 'export PATH="$HOME/.composer/vendor/bin:$PATH"' >> ~/.bashrc

And the referenced nginx.dockerfile

FROM nginx:stable-alpine

RUN apk add --update bash && rm -rf /var/cache/apk/*

WORKDIR /var/www/web

It's driving me batty... what in the bloody hell could I be doing wrong? If there is anything I've left out that you'd all like to know, please let me know and I'll update the post.


UPDATE

Thanks to @BMitch and you all who have commented thus far. I took my entire /docker build directory and moved it into a test folder, and then created empty /web, /moodle, and /moodledata directories before running the build command. It started to compile immediately.

What's curious to me is that the other coworkers DL'd the same Git repo that I have and did not have any of the same issues. Oooh... come to think of it... I bet I know what the issue is.

答案1

得分: 55

这是来自您的构建上下文(通常是您运行构建的目录,但可以在Compose文件中进行覆盖设置)。您的上下文目录中有大量文件或大文件,这些文件在执行构建之前会被发送。

您可以使用.dockerignore,它具有与.gitignore 几乎相同的格式,用于排除不希望在构建时发送的文件。并且使用 BuildKit(在最新版本的 Docker 中,如果您执行 export DOCKER_BUILDKIT=1 将启用它),它将仅在您明确复制文件时才发送上下文,而且仅当这些文件与缓存中的可用内容不同。

有关构建上下文的更多信息,请参阅:https://docs.docker.com/engine/reference/commandline/build/

还有最佳实践:https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

英文:

This is from your build context (that's often the directory where you run your build, but can be overridden as you've done in the compose file). You have a large number of files, or large files, in the context directory that is sent before performing the build.

You can use .dockerignore which has a nearly identical format to .gitignore to exclude files from being sent on build. And with BuildKit (enabled if you export DOCKER_BUILDKIT=1 in recent versions of docker) it will only send context when you explicitly copy files and then only when those files have changed from what is available in the cache.

For more on the build context, see: https://docs.docker.com/engine/reference/commandline/build/

There's also the best practices: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

huangapple
  • 本文由 发表于 2020年1月7日 00:04:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/59615266.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定