Docker-compose in multiple projects same services different container but shared volumes and data are in wrong projects

huangapple
go评论96阅读模式
英文:

Docker-compose in multiple projects same services different container but shared volumes and data are in wrong projects

问题

你有很多进行中的网站项目，每个项目都有自己的docker-compose.yml文件，它们基本上使用相同的服务（PHP、MySQL、Nginx等）。

你在.env文件中为每个项目使用独特的COMPOSE_PROJECT_NAME，并且它们有动态的容器名称：

  1. mysql:
  2.     image: 'mysql:8.0'
  3.     container_name: '${APP_NAME}_mysql'

我的问题如下：

  1. 对于project1创建的数据库在project2的mysql中可见，反之亦然。
  2. 为project2重建或销毁mysql会同时影响project1，因此我失去了数据库。
  3. 这变成了一场噩梦，因为我有不止两个项目。

更多信息：有时一些项目使用PHP7.x，而另一些使用PHP8.x，当我编辑PHP服务的Dockerfile并更改版本后重建时，所有项目现在都使用新的PHP版本，这显然可能会破坏它们。

环境：Windows 10 Pro + WSL2 Ubuntu

以下是完整的Docker Compose文件：

  1. version: '3'
  2. services:
  3.     # PHP Service
  4.     app:
  5.         # 配置省略
  6.     webserver:
  7.         # 配置省略
  8.     mysql:
  9.         # 配置省略
  10.     pma:
  11.         # 配置省略
  12.     redis:
  13.         # 配置省略
  14.     mailhog:
  15.         # 配置省略
  16. networks:
  17.     sail:
  18.         driver: bridge
  19. volumes:
  20.     sail-mysql:
  21.         driver: local
  22.     sail-redis:
  23.         driver: local

另外，这是Dockerfile的内容：

  1. # Dockerfile 配置省略

请让我知道你需要进一步了解的内容。

英文:

Hi I have many ongoing website projects, each with their own docker-compose.yml. They basically use the same services (PHP, MySQL, Nginx, etc).

I do use unique COMPOSE_PROJECT_NAME in .env for each project and they have dynamic container names :

  1.     mysql:
  2.         image: 'mysql:8.0'
  3.         container_name: '${APP_NAME}_mysql'
  1. /projects
  2.     /project1
  3.          /.env
  4.          /docker-compose.yml
  5.     /project2
  6.          /.env
  7.          /docker-compose.yml

My problem is the following :
database created for project1 is visible in the mysql of the project2 and vice versa
rebuilding or destroying mysql for project2, destroys it for project1 also, so I lose database.
this becomes a nightmare because I have a lot more than 2 projects.

More info: it also happens that some projects are in PHP7.x and others in PHP8.x, and when I edit my Dockerfile for the PHP service and change the version and rebuild, than all projects are now on the new PHP version, which obviously might break them.

Environment : windows10pro + wsl2 Ubuntu

Adding docker compose file for completeness:

  1.     # For more information: https://laravel.com/docs/sail
  2. version: '3'
  3. services:
  4.     #PHP Service
  5.     app:
  6.         build:
  7.             context: .
  8.             dockerfile: Dockerfile
  9.         image: digitalocean.com/php
  10.         container_name: ${APP_NAME}_app
  11.         restart: unless-stopped
  12.         extra_hosts:
  13.             - "host.docker.internal:${HOST_GATEWAY}" 
  14.         tty: true
  15.         environment:
  16.             SERVICE_NAME: app
  17.             SERVICE_TAGS: dev
  18.         working_dir: /var/www
  19.         volumes:
  20.             - ./:/var/www
  21.             - ./docker/php/local.ini:/usr/local/etc/php/conf.d/local.ini
  22.             - ./docker/php/docker-php-ext-xdebug.ini:/usr/local/etc/php/conf.d/xdebug.ini
  23.         networks:
  24.             - sail
  25.         depends_on:
  26.             - mysql
  27.             - redis
  28.             - mailhog
  30.     #Nginx Service
  31.     webserver:
  32.         image: nginx:alpine
  33.         container_name: ${APP_NAME}_webserver
  34.         restart: unless-stopped
  35.         tty: true
  36.         ports:
  37.             - "80:80"
  38.             - "443:443"
  39.         volumes:
  40.             - ./:/var/www
  41.             - ./docker/nginx/conf.d/:/etc/nginx/conf.d/
  42.             - ./docker/certs:/etc/nginx/certs
  43.         networks:
  44.             - sail
  45.         extra_hosts:
  46.             - "host.docker.internal:host-gateway"            
  47.     mysql:
  48.         image: mysql:5.7.22
  49.         container_name: ${COMPOSE_PROJECT_NAME}_db
  50.         restart: unless-stopped
  51.         tty: true
  52.         ports:
  53.             - '${FORWARD_DB_PORT:-3306}:3306'
  54.         environment:
  55.             MYSQL_DATABASE: '${DB_DATABASE}'
  56.             MYSQL_USER: '${DB_USERNAME}'
  57.             MYSQL_PASSWORD: '${DB_PASSWORD}'
  58.             MYSQL_ROOT_PASSWORD: '${DB_ROOT_PASSWORD}'
  59.             MYSQL_ROOT_HOST: "%"
  60.             MYSQL_ALLOW_EMPTY_PASSWORD: 1
  61.             SERVICE_TAGS: dev
  62.             SERVICE_NAME: mysql
  63.         volumes:
  64.             - /docker/mysql-dump:/var/lib/mysql
  65.             - ./docker/mysql/my.cnf:/etc/mysql/my.cnf
  66.         networks:
  67.             - sail
  68.         healthcheck:
  69.             test: ["CMD", "mysqladmin", "ping", "-p${DB_PASSWORD}"]
  70.             retries: 3
  71.             timeout: 5s
  72.     pma:
  73.         image: phpmyadmin/phpmyadmin
  74.         container_name: ${APP_NAME}_pma
  75.         restart: always
  76.         depends_on:
  77.             - mysql
  78.         environment:
  79.             PMA_HOST: ${DB_HOST}
  80.             MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
  81.             UPLOAD_LIMIT: 300M
  82.         ports:
  83.             - '8282:80'
  84.         networks:
  85.             - sail  
  86.     redis:
  87.         image: 'redis:alpine'
  88.         container_name: ${APP_NAME}_redis
  89.         ports:
  90.             - '${FORWARD_REDIS_PORT:-6379}:6379'
  91.         volumes:
  92.             - 'sail-redis:/data'
  93.         networks:
  94.             - sail
  95.         healthcheck:
  96.             test: ["CMD", "redis-cli", "ping"]
  97.             retries: 3
  98.             timeout: 5s
  99.     mailhog:
  100.         image: 'mailhog/mailhog:latest'
  101.         container_name: ${APP_NAME}_mailhog
  102.         ports:
  103.             - '${FORWARD_MAILHOG_PORT:-1025}:1025'
  104.             - '${FORWARD_MAILHOG_DASHBOARD_PORT:-8025}:8025'
  105.         networks:
  106.             - sail
  107. networks:
  108.     sail:
  109.         driver: bridge
  110. volumes:
  111.     sail-mysql:
  112.         driver: local
  113.     sail-redis:
  114.         driver: local

Adding Dockerfile

  1. FROM php:8.2-fpm
  3. # Copy composer.lock and composer.json
  4. # COPY composer.lock composer.json /var/www/
  6. # Set working directory
  7. WORKDIR /var/www
  9. # Install dependencies
  10. RUN apt-get update && apt-get install -y \
  11.     build-essential \
  12.     libpng-dev \
  13.     libjpeg62-turbo-dev \
  14.     libfreetype6-dev \
  15.     libwebp-dev \
  16.     locales \
  17.     libzip-dev \
  18.     jpegoptim optipng pngquant gifsicle \
  19.     vim \
  20.     unzip \
  21.     git \
  22.     curl \
  23.     libicu-dev \
  24.     g++ \
  25.     libxml2-dev \
  26.     libonig-dev
  28. # Clear cache
  29. RUN apt-get clean && rm -rf /var/lib/apt/lists/*
  31. # Install extensions
  32. RUN docker-php-ext-install pdo_mysql
  33. RUN docker-php-ext-install mbstring
  34. RUN docker-php-ext-install zip
  35. RUN docker-php-ext-install exif
  36. RUN docker-php-ext-install pcntl
  37. RUN docker-php-ext-configure gd --with-freetype --with-jpeg --with-webp
  38. RUN docker-php-ext-install gd
  39. RUN docker-php-ext-configure intl \
  40. && docker-php-ext-install intl
  41. RUN docker-php-ext-install soap
  42. RUN docker-php-ext-install bcmath
  43. RUN docker-php-ext-install mysqli pdo pdo_mysql && docker-php-ext-enable mysqli
  45. RUN yes | pecl install xdebug \
  46.     && echo "zend_extension=$(find /usr/local/lib/php/extensions/ -name xdebug.so)" > /usr/local/etc/php/conf.d/xdebug.ini \
  47.     && echo "xdebug.remote_enable=on" >> /usr/local/etc/php/conf.d/xdebug.ini \
  48.     && echo "xdebug.remote_autostart=off" >> /usr/local/etc/php/conf.d/xdebug.ini
  50. RUN apt-get -y update
  51. RUN apt-get -y install vim nano
  53. RUN apt-get -y update
  54. RUN apt-get -y install default-mysql-client
  56. # Install composer
  57. RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
  59. # Add user for laravel application
  60. RUN groupadd -g 1000 www
  61. RUN useradd -u 1000 -ms /bin/bash -g www www
  63. # Copy existing application directory contents
  64. COPY docker /var/www
  66. # Copy existing application directory permissions
  67. COPY --chown=www:www docker /var/www
  69. # Change current user to www
  70. USER www
  72. # Expose port 9000 and start php-fpm server
  73. EXPOSE 9000
  74. CMD ["php-fpm"]

答案1

得分: 1

你说你的数据库在设置中共享它们的数据。容器之间共享数据的唯一方法（忽略网络）是通过卷，这里有个问题。 "卷"，即 docker 命令的 -v/--volume 参数或 docker compose 的 volumes 属性，实际上是两个不同的东西：

  • 卷 存储在由 Docker 管理的主机文件系统的一部分（在 Linux 上为 /var/lib/docker/volumes/）。非 Docker 进程不应修改文件系统的这一部分。卷是在 Docker 中持久保存数据的最佳方式。

  • 绑定挂载 可以存储在主机系统的任何位置，甚至可以是重要的系统文件或目录。主机上的非 Docker 进程或 Docker 容器可以随时修改它们。

来自 Docker 手册。

前者，即"真正的" 卷，在主机上共享。因此，如果另一个 compose 文件使用相同的名称定义卷，它指向相同的存储空间。

这就是为什么 Redis 数据被共享的原因。奇怪的是，正如@SIMULATAN在问题评论中指出的那样，MySQL 使用绝对路径，因此数据是通过绑定挂载到主机系统的共同目录共享的。

解决方案要么是使用路径到唯一位置（通常是相对路径）进行绑定挂载，要么是使用卷的名称，这些名称不会共享在你不希望共享的地方。在 前面提到的文档 中列出了各自的优缺点。

如果选择第二个选项，可以设置卷名称甚至是"动态的"可能会有用：

  1. volumes:
  2.   mysql:
  3.     name: "${MY_VAR}" #覆盖上面一行中指定的名称 `data`

[1] 文档中说你可以定义匿名卷，这些卷不能共享，因为它们没有预定义的名称。

英文:

You said that your databases share their data in your setup. The only way (ignoring networking) that the containers could share data is via volumes and there is the catch. The 'volumes', as in the -v/--volume parameter of docker command or the docker compose volumes property, are really two different things:

> Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.

> Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.

From docker manual.

The former one, the "real" volumes, are shared on the host.<sup>[1]</sup> So if another compose file define volume with the same name it points to the same storage space.

That's the reason why the redis data are shared. Oddly enough, as pointed out by the @SIMULATAN in the Q comments, the mysql uses an absolute path therefore the data are shared via bind mounting to a common directory in the host system.

The solution is either to bind mount the data using a paths to unique locations (typically relative paths) or you could use the volumes with names that aren't shared where you don't want. There is pros and cons listed in the already mentioned docs.

If you go with the second option, it could be useful that the volume names could be even set "dynamicly":

  1. volumes:
  2.   mysql:
  3.     name: &quot;${MY_VAR}&quot; #overrides the name `data` specified in the line above

<sup>[1] The docs said you could define anonymous volume, which could not be shared as they don't have predefined name</sup>

huangapple
  • 本文由 发表于 2023年3月21日 02:01:42
  • 转载请务必保留本文链接：https://go.coder-hub.com/75793782.html
  • docker
  • docker-compose
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定