Docker-compose in multiple projects same services different container but shared volumes and data are in wrong projects

huangapple go评论51阅读模式
英文:

Docker-compose in multiple projects same services different container but shared volumes and data are in wrong projects

问题

你有很多进行中的网站项目,每个项目都有自己的docker-compose.yml文件,它们基本上使用相同的服务(PHP、MySQL、Nginx等)。

你在.env文件中为每个项目使用独特的COMPOSE_PROJECT_NAME,并且它们有动态的容器名称:

mysql:
    image: 'mysql:8.0'
    container_name: '${APP_NAME}_mysql'

我的问题如下:

  1. 对于project1创建的数据库在project2的mysql中可见,反之亦然。
  2. 为project2重建或销毁mysql会同时影响project1,因此我失去了数据库。
  3. 这变成了一场噩梦,因为我有不止两个项目。

更多信息:有时一些项目使用PHP7.x,而另一些使用PHP8.x,当我编辑PHP服务的Dockerfile并更改版本后重建时,所有项目现在都使用新的PHP版本,这显然可能会破坏它们。

环境:Windows 10 Pro + WSL2 Ubuntu

以下是完整的Docker Compose文件:

version: '3'
services:
    # PHP Service
    app:
        # 配置省略
    webserver:
        # 配置省略
    mysql:
        # 配置省略
    pma:
        # 配置省略
    redis:
        # 配置省略
    mailhog:
        # 配置省略
networks:
    sail:
        driver: bridge
volumes:
    sail-mysql:
        driver: local
    sail-redis:
        driver: local

另外,这是Dockerfile的内容:

# Dockerfile 配置省略

请让我知道你需要进一步了解的内容。

英文:

Hi I have many ongoing website projects, each with their own docker-compose.yml. They basically use the same services (PHP, MySQL, Nginx, etc).

I do use unique COMPOSE_PROJECT_NAME in .env for each project and they have dynamic container names :

    mysql:
        image: 'mysql:8.0'
        container_name: '${APP_NAME}_mysql'
/projects
    /project1
         /.env
         /docker-compose.yml
    /project2
         /.env
         /docker-compose.yml

My problem is the following :
database created for project1 is visible in the mysql of the project2 and vice versa
rebuilding or destroying mysql for project2, destroys it for project1 also, so I lose database.
this becomes a nightmare because I have a lot more than 2 projects.

More info: it also happens that some projects are in PHP7.x and others in PHP8.x, and when I edit my Dockerfile for the PHP service and change the version and rebuild, than all projects are now on the new PHP version, which obviously might break them.

Environment : windows10pro + wsl2 Ubuntu

Adding docker compose file for completeness:

    # For more information: https://laravel.com/docs/sail
version: '3'
services:
    #PHP Service
    app:
        build:
            context: .
            dockerfile: Dockerfile
        image: digitalocean.com/php
        container_name: ${APP_NAME}_app
        restart: unless-stopped
        extra_hosts:
            - "host.docker.internal:${HOST_GATEWAY}" 
        tty: true
        environment:
            SERVICE_NAME: app
            SERVICE_TAGS: dev
        working_dir: /var/www
        volumes:
            - ./:/var/www
            - ./docker/php/local.ini:/usr/local/etc/php/conf.d/local.ini
            - ./docker/php/docker-php-ext-xdebug.ini:/usr/local/etc/php/conf.d/xdebug.ini
        networks:
            - sail
        depends_on:
            - mysql
            - redis
            - mailhog

    #Nginx Service
    webserver:
        image: nginx:alpine
        container_name: ${APP_NAME}_webserver
        restart: unless-stopped
        tty: true
        ports:
            - "80:80"
            - "443:443"
        volumes:
            - ./:/var/www
            - ./docker/nginx/conf.d/:/etc/nginx/conf.d/
            - ./docker/certs:/etc/nginx/certs
        networks:
            - sail
        extra_hosts:
            - "host.docker.internal:host-gateway"            
    mysql:
        image: mysql:5.7.22
        container_name: ${COMPOSE_PROJECT_NAME}_db
        restart: unless-stopped
        tty: true
        ports:
            - '${FORWARD_DB_PORT:-3306}:3306'
        environment:
            MYSQL_DATABASE: '${DB_DATABASE}'
            MYSQL_USER: '${DB_USERNAME}'
            MYSQL_PASSWORD: '${DB_PASSWORD}'
            MYSQL_ROOT_PASSWORD: '${DB_ROOT_PASSWORD}'
            MYSQL_ROOT_HOST: "%"
            MYSQL_ALLOW_EMPTY_PASSWORD: 1
            SERVICE_TAGS: dev
            SERVICE_NAME: mysql
        volumes:
            - /docker/mysql-dump:/var/lib/mysql
            - ./docker/mysql/my.cnf:/etc/mysql/my.cnf
        networks:
            - sail
        healthcheck:
            test: ["CMD", "mysqladmin", "ping", "-p${DB_PASSWORD}"]
            retries: 3
            timeout: 5s
    pma:
        image: phpmyadmin/phpmyadmin
        container_name: ${APP_NAME}_pma
        restart: always
        depends_on:
            - mysql
        environment:
            PMA_HOST: ${DB_HOST}
            MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
            UPLOAD_LIMIT: 300M
        ports:
            - '8282:80'
        networks:
            - sail  
    redis:
        image: 'redis:alpine'
        container_name: ${APP_NAME}_redis
        ports:
            - '${FORWARD_REDIS_PORT:-6379}:6379'
        volumes:
            - 'sail-redis:/data'
        networks:
            - sail
        healthcheck:
            test: ["CMD", "redis-cli", "ping"]
            retries: 3
            timeout: 5s
    mailhog:
        image: 'mailhog/mailhog:latest'
        container_name: ${APP_NAME}_mailhog
        ports:
            - '${FORWARD_MAILHOG_PORT:-1025}:1025'
            - '${FORWARD_MAILHOG_DASHBOARD_PORT:-8025}:8025'
        networks:
            - sail
networks:
    sail:
        driver: bridge
volumes:
    sail-mysql:
        driver: local
    sail-redis:
        driver: local

Adding Dockerfile

FROM php:8.2-fpm

# Copy composer.lock and composer.json
# COPY composer.lock composer.json /var/www/

# Set working directory
WORKDIR /var/www

# Install dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    libpng-dev \
    libjpeg62-turbo-dev \
    libfreetype6-dev \
    libwebp-dev \
    locales \
    libzip-dev \
    jpegoptim optipng pngquant gifsicle \
    vim \
    unzip \
    git \
    curl \
    libicu-dev \
    g++ \
    libxml2-dev \
    libonig-dev

# Clear cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*

# Install extensions
RUN docker-php-ext-install pdo_mysql
RUN docker-php-ext-install mbstring
RUN docker-php-ext-install zip
RUN docker-php-ext-install exif
RUN docker-php-ext-install pcntl
RUN docker-php-ext-configure gd --with-freetype --with-jpeg --with-webp
RUN docker-php-ext-install gd
RUN docker-php-ext-configure intl \
&& docker-php-ext-install intl
RUN docker-php-ext-install soap
RUN docker-php-ext-install bcmath
RUN docker-php-ext-install mysqli pdo pdo_mysql && docker-php-ext-enable mysqli

RUN yes | pecl install xdebug \
    && echo "zend_extension=$(find /usr/local/lib/php/extensions/ -name xdebug.so)" > /usr/local/etc/php/conf.d/xdebug.ini \
    && echo "xdebug.remote_enable=on" >> /usr/local/etc/php/conf.d/xdebug.ini \
    && echo "xdebug.remote_autostart=off" >> /usr/local/etc/php/conf.d/xdebug.ini

RUN apt-get -y update
RUN apt-get -y install vim nano

RUN apt-get -y update
RUN apt-get -y install default-mysql-client

# Install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

# Add user for laravel application
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www

# Copy existing application directory contents
COPY docker /var/www

# Copy existing application directory permissions
COPY --chown=www:www docker /var/www

# Change current user to www
USER www

# Expose port 9000 and start php-fpm server
EXPOSE 9000
CMD ["php-fpm"]

答案1

得分: 1

你说你的数据库在设置中共享它们的数据。容器之间共享数据的唯一方法(忽略网络)是通过卷,这里有个问题。 "卷",即 docker 命令的 -v/--volume 参数或 docker compose 的 volumes 属性,实际上是两个不同的东西:

  • 存储在由 Docker 管理的主机文件系统的一部分(在 Linux 上为 /var/lib/docker/volumes/)。非 Docker 进程不应修改文件系统的这一部分。卷是在 Docker 中持久保存数据的最佳方式。

  • 绑定挂载 可以存储在主机系统的任何位置,甚至可以是重要的系统文件或目录。主机上的非 Docker 进程或 Docker 容器可以随时修改它们。

来自 Docker 手册

前者,即"真正的" ,在主机上共享。因此,如果另一个 compose 文件使用相同的名称定义卷,它指向相同的存储空间。

这就是为什么 Redis 数据被共享的原因。奇怪的是,正如@SIMULATAN在问题评论中指出的那样,MySQL 使用绝对路径,因此数据是通过绑定挂载到主机系统的共同目录共享的。

解决方案要么是使用路径到唯一位置(通常是相对路径)进行绑定挂载,要么是使用的名称,这些名称不会共享在你不希望共享的地方。在 前面提到的文档 中列出了各自的优缺点。

如果选择第二个选项,可以设置卷名称甚至是"动态的"可能会有用:

volumes:
  mysql:
    name: "${MY_VAR}" #覆盖上面一行中指定的名称 `data`

[1] 文档中说你可以定义匿名卷,这些卷不能共享,因为它们没有预定义的名称。

英文:

You said that your databases share their data in your setup. The only way (ignoring networking) that the containers could share data is via volumes and there is the catch. The 'volumes', as in the -v/--volume parameter of docker command or the docker compose volumes property, are really two different things:

> Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.

> Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.

From docker manual.

The former one, the "real" volumes, are shared on the host.<sup>[1]</sup> So if another compose file define volume with the same name it points to the same storage space.

That's the reason why the redis data are shared. Oddly enough, as pointed out by the @SIMULATAN in the Q comments, the mysql uses an absolute path therefore the data are shared via bind mounting to a common directory in the host system.

The solution is either to bind mount the data using a paths to unique locations (typically relative paths) or you could use the volumes with names that aren't shared where you don't want. There is pros and cons listed in the already mentioned docs.

If you go with the second option, it could be useful that the volume names could be even set "dynamicly":

volumes:
  mysql:
    name: &quot;${MY_VAR}&quot; #overrides the name `data` specified in the line above

<sup>[1] The docs said you could define anonymous volume, which could not be shared as they don't have predefined name</sup>

huangapple
  • 本文由 发表于 2023年3月21日 02:01:42
  • 转载请务必保留本文链接:https://go.coder-hub.com/75793782.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定