Docker data best practices

I've been using Docker for about 6 months. Up until last Friday I thought I had found the holy grail. I love all the benefits of Docker, where I can upgrade my container but leave my data intact.

On Friday, something happened which lead me to wonder if there is an even better way. When I signed into my account on Friday Docker asked me to upgrade, so I did. When Docker restarted I got a fatal error saying that Docker wasn't able to start. I ended up uninstalling Docker and reinstalling. No big deal, or so I thought. All of my data is gone. This leads me to a couple questions. For context, here is the database service definition in my docker-compose file.

db:
    image: mariadb:10.9
    volumes:
        - mariadb_data:/var/lib/mysql
    environment:
        TZ: 'America/Chicago'
        MYSQL_ALLOW_EMPTY_PASSWORD: 'no'
        MYSQL_ROOT_PASSWORD: 'rootpwd'
        MYSQL_USER: 'User'
        MYSQL_PASSWORD: 'UserPassword1234'
        MYSQL_DATABASE: 'Database'
    ports:
        - 3306:3306
    restart: always
    networks:
        - backend
    healthcheck:
        test: mysqladmin ping -h 127.0.0.1 -u $$MYSQL_USER --password=$$MYSQL_PASSWORD
        start_period: 5s
        interval: 5s
        timeout: 5s
        retries: 55

volumes:
mariadb_data:

So, with that being said. I have the following questions:

• is my data truly gone? Or is there some super secret archive that Docker maintains that I can retrieve?
• what is the best practice surrounding data and containers?
• ideally, what I would like to do is have a system set up where my data can exist. Then when django comes out with a new version, or when MariaDb comes out with an update, I can just update them. That's what I like about Docker. However, I hadn't thought of Docker itself having issues. Is there a way that I can update my containers regularly, but still leave my data intact even if Docker has an issue?

Your setup as you've described it seems fine, but your data is probably gone. Can you mysqlrestore it from a dump?

In Docker in general, there are two places you can store data. A bind mount refers to a path on the host system; this can sometimes introduce permission problems, and especially using Docker Desktop it can be slower. A named volume stores the data in space controlled by Docker, which can make it hard to access directly.

For database data, my typical recommendation is to prefer a named volume, since that avoids the potential permission and performance problems, and you can't really do much useful with the opaque database storage.

However:

> I ended up uninstalling Docker and reinstalling.

If you're using Docker Desktop and do this, it will destroy all Docker content. Containers are inherently temporary, and images can either be pulled or rebuilt, but this also destroys named-volume content. So...sorry, but the data is gone.

IME you should not normally need this "destroy everything" path but it does occasionally happen. (I haven't needed to for many years now, but I have sudo rm -rf /var/lib/docker on native Linux in the past, which similarly destroys named volumes.)

Even if you're using Docker, general computing practices like "make backups" still apply. You'll also have to worry about data integrity if you update the database, in the same way you would on bare metal.