docker compose / cache-from cache-to / buildx: The container name /buildx_buildkit is already in use

Motivation
I have a docker compose which has several services which potentially are so large that they impact ci times significantly due to loading from registry.
To reduce build & start times I I'm using --cache-to & --cache-from for many of the services.

Using caches requires docker buildx.

Problem
docker compose build leads to somewhat a parallel execution of buildx and a name conflict for the container name "/buildx_buildkit_*"

Scroll past the Solution section to see the files I'm using with the error message on the bottom

Solution
See fbjorn's answer
Also pay attention to the following: at the time of writing, usually the image docker:20.10.16-dind is being used for cicd dind pipelines. To make this scenario work, I had to use docker:dind (ie latest dind)

# dind.yml
version: '3.9'

services:
  dind:
    image: docker:dind
    privileged: true
    restart: always
    volumes:
      - /dockerbuild:/build/dockerbuild

# docker-compose.yml
version: '3.9'

services:
  my_service:
    build:
      context: .
      cache_from:
        - type=local,src=./${CACHE_DIR:-build-cache}/my_service
      cache_to:
        - type=local,dest=./${CACHE_DIR:-build-cache}/my_service
      dockerfile: Dockerfile
    image: ${MY_IMAGE_TAG:-dockerbuild:latest}
    command: tail -f /dev/null
    
  your_service:
    build:
      context: .
      cache_from:
        - type=local,src=./${CACHE_DIR:-build-cache}/your_service
      cache_to:
        - type=local,dest=./${CACHE_DIR:-build-cache}/your_service
      dockerfile: Dockerfile
    image: ${MY_IMAGE_TAG:-dockerbuild:latest}
    command: tail -f /dev/null
    
  # sqlserver:
  #   build: 
  #     context: ./Dockerfile_sqlserver
  #     cache_from:
  #       - type=local,src=./${CACHE_DIR:-build-cache}/mssql
  #     cache_to:
  #       - type=local,dest=./${CACHE_DIR:-build-cache}/mssql
  #   image: mcr.microsoft.com/mssql/server:2019-CU19-ubuntu-20.04
    

# Dockerfile
FROM alpine:latest

CMD ["/bin/sh"]

# snippet from gitlab-ci - I'm running these commands in the dind container.
script:
    - cd /build/dockerbuild
    - docker buildx create --use --name my_builder_1 
      # my_builder_1
    - docker compose build
      # /build/dockerbuild # docker compose build
      # [+] Building 38.4s (2/2) FINISHED
      #  => CANCELED [your_service internal] booting buildkit
      #  => => pulling image moby/buildkit:buildx-stable-1
      #  => => creating container buildx_buildkit_my_builder_10
      #  => ERROR [my_service internal] booting buildkit
      #  => => pulling image moby/buildkit:buildx-stable-1
      #  => => creating container buildx_buildkit_my_builder_10  s
      # ------
      #  > [my_service internal] booting buildkit:
      # ------
      # Error response from daemon: Conflict. The container name "/buildx_buildkit_my_builder_10" is already in use by container "...". 
      # You have to remove (or rename) that container to be able to reuse that name.

I was facing the similar problem by building multiple images at one. It also said The container name "/buildx_buildkit_*" is already in use by container <..>.

Solution for me was to bootstrap the builder when creating it by appending --bootstrap flag. Try changing your command to:

docker buildx create --use --bootstrap --name my_builder_1 

Then your build commands won't try to bootstrap the container because it's already done.