英文:
Not able to connect between Celery and Minio containers?
问题
This is the Docker Compose configuration you're using for your Django app:
postgres:
image: "postgres:14"
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
volumes:
- .:/opt/myapp
ports:
- "5432:5432"
redis:
image: redis:6
ports:
- "6379:6379"
minio:
image: minio/minio
command: minio server /export
ports:
- "9000:9000"
environment:
- MINIO_ACCESS_KEY=dev_access
- MINIO_SECRET_KEY=dev_secret
celery-worker:
build:
context: .
dockerfile: ./Dockerfile.celery
env_file:
- ./docker.dev.env
environment:
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
depends_on:
- redis
- postgres
command: celery -A myapp worker --loglevel INFO
You mentioned that you want to make the Celery container connect to http://minio:9000/
instead of localhost. To achieve this, you should modify the docker.dev.env
file as follows:
MINIO_STORAGE_ENDPOINT=minio:9000
MINIO_STORAGE_ACCESS_KEY=dev_access
MINIO_STORAGE_SECRET_KEY=dev_secret
MINIO_STORAGE_USE_HTTPS=False
MINIO_STORAGE_MEDIA_BUCKET_NAME=media-dev
MINIO_STORAGE_AUTO_CREATE_MEDIA_BUCKET=True
MINIO_STORAGE_MEDIA_URL=http://minio:9000/ # Update this line
DEFAULT_FILE_STORAGE=minio_storage.storage.MinioMediaStorage
By setting MINIO_STORAGE_MEDIA_URL
to http://minio:9000/
, you should be able to make the Celery container connect to your Minio container correctly. However, make sure your Minio settings and access permissions are configured properly to allow access from the Celery container.
英文:
In my Django app, I'm offloading resizing of uploaded videos to a celery worker. I'm also storing all media files in Minio S3 buckets.
The resizing goes well without any errors, however Celery can't connect to the Minio container. This is the logs from Celery:
celery-worker_1 | [2023-07-19 20:51:46,076: INFO/MainProcess] Task common.tasks.resize_video[c38a90eb-793e-4190-a681-94b69eafadc8] received
celery-worker_1 | ffmpeg version 4.3.6-0+deb11u1 Copyright (c) 2000-2023 the FFmpeg developers
celery-worker_1 | built with gcc 10 (Debian 10.2.1-6)
celery-worker_1 | ffmpeg version 4.3.6-0+deb11u1[tcp @ 0x55c7500727c0] Copyright (c) 2000-2023 the FFmpeg developers Connection to tcp://localhost:9000 failed: Cannot assign requested address
celery-worker_1 | http://localhost:9000/event_media/test_video.mp4: Cannot assign requested address
Question is: how do I make the Celery container to connect to http://minio:9000/
rather than to localhost?
This is the docker-compose.yml file I'm using:
postgres:
image: "postgres:14"
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
volumes:
- .:/opt/myapp
ports:
- "5432:5432"
redis:
image: redis:6
ports:
- "6379:6379"
minio:
image: minio/minio
command: minio server /export
ports:
- "9000:9000"
environment:
- MINIO_ACCESS_KEY=dev_access
- MINIO_SECRET_KEY=dev_secret
celery-worker:
build:
context: .
dockerfile: ./Dockerfile.celery
env_file:
- ./docker.dev.env
environment:
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
depends_on:
- redis
- postgres
command: celery -A myapp worker --loglevel INFO
This is contents of the docker.dev.env
file:
MINIO_STORAGE_ENDPOINT=minio:9000
MINIO_STORAGE_ACCESS_KEY=dev_access
MINIO_STORAGE_SECRET_KEY=dev_secret
MINIO_STORAGE_USE_HTTPS=False
MINIO_STORAGE_MEDIA_BUCKET_NAME=media-dev
MINIO_STORAGE_AUTO_CREATE_MEDIA_BUCKET=True
MINIO_STORAGE_MEDIA_URL=http://localhost:9000/
DEFAULT_FILE_STORAGE=minio_storage.storage.MinioMediaStorage
Edit: Tried to change
MINIO_STORAGE_MEDIA_URL=http://minio:9000/
Which instead gave me this following error:
celery-worker_1 | [http @ 0x55698dd429c0] HTTP error 403 Forbidden
celery-worker_1 | http://minio:9000/event_media/test_video.mp4: Server returned 403 Forbidden (access denied)
Still scratching my head
答案1
得分: 0
经过仔细查阅https://django-minio-storage.readthedocs.io/en/latest/usage/#django-settings-configuration上的文档,特别是设置键MINIO_STORAGE_MEDIA_URL
,我更新了docker.dev.env
:
从
MINIO_STORAGE_MEDIA_URL=http://localhost:9000/
到
MINIO_STORAGE_MEDIA_URL=http://minio:9000/media-dev
我学到的是:
localhost
指向了自己的容器,因此需要使用容器名称minio
(如在docker-compose.yml
中定义)。MINIO_STORAGE_MEDIA_URL
也应包含 minio 存储桶的名称,在这种情况下是“media-dev”,由MINIO_STORAGE_MEDIA_BUCKET_NAME
定义。
英文:
After pouring over the documentation at https://django-minio-storage.readthedocs.io/en/latest/usage/#django-settings-configuration, and specifically the setting key MINIO_STORAGE_MEDIA_URL
, I updated docker.dev.env
:
From
MINIO_STORAGE_MEDIA_URL=http://localhost:9000/
To
MINIO_STORAGE_MEDIA_URL=http://minio:9000/media-dev
What I learned was:
- that
localhost
points on its own container, thus need to use the container nameminio
instead (as defined indocker-compose.yml
). - The
MINIO_STORAGE_MEDIA_URL
should also contain the name of the minio bucket, in this case "media-dev" as defined byMINIO_STORAGE_MEDIA_BUCKET_NAME
.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论