英文:
why should i use docker image "confluentinc/kafka" to kafka cluster?
问题
我在本地使用以下命令进行Kafka集群的Docker容器化部署:
docker-compose up
每当我想要创建主题、检索已有主题或搜索主题中存储的数据时,我使用以下命令:
docker run --net=host --rm confluentinc/cp-kafka:latest kafka-topics --describe --topic bar --zookeeper localhost:32181
这些命令是在官方Kafka集群部署站点上使用的。然而,我真的想在本地存储中使用Kafka,而不是Kafka Docker镜像,如下所示:
kafka-topics --describe --topic bar --zookeeper localhost:32181
但是,如果我使用上面的代码,我会遇到以下错误:
Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
此外,如果我检查netstat -anp tcp
,我发现Kafka集群没有LISTEN端口。
我应该怎么做?请告诉我关于Docker的新手常见问题,因为我真的是Docker的新手
以下是YAML配置文件:
---
version: '2'
services:
zookeeper-1:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_CLIENT_PORT: 22181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
network_mode: host
extra_hosts:
- "moby:127.0.0.1"
zookeeper-2:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_SERVER_ID: 2
ZOOKEEPER_CLIENT_PORT: 32181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
network_mode: host
extra_hosts:
- "moby:127.0.0.1"
zookeeper-3:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_SERVER_ID: 3
ZOOKEEPER_CLIENT_PORT: 42181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
network_mode: host
extra_hosts:
- "moby:127.0.0.1"
kafka-1:
image: confluentinc/cp-kafka:latest
network_mode: host
depends_on:
- zookeeper-1
- zookeeper-2
- zookeeper-3
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
extra_hosts:
- "moby:127.0.0.1"
kafka-2:
image: confluentinc/cp-kafka:latest
network_mode: host
希望这对你有所帮助,如果你有任何其他问题,请随时提问。
英文:
I did
docker-compose up
for kafka clustering in my local.
and whenever i want to make topic, retrieve topics i have or search data stored in the topic i use
docker run --net=host --rm confluentinc/cp-kafka:latest kafka-topics --describe --topic bar --zookeeper localhost:32181
which is on the official kafka clustering deployment site.
However i really want to use Kafka in my local storage not the kafka docker image like
kafka-topics --describe --topic bar --zookeeper localhost:32181
if i use the code above i face this error.
Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:258)
Besides if i check netstat -anp tcp
there is no LISTEN ports from kafka cluster.
What am i supposed to do ?
and please let me know what i am missing now about docker (because i am really new to docker )
This is yaml configuration
---
version: '2'
services:
zookeeper-1:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_CLIENT_PORT: 22181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
network_mode: host
extra_hosts:
- "moby:127.0.0.1"
zookeeper-2:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_SERVER_ID: 2
ZOOKEEPER_CLIENT_PORT: 32181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
network_mode: host
extra_hosts:
- "moby:127.0.0.1"
zookeeper-3:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_SERVER_ID: 3
ZOOKEEPER_CLIENT_PORT: 42181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
network_mode: host
extra_hosts:
- "moby:127.0.0.1"
kafka-1:
image: confluentinc/cp-kafka:latest
network_mode: host
depends_on:
- zookeeper-1
- zookeeper-2
- zookeeper-3
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
extra_hosts:
- "moby:127.0.0.1"
kafka-2:
image: confluentinc/cp-kafka:latest
network_mode: host
"docker-compose.yml" 83L, 2321C
答案1
得分: 1
如果您正在主机上运行Kafka(和Zookeeper),那么不,您根本不需要Docker。
您不应在容器的任何连接字符串中使用 localhost
,而应使用容器的外部服务名称,并从配置中删除 network_mode
。
注意:在一台机器上运行多个代理/ Zookeeper 并没有什么帮助。
或者搜索存储在主题中的数据
您不应该使用Zookeeper来执行这项操作。
英文:
If you are running Kafka (and Zookeeper) on your host, then, no, you don't need Docker at all.
You should not use localhost
in any connection string from the container, rather use the external service names of the containers and remove network_mode
from the configurations.
Note: multiple brokers/zookeepers on one machine isn't helping anything.
> or search data stored in the topic
You shouldn't use zookeeper to do that
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论