You should use Docker image “confluentinc/kafka” 为了 Kafka 集群吗?

huangapple go评论76阅读模式
英文:

why should i use docker image "confluentinc/kafka" to kafka cluster?

问题

我在本地使用以下命令进行Kafka集群的Docker容器化部署:

docker-compose up

每当我想要创建主题、检索已有主题或搜索主题中存储的数据时,我使用以下命令:

docker run --net=host --rm confluentinc/cp-kafka:latest kafka-topics --describe --topic bar --zookeeper localhost:32181

这些命令是在官方Kafka集群部署站点上使用的。然而,我真的想在本地存储中使用Kafka,而不是Kafka Docker镜像,如下所示:

kafka-topics --describe --topic bar --zookeeper localhost:32181

但是,如果我使用上面的代码,我会遇到以下错误:

Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING

此外,如果我检查netstat -anp tcp,我发现Kafka集群没有LISTEN端口。

我应该怎么做?请告诉我关于Docker的新手常见问题,因为我真的是Docker的新手 You should use Docker image “confluentinc/kafka” 为了 Kafka 集群吗?

以下是YAML配置文件:

---
version: '2'
services:
  zookeeper-1:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 22181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  zookeeper-2:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 2
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  zookeeper-3:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 3
      ZOOKEEPER_CLIENT_PORT: 42181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  kafka-1:
    image: confluentinc/cp-kafka:latest
    network_mode: host
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
    extra_hosts:
      - "moby:127.0.0.1"

  kafka-2:
    image: confluentinc/cp-kafka:latest
    network_mode: host

希望这对你有所帮助,如果你有任何其他问题,请随时提问。

英文:

I did

docker-compose up

for kafka clustering in my local.

and whenever i want to make topic, retrieve topics i have or search data stored in the topic i use

docker run --net=host --rm confluentinc/cp-kafka:latest kafka-topics --describe --topic bar --zookeeper localhost:32181

which is on the official kafka clustering deployment site.

However i really want to use Kafka in my local storage not the kafka docker image like

kafka-topics --describe --topic bar --zookeeper localhost:32181

if i use the code above i face this error.

Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
        at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:258)

Besides if i check netstat -anp tcp
there is no LISTEN ports from kafka cluster.

What am i supposed to do ?
and please let me know what i am missing now about docker (because i am really new to docker You should use Docker image “confluentinc/kafka” 为了 Kafka 集群吗? )

This is yaml configuration

---
version: '2'
services:
  zookeeper-1:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 22181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  zookeeper-2:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 2
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  zookeeper-3:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 3
      ZOOKEEPER_CLIENT_PORT: 42181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  kafka-1:
    image: confluentinc/cp-kafka:latest
    network_mode: host
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
    extra_hosts:
      - "moby:127.0.0.1"

  kafka-2:
    image: confluentinc/cp-kafka:latest
    network_mode: host
"docker-compose.yml" 83L, 2321C

答案1

得分: 1

如果您正在主机上运行Kafka(和Zookeeper),那么不,您根本不需要Docker。

您不应在容器的任何连接字符串中使用 localhost,而应使用容器的外部服务名称,并从配置中删除 network_mode

注意:在一台机器上运行多个代理/ Zookeeper 并没有什么帮助。

或者搜索存储在主题中的数据

您不应该使用Zookeeper来执行这项操作。

英文:

If you are running Kafka (and Zookeeper) on your host, then, no, you don't need Docker at all.

You should not use localhost in any connection string from the container, rather use the external service names of the containers and remove network_mode from the configurations.

Note: multiple brokers/zookeepers on one machine isn't helping anything.

> or search data stored in the topic

You shouldn't use zookeeper to do that

huangapple
  • 本文由 发表于 2020年1月6日 17:35:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/59609677.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定