英文:
kafka replication factor less then broker count
问题
我有一个使用副本因子为2、分区数为1且经纪人数为3的Kafka主题。
如果在消费者中将所有3个经纪人指定为引导服务器，消息会被正确读取吗？
当消费者到达一个没有副本的节点时，消息是否可能不被消耗？还是不是这样工作的？
英文:
I have a Kafka topic created with replication factor=2, partition count = 1 and brokers count = 3.
Will messages be correctly read if all 3 brokers are specified as bootstrap servers in consumer?
Can it be, that messages are not consumed, when consumer gets to a node, that doesn't have a replica? Or it doesn't work that way?
答案1
得分: 2
Kafka代理在内部彼此通信。
这意味着，如果您的消费者只知道其中一个代理，它将与群集中的所有代理通信，以找出分区领导位于哪个代理上。然后您的消费者将从该代理消费。
然而，通常最好不仅提供一个代理地址，而是在您的消费者配置的bootstrap.servers中提供一个逗号分隔的地址列表，可以是一个或多个甚至是所有地址。这样，即使其中一个代理失效，您的消费者仍将能够获取数据。
如果在消费者的引导服务器中指定了所有3个代理，消息是否会被正确读取？
是的。然而，每个分区都有一个分区领导者，位于其中一个代理上，因此您的消费者只会从该代理消费数据。
当消费者到达没有副本的节点时，消息是否可能未被消耗？
不会。如上所述，代理之间正在进行通信，以找出分区领导位于哪个代理上。
英文:
Kafka brokers are communicating internally which each other.
That means, if your consumer only knows about one of the brokers it will communicate with all brokers in the cluster to figure out on which broker the partition leader is located. Then your consumer will consume from that broker.
However, it is usually best to not only provide one of the broker addresses but rather more than one or even all addresses as a comma separated list in your consumer configuration bootstrap.servers. That way, even if one of the brokers goes down, your consumer will still be able to fetch the data.
> Will messages be correctly read if all 3 brokers are specified as bootstrap servers in consumer?
Yes. However, each partition as a partition leader that sits on one of the brokers, so your consumer will only consume data from that one broker.
> Can it be, that messages are not consumed, when consumer gets to a node, that doesn't have a replica?
No. As explained above, the brokers are communicating with each other to figure out on which broker the partition leader is located.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论