英文:
Why kakfa topic partition is not receiving messages?
问题
我有一个包含3个代理（brokers）和一个包含8个分区（partitions）的Kafka集群。
一个使用Spring Boot编写的Java生产者，没有自定义的负载均衡规则，这意味着应该采用轮询方式。
问题在于，有一些分区没有接收到消息。我查明了4个消费者接收到了什么，即使它们处理了所有消息，但有一个消费者一直处于空闲状态，因为它只接收到了一条消息。
可能的问题是什么？
我使用的Kafka版本是0.10.1.1。
此案例中，我没有使用分区的复制功能。
英文:
I have a kafka cluster with 3 brokers and a topic with 8 partitions.
A producer written in java using spring boot and without custom rule for load balancing. It means it should do round robin.
The issue is that there are some partitions there are not receiving messages into it. I figured it out checking what the 4 consumers are receiving and even they are processing all messages there is a consumer idle all the time because it has received just one message.
What could be the issue?
Kafka version I'm using is 0.10.1.1
Additional note in this case I'm not using replicas for the partitions
答案1
得分: 1
> 这意味着它应该执行轮询。
仅当您的Kafka消息中没有keys时，它才会执行轮询。否则，消息会根据键的哈希值进行分区：
hash(key) % number_of_partitions
很常见的情况是，这会导致一些分区根本不会收到任何消息。想象一个情况，您使用的键只能有两个不同的值。在这种情况下，所有的数据都会流入仅两个分区，与主题中分区的数量无关。
英文:
> It means it should do round robin.
It will only do round robin, if you have no keys in your Kafka messages. Otherwise, the messages are partitioned based on a hash value of the key:
hash(key) % number_of_partitions
It is not unusal, that this will cause some partitions to not receive any messages at all. Imagine a case, where you are using a key that can only have two different values. In that case, all your data will flow into only two partitions, independent of the number of partitions in your topic.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论