英文:
Two instances consuming from topic
问题
我已部署了两个应用程序实例。这两个应用程序运行相同的代码，并且从相同的主题消费。
@KafkaListener( offsetReset = OffsetReset.EARLIEST, offsetStrategy = OffsetStrategy.DISABLED )
public class AppConsumer implements ConsumerRebalanceListener, KafkaConsumerAware {
@Topic("topic")
public void consumeAppInfo(@KafkaKey String name, @Body @Nullable String someString) {
...
}
}
我遇到了一个问题，只有一个应用程序会消费消息。这个主题只有一个分区，`分区 0`，我相信这是默认设置。
我尝试过向KafkaListener添加group-id和threads。有时候这似乎有效，而其他时候则无效。
@KafkaListener(groupId="myGroup", threads=10)
让两个应用程序都消费相同消息的最简单解决方案是什么？
英文:
I have deployed two instances of an application. Both applications runs the same code and consumes from the same topic.
@KafkaListener( offsetReset = OffsetReset.EARLIEST, offsetStrategy = OffsetStrategy.DISABLED )
public class AppConsumer implements ConsumerRebalanceListener, KafkaConsumerAware {
@Topic("topic")
public void consumeAppInfo(@KafkaKey String name, @Body @Nullable String someString) {
...
}
}
I have a problem where only one of the applications consumes the message. The topic has only one partition, partition 0, which i believe is default.
I have tried to add group-id and threads to the KafkaListener. This seems to work sometimes and other time not.
@KafkaListener(groupId="myGroup", threads=10)
What is the simplest solution to getting both applications to consume the same message?
答案1
得分: 2
你无法执行组操作，只需为每个应用程序分配一个单独的消费者标识，每个消费者都会消费所有消息（除非它们还被分配到一个组）。
组用于并行处理消息，组中的每个消费者都会被分配到一个分区，以处理消息。
英文:
You could not do the group and just give each application a separate consumer id each consumer consumes all messages (unless they are also assigned to a group).
Groups are used for parallel processing of messages each consumer in a group get assigned to a partition for processing messages.
More info => https://stackoverflow.com/questions/34550873/difference-between-groupid-and-consumerid-in-kafka-consumer
答案2
得分: 1
在 Kafka 中，每个分区只会分配给消费者群组中的一个消费者。如果您希望不同的应用程序从同一分区消费数据，您需要为每个不同的应用程序/实例指定不同的消费者群组。
英文:
In kafka, only one consumer within consumer group is assigned to each partition. If you want to consume the data from the same partition by different applications, you need to specify different consumer groups for each different application / instance.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论