两个实例从主题消费。

huangapple go评论79阅读模式
英文:

Two instances consuming from topic

问题

我已部署了两个应用程序实例这两个应用程序运行相同的代码并且从相同的主题消费

@KafkaListener( offsetReset = OffsetReset.EARLIEST, offsetStrategy = OffsetStrategy.DISABLED )
public class AppConsumer implements ConsumerRebalanceListener, KafkaConsumerAware {

@Topic("topic")
public void consumeAppInfo(@KafkaKey String name, @Body @Nullable String someString) {
     ...
}

}


我遇到了一个问题,只有一个应用程序会消费消息。这个主题只有一个分区,`分区 0`,我相信这是默认设置。

我尝试过向KafkaListener添加group-id和threads。有时候这似乎有效,而其他时候则无效。

@KafkaListener(groupId="myGroup", threads=10)


让两个应用程序都消费相同消息的最简单解决方案是什么?
英文:

I have deployed two instances of an application. Both applications runs the same code and consumes from the same topic.

@KafkaListener( offsetReset = OffsetReset.EARLIEST, offsetStrategy = OffsetStrategy.DISABLED )
public class AppConsumer implements ConsumerRebalanceListener, KafkaConsumerAware {

    @Topic("topic")
    public void consumeAppInfo(@KafkaKey String name, @Body @Nullable String someString) {
         ...
    }

}

I have a problem where only one of the applications consumes the message. The topic has only one partition, partition 0, which i believe is default.

I have tried to add group-id and threads to the KafkaListener. This seems to work sometimes and other time not.

@KafkaListener(groupId="myGroup", threads=10)

What is the simplest solution to getting both applications to consume the same message?

答案1

得分: 2

你无法执行组操作,只需为每个应用程序分配一个单独的消费者标识,每个消费者都会消费所有消息(除非它们还被分配到一个组)。

组用于并行处理消息,组中的每个消费者都会被分配到一个分区,以处理消息。

更多信息 => https://stackoverflow.com/questions/34550873/difference-between-groupid-and-consumerid-in-kafka-consumer

英文:

You could not do the group and just give each application a separate consumer id each consumer consumes all messages (unless they are also assigned to a group).

Groups are used for parallel processing of messages each consumer in a group get assigned to a partition for processing messages.

More info => https://stackoverflow.com/questions/34550873/difference-between-groupid-and-consumerid-in-kafka-consumer

答案2

得分: 1

在 Kafka 中,每个分区只会分配给消费者群组中的一个消费者。如果您希望不同的应用程序从同一分区消费数据,您需要为每个不同的应用程序/实例指定不同的消费者群组。

英文:

In kafka, only one consumer within consumer group is assigned to each partition. If you want to consume the data from the same partition by different applications, you need to specify different consumer groups for each different application / instance.

huangapple
  • 本文由 发表于 2020年8月24日 22:15:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/63562834.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定