英文:
Kafka - Tracking the data reads
问题
Kafka主题有多个消费者（来自不同的消费者组）时，谁会负责维护偏移量或已读取的数据（消费者组、主题、分区）？是Kafka还是消费者必须实现逻辑以跟踪已读取的数据？例如：如果Spark正在从Kafka读取数据，它会维护一个检查点来跟踪已读取的数据。
英文:
when a Kafka topic has multiple consumers (from different consumer group), who would maintain keeping track of offsets or the data read (consumer-group, topic, partition)? Is it Kafka or consumers have to implement a logic to keep track of the data read? for ex: if Spark is reading the data from Kafka it maintains a checkpoint keeping track of the data read.
答案1
得分: 1
Kafka有一个内部主题__consumer_offsets，当使用消费者组时，消费者将其偏移量提交到该主题。
英文:
Kafka has an internal topic __consumer_offsets that Consumers commit their offsets to when using a consumer group.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论