英文:
How to rewind and look at previous offset in a partition using Kafka Go client's Consumer
问题
我是你的中文翻译助手,以下是翻译好的内容:
我对Kafka还不熟悉。目前我正在尝试使用Confluent Inc的Github仓库中的这个Channel Consumer example进行实验。
据我所知,消费者被分成不同的组。每个组在分区中都有自己的偏移量。假设我在一个特定的主题(我们称之为owner_commands)中有40条消息。一个属于dog组的消费者加入并开始消费这40条消息。
当我断开连接并重新连接这个消费者时,我注意到消息不再显示。它显示已经到达文件末尾。然而,如果我用另一个属于不同组(比如cat)的消费者加入集群,我就可以再次读取那40条消息。
你知道在Kafka的Go API中,有没有一种方法让dog组的消费者倒回并重新播放那些消息?我查看了Kafka Golang API的源代码,但没有找到任何可以让我倒回并查看过去特定消息的指示。
谢谢。
英文:
I am new to Kafka. Currently I am experimenting with this Channel Consumer example from Confluent Inc's Github repo
From what I know, consumers are separated into groups. Each group has their own offset in the partition. Let's say I have 40 messages in a particular topic let's call it, owner_commands. A consumer, belongs to the dog group, joins and begins to consume those 40 messages.
When I disconnected and reconnected this consumer, I noticed that messages don't show up anymore. It says that I have reached the end of file. However, if I join the cluster with another consumer, which belongs to a different group (say cat) I get to read those 40 messages again.
Do you know if there is a way for consumers in the dog group to rewind and replay those messages again using Kafka's Go API. I looked at the source code for Kafka Golang API, I couldn't find anything that indicates to me that I can rewind and look at a particular message in the past.
Thank you
答案1
得分: 3
你可以使用CommitOffsets并将其提交回你想要回滚到的偏移量。下一次轮询将从该偏移量开始。
CommitOffsets的文档在这里:
http://docs.confluent.io/current/clients/confluent-kafka-go/index.html#Consumer.CommitOffsets
除了API之外,kafka-consumer-groups命令还提供了移动消费者组位置的功能。这是在Apache Kafka 0.11中发布的。
英文:
You could use CommitOffsets and just commit back to the offset you want to rewind to. The next poll will start from that offset.
CommitOffsets is documented here:
http://docs.confluent.io/current/clients/confluent-kafka-go/index.html#Consumer.CommitOffsets
Outside of the API, there's functionality in the kafka-consumer-groups command to move the position of consumer groups as well. This is released with Apache Kafka 0.11.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论