英文:
Out of order commit behaviour in case of errors
问题
我喜欢使用Spring Kafka记录消息监听器以及无序提交功能。由于处理记录可能会因相关网络调用等原因需要一些时间,我的想法是将消费的消息提交到线程池中。在那里,消息将异步处理,并通过调用确认方法来提交。
然而,我想知道在使用无序提交时Spring Kafka在出现错误时的行为。文档中指出,不应与此功能一起使用nack方法。此外,监听器容器会延迟提交,直到接收到所有确认并在此期间暂停消费者。
文档没有提供任何关于如果记录从未被确认会发生什么的提示,例如由于池中的线程内发生异常而传递回消息监听器。
所以我的问题是Spring Kafka在这些情况下的行为是怎样的?消息是否会重新播放,还是我需要自己处理重试?
感谢您的帮助!
英文:
I like to use a Spring Kafka record message listener with the out of order commit feature. As processing of records may take some time due to e.g. related network calls, my idea is to submit the consumed messages into a thread pool. There the messages will be processed asynchronously and commited by calling the acknowledge method.
However I wondered how Spring Kafka behaves in the case errors while using out of order commits. The documentation states, that the nack method must not be used together with this feature. Moreover, the listener container defers the commit until all acknowledgements are received and the consumer is paused in the meanwhile.
The documentation doesn't give any hint what happens if a record is never acknowledged, e.g. because of an exception inside a thread of the pool which is passed back to the message listener.
So my question is how Spring Kafka behaves in those situations? Are the messages going to replayed or do i need to take care of the retries on my own?
Thanks for your help!
答案1
得分: 0
容器将无限期暂停;您需要负责处理代码中的任何错误。容器不能继续运行,直到所有的空白都被填充。
如果您停止了带有缺失提交的容器,任何延迟提交都将丢失,并且在容器重新启动时,所有未提交的记录都将被重新传送。
通常最好通过将max.poll.records
减少到可以在max.poll.interval.ms
中处理的数量来避免无序提交。
英文:
The container will be paused indefinitely; you are responsible for handling any errors in your code. The container cannot continue until all the gaps are filled.
If you stop the container with missing commits, any deferred commits will be lost and all uncommitted records will be redelivered when the container is restarted.
It is generally better to avoid out of order commits by reducing the max.poll.records
to a number that can be handled in the max.poll.interval.ms
.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论