在Kafka Streams版本大于2.10的情况下,保留。

huangapple go评论75阅读模式
英文:

Retention on Kafka Streams > 2.10

问题

我正在加入KStreams,因此我需要设置JoinWindows。已连接的数据具有来自CustomTimestampExtractors的计算时间戳。数据可能是无序的。我可以使用已弃用的JoinWindows.until(long)方法设置retention,但由于它已被弃用,我正在寻找另一个解决方案。

我发现了windowstore.changelog.additional.retention.ms属性,我应该使用它吗?这将导致所有状态存储保留那么长时间。

英文:

I am joining to KStreams thus I need to set JoinWindows. The joined data has computed Timestamps from CustomTimestampExtractors. The Data could be out of order. I can set the retention with the deprecated JoinWindows.until(long)-method but since its deprecated I am looking for another solution.

I found the windowstore.changelog.additional.retention.ms-property am I supposed to use it ? This would lead to all state stores being retained that long.

答案1

得分: 0

根据文档所述:

> 使用JoinWindows#grace(Duration)替代

依我之见,文档在与以下内容相比时有些误导:

> 设置窗口维持持续时间(保留时间)为毫秒。此保留时间是窗口维持的最低保证下限。

但要注意,我希望我的StateStore保持尽可能长的时间,因此之前我在使用JoinWindows.until(Long.MAX_VALUE)。使用JoinWindows.grace(Duration.ofMillis(Long.Max_VALUE))会报告宽限期不能为负。因此,您必须使用较小的值。

英文:

As the Documentation states:

> Use JoinWindows#grace(Duration) instead

IMHO the doc is a bit misleading in contrast to:

> Set the window maintain duration (retention time) in milliseconds. This retention time is a guaranteed lower bound for how long a window will be maintained.

But be careful I wanted my StateStore to retain as long as possible thus I was using JoinWindows.until(Long.MAX_VALUE) before. Using JoinWindows.grace(Duration.ofMillis(Long.Max_VALUE)) does complain that grace period can't be negative. There for you have to use smaller values.

huangapple
  • 本文由 发表于 2020年10月13日 15:26:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/64330489.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定