Statefull doFn是否受Google Dataflow Runner支持?

huangapple go评论105阅读模式
英文:

Is Statefull doFn supported by google dataflow runner?

问题

我无法使批处理函数在Dataflow上运行。我在这里使用了Go中的示例中的函数:https://beam.apache.org/documentation/programming-guide/#state-timers-examples

该函数接收元素并触发一次OnTimer,然后一切都停止了(我可以通过日志看到)。

我正在流处理模式下。所以我的第一个问题是,Dataflow是否支持Go中的计时器?如果支持,我做错了什么?我简直是照着示例复制的。

英文:

I cannot make a batch fn work on Dataflow. I used the one in example here in Go: https://beam.apache.org/documentation/programming-guide/#state-timers-examples

The dofn receive element and trigger the OnTimer once and then everything get stuck (I can see it with logs).

I'am in streaming mode. So my first question is do Dataflow support Timers in Go? If yes what I'm doing wrong? I literrally copied the example.

答案1

得分: 1

在Dataflow中遇到相同问题的人,请在OnTimer()回调中添加以下行:

fn.OutputState.ClearTag(tp, timer.Tag) // 清除已触发的定时器标签。(用于暂时解决runner bug的临时方法。)

(此行在GitHub示例中存在,但在文档中不存在)

英文:

Ok, for everyone struggling with same issue on Dataflow, you have to add this line is the OnTimer() callback:

fn.OutputState.ClearTag(tp, timer.Tag) // Clean up the fired timer tag. (Temporary workaround for a runner bug.)

(this line is present in the example on github but not in the documentation)

huangapple
  • 本文由 发表于 2023年8月5日 00:32:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/76837741.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定