英文:
Is Statefull doFn supported by google dataflow runner?
问题
我无法使批处理函数在Dataflow上运行。我在这里使用了Go中的示例中的函数:https://beam.apache.org/documentation/programming-guide/#state-timers-examples
该函数接收元素并触发一次OnTimer,然后一切都停止了(我可以通过日志看到)。
我正在流处理模式下。所以我的第一个问题是,Dataflow是否支持Go中的计时器?如果支持,我做错了什么?我简直是照着示例复制的。
英文:
I cannot make a batch fn work on Dataflow. I used the one in example here in Go: https://beam.apache.org/documentation/programming-guide/#state-timers-examples
The dofn receive element and trigger the OnTimer once and then everything get stuck (I can see it with logs).
I'am in streaming mode. So my first question is do Dataflow support Timers in Go? If yes what I'm doing wrong? I literrally copied the example.
答案1
得分: 1
在Dataflow中遇到相同问题的人,请在OnTimer()回调中添加以下行:
fn.OutputState.ClearTag(tp, timer.Tag) // 清除已触发的定时器标签。(用于暂时解决runner bug的临时方法。)
(此行在GitHub示例中存在,但在文档中不存在)
英文:
Ok, for everyone struggling with same issue on Dataflow, you have to add this line is the OnTimer() callback:
fn.OutputState.ClearTag(tp, timer.Tag) // Clean up the fired timer tag. (Temporary workaround for a runner bug.)
(this line is present in the example on github but not in the documentation)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论