英文:
Flink Dashboard: Operator Throughput
问题
-
我有一个使用 Flink (1.9) 的程序，我想通过仪表板检查 Map 操作符实例的吞吐量。从已有的度量指标中，numRecordsInPerSecond 似乎是最有前途的，但我猜它可能不涉及处理时间。我错了吗？
-
我已经定义了自己的度量指标（吞吐量），基本上是通过将处理的记录数除以 OUT map(IN value) 函数的总执行时间来计算平均吞吐量。但这不会计算 map 函数之外的任何操作。
-
另一个想法是在 map 函数末尾添加一个计量器，但我想，如果数据源产生的记录不够快，计算出的吞吐量会变差，因为操作符大部分时间都空闲。这个理解正确吗？
请具体回答问题 1 和 2。另外，您通常是如何在您的程序中计算吞吐量的？
英文:
-
I have a program that uses flink (1.9) and I want to check the throughput of instances of a Map operator with the help of the dashboard. From the already existing metrics numRecordsInPerSecond seems to be the most promising, but I guess it doesn't involve processing time. Am I wrong?
-
I've defined my own metric (throughput) that essentially calculates the average throughput by dividing the number of records processed by the total execution time of the OUT map(IN value) function. But this does not count anything that happens outside the map function.
-
Another idea would be to add a meter at the end of the map function, but I suppose if the source doesn't produce records fast enough the throughput calculated will be worse just because the operator remains idle a lot of the time. Is this correct?
Please specifically answer 1 and 2. Also, how do you usually calculate the throughput in your programs?
答案1
得分: 3
Flink所有的Meter指标，比如numRecordsInPerSecond，都是以处理时间来衡量的。
我通常很满意地依赖这些内置指标来衡量吞吐量。但是对于输出操作，你可能希望在sink中添加一个自定义指标，因为Flink对于sinks的numRecordsOut和numRecordsOutPerSecond总是返回0。
英文:
All of Flink's Meter metrics, such as numRecordsInPerSecond, are measuring rates in terms of processing time.
I'm generally content to rely on these built-in metrics for measuring throughput. But you might want to add a custom metric in the sink, since Flink always returns 0 for numRecordsOut and numRecordsOutPerSecond for sinks.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论