英文:

Dataflow Flex Template max size allowed

问题

I trying to find the max size allowed for the dataflow run in flex mode and a way to check the size. 我试图找出在灵活模式下数据流运行的最大允许大小以及检查大小的方法。

I know the classic template has a limit of 10MB per template. 我知道经典模板每个模板有10MB的限制。

But Searching on the Google Cloud documentation I was not able to find if the size is the same as the classic one. 但是在Google Cloud文档上搜索，我无法找到大小是否与经典模板相同的信息。

On the classic template I can check the size of my template by checking the file size inside a bucket. 在经典模板上，我可以通过检查存储桶内的文件大小来检查模板的大小。

But for the flex template I can't do that because the flex templates runs as a Container file so its build when the job is running. 但对于灵活模板，我无法这样做，因为灵活模板作为容器文件运行，因此在作业运行时构建。

One thing I can do is check the Uber jar that my java project package before upload to GCP. 我可以做的一件事是在上传到GCP之前检查我的Java项目包中的Uber jar。

But I'm assuming the template is the max size of the graph that dataflow generate and not by the size of the application (my jars has around 90 ~ 120MB and runs fine in the GCP). 但我假设模板的大小是数据流生成的图形的最大大小，而不是应用程序的大小（我的jar文件大约为90~120MB，在GCP上运行良好）。

英文:

Hi I trying to find the max size allowed for the dataflow run in flex mode and a way to check the size.
I know the classic template has a limit of 10MB per template.

But Searching on the Google Cloud documentation I was not able to find if the size is the same as the classic one.

On the classic template I can check the size of my template by checking the file size inside a bucket.

But for the flex template I can't do that because the flex templates runs as a Container file so
its build when the job is running.
One thing I can do is check the Uber jar that my java project package before upload to GCP.

But I'm assuming the template is the max size of the graph that dataflow generate and not by the size of the application (my jars has around 90 ~ 120MB and runs fine in the GCP).

答案1

得分: 1

我认为Dataflow Flex Template没有大小限制。太大的JAR包可能会减少运行时间，但据我所知，Flex模板没有配额或限制。

英文:

I think there is no size limit for Dataflow Flex Template.

Too big fat jar can reduce the runtime, but as far as I know there is no quota or limit for Flex template.