英文:
How to write flink job output to a single location instead of two locations
问题
目前，我已经通过Docker将Flink运行为一个集群，使用了三个容器：一个用于jobmanager，另外两个用于taskmanagers。每当我提交一个作业时，作业的输出会写入这两个容器。我有兴趣找到一种解决方案，将输出汇总到一个单一位置，最好是在本地机器上。有没有办法实现这个目标？
英文:
Currently, I have Flink operating as a cluster through Docker, employing three containers: one for the jobmanager and two for the taskmanagers. Whenever I submit a job, the job's output gets written to both containers. I am interested in finding a solution to consolidate the output in a single location, preferably either on the local machine. Is there a way to achieve this?
答案1
得分: 1
你可以：
- 使用一个具有两倍多的插槽的单一任务管理器（仍然会得到多个文件，但它们将在同一台机器的同一目录下）
- 减少作业的并行性
- 减少汇聚操作的并行性
英文:
You could:
- use a single task manager with 2x as many slots (you'll still end up with multiple files, but they'll be in the same directory on the same machine)
- reduce the parallelism of the job
- reduce the parallelism of the sink
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论