英文:
How do I get a dataflow Job ID from within the pipeline or before the pipeline ran?
问题
I'd like to grab the job id here
在这里我想获取作业标识符
英文:
I want to grab the dataflow's Job ID as a string before actually running the pipeline or inside of it.
As an example let's say I have this:
DataflowOptions options = optionsConst.getOptions();
Pipeline p = Pipeline.create(options);
String jobID = doSomthing(); <-- I'd like to grab the job id here
PCollection<KV<String, Object>> refData = p.apply("BigTable Access", BigtableIO.read()
.withProjectId(options.getBigtableProjectId())
.withInstanceId(options.getBigtableInstanceId())
.withTableId(options.getLookupTableId()))
.apply("Keep tags valid versions", ParDo.of(new MethodThatDoesSomethingWithJobID(**jobId**))); <-- Or inside this
答案1
得分: 1
不可能做到这一点,因为作业 ID 是在您开始运行作业时创建的。您可以在开始运行管道后保存输出,或返回正在运行的作业列表。
英文:
Is not possible to do that, as the Job Id is created once you start to run it. What you can do is to save the output once you start run the pipeline, or to return the list job running
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论