英文:
How to automate data pipeline in Palantir Foundry?
问题
我是新来的Palantir Foundry。我需要知道如何自动化数据流程？如果我们在数据同步时进行数据摄取并设置计划，是否与在完成所有操作（如编码到数据转换等）并在数据血统处查看数据时进行数据流程调度相同？这两者是否相同？这是否意味着自动化数据流程？请有人能解释一下。谢谢。
英文:
I'm new to Palantir Foundery. I need to know how to automate the data pipeline? Is that same if we do the data ingestion to foundry while setting schedule at data sync? OR after all done (such as coding to data transformation etc.) when we see the data lineage at that point we can do the scheduling to the data pipeline. Is that both same ? Is that the meaning to say automate the data pipeline? Please some one can explain me. Thank you
答案1
得分: 1
在不清楚上下文的情况下，很难理解什么最适合，但通常情况下，“自动化数据管道”确实可以指调度。
关于调度器的文档可以在这里找到。调度器操作的对象是数据集。数据集可以通过从其他系统中摄取数据或通过各种方式定义的转换（如代码存储库、管道构建器等）来填充。
英文:
It is hard to understand what is best suited if the context of the ask is unclear to you in the first place, but usually "automating a data pipeline" can indeed refer to scheduling.
Docs about Schedulers can be found here. Schedulers are operating on Datasets. Datasets can be filled with data ingested from other systems or generated via transforms (defined in many ways: code repositories, pipeline builder, etc.).
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论