英文:
How to return data from KubernetesPodOperator
问题
在DAG脚本中：
result = KubernetesPodOperator(
task_id=task_id,
name=task_name,
cmds=["pipenv", "run", "python3", "myscript.py"],
env_vars=env_vars,
image=image # Docker镜像
)
在myscript.py中，结果将是一个整数：
def myfunction():
'''执行一系列操作'''
return result
myfunction()
出于某种原因，我需要运行myscript.py，它将返回DAG所需的结果。使用env_vars，数据可以从DAG脚本传递到myscript.py，但如何将数据传递回DAG脚本呢？备注：我知道xcom可以用于任务之间的通信，但不确定它是否适用于这种情况。
英文:
In dag script
result = KubernetesPodOperator(
task_id=task_id,
name=task_name,
cmds=["pipenv", "run", "python3", "myscript.py"],
env_vars=env_vars,
image=image # docker image
)
In myscript.py, the result would be an integer
def myfunction():
''' Perform a series of operations '''
return result
myfunction()
For certain reason, I need to run myscript.py in which it'll return a result needed by the dag. Using env_vars, data would be able to be passed from the dag script to myscript.py but how data can be passed back to the dag script? PS: I'm aware of xcom that can help to communicate between tasks but not sure if it applies for this case
答案1
得分: 0
KubernetesPodOperator处理XCom值与其他操作器不同。为了从您的Pod传递XCom值，您必须将do_xcom_push指定为True。这将创建一个与Pod并行运行的sidecar容器。Pod必须将XCom值写入此位置，位于/airflow/xcom/return.json路径下。
详细信息请参阅此处。
英文:
The KubernetesPodOperator handles XCom values differently than other operators. In order to pass a XCom value from your Pod you must specify the do_xcom_push as True. This will create a sidecar container that runs alongside the Pod. The Pod must write the XCom value into this location at the /airflow/xcom/return.json path.
see this for details
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论