英文:
DAG run marked as success while task failed
问题
def my_main(**kwargs):try:....except:print("错误：", pprint.pformat(traceback.format_exc(), indent=1))dag = DAG('my_main',description='my_main',start_date=days_ago(0),schedule_interval='@daily',catchup=False)task_checker = PythonOperator(task_id='task_checker_my_main', provide_context=True, python_callable=my_main, dag=dag)
英文:
I have some PythonOperator in my DAG which time to time throws error therefore simply it goes into except statement and entire process is stopped due to that. What i do not understand is my dag run is marked as success. Can anyone explains that to me please as i thought if code inside task throws an error the dag should be marked as failed, not? Why dag is marked success?
DAG (cutted):
import timeimport datetimeimport jsonimport pprintimport requestsfrom airflow import DAGfrom airflow.operators.dummy_operator import DummyOperatorfrom airflow.operators.python_operator import PythonOperatorfrom airflow.operators.python_operator import BranchPythonOperatordef my_main(**kwargs):try:....except:print("Error:", pprint.pformat(traceback.format_exc(), indent=1))dag = DAG('my_main',description='my_main',start_date=days_ago(0),schedule_interval='@daily',catchup=False)task_checker = PythonOperator(task_id='task_checker_my_main', provide_context=True, python_callable=my_main, dag=dag)
答案1
得分: 2
你需要触发异常。你“吞掉”了真正的异常，所以函数以无错误的方式结束。在打印后直接触发异常。def my_main(**kwargs):try:....except:print("错误:", pprint.pformat(traceback.format_exc(), indent=1))raise AirflowException("你的消息")
英文:
You need to raise Exception. you "swallow" the real exception so the function ends with no error . just raise exception after the print.
def my_main(**kwargs):try:....except:print("Error:", pprint.pformat(traceback.format_exc(), indent=1))raise AirflowException("your message")
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论