英文:
How to fix failed to import DAGs due to 30s timeout errors?
问题
我们有多个mw1.large MWAA环境，每天都会出现导入错误，错误信息为airflow.exceptions.AirflowTaskTimeout: DagBag import timeout for ..dag_name.py after 30.0s。
这些错误并不会阻塞任何操作，因为它们很快就消失了。错误指向了最佳实践文档，但我认为我们的DAG并不特别复杂。在所有环境中，我们每天至少会看到1个导入错误。
想知道是否有除了增加超时时间之外的其他建议。
英文:
We have multiple mw1.large MWAA environments and we are seeing import errors everyday which say
airflow.exceptions.AirflowTaskTimeout: DagBag import timeout for ..dag_name.py after 30.0s.
They are not blocking anything as they disappear quickly. The errors point to best practices doc but I don't think our DAGs are particularly complex. We see at least 1 import error every day in all environments.
Want to know if there are any other recommendations apart from increasing the timeout.
答案1
得分: 3
你可以尝试增加 dagbag 导入超时。同时，检查你的 DAGs 是否包含顶层代码，这被认为是 Airflow 的反模式。
英文:
You can try increasing the dagbag import timeout. Also, review your DAGs for top-level code, which is considered an Airflow antipattern.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论