英文:
Parse complex json in airflow using python
问题
以下是我的JSON。
{
"configuration": [
{
"task_name": "reading",
"novel": {
"name": "ABC",
"author": "aut1"
},
"story": {
"name": "CDE",
"author": "aut2"
}
},
{
"task_name": "singing",
"pop": {
"name": "rockit",
"author": "aut11"
},
"jazz": {
"name": "jazzy",
"author": "aut22"
}
}
]
}
我想从Airflow的管理员变量中读取此JSON，然后根据条件（task_name）进行迭代，并提取值并分配给变量。
由于我是Airflow和Python的新手，所以很难找到答案。欢迎任何输入。
英文:
Below given is my json.
{
"configuration": [
{
"task_name":"reading",
"novel":{
"name":"ABC",
"author":"aut1"
},
"story":{
"name":"CDE",
"author":"aut2"
}
},
{
"task_name":"singing",
"pop":{
"name":"rockit",
"author":"aut11"
},
"jazz":{
"name":"jazzy",
"author":"aut22"
}
}
]
}
I want to read this json from admin variables in airflow then based on condition(task_name) need to iterate through it and takeout values and assign to variables.
Since i am new in airflow and python it found difficult to get answer.
Welcomes any inputs
答案1
得分: 1
-
创建您的管理员变量。将JSON以纯文本形式放入“Val”值中，并指定一个“Key”（例如，“configuration”）。
-
在您的Python代码中导入“Variable”模型。
from airflow.models import Variable
- 使用“Variable”模型获取您的管理员变量，并将其反序列化为Python的“dict”，将“deserialize_json”设置为“True”。
configs = Variable.get("configuration", deserialize_json=True)
- 现在，您可以像普通的“dict”一样进行迭代。
for config in configs["configuration"]:
print(config["task_name"])
英文:
Follow these steps:
- Create your admin variable. Put your JSON in plain text in the
Valvalue and specify aKey(e.g. "configuration"). - Import the
Variablemodel in your Python code.
from airflow.models import Variable
- Get your admin var using the
Variablemodel and deserialize it to a Pythondictsettingdeserialize_jsontoTrue.
configs = Variable.get("configuration", deserialize_json=True)
- Now you can iterate it like a regular
dict.
for config in configs["configuration"]:
print(config["task_name"])
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论