如何在NIFI中使用Jolttransform规范进行映射？

huangapple
go评论150阅读模式
英文:

How to map using Jolttransform specification in NIFI?

问题

我正在准备jolt规范，但无法完成，请帮忙。

**注意：**有一个credits数组，该数组的值应与类型为credit record的模式字段进行映射。

输入：

{
  "schema": {
    "fields": [
      {
        "name": "user",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "billed",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "currency",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "currency_conversion_rate",
        "type": "FLOAT",
        "mode": "NULLABLE"
      },
      {
        "name": "usage_unit",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "adjustment_info_mode",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "adjustment_info_type",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "InvoiceMonth",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "credits",
        "type": "RECORD",
        "mode": "REPEATED",
        "fields": [
          {
            "name": "name",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "amount",
            "type": "FLOAT",
            "mode": "NULLABLE"
          },
          {
            "name": "full_name",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "id",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "type",
            "type": "STRING",
            "mode": "NULLABLE"
          }
        ]
      },
      {
        "name": "resource_name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "resource_global_name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "label",
        "type": "STRING",
        "mode": "NULLABLE"
      }
    ]
  },
  "InvoiceMonth": "202006",
  "credits": [
    {
      "v": {
        "f": [
          {
            "v": "ae391_credit:0"
          },
          {
            "v": "-0.010237"
          },
          {
            "v": "Committed dollar"
          },
          {
            "v": "ae31_credit:0"
          },
          {
            "v": "COMMITTED_BASE"
          }
        ]
      }
    },
    {
      "v": {
        "f": [
          {
            "v": "ae391_credit:0"
          },
          {
            "v": "-0.010237"
          },
          {
            "v": "Committed dollar"
          },
          {
            "v": "ae31_credit:0"
          },
          {
            "v": "COMMITTED_BASE"
          }
        ]
      }
    }
  ],
  "resource_name": "projects-0",
  "resource_global_name": "//compute.googleapis.com/",
  "label": ""
}

我尝试了这个jolt规范：

[
  {
    "operation": "shift",
    "spec": {
      "credits": {
        "*": {
          "*": {
            "*": {
              "*": {
                "@v": "[&3].@(5,schema.fields.fields[&].name)"
              }
            }
          }
        }
      }
    }
  }
]

预期输出：

[
  {
    "name": "ae391_credit:0",
    "amount": "-0.010237",
    "full_name": "Committed dollar",
    "id": "ae31_credit:0",
    "type": "COMMITTED_BASE"
  },
  {
    "name": "ae391_credit:0",
    "amount": "-0.010237",
    "full_name": "Committed dollar",
    "id": "ae31_credit:0",
    "type": "COMMITTED_BASE"
  }
]
英文:

I am preparing the jolt specification but I am unable to complete it, pls help out.

Note: There is a credits array, the values of that array should be mapped with schema fields which are of type credit record.

Input:

{
  "schema": {
    "fields": [
      {
        "name": "user",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "billed",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "currency",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "currency_conversion_rate",
        "type": "FLOAT",
        "mode": "NULLABLE"
      },
      {
        "name": "usage_unit",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "adjustment_info_mode",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "adjustment_info_type",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "InvoiceMonth",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "credits",
        "type": "RECORD",
        "mode": "REPEATED",
        "fields": [
          {
            "name": "name",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "amount",
            "type": "FLOAT",
            "mode": "NULLABLE"
          },
          {
            "name": "full_name",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "id",
            "type": "STRING",
            "mode": "NULLABLE"
          },
          {
            "name": "type",
            "type": "STRING",
            "mode": "NULLABLE"
          }
        ]
      },
      {
        "name": "resource_name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "resource_global_name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "label",
        "type": "STRING",
        "mode": "NULLABLE"
      }
    ]
  },
  "InvoiceMonth": "202006",
  "credits": [
    {
      "v": {
        "f": [
          {
            "v": "ae391_credit:0"
          },
          {
            "v": "-0.010237"
          },
          {
            "v": "Committed dollar "
          },
          {
            "v": "ae31_credit:0"
          },
          {
            "v": "COMMITTED_BASE"
          }
        ]
      }
    },
    {
      "v": {
        "f": [
          {
            "v": "ae391_credit:0"
          },
          {
            "v": "-0.010237"
          },
          {
            "v": "Committed dollar "
          },
          {
            "v": "ae31_credit:0"
          },
          {
            "v": "COMMITTED_BASE"
          }
        ]
      }
    }
  ],
  "resource_name": "projects-0",
  "resource_global_name": "//compute.googleapis.com/",
  "label": ""
}

I have tried this jolt:

[
  {
    "operation": "shift",
    "spec": {
      "credits": {
        "*": {
          "*": {
            "*": {
              "*": {
                "@v": "[&3].@(5,schema.fields.fields[&].name)"
              }
            }
          }
        }
      }
    }
  }
]

Expected Output:

[
  {
    "name": "ae391_credit:0",
    "amount": "-0.010237",
    "full_name": "Committed dollar",
    "id": "ae31_credit:0",
    "type": "COMMITTED_BASE"
  },
  {
    "name": "ae391_credit:0",
    "amount": "-0.010237",
    "full_name": "Committed dollar",
    "id": "ae31_credit:0",
    "type": "COMMITTED_BASE"
  }
]

答案1

得分: 1

你可以使用 shift 操作将内部的 fields 提取并在 credits 数组中使用另一个 shift 操作来获取名称作为键的值。

[
  {
    "operation": "shift",
    "spec": {
      "credits": "&",
      "schema": {
        "fields": {
          "*": {
            "fields": "&"
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "credits": {
        "*": {
          "*": {
            "*": {
              "*": {
                "*": "[&4].@(6,fields[&1].name)"
              }
            }
          }
        }
      }
    }
  }
]
英文:

You can use a shift operation and bring inner fields up and use another shift for getting the value of the name as your key in the credits array.

[
  {
    "operation": "shift",
    "spec": {
      "credits": "&",
      "schema": {
        "fields": {
          "*": {
            "fields": "&"
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "credits": {
        "*": {
          "*": {
            "*": {
              "*": {
                "*": "[&4].@(6,fields[&1].name)"
              }
            }
          }
        }
      }
    }
  }
]

huangapple
  • 本文由 发表于 2023年6月12日 16:52:41
  • 转载请务必保留本文链接：https://go.coder-hub.com/76454983.html
  • apache-nifi
  • jolt
  • json
  • transformation
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定