JSON到JSON的转换使用JOLTTransformJSON NiFi

huangapple go评论46阅读模式
英文:

JSON to JSON transform using JOLTTransformJSON NiFi

问题

I am using JOLTTransformJson Processor in Nifi.

My input is:

[
  {
    "col_name": "time",
    "data_type": "timestamp",
    "is_nullable": true
  },
  {
    "col_name": "otherData",
    "data_type": "string",
    "is_nullable": false
  }
]

I am using the below spec:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "col_name": "name",
        "data_type": "type[0]",
        "is_nullable": {
          "true": "type[1]",
          "false": "type[1]"
        }
      }
    }
  },
  {
    "operation": "default",
    "spec": {
      "*": {
        "type[1]": "notnull"
      }
    }
  }
]

Expected Output is :

{
  "type": "record",
  "name": "table_name",
  "fields": [
    {
      "name": "time",
      "type": [
        "timestamp",
        "null"
      ]
    },
    {
      "name&quot: "otherData",
      "type": [
        "string",
        "notnull"
      ]
    }
  ]
}

But getting the below one as the current result by combining all values in array like:

{
  "name": [
    "time",
    "otherData"
  ],
  "type": [
    [
      "timestamp",
      "int"
    ],
    null
  ]
}

Can someone please help what am I missing.

英文:

I am using JOLTTransformJson Processor in Nifi.

My input is:

[
  {
    "col_name": "time",
    "data_type": "timestamp",
    "is_nullable": true
  },
  {
    "col_name": "otherData",
    "data_type": "string",
    "is_nullable": false
  }
]

I am using the below spec:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "col_name": "name",
        "data_type": "type[0]",
        "is_nullable": {
          "true": "type[1]",
          "false": "type[1]"
        }
      }
    }
  },
  {
    "operation": "default",
    "spec": {
      "*": {
        "type[1]": "notnull"
      }
    }
  }
]

Expected Output is :

{
  "type": "record",
  "name": "table_name",
  "fields": [
    {
      "name": "time",
      "type": [
        "timestamp",
        "null"
      ]
    },
    {
      "name": "otherData",
      "type": [
        "string",
        "notnull"
      ]
    }
  ]
}

But getting the below one as the current result by combining all values in array like:

{
  "name": [
    "time",
    "otherData"
  ],
  "type": [
    [
      "timestamp",
      "int"
    ],
    null
  ]
}

Can someone please help what am I missing.

答案1

得分: 0

你可以使用以下的第一个 shift 转换规范来通过遍历数组中的对象,以便能够 重复 应用这些技巧:

[
  {
    "operation": "shift",
    "spec": {
      "#record": "type", // 使用左侧的 # 通配符形成的固定值
      "#table_name": "name",
      "*": {
        "col_*": "fields[#2].&(0,1)", // 复制当前(0级)星号的第一个替换
        "is_nullable": { // 条件逻辑从这里开始
          "@(1,data_type)": "fields[#3].type",
          "true": {
            "#null": "fields[#4].type"
          },
          "false": {
            "#notnull": "fields[#4].type"
          }
        }
      }
    }
  },
  { // 对字段数组中的属性进行排序
    "operation": "sort",
    "spec": {
      "fields": ""
    }
  },
  { // 对整个结果进行排序
    "operation": "shift",
    "spec": {
      "type": "&&",
      "name": "&&",
      "fields": "&&"
    }
  }
]

第二和第三个转换仅仅是为了按所需的方式排序属性/数组而添加的。

英文:

You can use the following first shift transformation spec through walking by the objects of the array in order to be able to repeatedly apply the techniques :

[
  {
    "operation": "shift",
    "spec": {
      "#record": "type", // fixed values formed by using # wildcards left-hand-side
      "#table_name": "name",
      "*": {
        "col_*": "fields[#2].&(0,1)", // replicate the 1st replacement of the asterisk from the current(0th) level
        "is_nullable": { // conditional logic starts here
          "@(1,data_type)": "fields[#3].type",
          "true": {
            "#null": "fields[#4].type"
          },
          "false": {
            "#notnull": "fields[#4].type"
          }
        }
      }
    }
  },
  { //to sort the attributes within the fields array
    "operation": "sort",
    "spec": {
      "fields": ""
    }
  },
  { //to sort whole result
    "operation": "shift",
    "spec": {
      "type": "&",
      "name": "&",
      "fields": "&"
    }
  }
]

the second and third transformation are just added to sort the attributes/arrays as desired

huangapple
  • 本文由 发表于 2023年2月6日 14:34:15
  • 转载请务必保留本文链接:https://go.coder-hub.com/75358042.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定