从JSON负载中删除元素的JOLT转换

huangapple go评论44阅读模式
英文:

Removing elements from JSON payload with JOLT transformation

问题

我对JOLT转换还很陌生,仍然在处理相当基本的问题时遇到了困难。

我有一个庞大的JSON数据,我需要简化它,只保留根级别的一个普通属性: "root_element": 14,以及数组元素的特定字段: "array_elements": [{"array_field": 103854}, {"array_field": 9975}, {"array_field": 47551}]

使用此JOLT转换:

{
    "operation": "shift",
    "spec": {
      "array_elements": {
        "*": {
          "array_field": "[&1].array_field"
        }
      }
    }
 }

我得到了以下结果:

[
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
]

然后我尝试将所需的根属性添加到我的转换:

{
    "operation": "shift",
    "spec": {
      "root_element": "root_element",
      "array_elements": {
        "*": {
          "array_field": "[&1].array_field"
        }
      }
    }
 }

但这只返回这个:

{
  "root_element": 14
}

我如何修改我的JOLT转换以生成此JSON?

{
 "root_element": 14,
 "array_elements":[
   {
     "array_field": 103854
   },
   {
     "array_field": 9975
   },
   {
     "array_field": 47551
   }
  ] 
}

在我的实际情况中还有更多的属性。例如:

{
  "root_element": 14,
  "another_root_element": "text",
  "array_elements": [
    {
      "array_field": 103854,
      "another_array_field": "more text"
    },
    {
      "array_field": 9975
    },
    {
      "array_field": 47551
    }
  ]
}

我需要移除"another"元素,但想象一下有多个"another"元素(数百个)以及各种名称(太多而且随机)。我只需要保留我描述的属性。感谢您!

英文:

I'm rather new to JOLT transformations so I'm still having trouble with pretty basic stuff.

I have a big JSON payload that I need to simplify by removing most data from it.
The only attributes I want to keep are a normal attribute at the root level:

"root_element" : 14,

and a specific field from array elements:

"array_elements": [
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
],

It's impractical to use the "remove" operation since there are a large number of attributes at both root level and within array_elements.

If I use this JOLT transformation:

{
    "operation": "shift",
    "spec": {
      "array_elements": {
        "*": {
          "array_field": "[&1].array_field"
        }
      }
    }
 }

I get the following result:

[
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
]

Then I try to add the required root attribute to my transformation:

{
    "operation": "shift",
    "spec": {
    "root_element" : "root_element",
      "array_elements": {
        "*": {
          "array_field": "[&1].array_field"
        }
      }
    }
 }

But this only returns this:

{
  "root_element" : 14
}

How can I modify my JOLT transformation in order to generate this JSON?

{
 "root_element" : 14,
"array_elements":[
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
 ] 
}

there are more attributes in-between for my real case. As an Example:

{
  "root_element": 14,
  "another_root_element": "text",
  "array_elements": [
    {
      "array_field": 103854,
      "another_array_field": "more text"
    },
    {
      "array_field": 9975
    },
    {
      "array_field": 47551
    }
  ]
}

I need to remove the "another" elements but imagine there are multiple "another" elements (hundreds) with various names (too many and random). I just need to keep the attributes I described.

Thanks in advance!

答案1

得分: 1

你正在尝试将array_field项目添加到一个数组中。我这样说是因为你写了[&1]

所以对于root_element,你想将其添加到一个带有root_element键的对象中。

现在假设,JOLT如何看到这一点?

你将root_element写为你在JOLT规范中想要获取的第一个键,所以JOLT会为你创建一个object并将root_element放入其中。

但对于下一个键(array_elements),你想将其放入一个数组中,但你当前有一个对象。因此,JOLT会忽略该键并为你创建一个只带有root_element键的对象,作为期望输出。

为了解决这个问题,你可以将array_elements放入名为arr的对象中。这样,你可以得到以下的JOLT规范:

[
  {
    "operation": "shift",
    "spec": {
      "root_element": "root_element",
      "array_elements": {
        "*": {
          "array_field": "arr[&1].array_field"
        }
      }
    }
 }
]

输出:

{
  "root_element": 14,
  "arr": [
    {
      "array_field": 103854
    },
    {
      "array_field": 9975
    },
    {
      "array_field": 47551
    }
  ]
}
英文:

You are trying to add array_field items to an array. I said that because you have written [&1].

So for root_element you want to add it to an object with the root_element key.

Now suppose, How JOLT can see that?

You write root_element as the first key you want to get in your JOLT spec, So JOLT creates an object for you and put root_element into that.

But for the next key (array_elements) you want to put that into an array but you have an object currently. So jolt ignore that key and create an object with just the root_element key in desired output for you.

For fixing that you can for example put array_elements in an object named arr, So you can have the following JOLT spec:

[
  {
    "operation": "shift",
    "spec": {
      "root_element": "root_element",
      "array_elements": {
        "*": {
          "array_field": "arr[&1].array_field"
        }
      }
    }
 }
]

Output:

{
  "root_element": 14,
  "arr": [
    {
      "array_field": 103854
    },
    {
      "array_field": 9975
    },
    {
      "array_field": 47551
    }
  ]
}

答案2

得分: 1

你可以使用以下转换,考虑其他属性:

[
  {
    "operation": "shift",
    "spec": {
      "root_element": "&",
      "array_elements": {
        "*": {
          "array_field": "&2[&1].&"
        }
      }
    }
  }
]

该组合 "&2[&1].&" 也可以替换为 "&2[#2].&" 作为抑制某些情况下生成 null 的替代方案。

英文:

You can use the following transformation considering those other attributes :

[
  {
    "operation": "shift",
    "spec": {
      "root_element": "&", // pick attribute named "root_element" only
      "array_elements": {
        "*": {
          // pick attributes named "array_field" only
          "array_field": "&2[&1].&" // &2 grabs value after going two levels up the tree
                                    // &  replicates the current key-value combination
        }
      }
    }
  }
]

the combination "&2[&1].&" might also be replaced with "&2[#2].&" as anternative to suppress the null generation for same cases you may encounter.

huangapple
  • 本文由 发表于 2023年4月17日 15:50:40
  • 转载请务必保留本文链接:https://go.coder-hub.com/76032819.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定