Jolt flattening of array introduces unwanted null array item

we have a requirement to flatten a JSON object. We have input JSON data that looks like this:

{
"changes": [
{
"l1_Entity_Id": 7004,
"l1_Entity_Nm": "Academic Administration2",
"visibility": false,
"L2_Entities": [
{
"L2_Entity_Id": 8003,
"L2_Entity_Nm": "Desktop Software2",
"visibility": 1,
"primary_triage": "EAAS",
"markham_triage": "MK_Team1",
"Faculty_Support": [
"Faculty1",
"Faculty2",
"Faculty4"
],
"hasReference": true
}
]
},
{
"l1_Entity_Id": 7002,
"l1_Entity_Nm": "Telephony5",
"visibility": false,
"L2_Entities": [
{
"L2_Entity_Id": 8005,
"L2_Entity_Nm": "Automatic Call Distribution3",
"visibility": 1,
"primary_triage": "Telecom",
"markham_triage": null,
"Faculty_Support": [
"Faculty3",
"Faculty4"
],
"hasReference": true
},
{
"L2_Entity_Id": 8004,
"L2_Entity_Nm": "Phone",
"visibility": 1,
"primary_triage": null,
"markham_triage": null,
"Faculty_Support": [
"Faculty1"
],
"hasReference": true
}
]
}
],
"methodName": "generateEntitiesChgReport"
}

Once shifted, we'd like output data like this:

[
{
"l1_Entity_Nm": "Academic Administration2",
"L1_visibility": false,
"L2_Entity_Id": 8003,
"L2_Entity_Nm": "Desktop Software2",
"aggregateNm": "Academic Administration2>Desktop Software2",
"L2_visibility": 1,
"primary_triage": "EAAS",
"isMarkham": true,
"markham_triage": "MK_Team1",
"Faculty_Support": [
"Faculty1",
"Faculty2",
"Faculty4"
]
},
{
"l1_Entity_Nm": "Telephony5",
"L1_visibility": false,
"L2_Entity_Id": 8005,
"L2_Entity_Nm": "Automatic Call Distribution3",
"aggregateNm": "Telephony5>Automatic Call Distribution3",
"L2_visibility": 1,
"primary_triage": "Telecom",
"isMarkham": false,
"markham_triage": null,
"Faculty_Support": [
"Faculty3",
"Faculty4"
]
},
{
"l1_Entity_Nm": "Telephony5",
"L2_Entity_Id": 8004,
"L2_Entity_Nm": "Phone",
"aggregateNm": "Telephony5>Phone",
"L2_visibility": 1,
"primary_triage": null,
"isMarkham": false,
"markham_triage": null,
"Faculty_Support": [
"Faculty1"
]
}
]

We almost got the desired output with the Jolt spec below:

[
{
"operation": "shift",
"spec": {
"changes": {
"*": {
"L2_Entities": {
"*": {
"@": "&[&3]",
"@(2,l1_Entity_Id)": "&[&3].l1_Entity_Id",
"@(2,l1_Entity_Nm)": "&[&3].l1_Entity_Nm",
"@(2,l1_visibility)": "&[&3].l1_visibility"
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]

However, it introduces an unwanted null after second json object as shown below:

[ {
"L2_Entity_Id" : 8003,
"L2_Entity_Nm" : "Desktop Software2",
"visibility" : 1,
"primary_triage" : "EAAS",
"markham_triage" : "MK_Team1",
"Faculty_Support" : [ "Faculty1", "Faculty2", "Faculty4" ],
"hasReference" : true,
"l1_Entity_Id" : 7004,
"l1_Entity_Nm" : "Academic Administration2"
}, {
"L2_Entity_Id" : 8005,
"L2_Entity_Nm" : "Automatic Call Distribution3",
"visibility" : 1,
"primary_triage" : "Telecom",
"markham_triage" : null,
"Faculty_Support" : [ "Faculty3", "Faculty4" ],
"hasReference" : true,
"l1_Entity_Id" : 7002,
"l1_Entity_Nm" : "Telephony5"
}, null, {
"L2_Entity_Id" : 8004,
"L2_Entity_Nm" : "Phone",
"visibility" : 1,
"primary_triage" : null,
"markham_triage" : null,
"Faculty_Support" : [ "Faculty1" ],
"hasReference" : true,
"l1_Entity_Id" : 7002,
"l1_Entity_Nm" : "Telephony5"
} ]

I hope someone can help us to resolve the issue. Thank you all for any help you might give us.

You can rather use * symbol than @ symbol along with non-square-bracketed ampersands such as

[
  {
    "operation": "shift",
    "spec": {
      "changes": {
        "*": {
          "L2_Entities": {
            "*": {
              "*": "&3.&1.&",
              "@2,l1_Entity_Id": "&3.&1.l1_Entity_Id",
              "@2,l1_Entity_Nm": "&3.&1.l1_Entity_Nm",
              "@2,l1_visibility": "&3.&1.l1_visibility"
            }
          }
        }
      }
    }
  },
  { // get rid of the object keys
    "operation": "shift",
    "spec": {
      "*": {
        "*": ""
      }
    }
  }
]

or you can make the transformation more dynamic in which no need to state each attributes individually such as

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": { // loop through all the "changes"
          "l1*": "&2.&1.Others.&", // accumulate the elements other than "L2_Entities" and "visibility" within an individual object caled "Others"
          "L2_Entities": "&2.&1.&"
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "L2_Entities": {
            "*": {
              "@2,Others": { "*": "&4.&2.&" }, // go two levels up the tree to grab the whole values from the "Others" object
              "*": "&3.&1.&"
            }
          }
        }
      }
    }
  },
  { // get rid of the object keys
    "operation": "shift",
    "spec": {
      "*": {
        "*": ""
      }
    }
  }
]

the demo 2 :