Filtering json using jq

huangapple go评论54阅读模式
英文:

Filtering json using jq

问题

在Azure DevOps管道中,通过Linux构建代理运行时,我希望使用jq从包含数据数组的JSON文件中提取特定的键/值,按日期对数组进行排序,并基于其中一个值过滤掉不必要的结果。

到目前为止,我有以下代码,它已经为我完成了很多工作。test2.json目前只是一个测试示例,用于概念验证。这将更改为API调用。

jq '[.value[] | {id: .id, date: .finishedDate, state: .state, result: .result}] | sort_by(.finishedDate)' $(Build.SourcesDirectory)/test2.json

上面的代码有效。它将test2.json中的较大数组减小到仅包含id、finishedDate、state和result字段。它还按finishedDate对结果进行了排序,这正是我需要的。下面是上述代码生成的JSON示例。

[
    {
        "id": 1902,
        "date": "2023-06-19T13:08:17.6903656Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1668,
        "date": "2023-06-19T13:00:18.9255899Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1621,
        "date": "2023-06-14T12:11:10.2714172Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1569,
        "date": "2023-06-14T08:20:34.0988263Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1566,
        "date": "2023-06-14T08:29:32.1229317Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1564,
        "date": "2023-06-14T07:48:54.1384016Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1557,
        "date": "2023-06-14T07:36:16.8346765Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1555,
        "date": "2023-06-14T07:24:19.5030833Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1553,
        "date": "2023-06-14T07:11:50.4071939Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1551,
        "date": "2023-06-14T06:58:34.3160857Z",
        "state": "completed",
        "result": "failed"
    }
]

我需要完成的最后一部分是只选择result等于"succeeded"的条目。根据我在线阅读的文档,我认为我需要添加类似于select(.result="succeeded")的内容,但取决于我将其添加到jq查询的位置,我要么会收到错误消息,例如找不到文件,要么得到与上面的结果相同,select(.result="succeeded")没有任何影响(请参见下面的jq查询)。

jq '[.value[] | select(.result="succeeded") | {id: .id, date: .finishedDate, state: .state, result: .result}] | sort_by(.finishedDate)' $(Build.SourcesDirectory)/test2.json

我对使用jq相对不熟悉,因此我想我可能遗漏了一些小细节,以使其正常工作,希望有人能提供帮助。

英文:

Within an Azure DevOps pipeline, running via a linux build agent, I'm looking to use jq to take specific keys/values from a json file containing an array of data, sort the array by date and filter out unnecessary results based on one of the values.

So far I have the below which does a lot of this for me. test2.json is just a test example for now while I do a proof of concept. This will change to an API call.

jq '[.value[] | {id: .id, date: .finishedDate, state: .state, result: .result}] | sort_by(.finishedDate)' $(Build.SourcesDirectory)/test2.json

The above code works. It reduces the larger array in test2.json to just the fields for id, finishedDate, state and result. It also sorted the result by finishedDate which is what I need it to do. Below is an example of the json off the back of the above code.

[
    {
        "id": 1902,
        "date": "2023-06-19T13:08:17.6903656Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1668,
        "date": "2023-06-19T13:00:18.9255899Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1621,
        "date": "2023-06-14T12:11:10.2714172Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1569,
        "date": "2023-06-14T08:20:34.0988263Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1566,
        "date": "2023-06-14T08:29:32.1229317Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1564,
        "date": "2023-06-14T07:48:54.1384016Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1557,
        "date": "2023-06-14T07:36:16.8346765Z",
        "state": "completed",
        "result": "succeeded"
    },
    {
        "id": 1555,
        "date": "2023-06-14T07:24:19.5030833Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1553,
        "date": "2023-06-14T07:11:50.4071939Z",
        "state": "completed",
        "result": "failed"
    },
    {
        "id": 1551,
        "date": "2023-06-14T06:58:34.3160857Z",
        "state": "completed",
        "result": "failed"
    }
]

The final part I need to do is select only entries where result = "succeeded". I believe I need to add something like select(.result="succeeded") based on the documentation I read online however depending on where I add this into my jq query I either get an error e.g. no such file found or I get the same results as above with the select(.result="succeeded") having no impact (see below jq query).

jq '[.value[] | select(.result="succeeded") | {id: .id, date: .finishedDate, state: .state, result: .result}] | sort_by(.finishedDate)' $(Build.SourcesDirectory)/test2.json

I'm fairly new to using jq so I imagine I'm missing something small to get this working and I'm hoping someone can help.

答案1

得分: 2

map(select(.result=="succeeded") | { id, date: .finishedDate, state, result }) | sort_by(.date)
英文:
map(select(.result=="succeeded") | { id, date: .finishedDate, state, result }) | sort_by(.date)

Gives the following:

[
  {
    "id": 1557,
    "date": "2023-06-14T07:36:16.8346765Z",
    "state": "completed",
    "result": "succeeded"
  },
  {
    "id": 1569,
    "date": "2023-06-14T08:20:34.0988263Z",
    "state": "completed",
    "result": "succeeded"
  },
  {
    "id": 1566,
    "date": "2023-06-14T08:29:32.1229317Z",
    "state": "completed",
    "result": "succeeded"
  },
  {
    "id": 1902,
    "date": "2023-06-19T13:08:17.6903656Z",
    "state": "completed",
    "result": "succeeded"
  }
]

  • id: .id can be shorten to just id
  • The select() needs to be inside the map()
  • sort_by() after the select() so we don't need to sort those we're removing

Online Demo

huangapple
  • 本文由 发表于 2023年6月21日 23:12:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/76524798.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定