grab IP addresses (but not subnets) from a command (or file) and output them one per line to a new file

huangapple go评论69阅读模式
英文:

grab IP addresses (but not subnets) from a command (or file) and output them one per line to a new file

问题

135.138.168.78
135.138.168.79
英文:
{
	"status": "success",
	"result": [{
		"address_details": [{
				"address": "135.138.168.78",
				"serviceType": "go_gateway",
				"addressType": "active",
				"create_time": 1631879023,
				"allow_listed": false
			},
			{
				"address": "135.138.168.79",
				"serviceType": "gp_gateway",
				"addressType": "active",
				"create_time": 1631879023,
				"allow_listed": false
			}
		],
		"zone": "Germany",
		"addresses": [
			"135.138.168.78",
			"135.138.168.79"
		],
		"zone_subnet": [
			"137.183.192.0/18",
			"208.27.0.0/16",
			"34.3.0.0/16",
			"34.199.0.0/16",
			"66.59.192.0/19",
			"165.11.128.0/17",
			"135.138.0.0/16",
			"130.141.0.0/16",
			"165.185.0.0/16"
		],
		"addresses_v6": [],
		"address_details_v6": []
	}]
}

Here is some sanitized fake example output of a command I'm using an API to pull.

I only need the actual IP addresses, not the subnets, and I only need the IP addresses once.

From the above example the hopeful output of this would just be the two IP addresses on two lines, like this

135.138.168.78
135.139.168.79

This is just one block of output from about 10 that are in the output/file, all formatted just like this.

Using my mediocre skillset and regex, I attempted to pull this information using grep and awk and had no real success, getting either multiples of the IP addresses in my output, or both the IPs and the subnets.

EDIT - updated with the validated JSON.

  • now getting an error using the command
    jq --raw-output '.addresses[]'

"jq: error (at <stdin>:37): Cannot iterate over null (null)
exit status 5"

tried adding a "?" after the square brackets and it outputs nothing.

tried doing '.result.addresses[]' or '.status.addresses[]'

get "jq: error (at <stdin>:37): Cannot index string with string "addresses"
exit status 5"

答案1

得分: 2

假设输入是有效的JSON(您提供的示例需要以下修复才能变成有效的JSON,参见https://www.json.org/的规范):

  • 一个文档不能仅以字段名开始,这里是&quot;address_details&quot;:,您必须使用花括号{打开一个对象 - 就像您在结尾处关闭它一样。

  • 字符串必须用双引号(&quot;)括起来。例如,对于&quot;zone&quot;: “Germany”,您可以看到“Germany”使用了不同的引号。

然后,以下stedolan/jq筛选器可以提取您想要的内容:

  • .addresses遍历到&quot;addresses&quot;字段名下的数组。
  • .[]迭代该数组的项(当与字段名一起缩写时,初始点被省略)
  • --raw-output标志将输出转换为原始文本(在那之前它们是JSON字符串,即如上所讨论,用引号括起来)
jq --raw-output &#39;.addresses[]&#39; file.json
135.138.168.79

如果您在调用环境中使用管道(一个程序生成JSON输出,您希望在不首先将其保存到文件中的情况下即时处理它),jq也可以从stdin中读取:

... | jq --raw-output &#39;.addresses[]&#39;

要轻松玩转jq,可以使用https://jqplay.org/。将jq筛选器(在这种情况下是.addresses[])输入到左上角的框中,将输入的JSON输入到左下角的框中,然后(仍然是JSON编码)的结果会立即显示在右侧。选中“Raw Output”复选框以应用我们之前使用的--raw-output标志。在输入框下方,您将看到一个准备好用于命令行的命令,由上面的输入组合而成。根据这个示例,它现在应该读作jq --raw-output &#39;.addresses[]&#39;。这是一个链接,可以直接进入这个最终状态:https://jqplay.org/s/BMmPjsL6sUT(在站点维护者决定重置站点时有效,这种情况每隔一段时间发生一次)。

要更深入地了解jq语言,请查看jqplay.org网站底部的(可点击的)示例,并参考https://stedolan.github.io/jq/manual/上的手册。

英文:

Assuming the input is valid JSON (your provided sample needs the following fixes to turn it into one (see the specs at https://www.json.org/):

  • a document cannot just start with a field name, here &quot;address_details&quot;:, you have to open an object with curly braces { - just like as you do close it at the end.

  • Strings have to be wrapped into double qoutes (&quot;). For &quot;zone&quot;: “Germany” you can see that “Germany” uses different ones.

), then the following stedolan/jq filter can extract what you want:

  • .addresses traverses to the array under the &quot;addresses&quot; field name.
  • .[] iterates through the items of that array (when contracting with a field name, the initial dot is omitted)
  • the --raw-output flag turns the outputs into raw text (until then they are JSON strings, i.e. wrapped into quotes as discussed above)
jq --raw-output &#39;.addresses[]&#39; file.json
135.138.168.78
135.138.168.79

If in your calling environment you are using a pipeline (a program produces the JSON output, and you want to process it on the fly without first saving it into a file), jq can also read in from stdin:

… | jq --raw-output &#39;.addresses[]&#39;

To easily play around with jq, there is https://jqplay.org/. Enter the jq filter (.addresses[] in this case) into the upper left box, and the input JSON into the lower left box, and the (still JSON-encoded) result is immediately shown on the right side. Tick the checkbox reading Raw Output to apply the --raw-output flag we used earlier. Below the input boxes you are shown a command ready to be used on the command line, compiled from the inputs you have made above. Following this example it should now read jq --raw-output &#39;.addresses[]&#39;. Here's a link that brings you right into this final state: https://jqplay.org/s/BMmPjsL6sUT (valid until the site maintainers decide to reset the site which happens every now and then.)

To dive more into the jq language, have a look at the (clickable) examples on the bottom of the jqplay.org site, and consult the manual at https://stedolan.github.io/jq/manual/.

答案2

得分: 1

cat | grep -Eo '"[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}"' | tr -d '"' | sort -u

应该可以完成这个任务。

英文:
cat &lt;file&gt; | grep -Eo &#39;&quot;[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}&quot;&#39; | tr -d &#39;&quot;&#39; | sort -u

should do it

答案3

得分: 0

jq --raw-output '.result | .[].addresses[]'; 这个命令是有效的,它可以在测试文件和真实数据中输出IP地址。

英文:

so, again, totally new to this. Trial and error got me to this command.

jq --raw-output &#39;.result | .[].addresses[]&#39;

It works. I have no idea if this is a well-formatted or thought out jq command, but it outputs the IPs in both my test file and the real data.

huangapple
  • 本文由 发表于 2023年2月24日 02:58:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/75549196.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定