合并两个YAML文件深度合并在Bash中使用yq。

huangapple go评论77阅读模式
英文:

Merge Two Yaml Files Deeply in Bash yq

问题

我有两个YAML文件:

file1.yaml

name: "name-a"
namespace: "name-a-space"

livenessProbe:
  path: is-alive

readinessProbe:
  path: is-ready

env:
  - name: key1
    value: test1
  - name: key2
    value: test2
  - name: KEY3
    value: test2

和file2.yaml

name: "name-b"
namespace: "name-b-space"

livenessProbe:
  path: is-alive1

readinessProbe:
  path: is-ready1

env:
  - name: key1
    value: test7

为了合并file1.yaml和file2.yaml,我正在使用yq v4.30.8

yq '. *= load("file2.yml")' file1.yml

问题是,关于env键,它会覆盖file1中file2的所有env元素,但我只需要覆盖key1的不同值,所以输出应该是

name: "name-b"
namespace: "name-b-space"

livenessProbe:
  path: is-alive1

readinessProbe:
  path: is-ready1

env:
  - name: key1
    value: test7
  - name: key2
    value: test2
  - name: KEY3
    value: test2

但我的代码生成

name: "name-b"
namespace: "name-b-space"

livenessProbe:
  path: is-alive1

readinessProbe:
  path: is-ready1

env:
  - name: key1
    value: test7

参考链接

英文:

I have two yaml files:

file1.yaml

name: "name-a"
namespace: "name-a-space"

livenessProbe:
  path: is-alive

readinessProbe:
  path: is-ready

env:
  - name: key1
    value: test1
  - name: key2
    value: test2
  - name: KEY3
    value: test2

and file2.yaml

name: "name-b"
namespace: "name-b-space"

livenessProbe:
  path: is-alive1

readinessProbe:
  path: is-ready1

env:
  - name: key1
    value: test7

And to merge file1.yaml and file2.yaml I'm using yq v4.30.8

yq '. *= load("file2.yml")' file1.yml based on this reference

The problem is that regarding env key it overwriting all elements of env in file2 on file1, but I need to to just overwrite the different value of key1
so the output should be

name: "name-b"
namespace: "name-b-space"

livenessProbe:
  path: is-alive1

readinessProbe:
  path: is-ready1

env:
  - name: key1
    value: test7
  - name: key2
    value: test2
  - name: KEY3
    value: test2

But my code generate

name: "name-b"
namespace: "name-b-space"

livenessProbe:
  path: is-alive1

readinessProbe:
  path: is-ready1

env:
  - name: key1
    value: test7

答案1

得分: 3

使用 *d合并、深度合并数组,它会合并数组 "类似对象,以其索引作为键",即 "合并数组中的第一项,然后不对[其他项]做任何操作":

yq '. *d load("file2.yaml")' file1.yaml
name: "name-b"
namespace: "name-b-space"
livenessProbe:
  path: is-alive1
readinessProbe:
  path: is-ready1
env:
  - name: key1
    value: test7
  - name: key2
    value: test2
  - name: KEY3
    value: test2

正如转到的链接所示,您希望通过匹配项目对象中的特定字段来合并.env中的数组。为此,您可以使用.name作为字段名称(在合并时变为 "键")将.env数组简化为一个对象,使用.[] as $item ireduce ({}; .[$item.name] = $item.value)。对于两侧的.env都执行此操作,然后使用常规合并 *。最后,使用map({ "name": key, "value": . })将索引对象映射回原始数组结构,使用多余的空格来突出显示重复的代码:

yq '
  ( .env |= (.[] as $item ireduce ({}; .[$item.name] = $item.value))) *
  (load("file2.yaml") | .env |= (.[] as $item ireduce ({}; .[$item.name] = $item.value)))
  | .env |= map({ "name": key, "value": . })
' file1.yaml

此外,还请查看手册中的合并对象数组,根据键匹配部分,该部分基于问题如何在"yq"中有条件地添加/替换数组元素

最后,还请考虑使用kislyuk/yq(这是yq的另一个实现),或者itchyny/gojq。这两个工具都是YAML处理器,前者在内部将YAML转换为JSON,然后在底层使用stedolan/jq,后者是Go中jq的重新实现,实现了本机的YAML功能。因此,两者都可以使用mikefarah/yq尚未实现的jq功能。例如:

# 使用 kislyuk/yq
yq -sy '
  map(.env |= INDEX(.name)) | .[0] * .[1] | .env |= map(.)
' file1.yaml file2.yaml

# 使用 itchyny/gojq
gojq -s --yaml-input --yaml-output '
  map(.env |= INDEX(.name)) | .[0] * .[1] | .env |= map(.)
' file1.yaml file2.yaml
英文:

Use *d to Merge, deeply-merging arrays which merges arrays "like objects, with indices as their key", i.e. "merge the first item in the array and do nothing with the [other ones].":

yq '. *d load("file2.yaml")' file1.yaml
name: "name-b"
namespace: "name-b-space"
livenessProbe:
  path: is-alive1
readinessProbe:
  path: is-ready1
env:
  - name: key1
    value: test7
  - name: key2
    value: test2
  - name: KEY3
    value: test2

As it turned out, you want the arrays in .env to be merged by matching a specific field within the item objects. To this end, you could ireduce the .env array into one object using .name as field names (which becomes the "key" when merging) using .[] as $item ireduce ({}; .[$item.name] = $item.value). Do this for .env on both sides, then use regular merging with *. Eventually, map the indexed object back into your original array structure using map({"name": key, "value": .}) (using unnecessary spaces to highlight duplicate code):

yq '
  (                     .env |= (.[] as $item ireduce ({}; .[$item.name] = $item.value))) *
  (load("file2.yaml") | .env |= (.[] as $item ireduce ({}; .[$item.name] = $item.value)))
  |                     .env |= map({"name": key, "value": .})
' file1.yaml

Also, take a look at how to Merge arrays of objects together, matching on a key in the manual, which is based on the question How to Add / Replace array elements with "yq" conditionally.

Finally, also consider using kislyuk/yq (which is "the other" implementation of yq), or itchyny/gojq. Both are YAML processors with the former internally translating to JSON, then using stedolan/jq under the hood, and the latter being a reimplementation of jq in Go, implementing native YAML functionality. Thus, both can make use of jq features not (yet) implemted by mikefarah/yq. For example:

# using kislyuk/yq
yq -sy '
  map(.env |= INDEX(.name)) | .[0] * .[1] | .env |= map(.)
' file1.yaml file2.yaml

# using itchyny/gojq
gojq -s --yaml-input --yaml-output '
  map(.env |= INDEX(.name)) | .[0] * .[1] | .env |= map(.)
' file1.yaml file2.yaml

huangapple
  • 本文由 发表于 2023年5月29日 01:02:07
  • 转载请务必保留本文链接:https://go.coder-hub.com/76352612.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定