如何通过代码向 YAML 文件添加新条目

huangapple go评论74阅读模式
英文:

How to add a new entry to yaml file via code

问题

我有一个yaml文件,我需要使用Go代码在运行时向其中添加数据。路径如下,我是指:

这是一个带有sif下的一个条目的yaml文件的示例:

spec:
  mec:
    tolerations:
    - effect: NoSchedule
      key: WorkGroup
      operator: Equal
      value: goxy
    resources:
      requests:
        cpu: 100m
        memory: 1Gi
    customConfig:
      sif:
        prom_exporter:
          type: prometheus_exporter
        snk_dev:   
          type: sk_hc_logs
          inputs:
            - tesslt
          ep: ${NT}
          dken: ${SN}
          encoding:
            codec: "json"
          index: us
          compression: gzip
          buffer:
            type: memory

在以下yaml路径下,我需要添加一个新的条目:

spec->mec->customConfig->sif 添加一个新的条目 snd_prd

snk_prod:   
  type: sk_hc_logs
  inputs:
    - tesslt
  ep: ${NT}
  dken: ${SN}
  encoding:
    codec: "json"
  index: us
  compression: gzip
  buffer:
    type: memory

我们正在使用kustomize,我想知道是否有一种通过代码来完成的方法,我是指提前准备好需要添加的文件,并在运行时将其添加进去。或者也可以使用https://github.com/go-yaml/yaml库来实现。

英文:

I’ve yaml file and I need to add to it data on runtime using go code
The path is like following, I mean

This is the yaml file with one entry under sif of snk_dev

spec:
  mec:
    tolerations:
    - effect: NoSchedule
      key: WorkGroup
      operator: Equal
      value: goxy
    resources:
      requests:
        cpu: 100m
        memory: 1Gi
    customConfig:
      sif:
        prom_exporter:
          type: prometheus_exporter
        snk_dev:   
          type: sk_hc_logs
          inputs:
            - tesslt
          ep: ${NT}
          dken: ${SN}
          encoding:
            codec: "json"
          index: us
          compression: gzip
          buffer:
            type: memory

under the following yaml path I need to add a new entry

spec->mec->customConfig->sif a new entry snd_prd

    snk_prod:   
      type: sk_hc_logs
      inputs:
        - tesslt
      ep: ${NT}
      dken: ${SN}
      encoding:
        codec: "json"
      index: us
      compression: gzip
      buffer:
        type: memory

We are using kustomize and I wonder if there is a way to do it via code, I mean prefer in advance the file that i need to add and to add it in runtime
Or maybe better of using the https://github.com/go-yaml/yaml

答案1

得分: 2

YAML编解码器支持将数据解码为map[string]any类型,并将这样的映射编码为YAML格式。

思路是先解码原始文档和额外的树,然后将额外的映射放置在所需路径下,最后再进行编码。

type YamlObject map[string]interface{}

func main() {
    // 解析初始文档
    doc := make(YamlObject)
    yaml.Unmarshal([]byte(document), &doc)
    // 解析额外的节点
    addon := make(YamlObject)
    yaml.Unmarshal([]byte(extra), &addon)
    // 根据路径查找节点
    node := findChild(doc, "spec", "mec", "customConfig", "sif")
    if node == nil {
        panic("不应该发生")
    }
    // 将额外文档中的键值对添加到指定路径下
    for key, val := range addon {
        (*node)[key] = val
    }
    // 输出修改后的文档
    outDoc, _ := yaml.Marshal(doc)
    println(string(outDoc))
}

func findChild(obj YamlObject, path ...string) *YamlObject {
    if len(path) == 0 {
        return &obj
    }
    key := path[0]
    child, ok := obj[key]
    if !ok {
        return nil
    }
    obj, ok = child.(YamlObject)
    if !ok {
        return nil
    }
    return findChild(obj, path[1:]...)
}

完整示例:https://go.dev/play/p/pTdXR53p0mq

输出结果:

spec:
    mec:
        customConfig:
            sif:
                prom_exporter:
                    type: prometheus_exporter
                snk_dev:
                    buffer:
                        type: memory
                    compression: gzip
                    dken: ${SN}
                    encoding:
                        codec: json
                    ep: ${NT}
                    index: us
                    inputs:
                        - tesslt
                    type: sk_hc_logs
                snk_prod:
                    buffer:
                        type: memory
                    compression: gzip
                    dken: ${SN}
                    encoding:
                        codec: json
                    ep: ${NT}
                    index: us
                    inputs:
                        - tesslt
                    type: sk_hc_logs
        resources:
            requests:
                cpu: 100m
                memory: 1Gi
        tolerations:
            - effect: NoSchedule
              key: WorkGroup
              operator: Equal
              value: goxy

YAML编解码器按字母顺序输出键。

英文:

Yaml codec supports decoding to map[string]any and encoding such map into yaml.

The idea is to decode both the document and the extra tree, then put the additional map under the required path and then encode back.

type YamlObject map[string]any

func main() {
    // Parse the initial document
	doc := make(YamlObject)
	yaml.Unmarshal([]byte(document), &doc)
    // Parse the additional nodes
	addon := make(YamlObject)
	yaml.Unmarshal([]byte(extra), &addon)
    // Find the node by the path
	node := findChild(doc, "spec", "mec", "customConfig", "sif")
	if node == nil {
		panic("Must not happen")
	}
    // Add the keys from the additional document
    // under the specified path
	for key, val := range addon {
		(*node)[key] = val
	}
    // Output the modified document
	outDoc, _ := yaml.Marshal(doc)
	println(string(outDoc))
}

func findChild(obj YamlObject, path ...string) *YamlObject {
	if len(path) == 0 {
		return &obj
	}
	key := path[0]
	child, ok := obj[key]
	if !ok {
		return nil
	}
	obj, ok = child.(YamlObject)
	if !ok {
		return nil
	}
	return findChild(obj, path[1:]...)
}

Full example https://go.dev/play/p/pTdXR53p0mq

Output:

spec:
    mec:
        customConfig:
            sif:
                prom_exporter:
                    type: prometheus_exporter
                snk_dev:
                    buffer:
                        type: memory
                    compression: gzip
                    dken: ${SN}
                    encoding:
                        codec: json
                    ep: ${NT}
                    index: us
                    inputs:
                        - tesslt
                    type: sk_hc_logs
                snk_prod:
                    buffer:
                        type: memory
                    compression: gzip
                    dken: ${SN}
                    encoding:
                        codec: json
                    ep: ${NT}
                    index: us
                    inputs:
                        - tesslt
                    type: sk_hc_logs
        resources:
            requests:
                cpu: 100m
                memory: 1Gi
        tolerations:
            - effect: NoSchedule
              key: WorkGroup
              operator: Equal
              value: goxy

YAML codec outouts keys in the alphabetic order

答案2

得分: 1

关键在于生成等效的Go结构体来模拟您的YAML,并使用gopkg.in/yaml.v3包中的Marshal/Unmarshal函数。

您可以使用像yaml-to-go这样的工具,自动生成所需的YAML结构体,然后在其基础上进行任何其他自定义。下面的答案使用了这样一个工具的定义。

您的YAML结构可以稍作改进,因为snk_devsnk_prod字段看起来很相似。您应该为这两个字段定义一个公共类型,并定义一个YAML对象列表,然后将其转换为该特定类型的结构体切片。但是,由于原始YAML将它们保留为不同的实体,您的结构体也需要不同。

根据您对答案的评论snk_devsnk_prod字段是动态派生的,因此将您的CustomConfig定义为map[string]interface{}以允许动态键名是有意义的。

package main

import (
	"fmt"
	"log"

	"gopkg.in/yaml.v3"
)

type YAMLData struct {
	Spec Spec `yaml:"spec"`
}
type Tolerations struct {
	Effect   string `yaml:"effect"`
	Key      string `yaml:"key"`
	Operator string `yaml:"operator"`
	Value    string `yaml:"value"`
}
type Requests struct {
	CPU    string `yaml:"cpu"`
	Memory string `yaml:"memory"`
}
type Resources struct {
	Requests Requests `yaml:"requests"`
}
type PromExporter struct {
	Type string `yaml:"type"`
}
type Encoding struct {
	Codec string `yaml:"codec"`
}
type Buffer struct {
	Type string `yaml:"type"`
}
type SifConfig struct {
	Type        string   `yaml:"type"`
	Inputs      []string `yaml:"inputs"`
	Ep          string   `yaml:"ep"`
	Dken        string   `yaml:"dken"`
	Encoding    Encoding `yaml:"encoding"`
	Index       string   `yaml:"index"`
	Compression string   `yaml:"compression"`
	Buffer      Buffer   `yaml:"buffer"`
}
type CustomConfig struct {
	Sif map[string]interface{} `yaml:"sif"`
}
type Mec struct {
	Tolerations  []Tolerations `yaml:"tolerations"`
	Resources    Resources     `yaml:"resources"`
	CustomConfig CustomConfig  `yaml:"customConfig"`
}
type Spec struct {
	Mec Mec `yaml:"mec"`
}

var data = `spec:
  mec:
    tolerations:
    - effect: NoSchedule
      key: WorkGroup
      operator: Equal
      value: goxy
    resources:
      requests:
        cpu: 100m
        memory: 1Gi
    customConfig:
      sif:
        prom_exporter:
          type: prometheus_exporter
        snk_dev:   
          type: sk_hc_logs
          inputs:
            - tesslt
          ep: ${NT}
          dken: ${SN}
          encoding:
            codec: "json"
          index: us
          compression: gzip
          buffer:
            type: memory
`

func main() {
	t := YAMLData{}
	err := yaml.Unmarshal([]byte(data), &t)
	if err != nil {
		log.Fatalf("error: %v", err)
	}

	config := &t.Spec.Mec.CustomConfig
	config.Sif["snk_prod"] = SifConfig{
		Type:        "sk_hc_logs",
		Inputs:      []string{"tesslt"},
		Ep:          "${NT}",
		Dken:        "${SN}",
		Encoding:    Encoding{Codec: "json"},
		Index:       "us",
		Compression: "gzip",
		Buffer:      Buffer{Type: "memory"},
	}

	yamlBytes, err := yaml.Marshal(t)
	if err != nil {
		log.Fatalf("error: %v", err)
	}
	fmt.Println(string(yamlBytes))

}

yamlBytes可以进一步用于写入单独的文件,上面的代码没有包含这部分。

Go playground

英文:

The key here is to generate the equivalent Go structs to model your YAML and use the Marhshal/Unmarshal functions from gopkg.in/yaml.v3 package.

You could use a tool like yaml-to-go, to autogenerate the structs needed for your YAML and then perform any more additional customisations on top of it. The answer below takes the definitions from such a tool.

Your YAML structure could be improved a bit, because snk_dev & snk_prod fields look alike. You should defining a common type for both these and define a list of YAML objects, which in-turn would have converted into a slice of structs of that particular type. But since the original YAML retains the two of them as different entities, your structs also need to be different.

Based on your comment to the answer, that the fields snk_dev & snk_prod are dynamically derived, it would make sense to define your CustomConfig to be a map[string]interface{} to allow for dynamic key names.

package main

import (
	"fmt"
	"log"

	"gopkg.in/yaml.v3"
)

type YAMLData struct {
	Spec Spec `yaml:"spec"`
}
type Tolerations struct {
	Effect   string `yaml:"effect"`
	Key      string `yaml:"key"`
	Operator string `yaml:"operator"`
	Value    string `yaml:"value"`
}
type Requests struct {
	CPU    string `yaml:"cpu"`
	Memory string `yaml:"memory"`
}
type Resources struct {
	Requests Requests `yaml:"requests"`
}
type PromExporter struct {
	Type string `yaml:"type"`
}
type Encoding struct {
	Codec string `yaml:"codec"`
}
type Buffer struct {
	Type string `yaml:"type"`
}
type SifConfig struct {
	Type        string   `yaml:"type"`
	Inputs      []string `yaml:"inputs"`
	Ep          string   `yaml:"ep"`
	Dken        string   `yaml:"dken"`
	Encoding    Encoding `yaml:"encoding"`
	Index       string   `yaml:"index"`
	Compression string   `yaml:"compression"`
	Buffer      Buffer   `yaml:"buffer"`
}
type CustomConfig struct {
	Sif map[string]interface{} `yaml:"sif"`
}
type Mec struct {
	Tolerations  []Tolerations `yaml:"tolerations"`
	Resources    Resources     `yaml:"resources"`
	CustomConfig CustomConfig  `yaml:"customConfig"`
}
type Spec struct {
	Mec Mec `yaml:"mec"`
}

var data = `spec:
  mec:
    tolerations:
    - effect: NoSchedule
      key: WorkGroup
      operator: Equal
      value: goxy
    resources:
      requests:
        cpu: 100m
        memory: 1Gi
    customConfig:
      sif:
        prom_exporter:
          type: prometheus_exporter
        snk_dev:   
          type: sk_hc_logs
          inputs:
            - tesslt
          ep: ${NT}
          dken: ${SN}
          encoding:
            codec: "json"
          index: us
          compression: gzip
          buffer:
            type: memory
`

func main() {
	t := YAMLData{}
	err := yaml.Unmarshal([]byte(data), &t)
	if err != nil {
		log.Fatalf("error: %v", err)
	}

	config := &t.Spec.Mec.CustomConfig
	config.Sif["snk_prod"] = SifConfig{
		Type:        "sk_hc_logs",
		Inputs:      []string{"tesslt"},
		Ep:          "${NT}",
		Dken:        "${SN}",
		Encoding:    Encoding{Codec: "json"},
		Index:       "us",
		Compression: "gzip",
		Buffer:      Buffer{Type: "memory"},
	}

	yamlBytes, err := yaml.Marshal(t)
	if err != nil {
		log.Fatalf("error: %v", err)
	}
	fmt.Println(string(yamlBytes))

}

The yamlBytes can be used further to be written as a separate file, which is left out of the above.

Go playground

huangapple
  • 本文由 发表于 2022年10月9日 20:03:13
  • 转载请务必保留本文链接:https://go.coder-hub.com/74004588.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定