如何让一个 Golang 脚本修改我的 Terraform（HCL 格式）文件中的一个值？

huangapple
go评论118阅读模式
英文:

How can I have a Golang script modify a value in my Terraform (HCL format) file?

问题

我正在尝试对一个 Terraform 文件进行一小部分自动化处理，该文件定义了一个 Azure 网络安全组。基本上，我有一个网站和 SSH 访问，我只想允许它们访问我的公共 IP 地址，我可以从 icanhazip.com 获取。我希望使用一个 Golang 脚本将我的 IP 写入 .tf 文件的相关部分（即设置 security_rule.source_address_prefixes 的值）。

我尝试使用 Golang 中的 hclsimple 库，还尝试过 gohcl、hclwrite 等其他库，但是在将 HCL 文件转换为 Golang 结构时遇到了困难。

我的 Terraform 文件（应该是 HCL 格式）如下所示：

  1. resource "azurerm_network_security_group" "my_nsg" {
  2.   name                = "my_nsg"
  3.   location            = "loc"
  4.   resource_group_name = "rgname"
  6.   security_rule       = [
  7.            {
  8.                access                                     = "Deny"
  9.                description                                = "Desc"
  10.                destination_address_prefix                 = "*"
  11.                destination_address_prefixes               = []
  12.                destination_application_security_group_ids = []
  13.                destination_port_range                     = ""
  14.                destination_port_ranges                    = [
  15.                    "123",
  16.                    "456",
  17.                    "789",
  18.                    "1001",
  19.                 ]
  20.                direction                                  = "Inbound"
  21.                name                                       = "AllowInboundThing"
  22.                priority                                   = 100
  23.                protocol                                   = "*"
  24.                source_address_prefix                      = "*"
  25.                source_address_prefixes                    = [
  26.                   # obtain from icanhazip.com
  27.                   "1.2.3.4"
  28.                ]
  29.                source_application_security_group_ids      = []
  30.                source_port_range                          = "*"
  31.                source_port_ranges                         = []
  32.             },
  33.            {
  34.                access                                     = "Allow"
  35.                description                                = "Grant acccess to App"
  36.                destination_address_prefix                 = "*"
  37.                destination_address_prefixes               = []
  38.                destination_application_security_group_ids = []
  39.                destination_port_range                     = ""
  40.                destination_port_ranges                    = [
  41.                    "443",
  42.                    "80",
  43.                 ]
  44.                direction                                  = "Inbound"
  45.                name                                       = "AllowIPInBound"
  46.                priority                                   = 200
  47.                protocol                                   = "*"
  48.                source_address_prefix                      = ""
  49.                source_address_prefixes                    = [
  50.                 # obtain from icanhazip.com
  51.                    "1.2.3.4"
  52.                 ]
  53.                source_application_security_group_ids      = []
  54.                source_port_range                          = "*"
  55.                source_port_ranges                         = []
  56.             }
  57.         ]
  58. }

至此，我已经编写了一个 Golang 脚本，试图将上述数据表示为结构体，然后解码 .tf 文件本身（我从 hclsimple 中本地复制了几个方法，以便它可以解码 .tf 文件，正如在其文档中建议的那样）。

  1. package main
  3. import (
  4. 	"fmt"
  5. 	"io"
  6. 	"io/ioutil"
  7. 	"log"
  8. 	"os"
  9. 	"path/filepath"
  10. 	"strings"
  12. 	"github.com/hashicorp/hcl/v2"
  13. 	"github.com/hashicorp/hcl/v2/gohcl"
  14. 	"github.com/hashicorp/hcl/v2/hclsimple"
  15. 	"github.com/hashicorp/hcl/v2/hclsyntax"
  16. 	"github.com/hashicorp/hcl/v2/json"
  17. )
  19. type Config struct {
  20. 	NetworkSecurityGroup []NetworkSecurityGroup `hcl:"resource,block"`
  21. }
  23. type NetworkSecurityGroup struct {
  24. 	Type              string         `hcl:"azurerm_network_security_group,label"`
  25. 	Name              string         `hcl:"mick-linux3-nsg,label"`
  26. 	NameAttr          string         `hcl:"name"`
  27. 	Location          string         `hcl:"location"`
  28. 	ResourceGroupName string         `hcl:"resource_group_name"`
  29. 	SecurityRule      []SecurityRule `hcl:"security_rule,block"`
  30. }
  32. type SecurityRule struct {
  33. 	Access                                 string   `hcl:"access"`
  34. 	Description                            string   `hcl:"description"`
  35. 	DestinationAddressPrefix               string   `hcl:"destination_address_prefix"`
  36. 	DestinationAddressPrefixes             []string `hcl:"destination_address_prefixes"`
  37. 	DestinationApplicationSecurityGroupIds []string `hcl:"destination_application_security_group_ids"`
  38. 	DestinationPortRange                   string   `hcl:"destination_port_range"`
  39. 	DestinationPortRanges                  []string `hcl:"destination_port_ranges"`
  40. 	Direction                              string   `hcl:"direction"`
  41. 	Name                                   string   `hcl:"name"`
  42. 	Priority                               int      `hcl:"priority"`
  43. 	Protocol                               string   `hcl:"protocol"`
  44. 	SourceAddressPrefix                    string   `hcl:"source_address_prefix"`
  45. 	SourceAddressPrefixes                  []string `hcl:"source_address_prefixes"`
  46. 	SourceApplicationSecurityGroupIds      []string `hcl:"source_application_security_group_ids"`
  47. 	SourcePortRange                        string   `hcl:"source_port_range"`
  48. 	SourcePortRanges                       []string `hcl:"source_port_ranges"`
  49. }
  51. func main() {
  52. 	// lets pass this in as a param?
  53. 	configFilePath := "nsg.tf"
  55. 	// create new Config struct
  56. 	var config Config
  58. 	// This decodes the TF file into the config struct, and hydrates the values
  59. 	err := MyDecodeFile(configFilePath, nil, &config)
  60. 	if err != nil {
  61. 		log.Fatalf("Failed to load configuration: %s", err)
  62. 	}
  63. 	log.Printf("Configuration is %#v", config)
  65. 	// let's read in the file contents
  66. 	file, err := os.Open(configFilePath)
  67. 	if err != nil {
  68. 		fmt.Printf("Failed to read file: %v\n", err)
  69. 		return
  70. 	}
  71. 	defer file.Close()
  73. 	// Read the file and output as a []bytes
  74. 	bytes, err := io.ReadAll(file)
  75. 	if err != nil {
  76. 		fmt.Println("Error reading file:", err)
  77. 		return
  78. 	}
  80. 	// Parse, decode and evaluate the config of the .tf file
  81. 	hclsimple.Decode(configFilePath, bytes, nil, &config)
  83. 	// iterate through the rules until we find one with
  84. 	// Description = "Grant acccess to Flask App"
  86. 	// CODE GO HERE
  87. 	for _, nsg := range config.NetworkSecurityGroup {
  88. 		fmt.Printf("security rule: %s", nsg.SecurityRule)
  89. 	}
  90. }
  92. // Basically copied from here https://github.com/hashicorp/hcl/blob/v2.16.2/hclsimple/hclsimple.go#L59
  93. // but modified to handle .tf files too
  94. func MyDecode(filename string, src []byte, ctx *hcl.EvalContext, target interface{}) error {
  95. 	var file *hcl.File
  96. 	var diags hcl.Diagnostics
  98. 	switch suffix := strings.ToLower(filepath.Ext(filename)); suffix {
  99. 	case ".tf":
  100. 		file, diags = hclsyntax.ParseConfig(src, filename, hcl.Pos{Line: 1, Column: 1})
  101. 	case ".hcl":
  102. 		file, diags = hclsyntax.ParseConfig(src, filename, hcl.Pos{Line: 1, Column: 1})
  103. 	case ".json":
  104. 		file, diags = json.Parse(src, filename)
  105. 	default:
  106. 		diags = diags.Append(&hcl.Diagnostic{
  107. 			Severity: hcl.DiagError,
  108. 			Summary:  "Unsupported file format",
  109. 			Detail:   fmt.Sprintf("Cannot read from %s: unrecognized file format suffix %q.", filename, suffix),
  110. 		})
  111. 		return diags
  112. 	}
  113. 	if diags.HasErrors() {
  114. 		return diags
  115. 	}
  117. 	diags = gohcl.DecodeBody(file.Body, ctx, target)
  118. 	if diags.HasErrors() {
  119. 		return diags
  120. 	}
  121. 	return nil
  122. }
  124. // Taken from here https://github.com/hashicorp/hcl/blob/v2.16.2/hclsimple/hclsimple.go#L89
  125. func MyDecodeFile(filename string, ctx *hcl.EvalContext, target interface{}) error {
  126. 	src, err := ioutil.ReadFile(filename)
  127. 	if err != nil {
  128. 		if os.IsNotExist(err) {
  129. 			return hcl.Diagnostics{
  130. 				{
  131. 					Severity: hcl.DiagError,
  132. 					Summary:  "Configuration file not found",
  133. 					Detail:   fmt.Sprintf("The configuration file %s does not exist.", filename),
  134. 				},
  135. 			}
  136. 		}
  137. 		return hcl.Diagnostics{
  138. 			{
  139. 				Severity: hcl.DiagError,
  140. 				Summary:  "Failed to read configuration",
  141. 				Detail:   fmt.Sprintf("Can't read %s: %s.", filename, err),
  142. 			},
  143. 		}
  144. 	}
  145. 	return MyDecode(filename, src, ctx, target)
  146. }

当我运行代码时，我遇到了定义 NetworkSecurityGroup.SecurityRule 的困难，并收到了上述代码中的错误：

  1. 2023/05/24 11:42:11 Failed to load configuration: nsg.tf:6,3-16: Unsupported argument; An argument named "security_rule" is not expected here. Did you mean to define a block of type "security_rule"?
  2. exit status 1

非常感谢您的任何建议。

英文:

I am attempting to do a small bit of automation on a Terraform file I have, that defines an Azure Network Security Group. Essentially I have a website & SSH access that I only want to allow to my public IP address, which I can obtain from icanhazip.com. I am hoping to write my IP into the relevant part of the .tf file with a Golang script (essentially set the value for security_rule.source_address_prefixes).

I am attempting to use hclsimple library in Golang, as well as having tried gohcl, hclwrite and others, but I'm not getting anywhere with converting the HCL file into Golang structs, essentially.

My Terraform file (HCL format I believe) is as follows:

  1. resource "azurerm_network_security_group" "my_nsg" {
  2. name                = "my_nsg"
  3. location            = "loc"
  4. resource_group_name = "rgname"
  5. security_rule       = [
  6. {
  7. access                                     = "Deny"
  8. description                                = "Desc"
  9. destination_address_prefix                 = "*"
  10. destination_address_prefixes               = []
  11. destination_application_security_group_ids = []
  12. destination_port_range                     = ""
  13. destination_port_ranges                    = [
  14. "123",
  15. "456",
  16. "789",
  17. "1001",
  18. ]
  19. direction                                  = "Inbound"
  20. name                                       = "AllowInboundThing"
  21. priority                                   = 100
  22. protocol                                   = "*"
  23. source_address_prefix                      = "*"
  24. source_address_prefixes                    = [
  25. # obtain from icanhazip.com
  26. "1.2.3.4"
  27. ]
  28. source_application_security_group_ids      = []
  29. source_port_range                          = "*"
  30. source_port_ranges                         = []
  31. },
  32. {
  33. access                                     = "Allow"
  34. description                                = "Grant acccess to App"
  35. destination_address_prefix                 = "*"
  36. destination_address_prefixes               = []
  37. destination_application_security_group_ids = []
  38. destination_port_range                     = ""
  39. destination_port_ranges                    = [
  40. "443",
  41. "80",
  42. ]
  43. direction                                  = "Inbound"
  44. name                                       = "AllowIPInBound"
  45. priority                                   = 200
  46. protocol                                   = "*"
  47. source_address_prefix                      = ""
  48. source_address_prefixes                    = [
  49. # obtain from icanhazip.com
  50. "1.2.3.4"
  51. ]
  52. source_application_security_group_ids      = []
  53. source_port_range                          = "*"
  54. source_port_ranges                         = []
  55. }
  56. ]
  57. }

And this is as far as I have got with my Golang script, in attempting to represent the above data as structs, and then decoding the .tf file itself (I copied a couple of methods locally from hclsimple in order to have it decode a .tf file, as suggested in their docs.

  1. package main
  2. import (
  3. "fmt"
  4. "io"
  5. "io/ioutil"
  6. "log"
  7. "os"
  8. "path/filepath"
  9. "strings"
  10. "github.com/hashicorp/hcl/v2"
  11. "github.com/hashicorp/hcl/v2/gohcl"
  12. "github.com/hashicorp/hcl/v2/hclsimple"
  13. "github.com/hashicorp/hcl/v2/hclsyntax"
  14. "github.com/hashicorp/hcl/v2/json"
  15. )
  16. type Config struct {
  17. NetworkSecurityGroup []NetworkSecurityGroup `hcl:"resource,block"`
  18. }
  19. type NetworkSecurityGroup struct {
  20. Type              string         `hcl:"azurerm_network_security_group,label"`
  21. Name              string         `hcl:"mick-linux3-nsg,label"`
  22. NameAttr          string         `hcl:"name"`
  23. Location          string         `hcl:"location"`
  24. ResourceGroupName string         `hcl:"resource_group_name"`
  25. SecurityRule      []SecurityRule `hcl:"security_rule,block"`
  26. }
  27. type SecurityRule struct {
  28. Access                                 string   `hcl:"access"`
  29. Description                            string   `hcl:"description"`
  30. DestinationAddressPrefix               string   `hcl:"destination_address_prefix"`
  31. DestinationAddressPrefixes             []string `hcl:"destination_address_prefixes"`
  32. DestinationApplicationSecurityGroupIds []string `hcl:"destination_application_security_group_ids"`
  33. DestinationPortRange                   string   `hcl:"destination_port_range"`
  34. DestinationPortRanges                  []string `hcl:"destination_port_ranges"`
  35. Direction                              string   `hcl:"direction"`
  36. Name                                   string   `hcl:"name"`
  37. Priority                               int      `hcl:"priority"`
  38. Protocol                               string   `hcl:"protocol"`
  39. SourceAddressPrefix                    string   `hcl:"source_address_prefix"`
  40. SourceAddressPrefixes                  []string `hcl:"source_address_prefixes"`
  41. SourceApplicationSecurityGroupIds      []string `hcl:"source_application_security_group_ids"`
  42. SourcePortRange                        string   `hcl:"source_port_range"`
  43. SourcePortRanges                       []string `hcl:"source_port_ranges"`
  44. }
  45. func main() {
  46. // lets pass this in as a param?
  47. configFilePath := "nsg.tf"
  48. // create new Config struct
  49. var config Config
  50. // This decodes the TF file into the config struct, and hydrates the values
  51. err := MyDecodeFile(configFilePath, nil, &config)
  52. if err != nil {
  53. log.Fatalf("Failed to load configuration: %s", err)
  54. }
  55. log.Printf("Configuration is %#v", config)
  56. // let's read in the file contents
  57. file, err := os.Open(configFilePath)
  58. if err != nil {
  59. fmt.Printf("Failed to read file: %v\n", err)
  60. return
  61. }
  62. defer file.Close()
  63. // Read the file and output as a []bytes
  64. bytes, err := io.ReadAll(file)
  65. if err != nil {
  66. fmt.Println("Error reading file:", err)
  67. return
  68. }
  69. // Parse, decode and evaluate the config of the .tf file
  70. hclsimple.Decode(configFilePath, bytes, nil, &config)
  71. // iterate through the rules until we find one with
  72. // Description = "Grant acccess to Flask App"
  73. // CODE GO HERE
  74. for _, nsg := range config.NetworkSecurityGroup {
  75. fmt.Printf("security rule: %s", nsg.SecurityRule)
  76. }
  77. }
  78. // Basically copied from here https://github.com/hashicorp/hcl/blob/v2.16.2/hclsimple/hclsimple.go#L59
  79. // but modified to handle .tf files too
  80. func MyDecode(filename string, src []byte, ctx *hcl.EvalContext, target interface{}) error {
  81. var file *hcl.File
  82. var diags hcl.Diagnostics
  83. switch suffix := strings.ToLower(filepath.Ext(filename)); suffix {
  84. case ".tf":
  85. file, diags = hclsyntax.ParseConfig(src, filename, hcl.Pos{Line: 1, Column: 1})
  86. case ".hcl":
  87. file, diags = hclsyntax.ParseConfig(src, filename, hcl.Pos{Line: 1, Column: 1})
  88. case ".json":
  89. file, diags = json.Parse(src, filename)
  90. default:
  91. diags = diags.Append(&hcl.Diagnostic{
  92. Severity: hcl.DiagError,
  93. Summary:  "Unsupported file format",
  94. Detail:   fmt.Sprintf("Cannot read from %s: unrecognized file format suffix %q.", filename, suffix),
  95. })
  96. return diags
  97. }
  98. if diags.HasErrors() {
  99. return diags
  100. }
  101. diags = gohcl.DecodeBody(file.Body, ctx, target)
  102. if diags.HasErrors() {
  103. return diags
  104. }
  105. return nil
  106. }
  107. // Taken from here https://github.com/hashicorp/hcl/blob/v2.16.2/hclsimple/hclsimple.go#L89
  108. func MyDecodeFile(filename string, ctx *hcl.EvalContext, target interface{}) error {
  109. src, err := ioutil.ReadFile(filename)
  110. if err != nil {
  111. if os.IsNotExist(err) {
  112. return hcl.Diagnostics{
  113. {
  114. Severity: hcl.DiagError,
  115. Summary:  "Configuration file not found",
  116. Detail:   fmt.Sprintf("The configuration file %s does not exist.", filename),
  117. },
  118. }
  119. }
  120. return hcl.Diagnostics{
  121. {
  122. Severity: hcl.DiagError,
  123. Summary:  "Failed to read configuration",
  124. Detail:   fmt.Sprintf("Can't read %s: %s.", filename, err),
  125. },
  126. }
  127. }
  128. return MyDecode(filename, src, ctx, target)
  129. }

When I run the code, essentially I am struggling to define NetworkSecurityGroup.SecurityRule, and receive the following error with the above code:

  1. 2023/05/24 11:42:11 Failed to load configuration: nsg.tf:6,3-16: Unsupported argument; An argument named "security_rule" is not expected here. Did you mean to define a block of type "security_rule"?
  2. exit status 1

Any advice much appreciated

答案1

得分: 0

所以目前使用https://github.com/minamijoyo/hcledit是不可能的（请参见https://github.com/minamijoyo/hcledit/issues/50 - 这表明hclwrite本身需要进行更改以便实现）。

所以我按照@Martin Atkins的建议进行了解决：

我创建了一个名为locals.tf的文件，其中包含一个本地变量，然后在NSG安全规则中引用该变量：

  1. locals {
  2.     my_ip = "1.2.3.4"
  3. }

现在，我只需获取我的IP并使用sed命令更新locals.tf文件中的值：

  1. my_ip=$(curl -s -4 icanhazip.com)
  2. sed -i "s|my_ip = \".*\"|my_ip = \"$my_ip\"|" locals.tf
英文:

So it's not currently possible with https://github.com/minamijoyo/hcledit (see here https://github.com/minamijoyo/hcledit/issues/50 - this suggests that hclwrite itself would need a change to facilitate)

So I've worked around as suggested by @Martin Atkins:

I've created a single locals.tf file containing a locals variable, which I then reference in the NSG security rules:

  1. locals {
  2. my_ip = "1.2.3.4"
  3. }

and now I simply obtain my IP and use sed to update the value in the locals.tf file

  1. my_ip=$(curl -s -4 icanhazip.com)
  2. sed -i "s|my_ip = \".*\"|my_ip = \"$my_ip\"|" locals.tf

答案2

得分: 0

slice应该表示为

  1. security_rule { item1 }
  2. security_rule { item2 }

而不是

  1. security_rule = [{ item1 }, { item2 }]

在HCL中。

还有一个新的HCL解组器：https://github.com/genelet/determined/。

"determined"的结果是：

  1. &main.Config{NetworkSecurityGroup:[]main.NetworkSecurityGroup{main.NetworkSecurityGroup{Type:"azurerm_network_security_group", Name:"my_nsg", NameAttr:"my_nsg", Location:"loc", ResourceGroupName:"rgname", SecurityRule:[]main.SecurityRule{main.SecurityRule{Access:"Deny", Description:"Desc", DestinationAddressPrefix:"*", DestinationAddressPrefixes:[]string{}, DestinationApplicationSecurityGroupIds:[]string{}, DestinationPortRange:"", DestinationPortRanges:[]string{"123", "456", "789", "1001"}, Direction:"Inbound", Name:"AllowInboundThing", Priority:100, Protocol:"*", SourceAddressPrefix:"*", SourceAddressPrefixes:[]string{"1.2.3.4"}, SourceApplicationSecurityGroupIds:[]string{}, SourcePortRange:"*", SourcePortRanges:[]string{}}, main.SecurityRule{Access:"Allow", Description:"Grant acccess to App", DestinationAddressPrefix:"*", DestinationAddressPrefixes:[]string{}, DestinationApplicationSecurityGroupIds:[]string{}, DestinationPortRange:"", DestinationPortRanges:[]string{"443", "80"}, Direction:"Inbound", Name:"AllowIPInBound", Priority:200, Protocol:"*", SourceAddressPrefix:"", SourceAddressPrefixes:[]string{"1.2.3.4"}, SourceApplicationSecurityGroupIds:[]string{}, SourcePortRange:"*", SourcePortRanges:[]string{}}}}}}
英文:

slice should be represented as

  1. security_rule { item1 }
  2. security_rule { item2 }

instead of

  1. security_rule = [{ item1 }, { item2 }]

in HCL.

There is also a new HCL unmarshaler: https://github.com/genelet/determined/.

The result from "determined" is:

  1. &main.Config{NetworkSecurityGroup:[]main.NetworkSecurityGroup{main.NetworkSecurityGroup{Type:"azurerm_network_security_group", Name:"my_nsg", NameAttr:"my_nsg", Location:"loc", ResourceGroupName:"rgname", SecurityRule:[]main.SecurityRule{main.SecurityRule{Access:"Deny", Description:"Desc", DestinationAddressPrefix:"*", DestinationAddressPrefixes:[]string{}, DestinationApplicationSecurityGroupIds:[]string{}, DestinationPortRange:"", DestinationPortRanges:[]string{"123", "456", "789", "1001"}, Direction:"Inbound", Name:"AllowInboundThing", Priority:100, Protocol:"*", SourceAddressPrefix:"*", SourceAddressPrefixes:[]string{"1.2.3.4"}, SourceApplicationSecurityGroupIds:[]string{}, SourcePortRange:"*", SourcePortRanges:[]string{}}, main.SecurityRule{Access:"Allow", Description:"Grant acccess to App", DestinationAddressPrefix:"*", DestinationAddressPrefixes:[]string{}, DestinationApplicationSecurityGroupIds:[]string{}, DestinationPortRange:"", DestinationPortRanges:[]string{"443", "80"}, Direction:"Inbound", Name:"AllowIPInBound", Priority:200, Protocol:"*", SourceAddressPrefix:"", SourceAddressPrefixes:[]string{"1.2.3.4"}, SourceApplicationSecurityGroupIds:[]string{}, SourcePortRange:"*", SourcePortRanges:[]string{}}}}}}

huangapple
  • 本文由 发表于 2023年5月24日 18:48:34
  • 转载请务必保留本文链接：https://go.coder-hub.com/76322692.html
  • go
  • hcl
  • terraform
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定