Terragrunt CICD初始化失败通过GitHub Actions

huangapple go评论67阅读模式
英文:

Terragrunt CICD init failure via GitHub Actions

问题

我在尝试使用Github Actions通过CI/CD运行Terragrunt时遇到了一些困难。我收到了以下错误消息。经过调查,我发现其他人也报告了这个问题,与依赖块有关。

我尝试了添加模拟输出,将模拟输出合并到状态中,升级/降级Terragrunt版本,引入一个Makefile以进入每个目录并运行init、plan和apply(因为似乎在运行所有命令时存在问题)等推荐的操作。

我可以成功地使用CLI按照依赖关系的特定顺序部署Terragrunt资源。首先需要设置VPC和托管区域,因为它们没有依赖关系。目前,资源是由CLI设置的,状态文件存在,因此CI/CD流水线的任何后续运行都应该正常工作,但它仍然失败。CLI和流水线都安装了相同版本的Terraform和Terragrunt。在切换Terragrunt版本时,我观察到无效字符'c'更改为无效字符':'

有人经历过这个问题并成功找到了解决方法吗?还是有关于如何更好地设置这个问题的建议?我是不是最好放弃Terragrunt?

1. 错误消息

********** Running init for dns_records **********
time=2023-05-29T17:05:33Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-hosted-zone.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/hosted_zone] 
time=2023-05-29T17:05:35Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod/*******/hosted_zone/terragrunt.hcl. Underlying error: invalid character 'c' looking for beginning of value
time=2023-05-29T17:05:35Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
********** Running init for subnet/mft-subnet **********
time=2023-05-29T17:05:35Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-vpc.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc] 
time=2023-05-29T17:05:37Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc/terragrunt.hcl. Underlying error: invalid character 'c' looking for beginning of value
time=2023-05-29T17:05:37Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
********** Running init for tgw-attach **********
time=2023-05-29T17:05:37Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-vpc.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc] 
time=2023-05-29T17:05:39Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc/terragrunt.hcl. Underlying error: invalid character 'c' looking for beginning of value
time=2023-05-29T17:05:39Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
********** Running init for route/tgw **********
time=2023-05-29T17:05:40Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-vpc.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc] 
time=2023-05-29T17:05:41Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc/terragrunt.hcl. Underlying error: invalid character 'c' looking for beginning of value
time=2023-05-29T17:05:41Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
make: *** [Makefile:14: init] Error 1
Error: Process completed with exit code 2.

2. 目录结构

.
├── Makefile
├── common.hcl
├── dns_records
│   ├── config.hcl
│   └── terragrunt.hcl
├── env.hcl
├── hosted_zone
│   ├── config.hcl
│   └── terragrunt.hcl
├── route
│   └── tgw
│       ├── config.hcl
│       └── terragrunt.hcl
├── subnet
│   └── mft-subnet
│       ├── config.hcl
│       └── terragrunt.hcl
├── tgw-attach
│   ├── config.hcl
│   └── terragrunt.hcl
└── vpc
    ├── config.hcl
    ├── override.tf
    └── terragrunt.hcl

3. Makefile

CURRENT_DIR := $(shell pwd)
MODULES := hosted_zone vpc dns_records subnet/mft-subnet tgw-attach route/tgw
.PHONY: check
check:
@echo "please specify rule"
all: init plan apply
.PHONY: init
init:
@for module in $(MODULES); do \
echo "********** Running init for $$module **********"; \
cd $(CURRENT_DIR)/$$module && terragrunt init; \
done
.PHONY: plan
plan:
<details>
<summary>英文:</summary>
I&#39;m having some difficulty attempting to run terragrunt via CI/CD using Github Actions. I get the below error message. Upon investigation, i found that this has been reported by others and relates to dependency blocks.
I have tried the recommended actions of adding mock outputs, merging the mock outputs to state, upgrading/downgrading the version of terragrunt, introducing a Makefile to &#39;cd&#39; into each directory and run an init, plan and apply (as there seem to be issues with the run-all command). 
I am able to successfully deploy the terragrunt resources in a particular order using CLI due to dependencies. The VPC and Hosted Zone need to be setup first as they have no dependencies. Currently, the resources are setup by CLI and the state file exists, so I any subsequent run of the CI/CD pipeline would work, but it still fails. Both CLI and Pipeline have the same version of terraform and terragrunt installed. On switching version of terragrunt i observed the **invalid character &#39;c&#39;** changes to an **invalid character &#39;:&#39;**.
Has anybody experienced this and managed to find a fix? or is there advice on how to better set this up? Am i better off getting rid of terragrunt?
**1. Error message**
********** Running init for dns_records **********
time=2023-05-29T17:05:33Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-hosted-zone.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/hosted_zone] 
time=2023-05-29T17:05:35Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/hosted_zone/terragrunt.hcl. Underlying error: invalid character &#39;c&#39; looking for beginning of value
time=2023-05-29T17:05:35Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
********** Running init for subnet/mft-subnet **********
time=2023-05-29T17:05:35Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-vpc.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc] 
time=2023-05-29T17:05:37Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc/terragrunt.hcl. Underlying error: invalid character &#39;c&#39; looking for beginning of value
time=2023-05-29T17:05:37Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
********** Running init for tgw-attach **********
time=2023-05-29T17:05:37Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-vpc.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc] 
time=2023-05-29T17:05:39Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc/terragrunt.hcl. Underlying error: invalid character &#39;c&#39; looking for beginning of value
time=2023-05-29T17:05:39Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
********** Running init for route/tgw **********
time=2023-05-29T17:05:40Z level=warning msg=No double-slash (//) found in source URL /***************/terraform-aws-vpc.git. Relative paths in downloaded Terraform code may not work. prefix=[/runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc] 
time=2023-05-29T17:05:41Z level=error msg=Could not parse output from terragrunt config /runner/_work/******-infra-terraform/******-infra-terraform/environments/preprod-******/vpc/terragrunt.hcl. Underlying error: invalid character &#39;c&#39; looking for beginning of value
time=2023-05-29T17:05:41Z level=error msg=Unable to determine underlying exit code, so Terragrunt will exit with error code 1
make: *** [Makefile:14: init] Error 1
Error: Process completed with exit code 2.
**2. Directory Structure**
.
├── Makefile
├── common.hcl
├── dns_records
│&#160;&#160; ├── config.hcl
│&#160;&#160; └── terragrunt.hcl
├── env.hcl
├── hosted_zone
│&#160;&#160; ├── config.hcl
│&#160;&#160; └── terragrunt.hcl
├── route
│&#160;&#160; └── tgw
│&#160;&#160;     ├── config.hcl
│&#160;&#160;     └── terragrunt.hcl
├── subnet
│&#160;&#160; └── mft-subnet
│&#160;&#160;     ├── config.hcl
│&#160;&#160;     └── terragrunt.hcl
├── tgw-attach
│&#160;&#160; ├── config.hcl
│&#160;&#160; └── terragrunt.hcl
└── vpc
├── config.hcl
├── override.tf
└── terragrunt.hcl
**3. Makefile**
The pipeline first runs a task for &#39;make init&#39;, then &#39;make plan&#39; and finally &#39;make apply&#39; but doesn&#39;t get past the make init.
CURRENT_DIR := $(shell pwd)
MODULES := hosted_zone vpc dns_records subnet/mft-subnet tgw-attach route/tgw
.PHONY: check
check:
@echo &quot;please specify rule&quot;
all: init plan apply
.PHONY: init
init:
@for module in $(MODULES); do \
echo &quot;********** Running init for $$module **********&quot;; \
cd $(CURRENT_DIR)/$$module &amp;&amp; terragrunt init; \
done
.PHONY: plan
plan:
@for module in $(MODULES); do \
echo &quot;********** Running plan for $$module **********&quot;; \
cd $(CURRENT_DIR)/$$module &amp;&amp; terragrunt plan; \
done
.PHONY: apply
apply:
@for module in $(MODULES); do \
echo &quot;********** Running apply for $$module **********&quot;; \
cd $(CURRENT_DIR)/$$module &amp;&amp; terragrunt apply -auto-approve; \
done
**4. terragrunt.hcl for dns_records**
include &quot;root&quot; {
path = find_in_parent_folders()
}
dependency &quot;hosted_zone&quot; {
config_path                             = &quot;../hosted_zone&quot;
mock_outputs_merge_strategy_with_state  = true
mock_outputs_allowed_terraform_commands = [&quot;validate&quot;, &quot;fmt&quot;, &quot;init&quot;, &quot;plan&quot;]
mock_outputs = {
zone_id = &quot;zone-id&quot;
}
}
locals {
# Load the data from common.hcl
common = read_terragrunt_config(&quot;../common.hcl&quot;)
env    = read_terragrunt_config(&quot;../env.hcl&quot;)
config = read_terragrunt_config(&quot;config.hcl&quot;)
}
# Set the generate config dynamically to the generate config in common.hcl
generate = local.common.generate
terraform {
source = &quot;git::git@github.com:***************/terraform-aws-route53-record.git?ref=master&quot;
}
inputs = {
zone_id = dependency.hosted_zone.outputs.zone_id
records = local.config.locals.records
tags    = local.env.locals.tags
}
**5. Workflow.yml**
name: Linting &amp; Validating
on:
pull_request:
types:
- opened
- edited
- synchronize
- reopened
branches:
- &quot;master&quot;
# Allow to trigger manually
workflow_dispatch:
env:
tf_version: &#39;latest&#39;
tg_version: &#39;latest&#39;
tf_working_dir: &#39;.&#39;
GIT_SSH_COMMAND: &quot;echo &#39;${{ secrets.GH_PRIVATE_KEY }}&#39; &gt; id_rsa
&amp;&amp; ssh-keyscan github.com &gt; known_hosts
&amp;&amp; chmod 600 id_rsa known_hosts
&amp;&amp; ssh -i ./id_rsa -o UserKnownHostsFile=./known_hosts&quot;
jobs:
check-pr:
name: check-pr
runs-on: self-hosted
strategy:
fail-fast: false
matrix:
include:
- { env: &quot;preprod&quot;, home: &quot;environments/preprod-********&quot;}
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.IE_********_TERRAFORM_USER_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.IE_********_TERRAFORM_USER_AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2
role-duration-seconds: 900
- name: Check out Git repository
uses: actions/checkout@v3
- name: Setup Terragrunt
uses: autero1/action-terragrunt@v1.3.1
with:
terragrunt_version: 0.42.5
- name: Print Terragrunt Version
run: terragrunt --version
- name: Setup node
uses: actions/setup-node@v3
with:
node-version: 16.19.1
- name: Setup Terraform
uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.4.5
- name: Print Terraform Version
run: terraform --version
- name: &#39;Terragrunt Format&#39;
run: |
cd ${{ matrix.home }}
terragrunt hclfmt --terragrunt-check
- name: &#39;Terragrunt Init&#39;
run: |
cd ${{ matrix.home }}
make init
- name: &#39;Terragrunt Plan&#39;          
run: |
cd ${{ matrix.home }}
make plan
</details>
# 答案1
**得分**: 2
以上问题的原因是 terraform_wrapper 标志引起的。因为我使用的是 GitHub Marketplace 上的 `hashicorp/setup-terraform@v2`,默认情况下 terraform_wrapper 被设置为 "true"。当我在我的 GitHub Actions workflow.yml 文件中添加了如下的 `terraform_wrapper: false` 后,问题得到解决。感谢另一个帖子中的 Brian 指出了这一点。
```yaml
steps:
- uses: hashicorp/setup-terraform@v2
with:
terraform_wrapper: false
英文:

It turns out the above issue was caused by the terraform_wrapper flag. Because i was using the hashicorp/setup-terraform@v2 from the GitHub marketplace, by default terraform_wrapper is set to "true". As soon as i added the terraform_wrapper: false per below in my GitHub Actions workflow.yml file, it resolved the issue. Thanks to Brian on another thread for pointing this out.

steps:
- uses: hashicorp/setup-terraform@v2
with:
terraform_wrapper: false

huangapple
  • 本文由 发表于 2023年5月30日 01:57:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/76359440.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定