英文:
How can I run a single Terraform GitHub Actions pipeline that applies multiple terraform resources (each with own state files) at same time?
问题
我在GitHub Actions中有一个流水线,当用户提出PR时,会运行Terraform Plan,一旦PR获批准,就会运行Apply。
流水线配置了一个Terraform工作目录,并仅为特定目录运行Terraform init,例如user-1/projectA硬编码到我的流水线中。
然而,同一仓库还包含其他文件夹,例如带有相同结构的projectB。还有其他用户文件夹,例如user-2;
user-project1
└ projectA
└ main.tf
└ backend.tf
└ variables.tfvars
└ projectB
└ main.tf
└ backend.tf
└ variables.tfvars
user-project2
└ projectA
└ main.tf
└ backend.tf
└ variables.tfvars
└ projectB
└ main.tf
└ backend.tf
└ variables.tfvars
我想要一个单一的流水线,可以在这个包含多个项目的monorepo中的任何更改上运行,例如,如果在上述任何文件中进行了任何更改。
背景是,我有一个包含多个GCP项目的monorepo。只有我可以访问此仓库并进行更改。当我收到一群用户的请求,需要在多个项目中进行一些更改(例如启用新的API)时,我希望进行所有这些更改,然后运行单一的流水线来将它们全部应用。
每个项目在远程后端中都有自己的状态文件。
我仅使用Terraform CLI,没有使用TF Cloud或Enterprise的选项。
英文:
I have a pipeline in GitHub Actions that runs a terraform Plan when a user raises a PR, and an Apply once that PR is approved.
The pipeline is configured to set a Terraform working directory and run Terraform init for a specific directory only e.g user-1/projectA is hard coded into my pipeline
However the same repo also contains additional folders, e.g projectB with the same structure as above. And also additional user folders, e.g user-2;
user-project1
└ projectA
└ main.tf
└ backend.tf
└ variables.tfvars
└ projectB
└ main.tf
└ backend.tf
└ variables.tfvars
user-project2
└ projectA
└ main.tf
└ backend.tf
└ variables.tfvars
└ projectB
└ main.tf
└ backend.tf
└ variables.tfvars
I'd like a single pipeline that will run for any change in this monorepo that contains multiple projects, e.g if any change is made in any of the files above.
The context is, I have a monorepo that contains multiple GCP projects. Only I have access to this repo and to make changes. When I get a request from a a bunch of users to make some changes across a number of projects (enabling a new API for example), I'd like to make all those changes and then run a single pipeline to get them all applied.
Each project has its own state file in a remote backend
I am using Terraform CLI only with no option to use TF Cloud or Enterprise
答案1
得分: 0
你可以使用可重复使用的工作流程,以及矩阵策略。在这种情况下,你将会有2个工作流程:
- 一个名为
main
的工作流程,包含2个作业:- 一个用于识别已更新的文件夹
- 一个用于调用可重复使用的工作流程以针对已更新的文件夹执行
terraform apply
命令。
- 一个名为
reusable workflow
的工作流程,它将接收一个folder
作为输入,然后在执行terraform apply
命令之前,使用cd
进入此文件夹。
matrix
将被填充为 main
工作流程上已更新的文件夹列表,并为每个已更新的文件夹调用可重复使用的工作流程。
以下是一个示例:
main workflow
on:
push:
paths:
- 'folder1/**'
- 'folder2/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
setup:
runs-on: ubuntu-latest
outputs:
folders: ${{ steps.array.outputs.array }}
steps:
- uses: actions/checkout@v3
- uses: dorny/paths-filter@v2
id: changes
with:
filters: |
folder1:
- 'folder1/**'
folder2:
- 'folder2/**'
- name: Build Array
id: array
run: |
myArray=()
if [ "${{ steps.changes.outputs.folder1 }}" = "true" ]
then
myArray+=("folder1")
fi
if [ "${{ steps.changes.outputs.folder2 }}" = "true" ]
then
myArray+=("folder2")
fi
myArray=$(jq --compact-output --null-input '$ARGS.positional' --args -- "${myArray[@]}")
echo "Updated folder list: $myArray"
echo "array=$myArray" >> $GITHUB_OUTPUT
shell: bash
build:
needs: [setup]
strategy:
fail-fast: false
matrix:
folder: ${{ fromJSON(needs.setup.outputs.folders) }}
uses: ./.github/workflows/reusable.yml
with:
path: ${{ matrix.folder }}
注意:你可以使用脚本来替代执行所有的 shell 命令,但我发现这种方式更容易,因为 dorny/paths-filter action
为每个文件夹生成一个输出。
reusable workflow
on:
workflow_call:
inputs:
path:
required: true
type: string
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ inputs.path }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- run: |
cd ${{ inputs.path }}
terraform apply
请确保按照示例中的格式和结构设置你的工作流程。
英文:
You could use a reusable workflow along with a matrix strategy.
In that case, you would have 2 workflows:
- a
main
workflow with 2 jobs- one that identify the updated folders
- one that call the reusable workflow for the updated folders.
- a
reusable workflow
that would receive afolder
as input, and thencd
to this folder before performing aterraform apply
command.
The matrix
would be filled with the list of updated folders on the main
workflow, calling the reusable workflow for each folder updated.
Here is an example:
main workflow
on:
push:
paths:
- 'folder1/**'
- 'folder2/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
setup:
runs-on: ubuntu-latest
outputs:
folders: ${{ steps.array.outputs.array }}
steps:
- uses: actions/checkout@v3
- uses: dorny/paths-filter@v2
id: changes
with:
filters: |
folder1:
- 'folder1/**'
folder2:
- 'folder2/**'
- name: Build Array
id: array
run: |
myArray=()
if [ "${{ steps.changes.outputs.folder1 }}" = "true" ]
then
myArray+=("folder1")
fi
if [ "${{ steps.changes.outputs.folder2 }}" = "true" ]
then
myArray+=("folder2")
fi
myArray=$(jq --compact-output --null-input '$ARGS.positional' --args -- "${myArray[@]}")
echo "Updated folder list: $myArray"
echo "array=$myArray" >> $GITHUB_OUTPUT
shell: bash
build:
needs: [setup]
strategy:
fail-fast: false
matrix:
folder: ${{ fromJSON(needs.setup.outputs.folders) }}
uses: ./.github/workflows/reusable.yml
with:
path: ${{ matrix.folder }}
Note: You may use a script instead of doing all the shell commands, but I found it easier that way as the dorny/paths-filter action
generates an output for each folder.
reusable workflow
on:
workflow_call:
inputs:
path:
required: true
type: string
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ inputs.path }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- run: |
cd ${{ inputs.path }}
terraform apply
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论