Azure DevOps中使用GitHub存储的代码进行Terraform初始化的问题

huangapple go评论79阅读模式
英文:

Azure DevOps Terraform init issue with code in Github

问题

我使用Azure DevOps Pipelines来运行Terraform代码。Terraform代码和模块存储在连接的GitHub仓库中,供Azure DevOps Pipelines使用。如果我不使用Terraform模块,一切都正常。但是,如果我尝试使用存储在同一仓库中的模块,我在Terraform init(下载模块时)时会遇到错误:

初始化模块...
正在下载 git::https://github.com/username/terraform.git?ref=v0.0.1 到存储...
│ 错误: 下载模块失败
│ 无法从“git::https://github.com/username/terraform.git?ref=v0.0.1”下载模块“storage”(modules.tf:1)源代码:错误
│ 下载 'https://github.com/username/terraform.git?ref=v0.0.1' 时出错:
│ /usr/bin/git 退出,代码为 128:正在克隆到 '.terraform/modules/storage'...
│ 致命错误:无法读取 'https://github.com' 的用户名:终端提示已禁用

出于某种原因,连接到GitHub的服务连接似乎不足以下载模块。但是检出仓库或没有模块的tf代码可以正常工作。

我的仓库看起来像这样:

  • azure-pipeline.yml
  • environments:
    • prod
    • dev
  • modules:
    • module_1
    • module_2

对于Terraform的安装、init等操作,我使用提供的插件。

pipeline.yml的一部分示例:

stages:
  - stage: prepare
    displayName: "TF - Init and Plan"
    jobs:
      - job: prepare
        displayName: "Prepare"
        steps:
          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
            displayName: 'Install Terraform 1.3.6'
            inputs:
              terraformVersion: 1.3.7
              
          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3@3
            displayName: 'Terraform : init'
            inputs:
              workingDirectory: '$(System.DefaultWorkingDirectory)/environments/dev'
              backendServiceArm: '<subscription_id>'
              backendAzureRmResourceGroupName: <backend_rg_name>
              backendAzureRmStorageAccountName: <backend_storage>
              backendAzureRmContainerName: <backend_container>
              backendAzureRmKey: dev/terrafrom.state

          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3@3
            displayName: 'Terraform : plan'
            inputs:
              command: plan
              workingDirectory: '$(System.DefaultWorkingDirectory)/environments/dev'
              environmentServiceNameAzureRM: '<subscription_id>'
              backendServiceArm: '<subscription_id>'
              backendAzureRmResourceGroupName: <backend_rg_name>
              backendAzureRmStorageAccountName: <backend_storage>
              backendAzureRmContainerName: <backend_container>
              backendAzureRmKey: dev/terrafrom.state

我已经尝试过 Solution 1Solution 2 了,但看起来它只适用于存储在Azure Repos中的代码。

我希望存在一种方法,可以在Azure DevOps中使用GitHub中的代码和模块。

英文:

I use Azure DevOps Pipelines to run terraform code.
The terraform code and modules stored in connected GitHub repo to Azure Devops Pipelines.
If I not use TF modules - all works fine.
But If I try to use modules which stored in the same repo - I got error during Terrafrom init (when modules are downloaded):

Initializing modules...
Downloading git::https://github.com/username/terraform.git?ref=v0.0.1 for storage...
╷
│ Error: Failed to download module
│ 
│ Could not download module &quot;storage&quot; (modules.tf:1) source code from
│ &quot;git::https://github.com/username/terraform.git?ref=v0.0.1&quot;: error
│ downloading &#39;https://github.com/username/terraform.git?ref=v0.0.1&#39;:
│ /usr/bin/git exited with 128: Cloning into &#39;.terraform/modules/storage&#39;...
│ fatal: could not read Username for &#39;https://github.com&#39;: terminal prompts
│ disabled

For some reason - service connection to GitHub are not enought to download modules. But checout repo, or tf code without modules works fine.

My repo looks like this:

  • azure-pipeline.yml
  • environments:
    • prod
    • dev
  • modules:
    • module_1
    • module_2

For Terraform install, init etc I use provided plugins.

Example of part of pipeline.yml

stages:
  - stage: prepare
    displayName: &quot;TF - Init and Plan&quot;
    jobs:
      - job: prepare
        displayName: &quot;Prepare&quot;
        steps:
          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
            displayName: &#39;Install Terraform 1.3.6&#39;
            inputs:
              terraformVersion: 1.3.7
              
          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3@3
            displayName: &#39;Terraform : init&#39;
            inputs:
              workingDirectory: &#39;$(System.DefaultWorkingDirectory)/environments/dev&#39;
              backendServiceArm: &#39;&lt;subscription_id&gt;&#39;
              backendAzureRmResourceGroupName: &lt;backend_rg_name&gt;
              backendAzureRmStorageAccountName: &lt;backend_storage&gt;
              backendAzureRmContainerName: &lt;backend_container&gt;
              backendAzureRmKey: dev/terrafrom.state

          - task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV3@3
            displayName: &#39;Terraform : plan&#39;
            inputs:
              command: plan
              workingDirectory: &#39;$(System.DefaultWorkingDirectory)/environments/dev&#39;
              environmentServiceNameAzureRM: &#39;&lt;subscription_id&gt;&#39;
              backendServiceArm: &#39;&lt;subscription_id&gt;&#39;
              backendAzureRmResourceGroupName: &lt;backend_rg_name&gt;
              backendAzureRmStorageAccountName: &lt;backend_storage&gt;
              backendAzureRmContainerName: &lt;backend_container&gt;
              backendAzureRmKey: dev/terrafrom.state

I already tried - Solution 1
and Solution 2

But looks like it works only for code which stored in Azure Repos.

I hope exist a way how I can use Azure DevOps with code and modules in GitHub.

答案1

得分: 1

由于这是一个私有GitHub存储库,根据错误消息,似乎缺少对GitHub存储库的授权。

根据Hashicorp模块源:GitHub文档,如果使用HTTP/HTTPS协议或任何其他使用用户名/密码凭据的协议,请配置Git凭据存储以选择适合您环境的凭据来源。

另外,您可以通过SSH密钥进行GitHub身份验证。最好使用部署密钥(Deploy Keys),这是一种SSH密钥,授予对单个存储库的访问权限。

步骤:

          # 为私有存储库模块安装SSH密钥{对所有具有有效部署密钥的私有Github存储库有效的配置}]
          - task: InstallSSHKey@0
            displayName: "为<repo_name>存储库安装SSH密钥"
            inputs:
              knownHostsEntry: $(knownHostsEntry)
              sshPublicKey: $(sshPublicKey)
              sshKeySecureFile: $(name_of_secure_file_in_library_group) # 存储了私有SSH密钥的位置
# 在变量组中
knownHostsEntry = github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==

sshPublicKey = "在`生成新的SSH密钥`步骤中生成的SSH公钥"
  • 修改您的terraform源URL,从https更改为git
module "module_name" {
  source = "github.com/username/terraform//modules/storage?ref=v0.0.4"
  ## 将此更改为 ##
  source = "git@github.com:username/terraform.git//modules/storage?ref=v0.5.1"
}

仍然有疑虑: 请使用此带有屏幕截图的逐步教程。

https://www.codewithadam.com/using-terraform-modules-from-github-in-azure-devops/

英文:

As this is a private GitHub repository by the error message it seems that the authorization to the GitHub repo is missing.

As per the Hashicorp Modules Sources: GitHub documentation, If using the HTTP/HTTPS protocol, or any other protocol that uses username/password credentials, configure Git Credentials Storage to select a suitable source of credentials for your environment.

Also, you can authenticate via SSH keys with GitHub. The best would be to use Deploy Keys, which is an SSH key that grants access to a single repository.

Steps:

          # Install SSH Key for private repo modules {config valid for all private Github Repos with a valid deploy key} ]
          - task: InstallSSHKey@0
            displayName: &quot;Install SSH key for &lt;repo_name&gt; Repo&quot;
            inputs:
              knownHostsEntry: $(knownHostsEntry)
              sshPublicKey: $(sshPublicKey)
              sshKeySecureFile: $(name_of_secure_file_in_library_group) # where private SSH key was upladed
# in Variable Group 
knownHostsEntry = github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==

sshPublicKey = &quot;SSH Public Key generated in the `Generating a new SSH key` step&quot;
  • Modify your terraform source URLs from https to git
module &quot;module_name&quot; {
  source = &quot;github.com/username/terraform//modules/storage?ref=v0.0.4&quot;
  ## CHANGE THIS TO ##
  source = &quot;git@github.com:username/terraform.git//modules/storage?ref=v0.5.1&quot;
}

Still having doubts: Please use this step-by-step tutorial with screenshots.

https://www.codewithadam.com/using-terraform-modules-from-github-in-azure-devops/

huangapple
  • 本文由 发表于 2023年1月9日 04:47:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/75051163.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定