如何使保险库(vault)的秘密ID可以多次重复使用?

huangapple go评论109阅读模式
英文:

How to make vault Secret ID can be reused multiple times?

问题

所以我有一个带有类似以下内容的Dockerfile的 PoC Vault(完整的存储库在这里):

  1. FROM hashicorp/vault
  2. RUN apk add --no-cache bash jq
  3. COPY reseller1-policy.hcl /vault/config/reseller1-policy.hcl
  4. COPY terraform-policy.hcl /vault/config/terraform-policy.hcl
  5. COPY init_vault.sh /init_vault.sh
  6. EXPOSE 8200
  7. ENTRYPOINT ["/init_vault.sh"]
  8. HEALTHCHECK \
  9. --start-period=5s \
  10. --interval=1s \
  11. --timeout=1s \
  12. --retries=30 \
  13. CMD ["/bin/sh", "-c", "[ -f /tmp/healthy ]"]

init_vault.sh 包含:

  1. #!/bin/sh
  2. set -e
  3. export VAULT_ADDR='http://127.0.0.1:8200'
  4. export VAULT_FORMAT='json'
  5. # 为开发 Vault 服务器生成一个新进程,并等待其上线
  6. # 参考:https://www.vaultproject.io/docs/concepts/dev-server
  7. vault server -dev -dev-listen-address="0.0.0.0:8200" &
  8. sleep 5s
  9. # 验证容器的本地 Vault CLI
  10. # 参考:https://www.vaultproject.io/docs/commands/login
  11. vault login -no-print "${VAULT_DEV_ROOT_TOKEN_ID}"
  12. # 添加策略
  13. # 参考:https://www.vaultproject.io/docs/concepts/policies
  14. vault policy write terraform-policy /vault/config/terraform-policy.hcl
  15. vault policy write reseller1-policy /vault/config/reseller1-policy.hcl
  16. # 启用 AppRole 认证方法
  17. # 参考:https://www.vaultproject.io/docs/auth/approle
  18. vault auth enable approle
  19. # 配置 AppRole
  20. # 参考:https://www.vaultproject.io/api/auth/approle#parameters
  21. vault write auth/approle/role/dummy_role \
  22. token_policies=reseller1-policy \
  23. token_num_uses=9000 \
  24. secret_id_ttl="32d" \
  25. token_ttl="32d" \
  26. token_max_ttl="32d"
  27. # 覆盖我们的角色 ID
  28. vault write auth/approle/role/dummy_role/role-id role_id="${APPROLE_ROLE_ID}"
  29. # 为 Terraform
  30. # 参考:https://www.vaultproject.io/docs/commands/token/create
  31. vault token create \
  32. -id="${TERRAFORM_TOKEN}" \
  33. -policy=terraform-policy \
  34. -ttl="32d"
  35. # 保持容器运行
  36. tail -f /dev/null & trap 'kill %1' TERM ; wait

使用 reseller1-policy.hcl

  1. # 此部分授予应用程序访问权限
  2. path "secret/data/dummy_config_yaml/reseller1/*" {
  3. capabilities = ["read"]
  4. }
  5. path "secret/dummy_config_yaml/reseller1/*" { # v1
  6. capabilities = ["read"]
  7. }

terraform-policy.hcl

  1. # 授予在“auth/approle/role/<role_name>/secret-id”路径上生成密钥 ID 的“update”权限
  2. path "auth/approle/role/dummy_role/secret-id" {
  3. capabilities = ["update"]
  4. }
  5. path "secret/data/dummy_config_yaml/*" {
  6. capabilities = ["create","update","read","patch","delete"]
  7. }
  8. path "secret/dummy_config_yaml/*" { # v1
  9. capabilities = ["create","update","read","patch","delete"]
  10. }
  11. path "secret/metadata/dummy_config_yaml/*" {
  12. capabilities = ["list"]
  13. }

这是使用 docker-compose.yml 启动的:

  1. version: '3.3'
  2. services:
  3. testvaultserver1:
  4. build: ./vault-server/
  5. cap_add:
  6. - IPC_LOCK
  7. environment:
  8. VAULT_DEV_ROOT_TOKEN_ID: root
  9. APPROLE_ROLE_ID: dummy_app
  10. TERRAFORM_TOKEN: dummyTerraformToken
  11. ports:
  12. - "8200:8200"

然后在 shell 上运行一些脚本 copy_config2vault_secret2tmp.sh

  1. TERRAFORM_TOKEN=`cat docker-compose.yml | grep TERRAFORM_TOKEN | cut -d':' -f2 | xargs echo -n`
  2. VAULT_ADDRESS="127.0.0.1:8200"
  3. # 检索 appsecret 的密钥,以便虚拟应用程序可以加载 /tmp/secret
  4. curl \
  5. --request POST \
  6. --header "X-Vault-Token: ${TERRAFORM_TOKEN}" \
  7. --header "X-Vault-Wrap-TTL: 32d" \
  8. "${VAULT_ADDRESS}/v1/auth/approle/role/dummy_role/secret-id" > /tmp/debug
  9. cat /tmp/debug | jq -r '.wrap_info.token' > /tmp/secret
  10. # 检查 appsecret 是否存在
  11. cat /tmp/debug
  12. cat /tmp/secret
  13. VAULT_DOCKER=`docker ps| grep vault | cut -d' ' -f 1`
  14. echo 'put secret'
  15. cat config.yaml | docker exec -i $VAULT_DOCKER vault -v kv put -address=http://127.0.0.1:8200 -mount=secret dummy_config_yaml/reseller1/region99 raw=-
  16. echo 'check secret length'
  17. docker exec -i $VAULT_DOCKER vault -v kv get -address=http://127.0.0.1:8200 -mount=secret dummy_config_yaml/reseller1/region99 | wc -l

然后创建一个程序来读取密钥并从 Vault 中检索 config.yaml

  1. package main
  2. import (
  3. "context"
  4. "fmt"
  5. "log"
  6. "time"
  7. vault "github.com/hashicorp/vault/api"
  8. "github.com/hashicorp/vault/api/auth/approle"
  9. )
  10. const AppRoleID = `dummy_app`
  11. func main() {
  12. conf, err := TryUseVault(`http://127.0.0.1:8200`, `secret/data/dummy_config_yaml/reseller1/region99`)
  13. if err != nil {
  14. log.Println(err)
  15. return
  16. }
  17. log.Println(conf)
  18. }
  19. func TryUseVault(address, configPath string) (string, error) {
  20. ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
  21. defer cancel()
  22. const secretFile = `/tmp/secret`
  23. config := vault.DefaultConfig() // modify for more granular configuration
  24. config.Address = address
  25. client, err := vault.NewClient(config)
  26. if err != nil {
  27. return ``, fmt.Errorf(`failed to create vault client: %w`, err)
  28. }
  29. approleSecretID := &approle.SecretID{
  30. FromFile: secretFile,
  31. }
  32. appRoleAuth, err := approle.NewAppRoleAuth(
  33. AppRoleID,
  34. approleSecretID,
  35. approle.WithWrappingToken(), // only required if the SecretID is response-wrapped
  36. )
  37. if err != nil {
  38. return ``, fmt.Errorf(`failed to create approle auth: %w`, err)
  39. }
  40. authInfo, err := client.Auth().Login(ctx, appRoleAuth)
  41. if err != nil {
  42. return ``, fmt.Errorf(`failed to login to vault: %w`, err)
  43. }
  44. if authInfo == nil {
  45. return ``, fmt.Errorf(`failed to login to vault: authInfo is nil`)
  46. }
  47. log.Println("connecting to vault: success!")
  48. secret, err := client.Logical().Read(configPath)
  49. if err != nil {
  50. return ``, fmt.Errorf(`failed to read secret from vault: %w`, err)
  51. }
  52. if secret == nil {
  53. return ``, fmt.Errorf(`failed to read secret from vault: secret is nil`)
  54. }
  55. if len(secret.Data) == 0 {
  56. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data is empty`)
  57. }
  58. data := secret.Data[`data`]
  59. if data == nil {
  60. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data is nil`)
  61. }
  62. m, ok := data.(map[string]interface{})
  63. if !ok {
  64. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data is not a map[string]interface{}`)
  65. }
  66. raw, ok := m[`raw`]
  67. if !ok {
  68. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data.raw is nil`)
  69. }
  70. rawStr, ok := raw.(string)
  71. if !ok {
  72. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data.raw is not a string`)
  73. }
  74. // set viper from string
  75. return rawStr, nil
  76. }

它工作得很好,但问题是,密钥只能使用一次

  1. $ ./copy_config2vault_secret2tmp.sh
  2. {"request_id":"","lease_id":"","renewable":false,"lease_duration":0,"data":null,"wrap_info":{"token":"hvs.CAESIDSE_hR3-CW1CLLotQoVAhes55vI1MCDemmbWbsAvDS6Gh4KHGh2cy5QdzQ0bzlxRTJ6MUZJUFRoeGpSWFRzV0E","accessor":"7jLABMbzGVHKPCKAd7qkPx5J","ttl":2764800,"creation_time":"2023-07-18T19:34:48.619332723Z","creation_path":"auth/approle/role/dummy_role/secret-id","wrapped_accessor":"2493fc83-aaf6-7553-dd04-2ccedc39a4b1"},"warnings":null,"auth":null}
  3. hvs.CAESIDSE_hR3-CW1CLLotQoVAhes55vI1MCDemmbWbsAvDS6Gh4KHGh2cy5QdzQ0bzlxRTJ6MUZJUFRoeGpSWFRzV0E
  4. put secret
  5. ================== Secret Path ==================
  6. secret/data/dummy_config_yaml/reseller1/region99
  7. ======= Metadata =======
  8. Key Value
  9. --- -----
  10. created_time 2023-07-18T19:34:48.827508755Z
  11. custom_metadata <nil>
  12. deletion_time n/a
  13. destroyed false
  14. version 9
  15. check secret length
  16. 19

检索一次工作正常:

  1. $ go run main.go
  2. 2023/07/19 02:35:52 connecting to vault: success!
  3. 2023/07/19 02:35:52
  4. this:
  5. is:
  6. some:
  7. secret: a35)*&BN)(*&%TN_@#

但是当我第二次运行它时,它总是报错(除非我再次运行获取密钥的 copy_config2vault_secret2tmp.sh 脚本):

  1. $ go run main.go
  2. 2023/07/19 02:36:06 failed to login to vault: unable to log in to auth method: unable to unwrap response wrapping token: Error making API request.
  3. URL: PUT http://127.0.0.1:8200/v1/sys/wrapping/unwrap
  4. Code: 400. Errors:
  5. * wrapping token is not valid or does not exist

密钥 ID 是否只能使用一次?如果不是,可能的原因是什么?

英文:

So I have a PoC Vault with Dockerfile something like this (full repo here):

  1. FROM hashicorp/vault
  2. RUN apk add --no-cache bash jq
  3. COPY reseller1-policy.hcl /vault/config/reseller1-policy.hcl
  4. COPY terraform-policy.hcl /vault/config/terraform-policy.hcl
  5. COPY init_vault.sh /init_vault.sh
  6. EXPOSE 8200
  7. ENTRYPOINT [ &quot;/init_vault.sh&quot; ]
  8. HEALTHCHECK \
  9. --start-period=5s \
  10. --interval=1s \
  11. --timeout=1s \
  12. --retries=30 \
  13. CMD [ &quot;/bin/sh&quot;, &quot;-c&quot;, &quot;[ -f /tmp/healthy ]&quot; ]

init_vault.sh contains:

  1. #!/bin/sh
  2. set -e
  3. export VAULT_ADDR=&#39;http://127.0.0.1:8200&#39;
  4. export VAULT_FORMAT=&#39;json&#39;
  5. # Spawn a new process for the development Vault server and wait for it to come online
  6. # ref: https://www.vaultproject.io/docs/concepts/dev-server
  7. vault server -dev -dev-listen-address=&quot;0.0.0.0:8200&quot; &amp;
  8. sleep 5s
  9. # authenticate container&#39;s local Vault CLI
  10. # ref: https://www.vaultproject.io/docs/commands/login
  11. vault login -no-print &quot;${VAULT_DEV_ROOT_TOKEN_ID}&quot;
  12. # add policy
  13. # ref: https://www.vaultproject.io/docs/concepts/policies
  14. vault policy write terraform-policy /vault/config/terraform-policy.hcl
  15. vault policy write reseller1-policy /vault/config/reseller1-policy.hcl
  16. # enable AppRole auth method
  17. # ref: https://www.vaultproject.io/docs/auth/approle
  18. vault auth enable approle
  19. # configure AppRole
  20. # ref: https://www.vaultproject.io/api/auth/approle#parameters
  21. vault write auth/approle/role/dummy_role \
  22. token_policies=reseller1-policy \
  23. token_num_uses=9000 \
  24. secret_id_ttl=&quot;32d&quot; \
  25. token_ttl=&quot;32d&quot; \
  26. token_max_ttl=&quot;32d&quot;
  27. # overwrite our role id
  28. vault write auth/approle/role/dummy_role/role-id role_id=&quot;${APPROLE_ROLE_ID}&quot;
  29. # for terraform
  30. # ref: https://www.vaultproject.io/docs/commands/token/create
  31. vault token create \
  32. -id=&quot;${TERRAFORM_TOKEN}&quot; \
  33. -policy=terraform-policy \
  34. -ttl=&quot;32d&quot;
  35. # keep container alive
  36. tail -f /dev/null &amp; trap &#39;kill %1&#39; TERM ; wait

with reseller1-policy.hcl:

  1. # This section grants access for the app
  2. path &quot;secret/data/dummy_config_yaml/reseller1/*&quot; {
  3. capabilities = [&quot;read&quot;]
  4. }
  5. path &quot;secret/dummy_config_yaml/reseller1/*&quot; { # v1
  6. capabilities = [&quot;read&quot;]
  7. }

and terraform-policy.hcl:

  1. # Grant &#39;update&#39; permission on the &#39;auth/approle/role/&lt;role_name&gt;/secret-id&#39; path for generating a secret id
  2. path &quot;auth/approle/role/dummy_role/secret-id&quot; {
  3. capabilities = [&quot;update&quot;]
  4. }
  5. path &quot;secret/data/dummy_config_yaml/*&quot; {
  6. capabilities = [&quot;create&quot;,&quot;update&quot;,&quot;read&quot;,&quot;patch&quot;,&quot;delete&quot;]
  7. }
  8. path &quot;secret/dummy_config_yaml/*&quot; { # v1
  9. capabilities = [&quot;create&quot;,&quot;update&quot;,&quot;read&quot;,&quot;patch&quot;,&quot;delete&quot;]
  10. }
  11. path &quot;secret/metadata/dummy_config_yaml/*&quot; {
  12. capabilities = [&quot;list&quot;]
  13. }

This was started with docker-compose.yml:

  1. version: &#39;3.3&#39;
  2. services:
  3. testvaultserver1:
  4. build: ./vault-server/
  5. cap_add:
  6. - IPC_LOCK
  7. environment:
  8. VAULT_DEV_ROOT_TOKEN_ID: root
  9. APPROLE_ROLE_ID: dummy_app
  10. TERRAFORM_TOKEN: dummyTerraformToken
  11. ports:
  12. - &quot;8200:8200&quot;

then run some script copy_config2vault_secret2tmp.sh on shell:

  1. TERRAFORM_TOKEN=`cat docker-compose.yml | grep TERRAFORM_TOKEN | cut -d&#39;:&#39; -f2 | xargs echo -n`
  2. VAULT_ADDRESS=&quot;127.0.0.1:8200&quot;
  3. # retrieve secret for appsecret so dummy app can load the /tmp/secret
  4. curl \
  5. --request POST \
  6. --header &quot;X-Vault-Token: ${TERRAFORM_TOKEN}&quot; \
  7. --header &quot;X-Vault-Wrap-TTL: 32d&quot; \
  8. &quot;${VAULT_ADDRESS}/v1/auth/approle/role/dummy_role/secret-id&quot; &gt; /tmp/debug
  9. cat /tmp/debug | jq -r &#39;.wrap_info.token&#39; &gt; /tmp/secret
  10. # check appsecret exists
  11. cat /tmp/debug
  12. cat /tmp/secret
  13. VAULT_DOCKER=`docker ps| grep vault | cut -d&#39; &#39; -f 1`
  14. echo &#39;put secret&#39;
  15. cat config.yaml | docker exec -i $VAULT_DOCKER vault -v kv put -address=http://127.0.0.1:8200 -mount=secret dummy_config_yaml/reseller1/region99 raw=-
  16. echo &#39;check secret length&#39;
  17. docker exec -i $VAULT_DOCKER vault -v kv get -address=http://127.0.0.1:8200 -mount=secret dummy_config_yaml/reseller1/region99 | wc -l

Then create a program to read the secret and retrieve the config.yaml from vault:

  1. package main
  2. import (
  3. &quot;context&quot;
  4. &quot;fmt&quot;
  5. &quot;log&quot;
  6. &quot;time&quot;
  7. vault &quot;github.com/hashicorp/vault/api&quot;
  8. &quot;github.com/hashicorp/vault/api/auth/approle&quot;
  9. )
  10. const AppRoleID = `dummy_app`
  11. func main() {
  12. conf, err := TryUseVault(`http://127.0.0.1:8200`, `secret/data/dummy_config_yaml/reseller1/region99`)
  13. if err != nil {
  14. log.Println(err)
  15. return
  16. }
  17. log.Println(conf)
  18. }
  19. func TryUseVault(address, configPath string) (string, error) {
  20. ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
  21. defer cancel()
  22. const secretFile = `/tmp/secret`
  23. config := vault.DefaultConfig() // modify for more granular configuration
  24. config.Address = address
  25. client, err := vault.NewClient(config)
  26. if err != nil {
  27. return ``, fmt.Errorf(`failed to create vault client: %w`, err)
  28. }
  29. approleSecretID := &amp;approle.SecretID{
  30. FromFile: secretFile,
  31. }
  32. appRoleAuth, err := approle.NewAppRoleAuth(
  33. AppRoleID,
  34. approleSecretID,
  35. approle.WithWrappingToken(), // only required if the SecretID is response-wrapped
  36. )
  37. if err != nil {
  38. return ``, fmt.Errorf(`failed to create approle auth: %w`, err)
  39. }
  40. authInfo, err := client.Auth().Login(ctx, appRoleAuth)
  41. if err != nil {
  42. return ``, fmt.Errorf(`failed to login to vault: %w`, err)
  43. }
  44. if authInfo == nil {
  45. return ``, fmt.Errorf(`failed to login to vault: authInfo is nil`)
  46. }
  47. log.Println(&quot;connecting to vault: success!&quot;)
  48. secret, err := client.Logical().Read(configPath)
  49. if err != nil {
  50. return ``, fmt.Errorf(`failed to read secret from vault: %w`, err)
  51. }
  52. if secret == nil {
  53. return ``, fmt.Errorf(`failed to read secret from vault: secret is nil`)
  54. }
  55. if len(secret.Data) == 0 {
  56. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data is empty`)
  57. }
  58. data := secret.Data[`data`]
  59. if data == nil {
  60. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data is nil`)
  61. }
  62. m, ok := data.(map[string]interface{})
  63. if !ok {
  64. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data is not a map[string]interface{}`)
  65. }
  66. raw, ok := m[`raw`]
  67. if !ok {
  68. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data.raw is nil`)
  69. }
  70. rawStr, ok := raw.(string)
  71. if !ok {
  72. return ``, fmt.Errorf(`failed to read secret from vault: secret.Data.data.raw is not a string`)
  73. }
  74. // set viper from string
  75. return rawStr, nil
  76. }

it works fine, but the problem is, the secret can only be used once

  1. $ ./copy_config2vault_secret2tmp.sh
  2. {&quot;request_id&quot;:&quot;&quot;,&quot;lease_id&quot;:&quot;&quot;,&quot;renewable&quot;:false,&quot;lease_duration&quot;:0,&quot;data&quot;:null,&quot;wrap_info&quot;:{&quot;token&quot;:&quot;hvs.CAESIDSE_hR3-CW1CLLotQoVAhes55vI1MCDemmbWbsAvDS6Gh4KHGh2cy5QdzQ0bzlxRTJ6MUZJUFRoeGpSWFRzV0E&quot;,&quot;accessor&quot;:&quot;7jLABMbzGVHKPCKAd7qkPx5J&quot;,&quot;ttl&quot;:2764800,&quot;creation_time&quot;:&quot;2023-07-18T19:34:48.619332723Z&quot;,&quot;creation_path&quot;:&quot;auth/approle/role/dummy_role/secret-id&quot;,&quot;wrapped_accessor&quot;:&quot;2493fc83-aaf6-7553-dd04-2ccedc39a4b1&quot;},&quot;warnings&quot;:null,&quot;auth&quot;:null}
  3. hvs.CAESIDSE_hR3-CW1CLLotQoVAhes55vI1MCDemmbWbsAvDS6Gh4KHGh2cy5QdzQ0bzlxRTJ6MUZJUFRoeGpSWFRzV0E
  4. put secret
  5. ================== Secret Path ==================
  6. secret/data/dummy_config_yaml/reseller1/region99
  7. ======= Metadata =======
  8. Key Value
  9. --- -----
  10. created_time 2023-07-18T19:34:48.827508755Z
  11. custom_metadata &lt;nil&gt;
  12. deletion_time n/a
  13. destroyed false
  14. version 9
  15. check secret length
  16. 19

retrieve it once works fine:

  1. $ go run main.go
  2. 2023/07/19 02:35:52 connecting to vault: success!
  3. 2023/07/19 02:35:52
  4. this:
  5. is:
  6. some:
  7. secret: a35)*&amp;BN)(*&amp;%TN_@#

But when I run it second time, it always error (unless I run the get secret copy_config2vault_secret2tmp.sh script again):

  1. $ go run main.go
  2. 2023/07/19 02:36:06 failed to login to vault: unable to log in to auth method: unable to unwrap response wrapping token: Error making API request.
  3. URL: PUT http://127.0.0.1:8200/v1/sys/wrapping/unwrap
  4. Code: 400. Errors:
  5. * wrapping token is not valid or does not exist

Is the secret ID can only be used only once by design? or if it's not, what's the possible cause of this?

答案1

得分: 2

包装令牌仅限单次使用。这就是为什么只有在运行Go应用程序之前执行copy_config2vault_secret2tmp.sh时才起作用。

根据Vault文档的参考:

当新创建的令牌被包装时,Vault将生成的令牌插入到单次使用令牌的小隔间中,并返回该单次使用的包装令牌。检索密钥需要对该包装令牌进行解包操作。

这个特定部分解释了包装令牌的使用,可能有助于理解细节:
https://developer.hashicorp.com/vault/tutorials/secrets-management/cubbyhole-response-wrapping#step-2-unwrap-the-secret

英文:

Wrapping tokens are limited to single use only. That's why it works only when You execute copy_config2vault_secret2tmp.sh before running Go app.

As a reference from Vault's documentation:

> When a newly created token is wrapped, Vault inserts the generated
> token into the cubbyhole of a single-use token, returning that
> single-use wrapping token. Retrieving the secret requires an unwrap
> operation against this wrapping token.

This specific part explaining use of wrapping token might help to understand the details:
https://developer.hashicorp.com/vault/tutorials/secrets-management/cubbyhole-response-wrapping#step-2-unwrap-the-secret

huangapple
  • 本文由 发表于 2023年7月19日 03:41:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/76716052.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定