Azure Synapse中的Azure函数链接服务(Terraform中)

huangapple go评论126阅读模式
英文:

Azure synapse linked service for Azure Function in Terraform

问题

我正在编写一个Terraform脚本来创建Azure Synapse工作区。我已经创建了一个Azure函数的链接服务,但在管道中无法使用它,出现了缺少函数密钥的错误。

这是我目前正在使用的内容。我确信问题出在type_properties_json参数中。

resource "azurerm_synapse_linked_service" "FunctionName" {
  name                 = "FunctionName"
  synapse_workspace_id = azurerm_synapse_workspace.synapse.id
  type                 = "AzureFunction"
  type_properties_json = <<JSON
    {
      "functionAppUrl": "https://${data.azurerm_function_app.FunctionName.default_hostname}",
      "authentication": "Anonymous",
      "functionKey": "${data.azurerm_function_app_host_keys.FunctionName.default_function_key}"
    }
  JSON
  depends_on = [
    azurerm_synapse_firewall_rule.allowAll,
    data.azurerm_function_app.FunctionName,
    data.azurerm_function_app_host_keys.FunctionName
  ]
}

这确实会创建一个链接服务,但当我在管道中使用它时,运行会失败并显示以下错误:

Azure函数活动缺少函数密钥。

在检查azurerm_function_app的输出后,我发现没有导出connectionString。

英文:

I'm writing a terraform script to create a azure synapse workspace.
I've created a linked service for Azure Function but I'm unable to use it in pipeline, where it gives me an error of missing function key.

This is what i'm using now. I'm sure the problem is in the type_properties_json parameter.

resource &quot;azurerm_synapse_linked_service&quot; &quot;FunctionName&quot; {
  name                 = &quot;FunctionName&quot;
  synapse_workspace_id = azurerm_synapse_workspace.synapse.id
  type                 = &quot;AzureFunction&quot;
  type_properties_json = &lt;&lt;JSON
    {
      &quot;functionAppUrl&quot;: &quot;https://${data.azurerm_function_app.FunctionName.default_hostname}&quot;,
      &quot;authentication&quot;: &quot;Anonymous&quot;,
      &quot;functionKey&quot;: &quot;${data.azurerm_function_app_host_keys.FunctionName.default_function_key}&quot;
    }
  JSON
  depends_on = [
    azurerm_synapse_firewall_rule.allowAll,
    data.azurerm_function_app.FunctionName,
    data.azurerm_function_app_host_keys.FunctionName
  ]
}

And this does create a linked service but when i use it in a pipeline, the run fails with the error

Azure function activity missing function key.

It appears to me after checking the output for azurerm_function_app there is no export for connectionString.

答案1

得分: 0

我尝试在我的环境中复现了这种情况。

尝试以下代码:

resource "azurerm_synapse_linked_service" "example" {
  name                 = "kavya-fnapplinked"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  type                 = "AzureFunction"
  type_properties_json = <<JSON
    {
      "functionAppUrl": "https://${data.azurerm_function_app.example.default_hostname}",
      "authentication": "Anonymous",
      "functionKey": "${data.azurerm_function_app_host_keys.example.default_function_key}"
    }
JSON
  depends_on = [
    azurerm_synapse_firewall_rule.allowAll,
    data.azurerm_function_app.example,
    data.azurerm_function_app_host_keys.example
  ]
}

由于 JSON 属性和函数密钥未按预期生成,因为它们不符合正确的格式,导致了错误。

对于这一点,注意两个重要的要点:

  1. 函数密钥只有在首次创建函数应用后才会生成,并且是敏感值。
  2. 敏感值的 JSON 格式必须如下所示:
"secret":
{
  "type": "SecureString",
  "value": "{value}"
}
  • 首先,我将函数密钥存储在密钥保管库中,以便它需要时间来生成并存储在保密中。

代码:

resource "azurerm_role_assignment" "role_assignment" {
  scope                = azurerm_storage_account.stfn.id
  role_definition_name = "Storage Blob Data Owner"
  principal_id         = data.azurerm_client_config.current.object_id
}

# 使用 Sleep 等待角色分配需要时间来传播
resource "time_sleep" "role_assignment_sleep" {
  create_duration = "60s"

  triggers = {
    role_assignment = azurerm_role_assignment.role_assignment.id
  }
}

resource "azurerm_storage_data_lake_gen2_filesystem" "example" {
  name               = "kavdatalakexample123"
  storage_account_id = azurerm_storage_account.stfn.id
  depends_on         = [time_sleep.role_assignment_sleep]
}

resource "azurerm_synapse_workspace" "example" {
  name                = "exmple-workspace"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_storage_account.example.location
  storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.example.id
  sql_administrator_login              = "sqladminuser"
  sql_administrator_login_password     = "H@Sh1CoR3!"
  managed_virtual_network_enabled      = true
  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_synapse_firewall_rule" "allowAll" {
  name                 = "allowAll"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  start_ip_address     = "0.0.0.0"
  end_ip_address       = "255.255.255.255"
}

resource "azurerm_storage_account" "stfn" {
  name                     = "kaexpleaccforfunct"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_storage_account.example.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_app_service_plan" "example" {
  name                = "exm-kavya-app-service-plan"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_storage_account.example.location
  kind                = "FunctionApp"
  sku {
    tier = "Dynamic"
    size = "Y1"
  }
}

resource "azurerm_function_app" "example" {
  name                      = "exm-kavya-function-app"
  resource_group_name = data.azurerm_resource_group.example.name
  location            = data.azurerm_resource_group.example.location
  storage_account_name       = azurerm_storage_account.stfn.name
  storage_account_access_key = azurerm_storage_account.stfn.primary_access_key
  app_service_plan_id       = azurerm_app_service_plan.example.id
}

data "azurerm_function_app_host_keys" "example" {
  resource_group_name = data.azurerm_resource_group.example.name
  name= azurerm_function_app.example.name
}

output "function_key" {
  value = data.azurerm_function_app_host_keys.example.default_function_key
  sensitive = true
}

output "function_appurl" {
  value = "https://${azurerm_function_app.example.default_hostname}"
}

resource "azurerm_key_vault" "example" {
  name                        = "kavyaexamplekeyvault"
  location                    = data.azurerm_resource_group.example.location
  resource_group_name         = data.azurerm_resource_group.example.name
  enabled_for_disk_encryption = true
  tenant_id                   = data.azurerm_client_config.current.tenant_id
  soft_delete_retention_days  = 7
  purge_protection_enabled    = false

  sku_name = "standard"

  access_policy {
    tenant_id = data.azurerm_client_config.current.tenant_id
    object_id = data.azurerm_client_config.current.object_id

    key_permissions = [
      "Create",
      "Get",
    ]

    secret_permissions = [
      "Set",
      "Get",
      "Delete",
      "Purge",
      "Recover",
      "List"
    ]

    storage_permissions = [
      "Get","Set"
    ]
  }
}

resource "azurerm_key_vault_secret" "example" {
  name         = "functionkey"
  value        = data.azurerm_function_app_host_keys.example.default_function_key
  key_vault_id = azurerm_key_vault.example.id
}

resource "azurerm_synapse_linked_service" "example" {
  name                 = "kav-fnapplinked"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  type                 = "AzureFunction"
  type_properties_json = <<JSON
    {
      "functionAppUrl": "https://${azurerm_function_app.example.default_hostname}",
      "authentication": "Anonymous",
      "functionKey": 
        {
          "type": "SecureString",
          "value": "${azurerm_key_vault_secret.example.value}"
        }
    }
JSON

  depends_on = [
    azurerm_synapse_firewall_rule.allowAll,
    azurerm_function_app.example,
    data.azurerm_function_app_host_keys.example
  ]
}

使用上述代码,我成功地创建了关联服务。

英文:

I tried to reproduce the scenario in my environment.

Tried below code:

resource &quot;azurerm_synapse_linked_service&quot; &quot;example&quot; {
name                 = &quot;kavya-fnapplinked&quot;
synapse_workspace_id = azurerm_synapse_workspace.example.id
type                 = &quot;AzureFunction&quot;
type_properties_json = &lt;&lt;JSON
{
&quot;functionAppUrl&quot;: &quot;https://${data.azurerm_function_app.example.default_hostname}&quot;,
&quot;authentication&quot;: &quot;Anonymous&quot;,
&quot;functionKey&quot;: &quot;${data.azurerm_function_app_host_keys.example.default_function_key}&quot;
}
JSON
depends_on = [
azurerm_synapse_firewall_rule.allowAll,
data.azurerm_function_app.example,
data.azurerm_function_app_host_keys.example
]
}

Got errors due to the json property and function key not being generated as expected as they are not in correct format.

Azure Synapse中的Azure函数链接服务(Terraform中)

> For this ,note two important points:
> 1. Function key is only generated after the function app is created first and is a sensitive value.
> 2. Json format for sensitive values must be in below format
>
> "secret":
> {
> "type": "SecureString",
> "value": “{value}"
> }

  • I have first stored the function key in keyvalut , so that it takes time to generate and get stored in secret.

Code:

resource &quot;azurerm_role_assignment&quot; &quot;role_assignment&quot; {
scope                = azurerm_storage_account.stfn.id
role_definition_name = &quot;Storage Blob Data Owner&quot;
principal_id         = data.azurerm_client_config.current.object_id
}
# used Sleep to wait for role assignment to take its time to propagate
resource &quot;time_sleep&quot; &quot;role_assignment_sleep&quot; {
create_duration = &quot;60s&quot;
triggers = {
role_assignment = azurerm_role_assignment.role_assignment.id
}
}
resource &quot;azurerm_storage_data_lake_gen2_filesystem&quot; &quot;example&quot; {
name               = &quot;kavdatalakexample123&quot;
storage_account_id = azurerm_storage_account.stfn.id
depends_on         = [time_sleep.role_assignment_sleep]
}
resource &quot;azurerm_synapse_workspace&quot; &quot;example&quot; {
name                = &quot;exmple-workspace&quot;
resource_group_name = data.azurerm_resource_group.example.name
location            = data.azurerm_storage_account.example.location
storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.example.id
sql_administrator_login              = &quot;sqladminuser&quot;
sql_administrator_login_password     = &quot;H@Sh1CoR3!&quot;
managed_virtual_network_enabled      = true
identity {
type = &quot;SystemAssigned&quot;
}
}
resource &quot;azurerm_synapse_firewall_rule&quot; &quot;allowAll&quot; {
name                 = &quot;allowAll&quot;
synapse_workspace_id = azurerm_synapse_workspace.example.id
start_ip_address     = &quot;0.0.0.0&quot;
end_ip_address       = &quot;255.255.255.255&quot;
}
resource &quot;azurerm_storage_account&quot; &quot;stfn&quot; {
name                     = &quot;kaexpleaccforfunct&quot;
resource_group_name = data.azurerm_resource_group.example.name
location            = data.azurerm_storage_account.example.location
account_tier             = &quot;Standard&quot;
account_replication_type = &quot;LRS&quot;
}
resource &quot;azurerm_app_service_plan&quot; &quot;example&quot; {
name                = &quot;exm-kavya-app-service-plan&quot;
resource_group_name = data.azurerm_resource_group.example.name
location            =data.azurerm_storage_account.example.location
kind                = &quot;FunctionApp&quot;
sku {
tier = &quot;Dynamic&quot;
size = &quot;Y1&quot;
}
}
resource &quot;azurerm_function_app&quot; &quot;example&quot; {
name                      = &quot;exm-kavya-function-app&quot;
resource_group_name = data.azurerm_resource_group.example.name
location            = data.azurerm_resource_group.example.location
// storage_connection_string = azurerm_storage_account.stfn.primary_connection_string
storage_account_name       = azurerm_storage_account.stfn.name
storage_account_access_key = azurerm_storage_account.stfn.primary_access_key
app_service_plan_id       = azurerm_app_service_plan.example.id
}
data &quot;azurerm_function_app_host_keys&quot; &quot;example&quot; {
resource_group_name = data.azurerm_resource_group.example.name
name= azurerm_function_app.example.name
}
output &quot;function_key&quot; {
value = data.azurerm_function_app_host_keys.example.default_function_key
sensitive = true
}
output &quot;function_appurl&quot; {
value = &quot;https://${azurerm_function_app.example.default_hostname}&quot;
}
resource &quot;azurerm_key_vault&quot; &quot;example&quot; {
name                        = &quot;kavyaexamplekeyvault&quot;
location                    = data.azurerm_resource_group.example.location
resource_group_name         = data.azurerm_resource_group.example.name
enabled_for_disk_encryption = true
tenant_id                   = data.azurerm_client_config.current.tenant_id
soft_delete_retention_days  = 7
purge_protection_enabled    = false
sku_name = &quot;standard&quot;
access_policy {
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
key_permissions = [
&quot;Create&quot;,
&quot;Get&quot;,
]
secret_permissions = [
&quot;Set&quot;,
&quot;Get&quot;,
&quot;Delete&quot;,
&quot;Purge&quot;,
&quot;Recover&quot;,
&quot;List&quot;
]
storage_permissions = [
&quot;Get&quot;,&quot;Set&quot;
]
}
}
resource &quot;azurerm_key_vault_secret&quot; &quot;example&quot; {
name         = &quot;functionkey&quot;
value        = data.azurerm_function_app_host_keys.example.default_function_key
key_vault_id = azurerm_key_vault.example.id
}
resource &quot;azurerm_synapse_linked_service&quot; &quot;example&quot; {
name                 = &quot;kav-fnapplinked&quot;
synapse_workspace_id = azurerm_synapse_workspace.example.id
type                 = &quot;AzureFunction&quot;
type_properties_json = &lt;&lt;JSON
{
&quot;functionAppUrl&quot;: &quot;https://${azurerm_function_app.example.default_hostname}&quot;,
&quot;authentication&quot;: &quot;Anonymous&quot;,
&quot;functionKey&quot;: 
{
&quot;type&quot;: &quot;SecureString&quot;,
&quot;value&quot;:  &quot;${azurerm_key_vault_secret.example.value}&quot;
}
&quot;authentication&quot;: &quot;Anonymous&quot;,
&quot;functionKey&quot;: 
{
&quot;type&quot;: &quot;SecureString&quot;,
&quot;value&quot;:  &quot;${azurerm_key_vault_secret.example.value}&quot;
}
}
JSON
depends_on = [
azurerm_synapse_firewall_rule.allowAll,
azurerm_function_app.example,
data.azurerm_function_app_host_keys.example
]
}

With abovecode I could successfully, create linked service.

Azure Synapse中的Azure函数链接服务(Terraform中)

Linked service for azure synapse workspace:

Azure Synapse中的Azure函数链接服务(Terraform中)

Reference: azure - Terraform issue creating the resource "azurerm_synapse_linked_service" specifically with the "type_properties_json" field - Stack Overflow

huangapple
  • 本文由 发表于 2023年2月8日 17:25:58
  • 转载请务必保留本文链接:https://go.coder-hub.com/75383633.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定