(Forbidden) The user, group or application 'appid="******" does not have secrets get permission on key vault '****Key;location=eastus'

I am using azure databricks notebook(python). I am trying to access the azure blob storage container files to azure databricks. To access the file I am trying to use the key Vault as use of hard coded account name and access is not good practice but I am getting following error:

(Forbidden) The user, group or application 'appid=-" ;iss=https://sts.windows.net/****/' does not have secrets get permission on key vault '***Key;location=eastus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287
Code: Forbidden
Message: The user, group or application 'appid=******;oid=*****;iss=https://sts.windows.net/****/' does not have secrets get permission on key vault '****Key;location=eastus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287
Inner error: {
    "code": "AccessDenied"
}

I have added secretscope to databricks. here is the screenshot of access policies:

here is my full code:

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
import pandas as pd
import io
import pyspark.sql
from datetime import datetime, timedelta

#set Azure Key Vaults details
key_vault_name = '***key'
secret_name = '******'

credential = DefaultAzureCredential()

#create Secret Client
secret_client = SecretClient(vault_url=f"https://{key_vault_name}.vault.azure.net/", credential=credential)

#define Azure Blob Storage connection details
storage_account_name = secret_client.get_secret('storage-account-name').value
storage_account_access_key = secret_client.get_secret('storage-account-access-key').value
container_name = '*****'
refined_container_name = '*****'

connection_string = secret_client.get_secret(secret_name).value

print('storage_account_name = ',storage_account_name)
print('storage_account_access_key = ',storage_account_access_key)
print('connection_string = ', connection_string)

I reproduce same thing in my environment .I got same error.

To resolve above error .Please follow below steps :

Step 1: Go to azure key vault -> Access policies -> + create and provide enough permissions.

Step2 : Create Secret Scope -> Go to Azure_Vault -> Enter the Vault_URI .(For Example, Vault_URI: https://<key_vault_name.>.vault.azure.net/) and Resource ID -> Go to Properties tab of an Azure Key Vault in your Azure portal you get both Vault URI and Resource ID.

Code:

from azure.keyvault.secrets import SecretClient
from azure.identity import ClientSecretCredential as cs
kv1_URI = "https://testvam.vault.azure.net/"
TENANT_ID1 = '72f988bf-86f1-41afxxxxxxxxxxxxx'
CLIENT1_ID = 'f4dab6c8-5009-48xxxxxxxxx'
CLIENT1_SECRET = 'UVh8Q~l6M55fxxxxxxxxxxxxxxxx'
credentials = cs(
		    tenant_id=TENANT_ID1,
		    client_id=CLIENT1_ID,
		    client_secret=CLIENT1_SECRET)

def set_secret(secret_name1,secret_value1):
    print(credentials)
    secret_client = SecretClient(vault_url=kv1_URI, credential=credentials)
    secret = secret_client.set_secret(secret_name1,secret_value1,enabled=True)
    secr_dic={}
    secr_dic['name']=secret.name
    secr_dic['value']=secret.value
    secr_dic['properties']=secret.properties.version
    return secr_dic

#Use Keyvalut Secret Value
x1=set_secret('dem','value')
print(x1)

Output:

You need to use dbutils(1)(2) to interact with databricks secret scoped instead of Azure key vault SDK.

(1) https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-utils#dbutils-secrets
(2) https://learn.microsoft.com/en-us/azure/databricks/security/secrets/example-secret-workflow#secret-example-notebook

On the Key Vault, add an access policy for the application id and add the "Get" secret permission.