英文:
How to Authenitcate Access to ADLS from Databricks without creating Service Principle with Using AD App Registrations to Mount a Drive
问题
It is possible to mount an ADLS Gen2 account/container without configuring service principal credentials to access storage. The following article states that it's possible to access an ADLS storage account without first configuring a service principal.
You can mount a drive in Databricks without registering an application by using the following code:
mount = "/mnt/lake"
if check(mount) == 1:
resultMsg = "<div>%s is already mounted. </div>" % mount
else:
dbutils.fs.mount(
source = f"abfss://root@{Lake}.dfs.core.windows.net/",
mount_point = mount,
extra_configs = configs)
resultMsg = "<div>%s was mounted. </div>" % mount
However, if you encounter the error "IllegalArgumentException: Unsupported Azure Scheme: abfss," it might be due to incorrect configuration or a problem with the URL you are using. Double-check your configuration and ensure that your storage account and container details are correct.
英文:
Is it possible to mount ADLS Gen2 account/container without configuring service principle credentials to access storage?
The following article states that it's possible to access a ADLS storage account with first configuring a service principle.
I normally mount a drive in Databricks by first creating a service principle by Registering an Azure AD application using App Registrations. I would use the following code to mount:
if check(mount)==1:
resultMsg = "<div>%s is already mounted. </div>" % mount
else:
dbutils.fs.mount(
source = f"abfss://root@{Lake}.dfs.core.windows.net/",
mount_point = mount,
extra_configs = configs)
resultMsg = "<div>%s was mounted. </div>" % mount
At present I am waiting for the administrator at my place of work to give me the permissions to register applications.
So, I was wondering if I could mount a drive from Databricks without first registering an application?
I tried @JayashankarGS suggestion as follows:
mount = "/mnt/lake"
if check(mount)==1:
resultMsg = "<div>%s is already mounted. </div>" % mount
else:
dbutils.fs.mount(
source = f"abfss://root@{Lake}.dfs.core.windows.net/",
mount_point = mount,
extra_configs = configs)
resultMsg = "<div>%s was mounted. </div>" % mount
But I got the error;
IllegalArgumentException: Unsupported Azure Scheme: abfss
答案1
得分: 1
你可以尝试使用账户密钥来挂载存储。
我在我的项目中也做了类似的事情!
你可以使用下面的代码作为参考:
Container_name = "root"
storage_account = "lake"
key = ""
url = "wasbs://" + container_name + "@" + storage_account + ".blob.core.windows.net/"
config = "fs.azure.account.key." + storage_account + ".blob.core.windows.net"
mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()
mounted_exist = False
for item in mounted_list:
if mount_folder in item[0]:
mounted_exist = True
break
if not mounted_exist:
dbutils.fs.mount(source=url, mount_point=mount_folder, extra_configs={config: key})
英文:
You can try to mount storage using account key
I did similar thing in my project as well!!
You can use below code as a reference
Container_name = "root"
storage_account = "lake"
key = ""
url = "wasbs://" + container_name + "@" + storage_account + ".blob.core.windows.net/"
config = "fs.azure.account.key." + storage_account + ".blob.core.windows.net"
mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()
mounted_exist = False
for item in mounted_list:
if mount_folder in item[0]:
mounted_exist = True
break
if not mounted_exist:
dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})
答案2
得分: 0
你使用存储帐户密钥来访问adls2中的数据。
下面是代码示例:
spark.conf.set("fs.azure.account.key.<存储帐户名称>.dfs.core.windows.net","你的存储帐户密钥")
dbutils.fs.ls("abfss://<容器名称>@<存储帐户名称>.dfs.core.windows.net/<目录>")
输出:
或者你也可以按照以下方法操作:
dbutils.fs.mount(source = "wasbs://data@jgsadls.blob.core.windows.net",
mount_point = "/mnt/iotdata",
extra_configs = {"fs.azure.account.key.jgsadls.blob.core.windows.net":"你的帐户密钥"})
dbutils.fs.ls("/mnt/iotdata/json_data/")
这是结果:
英文:
You use storage account key to access data in adls2.
Below is code for it.
spark.conf.set("fs.azure.account.key.<storage account name>.dfs.core.windows.net","Your stoarge account key")
dbutils.fs.ls("abfss://<container name>@<storage account name>.dfs.core.windows.net/<dir>")
Output:
Or you can follow below approach.
dbutils.fs.mount(source = "wasbs://data@jgsadls.blob.core.windows.net",
mount_point = "/mnt/iotdata",
extra_configs = {"fs.azure.account.key.jgsadls.blob.core.windows.net":"Your account key"})
dbutils.fs.ls("/mnt/iotdata/json_data/")
This is the result.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论