英文:
Databricks SQL reading Parquet file ceated by Copy activity in Azure Data Factory
问题
我正在尝试获取由Azure Data Factory复制活动写入的Parquet文件，将其"附加"到Databricks SQL。我使用了这个命令：
create table people_db.GLAccount USING PARQUET LOCATION "abfss://dlsxxxx.dfs.core.windows.net/datamesh/PricingAnalysis/rdv_60_134.vGLAccount.parquet"
但我收到了以下错误消息：
Abfss://dlsxxxx.core.windows.net/datamesh/PricingAnalysis/rdv_60_134.vGLAccount.parquet 具有无效的权限。
我不是Databricks专家，我注意到ADF-Copy将所有数据写入一个文件，而当我在Databricks中创建Parquet文件时，我得到一个包含多个文件的目录。因此，这可能与设置有关，但我需要一些关于在哪里进行更改/测试的指导...
Kr，Harry
英文:
I'm tyring to get a Parquet file written by a Azore Data Factory copy activity "attached" to Databricks SQL.
I used this command
create table people_db.GLAccount USING PARQUET LOCATION "abfss://dlsxxxx.dfs.core.windows.net/datamesh/PricingAnalysis/rdv_60_134.vGLAccount.parquet"
But I'm getting this error
Abfss://dlsxxxx.core.windows.net/datamesh/PricingAnalysis/rdv_60_134.vGLAccount.parquet has invalid authority.
I'm not a databricsk expert and I saw that the ADF-Copy writes all data into 1 file and when I create Parquet files in Databricks I'm getting a directory with several files into it. So this might all have something to do with setting, but I will need some guidance on where what to change/test...
Kr, Harry
答案1
得分: 0
最有可能的情况是，datamesh 是您在 ADLS 上的容器，是吗？如果是的话，那么URL应该是
abfss://datamesh@dlsxxxx.dfs.core.windows.net/PricingAnalysis/rdv_60_134.vGLAccount.parquet
一般来说，ABFSS的URL看起来像下面这样：
abfss://<容器>@<存储>.dfs.core.windows.net/<路径>
P.S. 您可以在Hadoop ABFSS集成文档中找到所有详细信息。
英文:
Most probably, the datamesh is your container on ADLS? If yes, then URL should be
abfss://datamesh@dlsxxxx.dfs.core.windows.net/PricingAnalysis/rdv_60_134.vGLAccount.parquet
In general, ABFSS urls look like following:
abfss://<container>@<storage>.dfs.core.windows.net/<path>
P.S. you can find all details in documentation for Hadoop ABFSS integration.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论