Azure Blob到SQL Server批量插入 – 没有文件访问权限

huangapple go评论69阅读模式
英文:

Azure Blob to SQL Server Bulk Insert - Don't have file access rights

问题

我正在尝试批量将一个平面文件从 Blob 存储插入到 SQL Server 数据库中。以下是我所做的:

  1. 使用存储账户的主访问密钥创建了一个安全连接到 Azure Blob 和 SQL Server。该连接指向特定的容器,文件会被暂存到“load-to-sql”文件夹中。
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'XYZ' 

CREATE DATABASE SCOPED CREDENTIAL ohhyeaa 
WITH IDENTITY = 'SHARED ACCESS SIGNATURE' 
SECRET = '整个容器的访问策略密钥' 

IF EXISTS ( SELECT * FROM sys.external_data_sources) 
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage 
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage 
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'CONTAINERaccount.blob.core.windows.net/load-to-sql', CREDENTIAL= MyAzureBlobStorageCredential) 
  1. 文件被加载到一个容器中。

  2. 执行:

BULK INSERT [dbo].file.csv
FROM 'StagingFolder_DataToSQL/project/date/file.csv' 
WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = '|', 
KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)

这给了我以下错误:

Msg 4860, Level 16, State 1, Line 1 无法进行批量加载。文件
"StagingFolder_DataToSQL///.csv" 不存在
或您没有文件访问权限。

我是服务器和正在加载数据的特定数据库的所有者。

在 Azure 门户中是否有我需要定义的访问设置,以允许加载文件?

在不同的 Azure 环境中,我已经成功地完成了这个过程,而且效果非常好。非常感谢您的帮助!谢谢!

英文:

I am trying to bulk insert a flat file from Blob Storage to a SQL Server Database. Here is what I have done:

  1. Created a secure connection to Azure Blob and SQL Server using the master access key on the overall storage account. The connection is pointed at the specific container the files get staged "load-to-sql".
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'XYZ' 

CREATE DATABASE SCOPED CREDENTIAL ohhyeaa 
WITH IDENTITY = 'SHARED ACCESS SIGNATURE' 
SECRET = 'access policy key from the entire container' 

IF EXISTS ( SELECT * FROM sys.external_data_sources) 
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage 
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage 
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'CONTAINERaccount.blob.core.windows.net/load-to-sql', CREDENTIAL= MyAzureBlobStorageCredential) 


  1. File is loaded into a container

  2. Execute:

     BULK INSERT [dbo].file.csv
     FROM 'StagingFolder_DataToSQL/project/date/file.csv' 
     WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = '|', 
     KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)
    

This gives me the following error:

> Msg 4860, Level 16, State 1, Line 1 Cannot bulk load. The file
> "StagingFolder_DataToSQL/<project>/<date>/<file>.csv" does not exist
> or you don't have file access rights.

I am the owner of the server and the specific database where data is getting loaded.

Are there access settings that I need to define in the Azure portal to allow for the file to load?

I have been successful with this process in a different Azure environment and it worked great. Help is greatly appreciated. Thanks!

答案1

得分: 1

不能批量加载。文件 "StagingFolder_DataToSQL///.csv" 不存在或您没有文件访问权限。

错误的原因是您正在使用的访问令牌没有对应加载对象的读取权限,或者文件路径不正确/文件路径区分大小写。

使用适当权限获取容器范围的SAS令牌:

Azure Blob到SQL Server批量插入 – 没有文件访问权限

我的SQL代码:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'strong_password'

CREATE DATABASE SCOPED CREDENTIAL ohhyeaa7 
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
SECRET = 'Container_scoped_SAS_token'

IF EXISTS ( SELECT * FROM sys.external_data_sources) 
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage 
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage 
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'https://<Storage_Account_name>.blob.core.windows.net/<Container_name>', CREDENTIAL= ohhyea7) 

BULK INSERT dbo.student
FROM 'stagged/sample/file3.csv'
WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = ',', 
KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)

Azure Blob到SQL Server批量插入 – 没有文件访问权限

英文:

> Cannot bulk load. The file "StagingFolder_DataToSQL///.csv" does not exist or you don't have file access rights.

The Cause of error is the access token you are using it doesn't have read permission on the object that should be loaded, or the file path is incorrect/Dose not exist file path is Case sensitive.

Get Container scoped SAS token with appropriate permissions:

Azure Blob到SQL Server批量插入 – 没有文件访问权限
My SQL Code:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = &#39;strong_password&#39; 

CREATE DATABASE SCOPED CREDENTIAL ohhyeaa7 
WITH IDENTITY = &#39;SHARED ACCESS SIGNATURE&#39;,
SECRET = &#39;Container_scoped_SAS_token&#39; 

IF EXISTS ( SELECT * FROM sys.external_data_sources) 
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage 
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage 
WITH ( TYPE = BLOB_STORAGE, LOCATION = &#39;https://&lt;Storage_Account_name&gt;.blob.core.windows.net/&lt;Container_name&gt;&#39;, CREDENTIAL= ohhyeaa7) 

BULK INSERT dbo.student
FROM &#39;stagged/sample/file3.csv&#39;
WITH (FORMAT = &#39;CSV&#39;, FIRSTROW = 2, FIELDTERMINATOR = &#39;,&#39;, 
KEEPIDENTITY, DATA_SOURCE = &#39;MyAzureBlobStorage&#39;, batchsize = 300000)

Azure Blob到SQL Server批量插入 – 没有文件访问权限

huangapple
  • 本文由 发表于 2023年7月28日 03:59:40
  • 转载请务必保留本文链接:https://go.coder-hub.com/76783034.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定