英文:
Azure Blob to SQL Server Bulk Insert - Don't have file access rights
问题
我正在尝试批量将一个平面文件从 Blob 存储插入到 SQL Server 数据库中。以下是我所做的:
- 使用存储账户的主访问密钥创建了一个安全连接到 Azure Blob 和 SQL Server。该连接指向特定的容器,文件会被暂存到“load-to-sql”文件夹中。
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'XYZ'
CREATE DATABASE SCOPED CREDENTIAL ohhyeaa
WITH IDENTITY = 'SHARED ACCESS SIGNATURE'
SECRET = '整个容器的访问策略密钥'
IF EXISTS ( SELECT * FROM sys.external_data_sources)
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'CONTAINERaccount.blob.core.windows.net/load-to-sql', CREDENTIAL= MyAzureBlobStorageCredential)
-
文件被加载到一个容器中。
-
执行:
BULK INSERT [dbo].file.csv
FROM 'StagingFolder_DataToSQL/project/date/file.csv'
WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = '|',
KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)
这给了我以下错误:
Msg 4860, Level 16, State 1, Line 1 无法进行批量加载。文件
"StagingFolder_DataToSQL// / .csv" 不存在
或您没有文件访问权限。
我是服务器和正在加载数据的特定数据库的所有者。
在 Azure 门户中是否有我需要定义的访问设置,以允许加载文件?
在不同的 Azure 环境中,我已经成功地完成了这个过程,而且效果非常好。非常感谢您的帮助!谢谢!
英文:
I am trying to bulk insert a flat file from Blob Storage to a SQL Server Database. Here is what I have done:
- Created a secure connection to Azure Blob and SQL Server using the master access key on the overall storage account. The connection is pointed at the specific container the files get staged "load-to-sql".
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'XYZ'
CREATE DATABASE SCOPED CREDENTIAL ohhyeaa
WITH IDENTITY = 'SHARED ACCESS SIGNATURE'
SECRET = 'access policy key from the entire container'
IF EXISTS ( SELECT * FROM sys.external_data_sources)
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'CONTAINERaccount.blob.core.windows.net/load-to-sql', CREDENTIAL= MyAzureBlobStorageCredential)
-
File is loaded into a container
-
Execute:
BULK INSERT [dbo].file.csv FROM 'StagingFolder_DataToSQL/project/date/file.csv' WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = '|', KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)
This gives me the following error:
> Msg 4860, Level 16, State 1, Line 1 Cannot bulk load. The file
> "StagingFolder_DataToSQL/<project>/<date>/<file>.csv" does not exist
> or you don't have file access rights.
I am the owner of the server and the specific database where data is getting loaded.
Are there access settings that I need to define in the Azure portal to allow for the file to load?
I have been successful with this process in a different Azure environment and it worked great. Help is greatly appreciated. Thanks!
答案1
得分: 1
不能批量加载。文件 "StagingFolder_DataToSQL///.csv" 不存在或您没有文件访问权限。
错误的原因是您正在使用的访问令牌没有对应加载对象的读取权限,或者文件路径不正确/文件路径区分大小写。
使用适当权限获取容器范围的SAS令牌:
我的SQL代码:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'strong_password'
CREATE DATABASE SCOPED CREDENTIAL ohhyeaa7
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
SECRET = 'Container_scoped_SAS_token'
IF EXISTS ( SELECT * FROM sys.external_data_sources)
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'https://<Storage_Account_name>.blob.core.windows.net/<Container_name>', CREDENTIAL= ohhyea7)
BULK INSERT dbo.student
FROM 'stagged/sample/file3.csv'
WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = ',',
KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)
英文:
> Cannot bulk load. The file "StagingFolder_DataToSQL///.csv" does not exist or you don't have file access rights.
The Cause of error is the access token you are using it doesn't have read permission on the object that should be loaded, or the file path is incorrect/Dose not exist file path is Case sensitive.
Get Container scoped SAS token with appropriate permissions:
My SQL Code:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'strong_password'
CREATE DATABASE SCOPED CREDENTIAL ohhyeaa7
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
SECRET = 'Container_scoped_SAS_token'
IF EXISTS ( SELECT * FROM sys.external_data_sources)
DROP EXTERNAL DATA SOURCE MyAzureBlobStorage
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'https://<Storage_Account_name>.blob.core.windows.net/<Container_name>', CREDENTIAL= ohhyeaa7)
BULK INSERT dbo.student
FROM 'stagged/sample/file3.csv'
WITH (FORMAT = 'CSV', FIRSTROW = 2, FIELDTERMINATOR = ',',
KEEPIDENTITY, DATA_SOURCE = 'MyAzureBlobStorage', batchsize = 300000)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论