英文:
Facing issue while merging existing table in snowflake with s3 stage parquet file
问题
我尝试执行以下脚本:
MERGE INTO data_stage.test_temp4 ex USING (
SELECT
$1:device_attr_hk,
$1:applicationruntime_raw,
$1:devicefamily_raw,
$1:deviceprofile_raw,
$1:manufacturer_raw,
$1:model_raw,
$1:application_runtime_os,
$1:device_profile,
$1:device_family,
$1:device_platform,
$1:manufacturer,
$1:model,
$1:device_type,
$1:creation_timestamp,
$1:change_hkey
FROM @dss.test_plus.test_plus_stage/dev/sub_solutions/test/ (file_format => 'parquet', PATTERN => '.*.parquet')
) ne ON ex.device_attr_hk = ne.device_attr_hk WHEN MATCHED THEN
UPDATE SET device_attr_hk = ne.device_attr_hk, applicationruntime_raw = ne.applicationruntime_raw, devicefamily_raw = ne.devicefamily_raw, deviceprofile_raw = ne.deviceprofile_raw, manufacturer_raw = ne.manufacturer_raw, model_raw = ne.model_raw, application_runtime_os = ne.application_runtime_os, device_profile = ne.device_profile, device_family = ne.device_family, device_platform = ne.device_platform, manufacturer = ne.manufacturer, model = ne.model, device_platform = ne.device_platform , manufacturer = ne.manufacturer, model = ne.model, device_type = ne.device_type, creation_timestamp = ne.creation_timestamp, change_hkey = ne.change_hkey WHEN NOT MATCHED THEN
INSERT
(device_attr_hk, applicationruntime_raw, devicefamily_raw, deviceprofile_raw, manufacturer_raw, model_raw, application_runtime_os, device_profile, device_family, device_platform, manufacturer, model, device_type, creation_timestamp, change_hkey) VALUES
(ne.device_attr_hk, ne.applicationruntime_raw, ne.devicefamily_raw, ne.deviceprofile_raw, ne.manufacturer_raw, ne.model_raw, ne.application_runtime_os, ne.device_profile, ne.device_family, ne.device_platform, ne.manufacturer, ne.model, ne.device_type, ne.creation_timestamp, ne.change_hkey);
commit;
但是我遇到了错误:SQL编译错误:文件格式'PARQUET' 不存在或未授权。
注意: 文件已经存在于stage: @dss.test_plus.test_plus_stage/dev/sub_solutions/test/,而且我能够访问它。
我怀疑为什么我无法访问此文件并进行合并操作,脚本中是否存在问题。
英文:
I tried to execute below script:
MERGE INTO data_stage.test_temp4 ex USING (
SELECT
$1:device_attr_hk,
$1:applicationruntime_raw,
$1:devicefamily_raw,
$1:deviceprofile_raw,
$1:manufacturer_raw,
$1:model_raw,
$1:application_runtime_os,
$1:device_profile,
$1:device_family,
$1:device_platform,
$1:manufacturer,
$1:model,
$1:device_type,
$1:creation_timestamp,
$1:change_hkey
FROM @dss.test_plus.test_plus_stage/dev/sub_solutions/test/ (file_format => 'parquet', PATTERN => '.*.parquet')
) ne ON ex.device_attr_hk = ne.device_attr_hk WHEN MATCHED THEN
UPDATE SET device_attr_hk = ne.device_attr_hk, applicationruntime_raw = ne.applicationruntime_raw, devicefamily_raw = ne.devicefamily_raw, deviceprofile_raw = ne.deviceprofile_raw, manufacturer_raw = ne.manufacturer_raw, model_raw = ne.model_raw, application_runtime_os = ne.application_runtime_os, device_profile = ne.device_profile, device_family = ne.device_family, device_platform = ne.device_platform, manufacturer = ne.manufacturer, model = ne.model, device_platform = ne.device_platform , manufacturer = ne.manufacturer, model = ne.model, device_type = ne.device_type, creation_timestamp = ne.creation_timestamp, change_hkey = ne.change_hkey WHEN NOT MATCHED THEN
INSERT
(device_attr_hk, applicationruntime_raw, devicefamily_raw, deviceprofile_raw, manufacturer_raw, model_raw, application_runtime_os, device_profile, device_family, device_platform, manufacturer, model, device_type, creation_timestamp, change_hkey) VALUES
(ne.device_attr_hk, ne.applicationruntime_raw, ne.devicefamily_raw, ne.deviceprofile_raw, ne.manufacturer_raw, ne.model_raw, ne.application_runtime_os, ne.device_profile, ne.device_family, ne.device_platform, ne.manufacturer, ne.model, ne.device_type, ne.creation_timestamp, ne.change_hkey);
commit;
But I am getting error: SQL compilation error: File format 'PARQUET' does not exist or not authorized.
Note: file already avialable on stage: @dss.test_plus.test_plus_stage/dev/sub_solutions/test/ And I have able to access it.
I have doubt , why I am not able to access this file an merge, is there any issue in script that written.
答案1
得分: 1
你之所以收到此错误是因为您尝试使用名称为'parquet'的file_format,但该文件格式不存在。
我建议您首先创建一个命名的file_format,并在命令中使用其名称。
步骤1. 创建一个命名的文件格式,例如:
CREATE OR REPLACE FILE FORMAT my_parquet_format
TYPE = PARQUET;
步骤2. 更改脚本中包含FROM
的行如下:
FROM @dss.test_plus.test_plus_stage/dev/sub_solutions/test/ (file_format =>'my_parquet_format', PATTERN =>'*.parquet')
然后您可以再次尝试。
英文:
You are getting this error because you try using the file_format with the name 'parquet' which does not exist
I suggest you create a named file_format first and use its name in your command.
Step 1. Create a named file format, e.g.:
CREATE OR REPLACE FILE FORMAT my_parquet_format
TYPE = PARQUET;
Step 2. Change the line with FROM
in your script as follows:
FROM @dss.test_plus.test_plus_stage/dev/sub_solutions/test/ (file_format => 'my_parquet_format', PATTERN => '.*.parquet')
Then you can try again.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论