Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

huangapple go评论71阅读模式
英文:

Auto Detect Data Types with Azure Synapse Analytics Copy Data (Import)

问题

我有一个与Azure Synapse Analytics(ASA)工作区关联的Azure Data Lake Storage Gen2帐户。存储帐户中有TIOBE测试数据,看起来像这样:

PartitionKey,Year,Month,Language,Rating
2023-04,2023,4,Python,14.51
2023-04,2023,4,C,14.41
2023-04,2023,4,Java,13.23
...

ASA工作区具有内置的复制数据工具。当我通过该工具预览我的TIOBE数据时,一切都看起来正常:

Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

不幸的是,当我到达目标/配置步骤时,列映射都硬编码为字符串,我无法更改它们:

Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

我向Microsoft的某人咨询过这个问题,我理解他说,更容易先创建具有正确数据类型的目标(接收)表,然后再使用复制数据工具导入数据。

我应该继续在使用ASA复制数据工具之前创建我的接收表,还是有更简单的方法来指定导入数据时的正确数据类型?

注意:我刚刚在SuperUser上提出了这个问题,被关闭为不相关,所以我删除了那个问题,现在在这里提问。

英文:

I have an Azure Data Lake Storage Gen2 account linked to an Azure Synapse Analytics (ASA) Workspace. The storage account has TIOBE Test data that looks like this:

PartitionKey,Year,Month,Language,Rating
2023-04,2023,4,Python,14.51
2023-04,2023,4,C,14.41
2023-04,2023,4,Java,13.23
...

ASA Workspaces have a built in Copy Data Tool. When I preview my TIOBE Data via that tool, everything looks fine:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

Unfortunately, when I get to the Destination / Configuration step, the Column Mappings are all hard wired to String and there is no way for me to change them:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

I asked someone at Microsoft about this, and I understood him to say it's easier to create the destination (sink) table with correct data types first and then use the Copy Data tool to import the data.

Should I continue to create my sink tables before using the ASA Copy Data Tool or is there an easier way to specify correct data types when importing data?

NOTE: I just asked this on SuperUser and it was closed as off topic, so I deleted that question and am now asking here.

答案1

得分: 1

根据我的理解,是的,先使用正确的数据类型创建目标表是一个不错的方法。原因是,当使用Azure Synapse Analytics复制数据工具时,它会根据源数据的前几行自动推断数据类型。

例如,我在ADLS中创建了一个数据集,并使用复制活动移动数据。
源:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)
源预览:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

接收端:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

在接收端创建SQL表:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)
在接收端创建表后的映射:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)
输出:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

英文:

As per my Understanding Yes, creating the destination table with the correct data types before is a good approach.
The reason is because that when you use the Azure Synapse Analytics Copy Data Tool, it automatically infers the data types based on the first few rows of the source data.

For example I have created a dataset in ADLS and using the copy activity I am moving the data.
Source:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)
Source Preview:

Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

Sink:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

Creating the SQL Table in the Sink:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)
Mapping after creating the table at the sink:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)
Output:
Auto Detect Data Types with Azure Synapse Analytics Copy Data(Import)

huangapple
  • 本文由 发表于 2023年5月18日 01:39:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/76274814.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定