英文:
Azure Data Factory Get Metadata activity returning "(404) not found" error when getting column count
问题
我正在尝试实现一个"获取元数据"活动,以返回我在一个单一的 Blob 存储容器中拥有的文件的列数。
"获取元数据"活动返回以下错误:Error
我对 Azure 数据工厂相当新,并且无法解决这个问题。以下是我的设置:
数据集:源数据集
名称- ten_eighty_split_CSV
连接- Blob 存储
架构- 从 Blob 存储文件导入
参数- "FileName";字符串;"@pipeline().parameters.SourceFile"
管道:
名称:ten eighty split
参数:"SourceFile";字符串;"@pipeline().parameters.SourceFile"
设置:并发性:1
"获取元数据"活动:获取元数据
唯一参数是"列数"
在调试时出现错误。我不确定该怎么办,(404) 未找到太广泛,我无法确定一个具体的解决方案。谢谢!
英文:
I am trying to implement a Get Metadata activity to return the column count of files I have in a single blob storage container.
Get Metadata activity is returning this error:
Error
I'm fairly new to Azure Data Factory and cannot solve this. Here's what I have:
Dataset:Source dataset
Name- ten_eighty_split_CSV
Connection- Blob storage
Schema- imported from blob storage file
Parameters- "FileName"; string; "@pipeline().parameters.SourceFile"
Pipeline:
Name: ten eighty split
Parameters: "SourceFile"; string; "@pipeline().parameters.SourceFile"
Settings: Concurrency: 1
Get Metadata activity: Get Metadata
Only argument is "Column count"
Throws the error upon debugging. I am not sure what to do, (404) not found is so broad I could not ascertain a specific solution. Thanks!
答案1
得分: 0
-
由于您提供了不正确的文件名(或)文件名不存在,所以发生错误。
-
由于您尝试使用创建的 blob 事件触发器来查找列数,您可以使用以下过程:
-
在配置获取元数据活动之后,创建存储事件触发器。转到“添加触发器 -> 选择触发器 -> 创建新的”。
- 单击“继续”。您将进入触发运行参数选项卡。在此处,将值设置为
@triggerBody().fileName
。
- 完成触发器的创建并发布管道。现在,每当文件上传到您的容器(您创建了存储事件触发器的容器),它将自动触发管道(无需调试)。如果容器为空,并且您尝试通过为sourceFile参数提供某些值进行调试,将会引发相同的错误。
- 将示例文件上传到您的容器。它将触发管道并提供所需的结果。
以下是我为我的容器创建的触发器 JSON:
{
"name": "trigger1",
"properties": {
"annotations": [],
"runtimeState": "Started",
"pipelines": [
{
"pipelineReference": {
"referenceName": "pipeline1",
"type": "PipelineReference"
},
"parameters": {
"sourceFile": "@triggerBody().fileName"
}
}
],
"type": "BlobEventsTrigger",
"typeProperties": {
"blobPathBeginsWith": "/data/blobs/",
"blobPathEndsWith": ".csv",
"ignoreEmptyBlobs": true,
"scope": "/subscriptions/b83c1ed3-c5b6-44fb-b5ba-2b83a074c23f/resourceGroups/<user>/providers/Microsoft.Storage/storageAccounts/blb1402",
"events": [
"Microsoft.Storage.BlobCreated"
]
}
}
}
英文:
-
The error occurs because you have given incorrect file name (or) name of a file that does not exist.
-
Since you are trying to use blob created event trigger to find the column count, you can use the procedure below:
-
After configuring the get metadata activity, create a storage event trigger. Go to
Add trigger -> choose trigger -> Create new
.
- Click on continue. You will get a Trigger Run Parameters tab. In this, give the value as
@triggerBody().fileName
.
- Complete the trigger creation and publish the pipeline. Now whenever the file is uploaded into your container (on top of which you created storage event trigger), it will trigger the pipeline automatically (no need to debug). If the container is empty and you try to debug by giving some value for sourceFile parameter, it would give the same error.
- Upload a sample file to your container. It will trigger the pipeline and give the desired result.
The following is the trigger JSON that I created for my container:
{
"name": "trigger1",
"properties": {
"annotations": [],
"runtimeState": "Started",
"pipelines": [
{
"pipelineReference": {
"referenceName": "pipeline1",
"type": "PipelineReference"
},
"parameters": {
"sourceFile": "@triggerBody().fileName"
}
}
],
"type": "BlobEventsTrigger",
"typeProperties": {
"blobPathBeginsWith": "/data/blobs/",
"blobPathEndsWith": ".csv",
"ignoreEmptyBlobs": true,
"scope": "/subscriptions/b83c1ed3-c5b6-44fb-b5ba-2b83a074c23f/resourceGroups/<user>/providers/Microsoft.Storage/storageAccounts/blb1402",
"events": [
"Microsoft.Storage.BlobCreated"
]
}
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论