Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

huangapple go评论69阅读模式
英文:

Azure Data Factory Get Metadata activity returning "(404) not found" error when getting column count

问题

我正在尝试实现一个"获取元数据"活动,以返回我在一个单一的 Blob 存储容器中拥有的文件的列数。

"获取元数据"活动返回以下错误:Error

我对 Azure 数据工厂相当新,并且无法解决这个问题。以下是我的设置:

数据集:源数据集

名称- ten_eighty_split_CSV

连接- Blob 存储

架构- 从 Blob 存储文件导入

参数- "FileName";字符串;"@pipeline().parameters.SourceFile"

管道:

名称:ten eighty split

参数:"SourceFile";字符串;"@pipeline().parameters.SourceFile"

设置:并发性:1

"获取元数据"活动:获取元数据

唯一参数是"列数"

在调试时出现错误。我不确定该怎么办,(404) 未找到太广泛,我无法确定一个具体的解决方案。谢谢!

英文:

I am trying to implement a Get Metadata activity to return the column count of files I have in a single blob storage container.

Get Metadata activity is returning this error:
Error

I'm fairly new to Azure Data Factory and cannot solve this. Here's what I have:

Dataset:Source dataset

Name- ten_eighty_split_CSV

Connection- Blob storage

Schema- imported from blob storage file

Parameters- "FileName"; string; "@pipeline().parameters.SourceFile"

Pipeline:

Name: ten eighty split

Parameters: "SourceFile"; string; "@pipeline().parameters.SourceFile"

Settings: Concurrency: 1

Get Metadata activity: Get Metadata

Only argument is "Column count"

Throws the error upon debugging. I am not sure what to do, (404) not found is so broad I could not ascertain a specific solution. Thanks!

答案1

得分: 0

  • 由于您提供了不正确的文件名(或)文件名不存在,所以发生错误。

  • 由于您尝试使用创建的 blob 事件触发器来查找列数,您可以使用以下过程:

  • 在配置获取元数据活动之后,创建存储事件触发器。转到“添加触发器 -> 选择触发器 -> 创建新的”。

Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

  • 单击“继续”。您将进入触发运行参数选项卡。在此处,将值设置为@triggerBody().fileName

Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

  • 完成触发器的创建并发布管道。现在,每当文件上传到您的容器(您创建了存储事件触发器的容器),它将自动触发管道(无需调试)。如果容器为空,并且您尝试通过为sourceFile参数提供某些值进行调试,将会引发相同的错误。
  • 将示例文件上传到您的容器。它将触发管道并提供所需的结果。

Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

以下是我为我的容器创建的触发器 JSON:

{
    "name": "trigger1",
    "properties": {
        "annotations": [],
        "runtimeState": "Started",
        "pipelines": [
            {
                "pipelineReference": {
                    "referenceName": "pipeline1",
                    "type": "PipelineReference"
                },
                "parameters": {
                    "sourceFile": "@triggerBody().fileName"
                }
            }
        ],
        "type": "BlobEventsTrigger",
        "typeProperties": {
            "blobPathBeginsWith": "/data/blobs/",
            "blobPathEndsWith": ".csv",
            "ignoreEmptyBlobs": true,
            "scope": "/subscriptions/b83c1ed3-c5b6-44fb-b5ba-2b83a074c23f/resourceGroups/<user>/providers/Microsoft.Storage/storageAccounts/blb1402",
            "events": [
                "Microsoft.Storage.BlobCreated"
            ]
        }
    }
}
英文:
  • The error occurs because you have given incorrect file name (or) name of a file that does not exist.

  • Since you are trying to use blob created event trigger to find the column count, you can use the procedure below:

  • After configuring the get metadata activity, create a storage event trigger. Go to Add trigger -&gt; choose trigger -&gt; Create new.

Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

  • Click on continue. You will get a Trigger Run Parameters tab. In this, give the value as @triggerBody().fileName.

Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

  • Complete the trigger creation and publish the pipeline. Now whenever the file is uploaded into your container (on top of which you created storage event trigger), it will trigger the pipeline automatically (no need to debug). If the container is empty and you try to debug by giving some value for sourceFile parameter, it would give the same error.
  • Upload a sample file to your container. It will trigger the pipeline and give the desired result.

Azure Data Factory Get Metadata activity在获取列数时返回”(404) not found”错误。

The following is the trigger JSON that I created for my container:

{
    &quot;name&quot;: &quot;trigger1&quot;,
    &quot;properties&quot;: {
        &quot;annotations&quot;: [],
        &quot;runtimeState&quot;: &quot;Started&quot;,
        &quot;pipelines&quot;: [
            {
                &quot;pipelineReference&quot;: {
                    &quot;referenceName&quot;: &quot;pipeline1&quot;,
                    &quot;type&quot;: &quot;PipelineReference&quot;
                },
                &quot;parameters&quot;: {
                    &quot;sourceFile&quot;: &quot;@triggerBody().fileName&quot;
                }
            }
        ],
        &quot;type&quot;: &quot;BlobEventsTrigger&quot;,
        &quot;typeProperties&quot;: {
            &quot;blobPathBeginsWith&quot;: &quot;/data/blobs/&quot;,
            &quot;blobPathEndsWith&quot;: &quot;.csv&quot;,
            &quot;ignoreEmptyBlobs&quot;: true,
            &quot;scope&quot;: &quot;/subscriptions/b83c1ed3-c5b6-44fb-b5ba-2b83a074c23f/resourceGroups/&lt;user&gt;/providers/Microsoft.Storage/storageAccounts/blb1402&quot;,
            &quot;events&quot;: [
                &quot;Microsoft.Storage.BlobCreated&quot;
            ]
        }
    }
}

huangapple
  • 本文由 发表于 2023年2月14日 06:06:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/75441660.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定