如何在Azure数据工厂的输出中包含字段的值

huangapple go评论52阅读模式
英文:

How to include the value of field in an Output with Azure Data Factory

问题

我想在输出湖中的数据时包含字段的值。

目前以下内容将输出以下文件格式

tablename_yyyyyMMdd_yyyyMMddhhmmss.csv

@concat(
pipeline().parameters.TableName,
'',
formatDateTime(utcNow(),'yyyyMMdd'),
'
',
formatDateTime(utcNow(),'yyyyMMddhhmmss'), '.csv'
)

我不再需要第一个formatDateTime(utcNow()函数。

相反,我想用特定字段名(例如first_name)的值替换它。因此,可能会变成以下样子:

@concat(
pipeline().parameters.TableName,
'',
first_name),
'
',
formatDateTime(utcNow(),'yyyyMMddhhmmss'), '.csv'
)

虽然我确信上面是不正确的,但希望你明白我的意思。

有什么想法?

我应该指出该字段将只对所有行具有单个值,例如如果有100行,且first_name是jonathan,则对所有100行都是相同的。

所以我需要帮助以获得以下输出
tablename_jonathan_yyyyMMddhhmmss.csv

我刚想到了一个主意...

是否可能将FirstName连接到TableName参数,如下图所示:

所以,实际表的预览如下图所示,
我想按照上面建议的方式将'first_name'添加到输出文件中,以便看起来像以下内容

tablename_jonathan_yyyyMMddhhmmss.csv

通过更新,根据潜在解决方案,我遇到了如图所示的错误

基本上,我遇到了错误:

The output of activity 'Lookup1' can't be referenced since it is either not an ancestor to the current activity or does not exist

英文:

I would like to include the value of a field when outputting data in the lake.

At present the following will out the following file format

tablename_yyyyyMMdd_yyyyMMddhhmmss.csv

@concat(
    pipeline().parameters.TableName,
    '_',
    formatDateTime(utcNow(),'yyyyMMdd'),
    '_',
    formatDateTime(utcNow(),'yyyyMMddhhmmss'), '.csv'
    )

如何在Azure数据工厂的输出中包含字段的值

I no longer need the first formatDateTime(utcNow() function.

Instead I would like to replace it with the value of a particular field name e.g first_name. Therefore, maybe it would change to something like the following:

@concat(
    pipeline().parameters.TableName,
    '_',
    first_name),
    '_',
    formatDateTime(utcNow(),'yyyyMMddhhmmss'), '.csv'
    )

Although, I'm sure the above is incorrect, but hopefully you get what I'm trying to say.

Any thoughts?

I should point out the field will only have a single value for all rows e.g. if there were 100 rows and the first_name was jonathan, it would be same for all 100 rows

So I would be needing help to get the following output
tablename_jonathan_yyyyMMddhhmmss.csv

I've just had a thought...

Would it be possible to concatentate the FirstName to the TableName parameter seen the image below:

如何在Azure数据工厂的输出中包含字段的值

So, a preview of the actual table looks like the image below,
I would like to add the 'first_name' to output file as suggested above, so that it would look like the following

tablename_jonathan_yyyyMMddhhmmss.csv

如何在Azure数据工厂的输出中包含字段的值

By way of an update, following a potential solution I'm getting the error shown in the image

如何在Azure数据工厂的输出中包含字段的值

Basically, I'm getting the error:

The output of activity 'Lookup1' can't be referenced since it is either not an ancestor to the current activity or does not exist

答案1

得分: 1

  • 由于“FirstName”列的表值都相同,并且您希望在接收文件名中使用该值,您可以首先在表上执行查找(仅限第一行)。这将提供如下图所示的数据:

如何在Azure数据工厂的输出中包含字段的值

  • 现在,使用此值与“TableName”参数(我的示例值为“sample1”)来构建所需的接收文件名。以下是我使用的动态内容:
@concat(pipeline().parameters.tableName,'_',activity('Lookup table').output.firstRow.firstName,'_',formatDateTime(utcNow(),'yyyyMMddhhmmss'),'.csv')

如何在Azure数据工厂的输出中包含字段的值

  • 您可以在目标路径中检查写入的文件名是否符合要求:

如何在Azure数据工厂的输出中包含字段的值

英文:
  • Since the table values for FirstName are all same and you want to use that value in the sink file name, you can first use look up (first row only) on the table. This would give data as shown in the below image:

如何在Azure数据工厂的输出中包含字段的值

  • Now, use this value with the TableName parameter (my sample value is sample1) to build the required sink filename. The following is the dynamic content that I used:
@concat(pipeline().parameters.tableName,'_',activity('Lookup table').output.firstRow.firstName,'_',formatDateTime(utcNow(),'yyyyMMddhhmmss'),'.csv')

如何在Azure数据工厂的输出中包含字段的值

  • You can check in your destination path whether the name of the file written matches the requirement:

如何在Azure数据工厂的输出中包含字段的值

huangapple
  • 本文由 发表于 2023年6月14日 23:31:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/76475242.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定