英文:
Need to generate JSON file in Azure Data Factory with dynamic Key
问题
以下是翻译好的内容:
我在CSV文件中有以下数据。
ID | userId | Name |
---|---|---|
719A070E-4874-E811-9CCE-02152146006A | 123 | Joe |
5d7d0a74-f1b9-483a-8e9f-c7f45ccfeda0 | 456 | Mike |
现在,我需要使用Azure数据工厂生成以下JSON,
[
{
"719A070E-4874-E811-9CCE-02152146006A":{
"userId":"123",
"Name":"Joe"
}
},
{
"5d7d0a74-f1b9-483a-8e9f-c7f45ccfeda0":{
"userId":"456",
"Name":"Mike"
}
}
]
我已经尝试了所有可能的方法,我能够创建内部JSON,使用用户ID和姓名。但我无法弄清楚如何设置JSON中的动态键,该键必须从CSV中的ID列派生。
英文:
I have a below data in csv file.
ID | userId | Name |
---|---|---|
719A070E-4874-E811-9CCE-02152146006A | 123 | Joe |
5d7d0a74-f1b9-483a-8e9f-c7f45ccfeda0 | 456 | Mike |
Now, I need to generate below json using Azure Data Factory,
[
{
"719A070E-4874-E811-9CCE-02152146006A":{
"userId":"123",
"Name":"Joe"
}
},
{
"5d7d0a74-f1b9-483a-8e9f-c7f45ccfeda0":{
"userId":"456",
"Name":"Mike"
}
}
]
I have tried all the possible ways, I can able to form the inner json using userid and name. But I cannot able to figure out how to set the dynamic KEY in JSON which has to be derived from ID column in csv.
答案1
得分: 1
我可以使用ForEach和附加变量活动来实现您的要求。
我采用了与您的CSV相同的数据并将其提供给了一个查找活动(在查找活动中取消选中“第一行”复选框)。在CSV数据集中,请取消选中“第一行作为标题”的复选框。
这将是查找输出数组:
然后,在变量中创建一个数组变量。创建一个ForEach活动,并为其提供以下表达式(在这里选中“Sequential”复选框):
@skip(activity('Lookup1').output.value, 1)
在ForEach内部,创建一个附加变量活动,其中包含以下表达式:
@json(concat('{"', item().Prop_0, '":{"', activity('Lookup1').output.value[0].Prop_1, '":"', item().Prop_1, '","', activity('Lookup1').output.value[0].Prop_2, '":"', item().Prop_2, '"}}'))
在这里,上述表达式在每次迭代中生成对象并附加到数组变量中(这是结果数组)。
为了显示输出,我使用了另一个数组变量,并在ForEach之后将结果数组的值分配给它。
您可以参考以下是我的管道JSON:
{
"name": "pipeline1",
"properties": {
"activities": [
{
"name": "Lookup1",
"type": "Lookup",
"dependsOn": [],
"policy": {
"timeout": "0.12:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "DelimitedTextSource",
"storeSettings": {
"type": "AzureBlobFSReadSettings",
"recursive": true,
"enablePartitionDiscovery": false
},
"formatSettings": {
"type": "DelimitedTextReadSettings"
}
},
"dataset": {
"referenceName": "sourcecsv",
"type": "DatasetReference"
},
"firstRowOnly": false
}
},
{
"name": "ForEach1",
"type": "ForEach",
"dependsOn": [
{
"activity": "Lookup1",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"items": {
"value": "@skip(activity('Lookup1').output.value, 1)",
"type": "Expression"
},
"isSequential": true,
"activities": [
{
"name": "Append variable1",
"type": "AppendVariable",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"variableName": "arr",
"value": {
"value": "@json(concat('{"', item().Prop_0, '":{"', activity('Lookup1').output.value[0].Prop_1, '":"', item().Prop_1, '","', activity('Lookup1').output.value[0].Prop_2, '":"', item().Prop_2, '"}}'))",
"type": "Expression"
}
}
}
]
}
},
{
"name": "Set variable1",
"type": "SetVariable",
"dependsOn": [
{
"activity": "ForEach1",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"variableName": "newarr",
"value": {
"value": "@variables('arr')",
"type": "Expression"
}
}
}
],
"variables": {
"arr": {
"type": "Array"
},
"newarr": {
"type": "Array"
}
},
"annotations": []
}
}
英文:
I am able to achieve your requirement using ForEach and append variable activities.
I took csv same data like yours and given it to a lookup activity(uncheck the first row
in lookup activity). Here in the csv dataset, uncheck the first row as header
checkbox.
This will be the lookup output array:
Then, in the variables create an array variable. Create a ForEach activity and give the following expression for it(check the Sequential checkbox here).
@skip(activity('Lookup1').output.value, 1)
Inside ForEach, create an append variable activity with following expression in it.
@json(concat('{"',item().Prop_0,'":{"',activity('Lookup1').output.value[0].Prop_1,'":"',item().Prop_1,'","',activity('Lookup1').output.value[0].Prop_2,'":"',item().Prop_2,'"}}'))
Here, the above expression generates the object in each iteration and appends to the array variable(This is result array).
For showing the output, I have used another array variable and assigned the result array value to it after Foreach.
My Pipeline JSON for your reference:
{
"name": "pipeline1",
"properties": {
"activities": [
{
"name": "Lookup1",
"type": "Lookup",
"dependsOn": [],
"policy": {
"timeout": "0.12:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "DelimitedTextSource",
"storeSettings": {
"type": "AzureBlobFSReadSettings",
"recursive": true,
"enablePartitionDiscovery": false
},
"formatSettings": {
"type": "DelimitedTextReadSettings"
}
},
"dataset": {
"referenceName": "sourcecsv",
"type": "DatasetReference"
},
"firstRowOnly": false
}
},
{
"name": "ForEach1",
"type": "ForEach",
"dependsOn": [
{
"activity": "Lookup1",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"items": {
"value": "@skip(activity('Lookup1').output.value, 1)",
"type": "Expression"
},
"isSequential": true,
"activities": [
{
"name": "Append variable1",
"type": "AppendVariable",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"variableName": "arr",
"value": {
"value": "@json(concat('{\"',item().Prop_0,'\":{\"',activity('Lookup1').output.value[0].Prop_1,'\":\"',item().Prop_1,'\",\"',activity('Lookup1').output.value[0].Prop_2,'\":\"',item().Prop_2,'\"}}'))",
"type": "Expression"
}
}
}
]
}
},
{
"name": "Set variable1",
"type": "SetVariable",
"dependsOn": [
{
"activity": "ForEach1",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"variableName": "newarr",
"value": {
"value": "@variables('arr')",
"type": "Expression"
}
}
}
],
"variables": {
"arr": {
"type": "Array"
},
"newarr": {
"type": "Array"
}
},
"annotations": []
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论