英文:
JOLT: How to convert datetimestamp to epoch on dynamic json keys
问题
My Input :
{
"updated_on": "2023-03-14 03:56:22",
"created_on": "2023-03-14 03:56:22"
}
My Jolt Spec :
[
{
"operation": "modify-overwrite-beta",
"spec": {
"assign": "${assign:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"updated_on": "${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"i_date": "${i_date:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"created_on": "${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
}
}
]
Desired Output :
{
"updated_on": "1678766182",
"created_on": "1678766182"
}
But i am still not understand why i am getting error whenever any one or two of datetimestamp field missing out of four
英文:
[Edited] I have a use case, where i have to convert the datetimestamp fields to epoch but the fields are not standardized as in some flowfiles we may not receive the datetime fields, My conversion is working perfectly using the EvaluateJsonPath and JoltTransformation JSON.
My Input :
{
"updated_on": "2023-03-14 03:56:22",
"created_on": "2023-03-14 03:56:22"
}
My Jolt Spec :
[
{
"operation": "modify-overwrite-beta",
"spec": {
"assign": "${assign:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"updated_on": "${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"i_date": "${i_date:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"created_on": "${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
}
}
]
Desired Output :
{
"updated_on": "1678766182",
"created_on": "1678766182"
}
But i am still not understand why i am getting error whenever any one or two of datetimestamp field missing out of four
答案1
得分: 0
让我们重新复制您的案例;
通过添加四个流程:
GetFile:
-
输入目录 - [已定义](示例:C:\ApacheNifi\App\JSON)
其中有一个名为flowfile II的flowfile.json -
保留源文件 - true
EvaluateJSONPath:
- 目标 - flowfile-attribute // 用于在下一个(Jolt)转换中单独使用
带有以下属性:
属性 值
created_on $.created_on
updated_on $.updated_on
- 路径未找到行为 - 跳过 => 如果相关属性(在此情况下是updated_on)在flowfile.json文件中不存在,那么属性名称将存在,但值为**""(例如*{"created_on":"1678755382000","updated_on":""}***)。
JoltTransformJSON:
- 规范:
[
{
"operation": "modify-overwrite-beta", // 用这个替换"默认"很重要,以便能够覆盖现有属性的值
"spec": {
"updated_on":"${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"created_on":"${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
}
}
]
- LogAttribute
除了LogAttribute,每个流程都运行一次。
结果是,您可以从最后一个连接上下文菜单的List queue中获得与您为flowfile II定义的期望结果类似的结果。
英文:
Let' reproduce your case;
By adding four processes :
GetFile :
-
Input Directory - [defined] ( example : C:\ApacheNifi\App\JSON )
where there's a flowfile.json with flowfile II as the content -
Keep Source File - true
EvaluateJSONPath :
-
Destination - flowfile-attribute //to be individually used within the next(Jolt) transformation
with added properties :
Property Value ----------- ------------- created_on $.created_on updated_on $.updated_on
-
Path not found behavior - skip => there will be the key name of the attribute while the value is "" (eg. {"created_on":"1678755382000","updated_on":""}) if the related attribute ( in this case updated_on ) doesn't exist within the flowfile.json file.
JoltTransformJSON :
- Specification :
[
{
"operation": "modify-overwrite-beta", // replacing "default" with this is important in order to
// be able to override the values of the existing attributes
"spec": {
"updated_on":"${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
"created_on":"${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
}
}
]
- LogAttribute
Run once each process except for LogAttribute
As a result, you can get a result similar to the one you've defined as the desired result for the flowfile II from the List queue in the last Connection's context menu.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论