将datetimestamp转换为动态JSON键上的epoch的方法:

huangapple go评论71阅读模式
英文:

JOLT: How to convert datetimestamp to epoch on dynamic json keys

问题

My Input :

{    
  "updated_on": "2023-03-14 03:56:22",
  "created_on": "2023-03-14 03:56:22"
}

My Jolt Spec :

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "assign": "${assign:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "updated_on": "${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "i_date": "${i_date:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "created_on": "${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
    }
  }
]

Desired Output :

{
  "updated_on": "1678766182",
  "created_on": "1678766182"
}

将datetimestamp转换为动态JSON键上的epoch的方法:

But i am still not understand why i am getting error whenever any one or two of datetimestamp field missing out of four

英文:

[Edited] I have a use case, where i have to convert the datetimestamp fields to epoch but the fields are not standardized as in some flowfiles we may not receive the datetime fields, My conversion is working perfectly using the EvaluateJsonPath and JoltTransformation JSON.

My Input :

{    
  "updated_on": "2023-03-14 03:56:22",
  "created_on": "2023-03-14 03:56:22"
}

My Jolt Spec :

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "assign": "${assign:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "updated_on": "${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "i_date": "${i_date:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "created_on": "${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
    }
  }
]

Desired Output :

{
  "updated_on": "1678766182",
  "created_on": "1678766182"
}

将datetimestamp转换为动态JSON键上的epoch的方法:

But i am still not understand why i am getting error whenever any one or two of datetimestamp field missing out of four

答案1

得分: 0

让我们重新复制您的案例;

通过添加四个流程:

GetFile

  • 输入目录 - [已定义](示例:C:\ApacheNifi\App\JSON
    其中有一个名为flowfile II的flowfile.json

  • 保留源文件 - true

EvaluateJSONPath

  • 目标 - flowfile-attribute // 用于在下一个(Jolt)转换中单独使用

带有以下属性:

属性 值


created_on $.created_on
updated_on $.updated_on

  • 路径未找到行为 - 跳过 => 如果相关属性(在此情况下是updated_on)在flowfile.json文件中不存在,那么属性名称将存在,但值为**""(例如*{"created_on":"1678755382000","updated_on":""}***)。

JoltTransformJSON

  • 规范
[
  {
    "operation": "modify-overwrite-beta", // 用这个替换"默认"很重要,以便能够覆盖现有属性的值
    "spec": {      
      "updated_on":"${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "created_on":"${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"      
    }
  }
]  		
  • LogAttribute

将datetimestamp转换为动态JSON键上的epoch的方法:

除了LogAttribute,每个流程都运行一次。

结果是,您可以从最后一个连接上下文菜单的List queue中获得与您为flowfile II定义的期望结果类似的结果。

英文:

Let' reproduce your case;

By adding four processes :

GetFile :

  • Input Directory - [defined] ( example : C:\ApacheNifi\App\JSON )
    where there's a flowfile.json with flowfile II as the content

  • Keep Source File - true

EvaluateJSONPath :

  • Destination - flowfile-attribute //to be individually used within the next(Jolt) transformation

    with added properties :

       Property    Value
       ----------- -------------
       created_on  $.created_on
       updated_on  $.updated_on
    
  • Path not found behavior - skip => there will be the key name of the attribute while the value is "" (eg. {"created_on":"1678755382000","updated_on":""}) if the related attribute ( in this case updated_on ) doesn't exist within the flowfile.json file.

JoltTransformJSON :

  • Specification :
[
  {
    "operation": "modify-overwrite-beta", // replacing "default" with this is important in order to 
                                          // be able to override the values of the existing attributes
    "spec": {      
      "updated_on":"${updated_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}",
      "created_on":"${created_on:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"      
    }
  }
]  		
  • LogAttribute

将datetimestamp转换为动态JSON键上的epoch的方法:

Run once each process except for LogAttribute

As a result, you can get a result similar to the one you've defined as the desired result for the flowfile II from the List queue in the last Connection's context menu.

huangapple
  • 本文由 发表于 2023年7月13日 19:31:41
  • 转载请务必保留本文链接:https://go.coder-hub.com/76678896.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定