如何使用Apache NIFI处理REST API分页。

huangapple go评论76阅读模式
英文:

How to handle REST API Pagenation with Apache NIFI

问题

有一个RESTAPI端点用于查询在一段时间内修改过的文档。
由于时间跨度可能较大,该查询的响应包含一个类似于number-of-pages、current-page-number的字段。响应还会返回10个文档的详细信息。基本上,页面大小固定为10。

在Python代码中,我可以编写一个循环遍历所有页面号以发送多个请求,以获取所有文档的详细信息,因为我知道number-of-pages在相同的日期范围内不会改变。

在NIFI中如何实现这一点?如果有示例的NIFI模板,那就更好了。

英文:

There is RESTAPI endpoint to query for the documents modified over a period of time.
As the the period could be large, the response for that query has a field like number-of-pages, current-page-numer. It will also return 10 documents detail in that response. Basically the page size is fixed at 10.

In python code, I could write a code to loop through all the page numbers to send multi requests to get all the documents' detail, since I know the number-of-pages and it won't change given the same date range.

How to do that in NIFI? If there is a sample nifi template, that will be even better.

答案1

得分: 1

这是一个循环和设置属性的过程。

示例在此处:

https://github.com/tspannhw/FLiPN-AirQuality-REST/blob/main/AirQuality%20(1).json

英文:

It's a loop and setting attributes

Example here

https://github.com/tspannhw/FLiPN-AirQuality-REST/blob/main/AirQuality%20(1).json

huangapple
  • 本文由 发表于 2023年3月21日 02:58:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/75794268.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定