英文:
Possible alternative to cron job based schedulers
问题
我们正在多个Java微服务中处理来自数据库的数据集(也称为项目)。我们获得一个包含所有这些项目的任务。我们的服务将这些项目作为kafka事件处理。一个项目在服务之间是一个事件。
每个服务对这些项目执行一种特定的逻辑。在一个特定的服务中,逻辑是等待所有项目被消耗完毕,然后通过基于cron作业的调度程序触发业务逻辑。
如何移除这个调度程序,也许用一些更异步的东西来代替?
我提出了创建一个生产者的方法,该生产者创建一个事件,表示任务中所有项目的消耗已完成,以便逻辑可以继续。还有其他的想法吗?
英文:
We are processing data sets(aka item) from database in multiple Java micro services. We get a task which has all these items. Our services process these items as kafka event. One item is one event between services.
Each service performs a said logic on the items. In one particular service, the logic is waiting for all items to get consumed and then trigger a business logic through a cron job based scheduler.
How can I remove this scheduler and maybe implement something more async in place?
I came up with the approach of creating a producer that creates an event signifying the consumption of all items for a task so that the logic can follow. Any more ideas?
答案1
得分: 1
你的想法很好。我假设你在所有项目都被消耗时创建一个事件,并通过某个队列(或Kafka主题)发送它,你的服务监听这个事件。我认为这是一个非常好的、可靠的解决方案。
但是,我不确定你正在寻找什么其他想法。
- 一个选项是使用Quartz企业作业调度器。
- 另一个选项类似于Spring提供的Cron调度器,但比Cron表达式更容易阅读。它包含了由我编写和维护的开源MgntUtils库。这里是Java文档页面,描述了这个想法。该库可以作为maven工件获取,或者在GitHub上获取,包括源代码和Java文档。此外,源代码包含一个可以立即运行的示例,演示如何使用它(请查看
com.mgnt.lifecycle.management.backgroundrunner.example.BackgroundRunnerUsageExample
类)。
英文:
Your idea is good. I assume that you create an event upon when all items are consumed and send it through some queue (or Kafka topic) where your service listens. I think this is very good and sound solution <br>
However, I am not sure what type of other ideas you are looking for.
- One option would be to use Quartz enterprise Job scheduler
- The other one is similar to Cron scheduler that Spring provides, but it is more humanly readable compare to corn expression. It comes with Open Source MgntUtils library written and maintained by me. Here is Javadoc page that describes the idea. The library can be obtained as a maven artifact, or on GitHub, with source code and Javadoc. Also source code includes working runnable out of the box example on how to use it. (See class
com.mgnt.lifecycle.management.backgroundrunner.example.BackgroundRunnerUsageExample
)
答案2
得分: 0
你可以尝试在Kafka Streams API中创建一个KTable,并实现一个Punctuator来迭代该表并处理它所见到的所有事件。
英文:
You could try creating a KTable in Kafka Streams API and implement a Punctuator that'll iterate the table and process all events it has seen
答案3
得分: 0
org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler
怎么样?
Runnable task = createRunnable(s2);
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.initialize();
scheduler.setPoolSize(2);
scheduler.schedule(task, date);
英文:
What about org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler
?
Runnable task = createRunnable(s2);
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.initialize();
scheduler.setPoolSize(2);
scheduler.schedule(task, date);
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论