英文:
Spring Quartz Scheduler not working in ECS Cluster
问题
我正在使用 Spring Boot(2.2.7.RELEASE) 和 Quartz Scheduler(2.3.2) 开发调度应用程序。该应用部署在 AWS ECS 集群，并在基于负载的多个 EC2 实例中运行。我正在使用 AWS RDS Mysql 数据库，并创建了所需的 Quartz 表。我已经使用了以下的属性文件来配置 Quartz。
quartz.properties-
org.quartz.scheduler.instanceName = MyClusteredScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 25
org.quartz.threadPool.threadPriority = 5
org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties = true
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000
应用程序启动日志 -
2020-10-13 23:33:02.853 INFO 938 --- [ main] org.quartz.core.QuartzScheduler : 已创建 Quartz Scheduler v.2.3.2。
2020-10-13 23:33:02.856 INFO 938 --- [ main] o.s.s.quartz.LocalDataSourceJobStore : 使用基于数据库表的数据访问锁定（同步）。
2020-10-13 23:33:02.858 INFO 938 --- [ main] o.s.s.quartz.LocalDataSourceJobStore : 已初始化 JobStoreCMT。
2020-10-13 23:33:02.859 INFO 938 --- [ main] org.quartz.core.QuartzScheduler : 调度程序元数据：Quartz Scheduler（v2.3.2）'quartzScheduler'，实例ID为'NON_CLUSTERED'
调度程序类：'org.quartz.core.QuartzScheduler' - 本地运行。
未启动。
当前处于待机模式。
已执行的作业数：0
使用线程池'org.quartz.simpl.SimpleThreadPool' - 共有 10 个线程。
使用作业存储'org.springframework.scheduling.quartz.LocalDataSourceJobStore'
- 支持持久性，并且未进行集群化。
问题是多个 EC2 实例在同一时间内选择了相同的作业。因此导致结果错误。
请问有人能告诉我在这里漏掉了什么吗？我需要做些什么才能使它在 ECS 集群中正常工作？每次启动 Spring 应用程序时，日志消息都显示'未进行集群化'。
英文:
I am working on a scheduling application using Spring Boot(2.2.7.RELEASE), Quartz Scheduler(2.3.2). Application deployed in AWS ECS Cluster and running in multiple EC2 instances based on the load. I am using the AWS RDS Mysql database and created required quartz tables. I have used the following properties file for quartz.
quartz.properties-
org.quartz.scheduler.instanceName = MyClusteredScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 25
org.quartz.threadPool.threadPriority = 5
org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties = true
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.jobStore.clusterCheckinInterval = 20000
Application Starting log -
2020-10-13 23:33:02.853 INFO 938 --- [ main] org.quartz.core.QuartzScheduler : Quartz Scheduler v.2.3.2 created.
2020-10-13 23:33:02.856 INFO 938 --- [ main] o.s.s.quartz.LocalDataSourceJobStore : Using db table-based data access locking (synchronization).
2020-10-13 23:33:02.858 INFO 938 --- [ main] o.s.s.quartz.LocalDataSourceJobStore : JobStoreCMT initialized.
2020-10-13 23:33:02.859 INFO 938 --- [ main] org.quartz.core.QuartzScheduler : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'quartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore'
- which supports persistence. and is not clustered.
Problem is multiple EC2 instances are picking up the same job at same time. Because of that getting the wrong result.
Can somebody please let me know what I am missing here? What I need to do here to make it work in the ECS cluster. Every time when the Spring application is getting started getting, it's showing the log message is not clustered.
答案1
得分: 0
这个问题已经解决。我已经将以下配置细节添加到 application.properties 文件中，并移除了 quartz.properties 文件。
spring.quartz.job-store-type = jdbc
spring.quartz.properties.org.quartz.scheduler.instanceName = MyClusteredScheduler
spring.quartz.properties.org.quartz.jobStore.isClustered = true
spring.quartz.properties.org.quartz.scheduler.instanceId = AUTO
spring.quartz.properties.org.quartz.jobStore.useProperties = true
spring.quartz.properties.org.quartz.jobStore.tablePrefix = QRTZ_
spring.quartz.properties.org.quartz.jobStore.isClustered = true
spring.quartz.properties.org.quartz.jobStore.clusterCheckinInterval = 20000
现在，在 ECS 中批处理作业正常运行，并且在任一时间只有一个作业在 ECS 集群中运行。
英文:
This's been resolved. I have added the following configuration details to the application.properties file and removed the quartz.properties file
spring.quartz.job-store-type = jdbc
spring.quartz.properties.org.quartz.scheduler.instanceName = MyClusteredScheduler
spring.quartz.properties.org.quartz.jobStore.isClustered = true
spring.quartz.properties.org.quartz.scheduler.instanceId = AUTO
spring.quartz.properties.org.quartz.jobStore.useProperties = true
spring.quartz.properties.org.quartz.jobStore.tablePrefix = QRTZ_
spring.quartz.properties.org.quartz.jobStore.isClustered = true
spring.quartz.properties.org.quartz.jobStore.clusterCheckinInterval = 20000
Now the batch job is working fine in ECS and at a time only one job is running in the ECS cluster.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论