如何扩展超过1个实例并处理Spring中的定时任务?

huangapple go评论76阅读模式
英文:

How to scale more than 1 instance and deal with scheduled task in spring?

问题

以下是你要的翻译部分:

我每天都在欧洲/巴黎时间上午8点通过Spring Boot向Android和iOS应用程序发送推送通知。

如果我运行多个实例,通知将会被多次发送。我正在考虑将每天的通知存储在数据库中,并进行检查,但我担心它仍然会多次运行,以下是我正在做的:

@Component
public class ScheduledTasks {

    private static final Logger log = LoggerFactory.getLogger(ScheduledTasks.class);

    private static final SimpleDateFormat dateFormat = new SimpleDateFormat("HH:mm:ss");

    @Autowired
    private ExpoPushTokenRepository expoPushTokenRepository;

    @Autowired
    private ExpoPushNotificationService expoPushNotificationService;

    @Autowired
    private MessageSource messageSource;

    // TODO: 如果实例数 > 1,这将运行多次,将发送的通知保存到数据库并防止多次发送。
    @Scheduled(cron = "${cron.promotions.notification}", zone = "Europe/Paris")
    public void sendNewPromotionsNotification() {
        List<ExpoPushToken> expoPushTokenList = expoPushTokenRepository.findAll();
        ArrayList<NotifyRequest> notifyRequestList = new ArrayList<>();
        for (ExpoPushToken expoPushToken : expoPushTokenList) {
            NotifyRequest notifyRequest = new NotifyRequest(
                    expoPushToken.getToken(),
                    "This is a test title",
                    "This is a test subtitle",
                    "This is a test body"
            );
            notifyRequestList.add(notifyRequest);
        }

        expoPushNotificationService.sendPushNotificationToList(notifyRequestList);
        log.info("{} 已向 " + expoPushTokenList.size() + " 位用户发送推送通知", dateFormat.format(new Date()));
    }
}

有谁有关于如何安全地防止这种情况的想法吗?

英文:

I am having a push notifications being send to android and ios application through spring boot every day at 8am Europe/Paris.

If I run multiple instances, the notifications will send multiple times. I am thinking to send every day notifications send on the database, and check them but I am worried it still run multiple times, this is what I am doing:

@Component
public class ScheduledTasks {

    private static final Logger log = LoggerFactory.getLogger(ScheduledTasks.class);

    private static final SimpleDateFormat dateFormat = new SimpleDateFormat(&quot;HH:mm:ss&quot;);

    @Autowired
    private ExpoPushTokenRepository expoPushTokenRepository;

    @Autowired
    private ExpoPushNotificationService expoPushNotificationService;

    @Autowired
    private MessageSource messageSource;

    // TODO: if instances &gt; 1, this will run multiple times, save to database the notifications send and prevent multiple sending.
    @Scheduled(cron = &quot;${cron.promotions.notification}&quot;, zone = &quot;Europe/Paris&quot;)
    public void sendNewPromotionsNotification() {
        List&lt;ExpoPushToken&gt; expoPushTokenList = expoPushTokenRepository.findAll();
        ArrayList&lt;NotifyRequest&gt; notifyRequestList = new ArrayList&lt;&gt;();
        for (ExpoPushToken expoPushToken : expoPushTokenList) {
            NotifyRequest notifyRequest = new NotifyRequest(
                    expoPushToken.getToken(),
                    &quot;This is a test title&quot;,
                    &quot;This is a test subtitle&quot;,
                    &quot;This is a test body&quot;
            );
            notifyRequestList.add(notifyRequest);
        }

        expoPushNotificationService.sendPushNotificationToList(notifyRequestList);
        log.info(&quot;{} Send push notification to &quot; + expoPushTokenList.size() + &quot; userse&quot;, dateFormat.format(new Date()));
    }
}

Does anybody have an idea on how I can prevent that safely?

答案1

得分: 5

Quartz将是我在手头任务中主要与数据库无关的解决方案,但已被排除在外,因此我们将不讨论它。

相反,我们要探讨的解决方案做出以下假设:

  • 使用Postgres &gt;= 9.5(因为我们将使用在Postgresl 9.5中引入的SKIP LOCKED)。
  • 可以运行原生查询。

在这些条件下,我们可以通过以下查询从应用程序的多个实例中检索通知的批次:

SELECT * FROM expo_push_token FOR UPDATE SKIP LOCKED LIMIT 100;

这将从表expo_push_token中检索并锁定最多100条记录。如果应用程序的两个实例同时执行此查询,接收到的结果将是不相交的。100只是一些示例值。我们可能需要微调此值以适应我们的用例。锁定会在当前事务结束之前保持活动状态。

在实例获取了一批通知之后,还必须从表中删除其锁定的条目,或以其他方式标记此条目已被处理(如果我们选择这种方法,则必须修改上述查询以过滤掉已处理的条目),然后关闭当前事务以释放锁定。然后,应用程序的每个实例将重复此查询,直到查询返回零条目为止。

还有一种替代方法:实例首先获取要发送的一批通知,保持与数据库的事务打开状态(从而继续保持对数据库的锁定),发送其通知,然后删除/更新条目并关闭事务。

这两种解决方案具有不同的优缺点:

  • 第一种解决方案使事务保持短暂。但是,如果应用程序在发送通知过程中崩溃,在此运行中未发送的批次部分将丢失。
  • 第二种解决方案保持事务打开,可能会持续很长时间。如果在发送通知过程中崩溃,所有条目都将被解锁,并且其批次将被重新处理,这可能会导致某些通知被发送两次。

为了使这个解决方案工作,我们还需要一种填充表expo_push_token所需数据的作业。此作业应在之前运行,即其执行不应与通知发送过程重叠。

英文:

Quartz would be my mostly database-agnostic solution for the task at hand, but was ruled out, so we are not going to discuss it.

The solution we are going to explore instead makes the following assumptions:

Under this conditions, we can retrieve batches of notifications from multiple instances of the application running through the following query:

SELECT * FROM expo_push_token FOR UPDATE SKIP LOCKED LIMIT 100;

This will retrieve and lock up to 100 entries from the table expo_push_token. If two instances of the application execute this query simultaneously, the received results will be disjoint. 100 is just some sample value. We may want to fine-tune this value for our use case. The locks stay active until the current transaction ends.

After an instance has fetched a batch of notifications, it has to also delete the entries it locked from the table or otherwise mark that this entry has been processed (if we go down this route, we have to modify the query above to filter-out already processed entires) and close the current transaction to release the locks. Each instance of the application would then repeat this query until the query returns zero entries.

There is also an alternative approach: an instance first fetches a batch size of notifications to send, keeps the transaction to the database open (thus continues holding the lock on the database), sends out its notification and then deletes/updates the entries and closes the transactions.

The two solutions have different strengths/weaknesses:

  • the first solutions keeps the transaction short. But if the application crashes in the middle of sending out notificatiosn, the part of its batch that was not send out is lost in this run.
  • the second solution keeps the transaction open, for possibly a long time. If it crashes in the middle fo sending out notifications, all entries will be unlocked and its batch would be re-processed, possibly resulting in some notifications being sent out twice.

For this solution to work, we also need some kind of job that fills table expo_push_token with the data we need. This job should run beforehand, i.e. its execution should not overlap with the notification sending process.

huangapple
  • 本文由 发表于 2020年8月15日 02:51:29
  • 转载请务必保留本文链接:https://go.coder-hub.com/63418536.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定