多小时的 @transaction.atomic Django 后台进程应该关注哪些问题?

huangapple go评论70阅读模式
英文:

What concerns should I have for a multiple hour long @transaction.atomic django background process?

问题

我有一个运行时间可能超过5小时的Celery后台任务,主要用于创建新对象,大约会生成约10万个新对象。如果在此过程中出现问题,我希望能够将其全部撤销,就好像从未发生过一样。我确保这一点的策略是将后台任务包装在@transaction.atomic中。由于这是一个如此长时间的事务,有没有什么特别需要关注的地方?如果在4小时后发生故障,我是否需要担心我的Postgres数据库在生产环境中会冻结?还有其他方法我应该采用,或者transaction.atomic是最佳策略吗?

我目前将其运行在@transaction.atomic块中。在开发环境中,它目前运行正常,但我担心它可能会在生产环境中引发问题。

英文:

I have a celery background task that runs and can take 5+ hours to complete. It is primarily creating new objects and it will generate around 100k or so new objects. If something were to go wrong during this I would like to roll it all back as if it never happened. My strategy for ensuring this is to wrap the background task in a @transaction.atomic. Since this is such a long running transaction is there anything I should be particularly concerned about? If I happen to fail after 4 hours do I need to worry about my Postgres database freezing up in production? Is there another way I should be doing this or is transaction.atomic the best strategy?

I currently have it running in a @transaction.atomic block. It currently works fine in development but I have concerns that it may cause issues in production.

答案1

得分: 1

以下是翻译好的内容:

有各种可能会引发的问题,但它是否会引发其中任何问题取决于我们没有的许多细节。

如果它回滚,它不会导致 PostgreSQL 简单地冻结。只有当其他“自然”查询或 VACUUM 遇到某个特定的数据时,该数据将被逐步回滚。需要检查某个数据片段的查询不会因等待其他 99,999 个对象被清理而冻结。

但另一个事务想要插入或更新一个元组,这将导致与大事务所做的某些内容发生约束违反的情况,将被阻塞,直到大事务完成。

英文:

There are a variety of problems it might cause, but whether it does cause any of them depends on a lot of details we don't have.

If it rolls back, it won't cause PostgreSQL to simply freeze up. Any particular piece of data will be rolled back incrementally when it is encountered--either by other "natural" queries, or by VACUUM. A query needing to inspect one piece of data won't freeze waiting for 99,999 other objects to also get cleaned up.

But another transaction wanting to insert or update a tuple which would cause a constraint violation against something the large transaction has done will block until the large transaction finishes.

huangapple
  • 本文由 发表于 2023年4月17日 23:14:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/76036659.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定