英文:
How to serve the old data while updating it in Redis with high availability?
问题
我有一个Spring Boot应用程序,定期更新Redis中的哈希数据,并通过REST API公开这些数据。哈希包含大约800-900,000条记录。
每次迭代,我都在Redis事务中使用MULTI/EXEC
删除键并再次插入它们。
这里的关键问题是,插入数据需要大约25-30秒,而我需要在这25-30秒内提供旧数据。
目前,我们正在使用以下命令来更新数据:
MULTI
DEL "key"
HSET "key" "x" "value"
HSET "key" "y" "value"
..
..
EXEC
如你所知,MULTI
后的所有命令都会排队在Redis中,一旦执行EXEC
命令,所有排队的命令都会执行。
不同的是,我认为下面的算法对于可用性可能更好:
MULTI
HSET "key_temp" "x" "value"
HSET "key_temp" "y" "value"
..
..
DEL "key"
RENAMENX "key_temp" "key"
EXEC
在解决方案1中,执行EXEC
命令期间,我首先删除了键,在这段时间内(25秒),API无法提供旧数据。
在解决方案2中,插入新数据的同时将提供旧数据,只有在执行DELETE
和RENAMENX
命令时,数据可能不可用,这可能需要1-2毫秒。
我想知道你的反馈,以及是否有更好的解决方案来解决这个问题?谢谢。
英文:
I have a spring boot application that periodically updates hash data in redis and exposes this data via a REST API. Hash contains about 800-900k records.
Every iteration I'm deleting the key and inserting them again in a redis transaction with multi/exec.
The key point here is that the availability, inserting data takes about 25-30 seconds and I need to serve the old data during that 25-30 seconds.
Currently we are using the commands below to update the data;
MULTI
DEL "key"
HSET "key" "x" "value"
HSET "key" "y" "value"
..
..
EXEC
As you know, all commands after multi are queued in redis and once the EXEC command is executed, all queued commands are executed.
Differently, I thought that the below algorithm would be better for availability;
MULTI
HSET "key_temp" "x" "value"
HSET "key_temp" "y" "value"
..
..
DELETE "key"
RENAMENX "key_temp" "key"
EXEC
In solution 1, during exec command execution, I am deleting the key first, and in that time (25 seconds), API is not able to serve the old data.
In solution 2, the old data will be served while inserting new one, and the data only will not be available in DELETE and RENAMENX command execution which probably takes 1-2 ms.
I wanted to know your feedback and if there is better solution for this problem?
Thank you.
答案1
得分: 0
Option2 看起来不错。
有一些可以改进的地方:
- 不需要在事务中包括 HSET
HSET "key_temp" "x" "value"
HSET "key_temp" "y" "value"
..
..
DELETE "key"
MULTI
DEL "key"
RENAMENX "key_temp" "key"
EXEC
- 在同一个 HSET 中更新多个字段
HSET "key_temp" "x" "value" "y" "value"
英文:
Option2 looks good.
Few things you can improve:
- no need to include the HSET in the transaction
HSET "key_temp" "x" "value"
HSET "key_temp" "y" "value"
..
..
DELETE "key"
MULTI
DEL "key"
RENAMENX "key_temp" "key"
EXEC
- Update multi fields in the same HSET
HSET "key_temp" "x" "value" "y" "value"
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论