英文:
Why does my redis cache return only part of the data
问题
这个函数实现了使用Redis缓存的Postgres。当我发出get请求时,第一个结果返回了所有的数据,但是当我发出下一个请求时,数据中的某些字段丢失了。
这是我的User结构体:
type User struct {
ID uuid.UUID `json:"id"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
OtherName string `json:"other_name"`
Username string `json:"user_name"`
Password string `json:"password"`
Email string `json:"email"`
ProfileImage string `json:"profile_image"`
Status string `json:"status"`
IsAdmin bool `json:"is_admin"`
Role string `json:"role"`
Gender string `json:"gender"`
PhoneNumber string `json:"phone_number"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
请注意,这只是代码的翻译部分,没有其他内容。
英文:
This function checks implements Postgres with Redis Cache
when I make the get request, with the first result all the data is returned, but when I make the next one certain fields of the data is missing
func (usr *UserImplementation) GetAllUsers(ctx context.Context) ([]models.User, error) {
cachedUsers, err := databaseapi.Redis_CacheDB_Api()
if err != nil {
return nil, fmt.Errorf("error connecting to Redis cache: %v", err)
}
// pipe := cachedUsers.TxPipeline()
cachedData, err := cachedUsers.Get(ctx, "users").Result()
if err != nil && err != redis.Nil {
return nil, fmt.Errorf("error retrieving cached data: %v", err)
}
if cachedData != "" {
// Data found in the cache, return the cached data
var cachedUsers []models.User
err := json.Unmarshal([]byte(cachedData), &cachedUsers)
if err != nil {
return nil, fmt.Errorf("error unmarshaling cached data: %v", err)
}
return cachedUsers, nil
}
users, err := usr.pg.Postgres_DB_Api().DB.GetAllUsers(ctx)
if err != nil {
return nil, fmt.Errorf("error: %v", err.Error())
}
cacheData, err := json.Marshal(users)
if err != nil {
return nil, fmt.Errorf("error marshaling data for caching: %v", err)
}
expiration := time.Hour
err = cachedUsers.Set(ctx, "users", string(cacheData), expiration).Err()
if err != nil {
return nil, fmt.Errorf("error caching data: %v", err)
}
return models.DatabaseUsersToUsers(users), nil
}
This is my User Struct: type User struct {
ID uuid.UUID json:"id"
FirstName string json:"first_name"
LastName string json:"last_name"
OtherName string json:"other_name"
Username string json:"user_name"
Password string json:"password"
Email string json:"email"
ProfileImage string json:"profile_image"
Status string json:"status"
IsAdmin bool json:"is_admin"
Role string json:"role"
Gender string json:"gender"
PhoneNumber string json:"phone_number"
CreatedAt time.Time json:"created_at"
UpdatedAt time.Time json:"updated_at"
}
答案1
得分: 0
我整理了一个小例子来帮助你。作为前提,我简化了你的例子,只提供了这里重要的部分。由于程序的某些部分没有被分享,我不得不做一些猜测。如果这个例子对你没有用,只需告诉我缺少什么,我会更新我的回答。让我们从我用来在本地使用Docker运行Postgres/Redis的命令开始。
设置
我使用的命令是:
docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest
现在,让我们转到代码部分。
程序
package main
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/redis/go-redis/v9"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type user struct {
ID int `json:"id"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
// 1. 实例化客户端
dsn := "host=localhost port=54322 user=postgres password=postgres"
db, err := gorm.Open(postgres.Open(dsn))
if err != nil {
panic(err)
}
redisClient := redis.NewClient(&redis.Options{
Addr: ":6379",
Password: "",
DB: 0,
})
// 2. 自动迁移对象并填充虚拟数据
db.AutoMigrate(&user{})
db.Create(&user{ID: 1, FirstName: "John", LastName: "Doe"})
db.Create(&user{ID: 2, FirstName: "Suzy", LastName: "White"})
// 3. 尝试从缓存中获取数据
var users []user
cachedUsers, err := redisClient.Get(context.Background(), "users").Result()
if err != nil && err != redis.Nil {
panic(fmt.Errorf("err retrieving cached data: %v", err))
}
if cachedUsers != "" {
if err := json.Unmarshal([]byte(cachedUsers), &users); err != nil {
panic(fmt.Errorf("err while unmarshaling data: %v", err))
}
fmt.Println("users taken from Redis")
for _, v := range users {
fmt.Println(v)
}
return
}
// 4. 从数据库中获取数据
if err := db.Model(&user{}).Find(&users).Error; err != nil {
panic(fmt.Errorf("err while retrieving from DB: %v", err))
}
// 5. 将数据存入缓存
rawUsers, err := json.Marshal(users)
if err != nil {
panic(fmt.Errorf("err while marshaling users: %v", err))
}
if err := redisClient.Set(context.Background(), "users", rawUsers, time.Minute*15).Err(); err != nil {
panic(fmt.Errorf("err while setting key in cache: %v", err))
}
fmt.Println("users taken from DB")
for _, v := range users {
fmt.Println(v)
}
}
让我们仔细看一下每个部分(通过编号的注释进行划分):
- 在客户端初始化中没有太多要说的。我们初始化了指向本地实例的客户端。
- 然后,我们使用一些虚拟数据设置了数据库。
- 我们尝试从Redis实例中获取数据。如果找到了数据,我们将其打印出来并终止程序。
- 如果在缓存中没有找到数据,那么我们从数据库中获取数据。
- 最后,如果我们从数据库中获取了数据,可以安全地假设我们也应该将它们放入缓存中。
在这两种情况下,我们获取了相同数量和相同字段的数据。因此,如果你按照这个例子并根据你的模型和项目类型进行调整(我的不是一个Web项目),你应该没问题。如果你仍然遇到问题,请告诉我。谢谢!
英文:
I put together a small example to try to help you. As a premise, I simplified your example just to provide what matters here. As some parts of the program haven't been shared, I had to do some guesses. If the example is not useful, just let me know what's missing and I'll update my answer. Let's start with the commands I used to run Postgres/Redis locally with Docker.
The setup
The commands I used were:
docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest
Now, let's switch to the code.
The program
package main
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/redis/go-redis/v9"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type user struct {
ID int `json:"id"`
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
// 1. instantiate clients
dsn := "host=localhost port=54322 user=postgres password=postgres"
db, err := gorm.Open(postgres.Open(dsn))
if err != nil {
panic(err)
}
redisClient := redis.NewClient(&redis.Options{
Addr: ":6379",
Password: "",
DB: 0,
})
// 2. automigrate objects & seed dummy data
db.AutoMigrate(&user{})
db.Create(&user{ID: 1, FirstName: "John", LastName: "Doe"})
db.Create(&user{ID: 2, FirstName: "Suzy", LastName: "White"})
// 3. attempt to retrieve from cache
var users []user
cachedUsers, err := redisClient.Get(context.Background(), "users").Result()
if err != nil && err != redis.Nil {
panic(fmt.Errorf("err retrieving cached data: %v", err))
}
if cachedUsers != "" {
if err := json.Unmarshal([]byte(cachedUsers), &users); err != nil {
panic(fmt.Errorf("err while unmarshaling data: %v", err))
}
fmt.Println("users taken from Redis")
for _, v := range users {
fmt.Println(v)
}
return
}
// 4. retrieve from the DB
if err := db.Model(&user{}).Find(&users).Error; err != nil {
panic(fmt.Errorf("err while retrieving from DB: %v", err))
}
// 5. set the key within the cache
rawUsers, err := json.Marshal(users)
if err != nil {
panic(fmt.Errorf("err while marshaling users: %v", err))
}
if err := redisClient.Set(context.Background(), "users", rawUsers, time.Minute*15).Err(); err != nil {
panic(fmt.Errorf("err while setting key in cache: %v", err))
}
fmt.Println("users taken from DB")
for _, v := range users {
fmt.Println(v)
}
}
Let's take a closer look at each section (divided by numbered comments):
- In the client initialization there isn't much to say. We initialized clients pointing toward our local instances
- Then, we set the DB up with some dummy data
- We try to get data from the Redis instance. If we find them, we print them out and we terminate the program
- If no data are found in the cache, then we get them from the DB
- Finally, if we get from the DB, it's safe to assume that we should put them into the cache as well
In both cases, we get the same amount of data with the same fields. So if you stick to this example and adjust it to your models and kind of project (mine is not a web project) you should be good. In case you still face issues, let me know. Thanks!
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论