为什么我的 Redis 缓存只返回部分数据?

huangapple go评论71阅读模式
英文:

Why does my redis cache return only part of the data

问题

这个函数实现了使用Redis缓存的Postgres。当我发出get请求时,第一个结果返回了所有的数据,但是当我发出下一个请求时,数据中的某些字段丢失了。

这是我的User结构体:

type User struct {
	ID           uuid.UUID `json:"id"`
	FirstName    string    `json:"first_name"`
	LastName     string    `json:"last_name"`
	OtherName    string    `json:"other_name"`
	Username     string    `json:"user_name"`
	Password     string    `json:"password"`
	Email        string    `json:"email"`
	ProfileImage string    `json:"profile_image"`
	Status       string    `json:"status"`
	IsAdmin      bool      `json:"is_admin"`
	Role         string    `json:"role"`
	Gender       string    `json:"gender"`
	PhoneNumber  string    `json:"phone_number"`
	CreatedAt    time.Time `json:"created_at"`
	UpdatedAt    time.Time `json:"updated_at"`
}

请注意,这只是代码的翻译部分,没有其他内容。

英文:

This function checks implements Postgres with Redis Cache
when I make the get request, with the first result all the data is returned, but when I make the next one certain fields of the data is missing

func (usr *UserImplementation) GetAllUsers(ctx context.Context) ([]models.User, error) {

	cachedUsers, err := databaseapi.Redis_CacheDB_Api()
	if err != nil {
		return nil, fmt.Errorf("error connecting to Redis cache: %v", err)
	}

	// pipe := cachedUsers.TxPipeline()

	cachedData, err := cachedUsers.Get(ctx, "users").Result()
	if err != nil && err != redis.Nil {
		return nil, fmt.Errorf("error retrieving cached data: %v", err)
	}

	if cachedData != "" {
		// Data found in the cache, return the cached data
		var cachedUsers []models.User
		err := json.Unmarshal([]byte(cachedData), &cachedUsers)
		if err != nil {
			return nil, fmt.Errorf("error unmarshaling cached data: %v", err)
		}
		return cachedUsers, nil
	}

	users, err := usr.pg.Postgres_DB_Api().DB.GetAllUsers(ctx)
	if err != nil {
		return nil, fmt.Errorf("error: %v", err.Error())
	}

	cacheData, err := json.Marshal(users)
	if err != nil {
		return nil, fmt.Errorf("error marshaling data for caching: %v", err)
	}

	expiration := time.Hour

	err = cachedUsers.Set(ctx, "users", string(cacheData), expiration).Err()
	if err != nil {
		return nil, fmt.Errorf("error caching data: %v", err)
	}

	return models.DatabaseUsersToUsers(users), nil

}

This is my User Struct: type User struct {
ID uuid.UUID json:"id"
FirstName string json:"first_name"
LastName string json:"last_name"
OtherName string json:"other_name"
Username string json:"user_name"
Password string json:"password"
Email string json:"email"
ProfileImage string json:"profile_image"
Status string json:"status"
IsAdmin bool json:"is_admin"
Role string json:"role"
Gender string json:"gender"
PhoneNumber string json:"phone_number"
CreatedAt time.Time json:"created_at"
UpdatedAt time.Time json:"updated_at"
}

答案1

得分: 0

我整理了一个小例子来帮助你。作为前提,我简化了你的例子,只提供了这里重要的部分。由于程序的某些部分没有被分享,我不得不做一些猜测。如果这个例子对你没有用,只需告诉我缺少什么,我会更新我的回答。让我们从我用来在本地使用Docker运行Postgres/Redis的命令开始。

设置

我使用的命令是:

  1. docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
  2. docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

现在,让我们转到代码部分。

程序

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"time"

	"github.com/redis/go-redis/v9"
	"gorm.io/driver/postgres"
	"gorm.io/gorm"
)

type user struct {
	ID        int    `json:"id"`
	FirstName string `json:"first_name"`
	LastName  string `json:"last_name"`
}

func main() {
	// 1. 实例化客户端
	dsn := "host=localhost port=54322 user=postgres password=postgres"
	db, err := gorm.Open(postgres.Open(dsn))
	if err != nil {
		panic(err)
	}
	redisClient := redis.NewClient(&redis.Options{
		Addr:     ":6379",
		Password: "",
		DB:       0,
	})

	// 2. 自动迁移对象并填充虚拟数据
	db.AutoMigrate(&user{})
	db.Create(&user{ID: 1, FirstName: "John", LastName: "Doe"})
	db.Create(&user{ID: 2, FirstName: "Suzy", LastName: "White"})

	// 3. 尝试从缓存中获取数据
	var users []user
	cachedUsers, err := redisClient.Get(context.Background(), "users").Result()
	if err != nil && err != redis.Nil {
		panic(fmt.Errorf("err retrieving cached data: %v", err))
	}
	if cachedUsers != "" {
		if err := json.Unmarshal([]byte(cachedUsers), &users); err != nil {
			panic(fmt.Errorf("err while unmarshaling data: %v", err))
		}
		fmt.Println("users taken from Redis")
		for _, v := range users {
			fmt.Println(v)
		}
		return
	}

	// 4. 从数据库中获取数据
	if err := db.Model(&user{}).Find(&users).Error; err != nil {
		panic(fmt.Errorf("err while retrieving from DB: %v", err))
	}

	// 5. 将数据存入缓存
	rawUsers, err := json.Marshal(users)
	if err != nil {
		panic(fmt.Errorf("err while marshaling users: %v", err))
	}
	if err := redisClient.Set(context.Background(), "users", rawUsers, time.Minute*15).Err(); err != nil {
		panic(fmt.Errorf("err while setting key in cache: %v", err))
	}
	fmt.Println("users taken from DB")
	for _, v := range users {
		fmt.Println(v)
	}
}

让我们仔细看一下每个部分(通过编号的注释进行划分):

  1. 在客户端初始化中没有太多要说的。我们初始化了指向本地实例的客户端。
  2. 然后,我们使用一些虚拟数据设置了数据库。
  3. 我们尝试从Redis实例中获取数据。如果找到了数据,我们将其打印出来并终止程序。
  4. 如果在缓存中没有找到数据,那么我们从数据库中获取数据。
  5. 最后,如果我们从数据库中获取了数据,可以安全地假设我们也应该将它们放入缓存中。

在这两种情况下,我们获取了相同数量和相同字段的数据。因此,如果你按照这个例子并根据你的模型和项目类型进行调整(我的不是一个Web项目),你应该没问题。如果你仍然遇到问题,请告诉我。谢谢!

英文:

I put together a small example to try to help you. As a premise, I simplified your example just to provide what matters here. As some parts of the program haven't been shared, I had to do some guesses. If the example is not useful, just let me know what's missing and I'll update my answer. Let's start with the commands I used to run Postgres/Redis locally with Docker.

The setup

The commands I used were:

  1. docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
  2. docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

Now, let's switch to the code.

The program

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"time"

	"github.com/redis/go-redis/v9"
	"gorm.io/driver/postgres"
	"gorm.io/gorm"
)

type user struct {
	ID        int    `json:"id"`
	FirstName string `json:"first_name"`
	LastName  string `json:"last_name"`
}

func main() {
	// 1. instantiate clients
	dsn := "host=localhost port=54322 user=postgres password=postgres"
	db, err := gorm.Open(postgres.Open(dsn))
	if err != nil {
		panic(err)
	}
	redisClient := redis.NewClient(&redis.Options{
		Addr:     ":6379",
		Password: "",
		DB:       0,
	})

	// 2. automigrate objects & seed dummy data
	db.AutoMigrate(&user{})
	db.Create(&user{ID: 1, FirstName: "John", LastName: "Doe"})
	db.Create(&user{ID: 2, FirstName: "Suzy", LastName: "White"})

	// 3. attempt to retrieve from cache
	var users []user
	cachedUsers, err := redisClient.Get(context.Background(), "users").Result()
	if err != nil && err != redis.Nil {
		panic(fmt.Errorf("err retrieving cached data: %v", err))
	}
	if cachedUsers != "" {
		if err := json.Unmarshal([]byte(cachedUsers), &users); err != nil {
			panic(fmt.Errorf("err while unmarshaling data: %v", err))
		}
		fmt.Println("users taken from Redis")
		for _, v := range users {
			fmt.Println(v)
		}
		return
	}

	// 4. retrieve from the DB
	if err := db.Model(&user{}).Find(&users).Error; err != nil {
		panic(fmt.Errorf("err while retrieving from DB: %v", err))
	}

	// 5. set the key within the cache
	rawUsers, err := json.Marshal(users)
	if err != nil {
		panic(fmt.Errorf("err while marshaling users: %v", err))
	}
	if err := redisClient.Set(context.Background(), "users", rawUsers, time.Minute*15).Err(); err != nil {
		panic(fmt.Errorf("err while setting key in cache: %v", err))
	}
	fmt.Println("users taken from DB")
	for _, v := range users {
		fmt.Println(v)
	}
}

Let's take a closer look at each section (divided by numbered comments):

  1. In the client initialization there isn't much to say. We initialized clients pointing toward our local instances
  2. Then, we set the DB up with some dummy data
  3. We try to get data from the Redis instance. If we find them, we print them out and we terminate the program
  4. If no data are found in the cache, then we get them from the DB
  5. Finally, if we get from the DB, it's safe to assume that we should put them into the cache as well

In both cases, we get the same amount of data with the same fields. So if you stick to this example and adjust it to your models and kind of project (mine is not a web project) you should be good. In case you still face issues, let me know. Thanks!

huangapple
  • 本文由 发表于 2023年6月19日 04:57:30
  • 转载请务必保留本文链接:https://go.coder-hub.com/76502503.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定