模拟 SQL 查询 Golang

huangapple go评论82阅读模式
英文:

Mock sql query Golang

问题

我有一个函数:

func (db *DBSqlx) GetRefreshToken(oldToken string, tx *sqlx.Tx) (string, error) {
    var refreshToken string

    err := db.TemplateGet(
        tx,
        &refreshToken,
        `query`,
        oldToken,
    )

    if err != nil {
        return "", errors.Wrap(err, "failed to query refresh token for the specified token")
    }

    return refreshToken, err
}

如何使用模拟响应编写此函数的测试?

DB 的类型为:

type DBSqlx struct {
    PgConn    *sqlx.DB
    PgConnCtx context.Context
}

我尝试编写了以下代码,但我不明白如何正确使用该包。

db, mock, err := sqlmock.New()
if err != nil {
    t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
defer db.Close()

mock.ExpectQuery("query").WillReturnRows()
英文:

I have a function :

    func (db *DBSqlx) GetRefreshToken(oldToken string, tx *sqlx.Tx) (string, error) {
	var refreshToken string

	err := db.TemplateGet(
		tx,
		&refreshToken,
		`query`,
		oldToken,
	)

	if err != nil {
		return "", errors.Wrap(err, "не удалось запросить рефреш токен для указанного токена")
	}

	return refreshToken, err
}

How to write a test for this function with mock response?

DB has type :

    type DBSqlx struct {
    PgConn    *sqlx.DB
    PgConnCtx context.Context
}

I tried to write this code. But I don't understand how to use the package correctly.

db, mock, err := sqlmock.New()
if err != nil {
	t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
defer db.Close()

mock.ExpectQuery("query").WillReturnRows()

答案1

得分: 0

你可以使用类似https://github.com/DATA-DOG/go-sqlmock的工具来模拟数据库查询,并通过对数据库的响应进行各种控制。

但需要注意的是,在测试中模拟数据库查询通常不被认为是良好的测试实践,因为它很可能测试的是你程序的实际实现而不是其行为。

英文:

You could use something like https://github.com/DATA-DOG/go-sqlmock to mock database queries with all kinds of control over responses by the database.

Though it should be noted, that mocking database queries in tests is generally not considered good testing practice because it is most likely testing the actual implementation of your program and not it's behavior.

答案2

得分: 0

你可以使用一些包装器来抽象存储和底层数据库处理程序(以便与纯数据库和事务一起使用),然后用另一个存根接口替换它。甚至不需要将额外的库包含到你的代码库中。

你应该注意真实数据库中的潜在序列化问题、NULL 值等,并使用 https://github.com/ory/dockertest 进行一些使用真实数据的集成测试。

但对于简单的情况,包装器就足够了。

// 根包
type TokenStorage interface {
	GetToken(ctx context.Context, oldToken string) (string, error)
}

// yourtestfile_test.go
type TokenStorageStub struct {}

func (t *TokenStorageStub) GetToken(ctx context.Context, oldToken string) (string, error) {
	b := make([]byte, 16)
	n, err := rand.Read(b)
	if n != len(b) || err != nil {
		return "", fmt.Errorf("无法成功从系统 CSPRNG 中读取。")
	}
	return hex.EncodeToString(b), nil
}

// postgres || mysql || sqlite 包

// TokenStorage 使用 postgres 实现根接口。
type TokenStorage struct {
	db ExtendedDB
}

func NewTokenStorage(db ExtendedDB) (*TokenStorage, error) {
	if db == nil {
		return nil, errors.New("提供的 db 句柄为空")
	}
	return &TokenStorage{db: db}, nil
}

func (s *TokenStorage) GetToken(ctx context.Context, oldToken string) (string, error) {
	const query = `SELECT ...`
	var token string
	if err := s.db.QueryRowContext(ctx, query, oldToken).Scan(&token); err != nil {
		if errors.Is(err, sql.ErrNoRows) {
			return nil, rootpkg.ErrTokenNotFound
		}
		return nil, fmt.Errorf("postgres: 使用旧令牌获取令牌时出现问题: %w", err)
	}
	return token, nil
}

// Queryer 是用于选择查询的接口。
type Queryer interface {
	QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
	QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
}

// Execer 是用于执行查询的接口。
type Execer interface {
	ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
}

// ExtendedDB 是一个可以使用上下文进行查询和执行的联合接口。
type ExtendedDB interface {
	Queryer
	Execer
}
英文:

You can abstract your storage and underlying db handle (for making it useful with pure db and tx) using some wrapper and then substitute it with another stub interface. It does not even need to include additional libraries to your codebase.

You should keep in mind potential serialization issues with real database, NULL values and etc, adding some intergration testing with real data using https://github.com/ory/dockertest

But for simple cases wrapper is enough

// root package 
type TokenStorage interface {
	GetToken(ctx context.Context, oldToken string) (string, error)
}

// yourtestfile_test.go
type TokenStorageStub struct {}

func (t *TokenStorageStub) GetToken(ctx context.Context, oldToken string) (string, error) {
	b := make([]byte, 16)
	n, err := rand.Read(b)
	if n != len(b) || err != nil {
		return "", fmt.Errorf("could not successfully read from the system CSPRNG.")
	}
	return hex.EncodeToString(b), nil
}

// postgres || mysql || sqlite package 

// TokenStorage impelements root interface using postgres. 
type TokenStorage struct {
	db ExtendedDB 
}

func NewTokenStorage(db ExtendedDB) (*TokenStorage, error) {
	if db == nil {
		return nil, errors.New("provided db handle is nil")
	}
	return &TokenStorage{db: db}, nil 
}

func (s *TokenStorage) GetToken(ctx context.Context, oldToken string) (string, error) {
	const query = `SELECT ...`
	var token string 
	if err := s.db.QueryRowContext(ctx, query, oldToken).Scan(&token); err != nil {
		if errors.Is(err, sql.ErrNoRows) {
			return nil, rootpkg.ErrTokenNotFound
		}
		return nil, fmt.Errorf("postgres: problem while get token using old token: %w", err)
	}
	return token, nil
}

// Queryer is an interface used for selection queries.
type Queryer interface {
	QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
	QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
}

// Execer is an interface used for executing queries.
type Execer interface {
	ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
}

// ExtendedDB is a union interface which can query, and exec, with Context.
type ExtendedDB interface {
	Queryer
	Execer
}

答案3

得分: 0

这是关于现有大多数 Golang 中的 SQL 客户端存在的问题,它们并不真正被设计成可模拟的,所以我们通常使用的方式(使用接口进行解耦)无法起作用。

更明确地说,为什么使用接口进行解耦,然后对接口进行模拟不起作用的原因是,Query 和 Exec 函数返回的是具有私有属性的结构体,你无法更改这些属性。

例如,如果你创建了这样一个接口:

type MyDB interface{
    Exec(query string, args ...any) (sql.Result, error)
    Query(query string, args ...any) (*sql.Rows, error)
}

你可以像这样编写一个模拟:

type MyDBMock struct{
    // ...
}

func (m MyDBMock) Exec(query string, args ...any) (sql.Result, error) {
    // ... 进行一些模拟操作 ...
    return sql.Result{
        // 这里不需要做任何操作,因为所有属性都是私有的
    }
}

func (m MyDBMock) Query(query string, args ...any) (*sql.Rows, error) {
    // ... 进行一些模拟操作 ...
    return &sql.Rows{
        // 这里不需要做任何操作,因为所有属性都是私有的
    }
}

但是由于你无法设置 sql.Rowssql.Results 的值,只能返回这两个结构体的零值,这在编写测试时几乎不是你所需要的。

database/sql 最常用的模拟技术

另一种选择是模拟用于连接数据库的整个驱动程序,这需要付出很多工作但收益很少。

幸运的是,正如其他答案中提到的 https://github.com/DATA-DOG/go-sqlmock 库所做的那样,它在这方面做得很好。实际上,这是我所知道的使用现有的 sql 库进行模拟的唯一方法。

或许不模拟是一个更好的选择

在编写测试时,模拟数据库请求被广泛认为是一种不好的做法,因为了解你的 SQL 代码是否有效的最佳/唯一方法是将其发送到真实的数据库实例并在那里运行。

因此,模拟可能会导致你在测试中得到错误的结果,然后在生产环境中遇到问题。

如果你想采用这种方法,有一种有趣的技术可以使用 https://github.com/ory/dockertest,你可以直接从 Golang 代码中启动一个 Docker 容器内的数据库实例,然后在测试结束时停止该容器。

这样可以让你获得大部分单元测试的好处,同时实际上允许你的查询在真实的数据库中运行。这是我测试数据库的首选方法。

这里有一个我在 postgres 中的示例 here,我实际上在 here 中为每个最大的 SQL DBMS 都有一个示例,如果有帮助的话。

使用实际帮助你进行模拟的库

> 免责声明:在本节中,我提到了我自己编写的一个库

有些情况下,你可能希望模拟数据库调用,以便可以在更小更快的测试中测试你的业务逻辑,然后在其他较慢的测试中测试你的实际查询。虽然这种情况并不常见,但我确实为 Go 编写了一个非常好的 SQL 库,除了其他功能外,还提供了一个开箱即用的模拟功能,所以这也是一个选择:

https://github.com/vingarcia/ksql

这里解释了如何使用该库的内置模拟功能:

https://github.com/VinGarcia/ksql/wiki/Testing-Tools-and-ksql.Mock

英文:

This is a problem with most existing sql clients in Golang, they were not really created to be mockable so writing a mock as we usually do (using an interface to decouple from it) doesn't work.

To be more clear about exactly why using an interface to decouple and then mocking the interface doesn't work the reason is because the Query and Exec functions return structs with private attributes that you cannot change.

So for example if you create an interface like this one:

type MyDB interface{
    Exec(query string, args ...any) (sql.Result, error)
    Query(query string, args ...any) (*sql.Rows, error)
}

You can write a mock for it like this:

type MyDBMock struct{
    // ...
}

func (m MyDBMock) Exec(query string, args ...any) (sql.Result, error) {
    // ... do some mocky thing ...
    return sql.Result{
        // nothing to do here, because all atributes are private
    }
}

func (m MyDBMock) Query(query string, args ...any) (*sql.Rows, error) {
    // ... do some mocky thing ...
    return &sql.Rows{
        // nothing to do here, because all atributes are private
    }
}

But since you can't set the values of sql.Rows nor sql.Results it is only possible to return the zero values for these two structs which is hardly what you need if you are writing tests.

The most used mocking technique for database/sql

The alternative is to mock the whole driver that was used to connect to the database which is a lot of work for little profit.

Fortunately that's exactly what the https://github.com/DATA-DOG/go-sqlmock library that was mentioned in other answers does, and it does a good job at that. And it's actually the only way I know how to do it with the existing sql libraries.

Maybe not mocking is a better idea

It is widely considered a bad practice to mock database requests when writing tests, because the best/only way of knowing if you SQL code is working is to send it to a real instance of the database and run it there.

So mocking it might just cause you to have a false positive on your tests and then have problems on production.

If you want to go this route, there is an interesting technique using the https://github.com/ory/dockertest where you can just start your database instance inside a docker container directly from the Golang code, and then stop this machine when the test is over.

This gives you most of the benefits of a unit test but actually allows you to have your queries running in a real database. This is my favorite approach for testing the database.

There is an example of how I do it for postgres here and I actually have one example for each of the biggest SQL DBMSs in here if that helps.

Using a library that actually helps you to mock it

> Disclaimer: I am referring to a library I wrote myself in this section

There are some situations where you might want to mock your database calls just so you can test your business logic in smaller faster tests and then have other slower tests testing your actual queries. This is not often the case but I did write a very good sql library for Go that among many other things also provides a mock you can out of the box, so that is also an option:

https://github.com/vingarcia/ksql

And here its explained how to use the built-in mock of this library:

https://github.com/VinGarcia/ksql/wiki/Testing-Tools-and-ksql.Mock

huangapple
  • 本文由 发表于 2022年8月23日 15:24:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/73454661.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定