处理在测试分页查询取消时的不精确时间

huangapple go评论70阅读模式
英文:

Handling imprecise timing when testing cancellation of paged query

问题

我有一个对象,用于进行分页SQL查询,并允许异步运行查询:

type PagedQuery[T any] struct {
	Results   chan []*T
	Errors    chan error
	Done      chan error
	Quit      chan error
	client    *sql.DB
}

func NewPagedQuery[T any](client *sql.DB) *PagedQuery[T] {
	return &PagedQuery[T]{
		Results:   make(chan []*T, 1),
		Errors:    make(chan error, 1),
		Done:      make(chan error, 1),
		Quit:      make(chan error, 1),
		client:    client,
	}
}

func (paged *PagedQuery[T]) requestAsync(ctx context.Context, queries ...*Query) {

	conn, err := client.Conn(ctx)
	if err != nil {
		paged.Errors <- err
		return
	}

	defer func() {
		conn.Close()
		paged.Done <- nil
	}()

	for i, query := range queries {
		select {
		case <-ctx.Done():
			return
		case <-paged.Quit:
            return
		default:
		}

		rows, err := conn.QueryContext(ctx, query.String, query.Arguments...)
		if err != nil {
			paged.Errors <- err
			return
		}

		data, err := sql.ReadRows[T](rows)
		if err != nil {
			paged.Errors <- err
			return
		}

		paged.Results <- data
	}
}

我正在尝试测试这段代码,特别是取消部分。我的测试代码如下:

svc, mock := createServiceMock("TEST_DATABASE", "TEST_SCHEMA")

mock.ExpectQuery(regexp.QuoteMeta("TEST QUERY")).
	WithArgs(...).
	WillReturnRows(mock.NewRows([]string{"t", "v", "o", "c", "h", "l", "vw", "n"}))

ctx, cancel := context.WithCancel(context.Background())
go svc.requestAsync(ctx, query1, query2, query3, query4)

time.Sleep(50 * time.Millisecond)
cancel()

results := make([]data, 0)
loop:
for {
    select {
    case <-query.Done:
		break loop
    case err := <-query.Errors:
		Expect(err).ShouldNot(HaveOccurred())
	case r := <-query.Results:
		results = append(results, r...)
    }
}

Expect(results).Should(BeEmpty())
Expect(mock.ExpectationsWereMet()).ShouldNot(HaveOccurred()) // fails here

我遇到的问题是,这个测试有时会在我标注的那一行失败,因为当调用cancel()时,执行不一定在检查<-ctx.Done<-Quitswitch语句处。执行可能在循环中的任何位置,直到我将结果发送到Results通道为止。然而,这是没有意义的,因为执行应该阻塞,直到我从Results通道接收到数据,而我在调用cancel()之后才这样做。此外,我依赖于用于SQL测试的sqlmock包,该包不允许对SQL查询进行任何模糊检查。为什么会出现这个失败,我该如何修复它?

英文:

I have an object I'm using to make paged SQL queries that allows for the queries to be run asynchronously:

type PagedQuery[T any] struct {
Results   chan []*T
Errors    chan error
Done      chan error
Quit      chan error
client    *sql.DB
}
func NewPagedQuery[T any](client *sql.DB) *PagedQuery[T] {
return &amp;PagedQuery[T]{
Results:   make(chan []*T, 1),
Errors:    make(chan error, 1),
Done:      make(chan error, 1),
Quit:      make(chan error, 1),
client:    client,
}
}
func (paged *PagedQuery[T]) requestAsync(ctx context.Context, queries ...*Query) {
conn, err := client.Conn(ctx)
if err != nil {
paged.Errors &lt;- err
return
}
defer func() {
conn.Close()
paged.Done &lt;- nil
}()
for i, query := range queries {
select {
case &lt;-ctx.Done():
return
case &lt;-paged.Quit:
return
default:
}
rows, err := conn.QueryContext(ctx, query.String, query.Arguments...)
if err != nil {
paged.Errors &lt;- err
return
}
data, err := sql.ReadRows[T](rows)
if err != nil {
paged.Errors &lt;- err
return
}
paged.Results &lt;- data
}
}

I'm trying to test this code, specifically the cancellation part. My test code looks like this:

svc, mock := createServiceMock(&quot;TEST_DATABASE&quot;, &quot;TEST_SCHEMA&quot;)
mock.ExpectQuery(regexp.QuoteMeta(&quot;TEST QUERY&quot;)).
WithArgs(...).
WillReturnRows(mock.NewRows([]string{&quot;t&quot;, &quot;v&quot;, &quot;o&quot;, &quot;c&quot;, &quot;h&quot;, &quot;l&quot;, &quot;vw&quot;, &quot;n&quot;}))
ctx, cancel := context.WithCancel(context.Background())
go svc.requestAsync(ctx, query1, query2, query3, query4)
time.Sleep(50 * time.Millisecond)
cancel()
results := make([]data, 0)
loop:
for {
select {
case &lt;-query.Done:
break loop
case err := &lt;-query.Errors:
Expect(err).ShouldNot(HaveOccurred())
case r := &lt;-query.Results:
results = append(results, r...)
}
}
Expect(results).Should(BeEmpty())
Expect(mock.ExpectationsWereMet()).ShouldNot(HaveOccurred()) // fails here

The issue I'm running into is that this test fails occaisionally at the line indicated by my comment, because when cancel() is called, execution isn't guaranteed to be at the switch statement where I check for &lt;-ctx.Done or &lt;-Quit. Execution could be anywhere in the loop up to where I send the results to the Results channel. Except that doesn't make sense because execution should block until I receive data from the Results channel, which I don't do until after I call cancel(). Furthermore, I'm relying on the sqlmock package for SQL testing which doesn't allow for any sort of fuzzy checking where SQL queries are concerned. Why am I getting this failure and how can I fix it?

答案1

得分: 1

我的问题是由于我对Go通道的理解不足导致的。我以为通过创建一个chan([]*T, 1),意味着当通道满时(即包含一个元素时)会阻塞,但事实并非如此。实际上,阻塞发生在我尝试在通道的缓冲区满时发送数据时。所以,通过像这样修改Results

func NewPagedQuery[T any](client *sql.DB) *PagedQuery[T] {
    return &PagedQuery[T]{
        Results:   make(chan []*T),    // 在这里移除缓冲区
        Errors:    make(chan error, 1),
        Done:      make(chan error, 1),
        Quit:      make(chan error, 1),
        client:    client,
    }
}

我可以确保通道在接收到其中的数据之前会阻塞。这一个改变修复了所有测试中的问题。

英文:

My issue was the result of my own lack of understanding around Go channels. I assumed that, by creating a chan([]*T, 1) meant that the channel would block when it was full (i.e. when it contained a single item) but that is not the case. Rather, the block occurs when I attempted to send to the channel when its buffer was full. So, by modifying Results like this:

func NewPagedQuery[T any](client *sql.DB) *PagedQuery[T] {
return &amp;PagedQuery[T]{
Results:   make(chan []*T),    // Remove buffer here
Errors:    make(chan error, 1),
Done:      make(chan error, 1),
Quit:      make(chan error, 1),
client:    client,
}
}

I can ensure that the channel blocks until the data it contains is received. This one change fixed all the problems with testing.

huangapple
  • 本文由 发表于 2022年12月14日 08:57:11
  • 转载请务必保留本文链接:https://go.coder-hub.com/74792530.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定