BigQuery Golang查询的maxResults参数

huangapple go评论93阅读模式
英文:

BigQuery Golang maxResults for Queries

问题

我想在使用golang的BigQuery库时能够指定maxResults。不过目前还不清楚如何实现这一点。在文档中没有看到这个选项,并且我已经浏览了源代码,但只看到一些似乎与查询无关的零星使用。有没有办法绕过这个问题?

英文:

I would like to be able to specify maxResults when using the golang BigQuery library. It isn't clear how to do this, though. I don't see it as an option in the documentation, and I have browsed the source to try to find it but I only see some sporadic usage in seemingly functionality not related to queries. Is there a way to circumvent this issue?

答案1

得分: 1

我认为SDK中没有实现这个方法,但是在查找了一下后,我找到了这个方法:request

你可以尝试执行一个HTTP GET请求,并指定参数(你可以在这里找到使用参数的示例:query_parameters

英文:

I think there is no implemented method in the SDK for that but after looking a bit, I found this one: request

You could try to execute an HTTP GET specifying the parameters (you can find an example of the use of parameters here: query_parameters)

答案2

得分: 0

默认情况下,Google API迭代器会为您管理页面大小。RowIterator默认返回一行数据,内部支持由后端选择适当大小的获取页面。

然而,如果您想指定一个固定的最大页面大小,您可以使用google.golang.org/api/iterator包来指定特定大小进行分页迭代。在这种情况下,大小对应于BigQuery查询API的maxResults

有关高级迭代器用法的更多一般信息,请参阅https://github.com/googleapis/google-cloud-go/wiki/Iterator-Guidelines。

下面是一个快速测试,演示了在bigquery中使用RowIterator进行分页迭代的示例。它执行一个查询,返回十月份的每一天的一行数据:

func TestQueryPager(t *testing.T) {
	ctx := context.Background()

	pageSize := 5

	client, err := bigquery.NewClient(ctx, "your-project-id here")
	if err != nil {
		t.Fatal(err)
	}
	defer client.Close()

	q := client.Query("SELECT * FROM UNNEST(GENERATE_DATE_ARRAY('2022-10-01','2022-10-31', INTERVAL 1 DAY)) as d")
	it, err := q.Read(ctx)
	if err != nil {
		t.Fatalf("query failure: %v", err)
	}
	pager := iterator.NewPager(it, pageSize, "")
	var fetchedPages int
	for {
		var rows [][]bigquery.Value
		nextToken, err := pager.NextPage(&rows)
		if err != nil {
			t.Fatalf("NextPage: %v", err)
		}
		fetchedPages = fetchedPages + 1
		if len(rows) > pageSize {
			t.Errorf("page size exceeded, got %d want %d", len(rows), pageSize)
		}
		t.Logf("(next token %s) page size: %d", nextToken, len(rows))
		if nextToken == "" {
			break
		}
	}

	wantPages := 7
	if fetchedPages != wantPages {
		t.Fatalf("fetched %d pages, wanted %d pages", fetchedPages, wantPages)
	}
}

希望对您有所帮助!

英文:

By default the google API iterators manage page size for you. The RowIterator returns a single row by default, backed internally by fetched pages that rely on the backend to select an appropriate size.

If however you want to specify a fixed max page size, you can use the google.golang.org/api/iterator package to iterate by pages while specifying a specific size. The size, in this case, corresponds to maxResults for BigQuery's query APIs.

See https://github.com/googleapis/google-cloud-go/wiki/Iterator-Guidelines for more general information about advanced iterator usage.

Here's a quick test to demonstrate with the RowIterator in bigquery. It executes a query that returns a row for each day in October, and iterates by page:

func TestQueryPager(t *testing.T) {
	ctx := context.Background()

	pageSize := 5

	client, err := bigquery.NewClient(ctx, "your-project-id here")
	if err != nil {
		t.Fatal(err)
	}
	defer client.Close()

	q := client.Query("SELECT * FROM UNNEST(GENERATE_DATE_ARRAY('2022-10-01','2022-10-31', INTERVAL 1 DAY)) as d")
	it, err := q.Read(ctx)
	if err != nil {
		t.Fatalf("query failure: %v", err)
	}
	pager := iterator.NewPager(it, pageSize, "")
	var fetchedPages int
	for {
		var rows [][]bigquery.Value
		nextToken, err := pager.NextPage(&rows)
		if err != nil {
			t.Fatalf("NextPage: %v", err)
		}
		fetchedPages = fetchedPages + 1
		if len(rows) > pageSize {
			t.Errorf("page size exceeded, got %d want %d", len(rows), pageSize)
		}
		t.Logf("(next token %s) page size: %d", nextToken, len(rows))
		if nextToken == "" {
			break
		}
	}

	wantPages := 7
	if fetchedPages != wantPages {
		t.Fatalf("fetched %d pages, wanted %d pages", fetchedPages, wantPages)
	}
}

huangapple
  • 本文由 发表于 2022年10月24日 20:08:41
  • 转载请务必保留本文链接:https://go.coder-hub.com/74180891.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定