无效的上传请求

huangapple go评论93阅读模式
英文:

Invalid Upload Request

问题

当我尝试使用Google API Client libraries for Go将数据上传到BigQuery时,我得到了以下响应。

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "badRequest",
    "message": "Invalid Upload Request"
   }
  ],
  "code": 400,
  "message": "Invalid Upload Request"
 }
}

我的作业描述如下:

j := &bigquery.Job{
	Configuration: &bigquery.JobConfiguration{
		Load: &bigquery.JobConfigurationLoad{
			DestinationTable: &bigquery.TableReference{
				projectId,
				"xyz",
				name + "_" + yyyymmdd,
			},
			SkipLeadingRows: 1,
			FieldDelimiter:  "|",
			MaxBadRecords:   3,
			Schema: &bigquery.TableSchema{
				[]*bigquery.TableFieldSchema{
					{Name: "f1", Type: "STRING"},
					{Name: "f2", Type: "STRING"},
					{Name: "f3", Type: "STRING"},
					{Name: "f4", Type: "STRING"},
					{Name: "f5", Type: "STRING"},
					{Name: "f6", Type: "STRING"},
					{Name: "f7", Type: "STRING"},
					{Name: "f8", Type: "STRING"},
					{Name: "f9", Type: "STRING"},
					{Name: "f10", Type: "STRING"},
					{Name: "f11", Type: "STRING"},
					{Name: "f12", Type: "STRING"},
					{Name: "f13", Type: "STRING"},
					{Name: "f14", Type: "STRING"},
					{Name: "f15", Type: "STRING"},
					{Name: "f16", Type: "STRING"},
					{Name: "f17", Type: "STRING"},
				},
			},
		},
	},
}

值得一提的是,使用这个库的其他调用都正常工作,所以我排除了oauth问题等。

编辑:我在请求发送之前捕获了请求。请注意,这是在http.Client能够添加oauth头之前。实际请求很大,所以我缩短了它并删除了标识符。

POST /upload/bigquery/v2/projects/.../jobs HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5
Content-Length: 56369074
Content-Type: multipart/related; boundary=2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: application/json

{"configuration":{"load":{"destinationTable":{"projectId":"...","datasetId":"...","tableId":"..."},"skipLeadingRows":1,"fieldDelimiter":"|","maxBadRecords":3,"schema":{"fields":[{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"}]}}}}

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: text/plain; charset=utf-8

H0|H1|H2|H3|H4|H5|H6|H7|H8|H9|H10|H11|H12|H13|H14|H15|H16
f0_0|f0_1|f0_2|f0_3|f0_4|f0_5|f0_6|f0_7|f0_8|f0_9|f0_10|f0_11|f0_12|f0_13|f0_14|f0_15|f0_16
f1_0|f1_1|f1_2|f1_3|f1_4|f1_5|f1_6|f1_7|f1_8|f1_9|f1_10|f1_11|f1_12|f1_13|f1_14|f1_15|f1_16
# Three lines
# of comments
# in the trailer

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf--
英文:

I'm getting the following response, when trying to upload data to BigQuery using the Google API Client libraries for Go.

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "badRequest",
    "message": "Invalid Upload Request"
   }
  ],
  "code": 400,
  "message": "Invalid Upload Request"
 }
}

My job descriptor looks like this:

j := &bigquery.Job{
	Configuration: &bigquery.JobConfiguration{
		Load: &bigquery.JobConfigurationLoad{
			DestinationTable: &bigquery.TableReference{
				projectId,
				"xyz",
				name + "_" + yyyymmdd,
			},
			SkipLeadingRows: 1,
			FieldDelimiter:  "|",
			MaxBadRecords:   3,
			Schema: &bigquery.TableSchema{
				[]*bigquery.TableFieldSchema{
					{Name: "f1", Type: "STRING"},
					{Name: "f2", Type: "STRING"},
					{Name: "f3", Type: "STRING"},
					{Name: "f4", Type: "STRING"},
					{Name: "f5", Type: "STRING"},
					{Name: "f6", Type: "STRING"},
					{Name: "f7", Type: "STRING"},
					{Name: "f8", Type: "STRING"},
					{Name: "f9", Type: "STRING"},
					{Name: "f10", Type: "STRING"},
					{Name: "f11", Type: "STRING"},
					{Name: "f12", Type: "STRING"},
					{Name: "f13", Type: "STRING"},
					{Name: "f14", Type: "STRING"},
					{Name: "f15", Type: "STRING"},
					{Name: "f16", Type: "STRING"},
					{Name: "f17", Type: "STRING"},
				},
			},
		},
	},
}

FWIW, other calls using this library work fine, so I have ruled out oauth problems, etc.

EDIT: I captured the request before it went out. Please note that this is before the http.Client is able to add the oauth headers. The actual request is large, so I have shortened it and removed the identifiers.

POST /upload/bigquery/v2/projects/.../jobs HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5
Content-Length: 56369074
Content-Type: multipart/related; boundary=2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: application/json

{"configuration":{"load":{"destinationTable":{"projectId":"...","datasetId":"...","tableId":"..."},"skipLeadingRows":1,"fieldDelimiter":"|","maxBadRecords":3,"schema":{"fields":[{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"}]}}}}

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: text/plain; charset=utf-8

H0|H1|H2|H3|H4|H5|H6|H7|H8|H9|H10|H11|H12|H13|H14|H15|H16
f0_0|f0_1|f0_2|f0_3|f0_4|f0_5|f0_6|f0_7|f0_8|f0_9|f0_10|f0_11|f0_12|f0_13|f0_14|f0_15|f0_16
f1_0|f1_1|f1_2|f1_3|f1_4|f1_5|f1_6|f1_7|f1_8|f1_9|f1_10|f1_11|f1_12|f1_13|f1_14|f1_15|f1_16
# Three lines
# of comments
# in the trailer

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf--

答案1

得分: 1

这是由于库中的一个错误导致的,该错误没有发送所需的“uploadType”查询参数。请参见这些差异,以获取修复该错误的补丁。

然而,这暴露了库中的另一个错误,我目前正在与作者一起修复。当修复完成后,我会更新这个答案。

编辑:截至此补丁,上述引用的错误已经修复。

英文:

It turns out that this was caused by a bug in the library that was not sending the required "uploadType" query parameter. See these diffs for a patch that fixes the bug.

However this exposed another bug in the library that I am currently working with the author to fix. I will update this answer when that fix is in.

EDIT: As of this patch, the bug referenced above is fixed.

答案2

得分: 0

让我先说明一点,我不是Go专家。
话虽如此,看起来问题出在你上传的数据上。看起来你正在进行媒体上传请求,因为在作业配置中没有提供源Google Storage路径,并且你发送的错误来自媒体上传器。

然而,你没有展示对JobInsertCall.Media的调用 - 你是如何设置Media字段的?你能验证它是有效的数据,或者尝试使用虚拟数据吗?

英文:

Let me start with the caveat that I'm not a Go expert.
That said, it looks like the problem is with the data that you're uploading. It looks like you're doing a media upload request, since you're not providing a source Google Storage path in the job configuration, and the error you sent comes from the media uploader.

However, you don't show the call to JobInsertCall.Media -- how are you setting the Media field? Can you verify that it is valid data, or try with dummy data?

huangapple
  • 本文由 发表于 2012年5月10日 03:33:59
  • 转载请务必保留本文链接:https://go.coder-hub.com/10522892.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定