英文:
schema error when using fs.createWriteStream to write data to bigquery (node.js)
问题
错误:作业或表上未指定模式
。
不知道为什么会发生这个错误。代码来自文档。我还尝试了以下不同格式的方式,如 fs.createWriteStream({sourceFormat: "json"})
,但结果仍然是相同的错误。
const { BigQuery } = require("@google-cloud/bigquery");
const bigquery = new BigQuery();
const dataset = bigquery.dataset("firebase_test_data");
const table = dataset.table("flattened_data");
const fs = require("fs");
fs.createReadStream("./data.json")
.pipe(table.createWriteStream("json"))
.on("job", (job) => {
// `job` 是一个可用于检查请求状态的 Job 对象。
console.log(job);
})
.on("complete", (job) => {
// 作业已成功完成。
});
英文:
Error : No schema specified on job or table
.
No idea why this error is happening. The code is from documentation. I have also tried following a different format such fs.createWriteStream({sourceFormat: "json"}) - but it results in the same error.
const { BigQuery } = require("@google-cloud/bigquery");
const bigquery = new BigQuery();
const dataset = bigquery.dataset("firebase_test_data");
const table = dataset.table("flattened_data");
const fs = require("fs");
fs.createReadStream("./data.json")
.pipe(table.createWriteStream("json"))
.on("job", (job) => {
// `job` is a Job object that can be used to check the status of the
// request.
console.log(job);
})
.on("complete", (job) => {
// The job has completed successfully.
});
答案1
得分: 0
在const table = dataset.table("flattened_data");
中定义的表格由于没有正确的模式而导致出现此错误,而您正在传递给data.json的模式是不合适的。
根据Google文档尝试了以下代码,并在BigQuery中指定了表格的模式,成功将数据加载到表格中。
const { BigQuery } = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const dataset = bigquery.dataset('my-dataset');
const table = dataset.table('my-table');
//-
// 从JSON文件加载数据。
//-
const fs = require('fs');
fs.createReadStream('/path-to-json/data.json')
.pipe(table.createWriteStream('json'))
.on('job', (job) => {
// `job`是一个Job对象,可用于检查请求的状态。
})
.on('complete', (job) => {
// 作业已成功完成。
});
<details>
<summary>英文:</summary>
was getting this error since the table defined in `const table = dataset.table("flattened_data");` doesn’t have the appropriate schema which you are passing in data.json.
tried the following code as per the [Google documentation][1] and specified the schema of table in BigQuery which successfully loads data into the table.
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const dataset = bigquery.dataset('my-dataset');
const table = dataset.table('my-table');
//-
// Load data from a JSON file.
//-
const fs = require('fs');
fs.createReadStream('/path-to-json/data.json')
.pipe(table.createWriteStream('json'))
.on('job', (job) => {
// `job` is a Job object that can be used to check the status of the
// request.
})
.on('complete', (job) => {
// The job has completed successfully.
});
[1]: https://googleapis.dev/nodejs/bigquery/latest/Table.html#createWriteStream-examples
</details>
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论