英文:
Getting error while trying to Import data from MongoDB to ElasticSearch using Nodejs
问题
我正在尝试从MongoDB导入小型文档到ElasticSearch,但出现错误:
{
"index": {
"_index": "impact-fulltext",
"_id": "t2oLkoUBwNXsTufYzszL",
"status": 400,
"error": {
"type": "mapper_parsing_exception",
"reason": "failed to parse field [_id] of type [_id] in document with id 't2oLkoUBwNXsTufYzszL'. Preview of field's value: '605315a3b4f719d00f69f2d3'",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
}
}
}
}
我感到困惑,因为我已经定义了_id
,但仍然出现错误。
db.collection("article_beta")
.find()
.limit(100)
.toArray((err, docs) => {
if (err) throw err;
esClient.bulk(
{
body: docs.flatMap((doc) => [
{
index: {
_index: "impact-fulltext",
_id: doc._id.$oid,
},
},
doc,
]),
},
(err, resp) => {
if (err) throw err;
console.log(resp);
client.close();
}
);
});
英文:
I am trying to import small documents from MongoDB to ElasticSearch but getting an error
{
"index": {
"_index": "impact-fulltext",
"_id": "t2oLkoUBwNXsTufYzszL",
"status": 400,
"error": {
"type": "mapper_parsing_exception",
"reason": "failed to parse field [_id] of type [_id] in document with id \u0027t2oLkoUBwNXsTufYzszL\u0027. Preview of field\u0027s value: \u0027605315a3b4f719d00f69f2d3\u0027",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
}
}
}
}
I am clueless as I also defined the _id
but still getting the error.
db.collection("article_beta")
.find()
.limit(100)
.toArray((err, docs) => {
if (err) throw err;
esClient.bulk(
{
body: docs.flatMap((doc) => [
{
index: {
_index: "impact-fulltext",
_id: doc._id.$oid,
},
},
doc,
]),
},
(err, resp) => {
if (err) throw err;
console.log(resp);
client.close();
}
);
});
答案1
得分: 2
在Elasticsearch中,文档包含一些元信息和文档的源数据。因此,文档的结构如下:
{
"_index": "index-name",
"_id": "document-id-stored-by-elasticsearch",
"_source": {
// 文档本身
}
}
因此,在文档内部,您不能像下面这样使用_id
字段,因为_id
字段是保留字段:
POST myindex/_doc
{
"_id": "123123",
"field1": "value1"
}
所以这个请求会产生以下错误:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [_id] of type [_id] in document with id '16Y6koUBnxYxC21BP-tS'. Preview of field's value: '123123'",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
}
},
"status": 400
}
在您的示例中,doc
变量的值来自MongoDB,其中有一个字段名为_id
。您需要将该字段的名称更改为id
。您可以将以下代码放在flatMap
函数内的匿名函数中:
db.collection("article_beta")
.find()
.limit(100)
.toArray((err, docs) => {
if (err) throw err;
esClient.bulk(
{
body: docs.flatMap((doc) => {
doc.id = doc._id // <---- 添加的代码
delete doc._id // <---- 添加的代码
return [
{
index: {
_index: "impact-fulltext",
_id: doc.id.$oid,
},
},
doc,
]
}),
},
(err, resp) => {
if (err) throw err;
console.log(resp);
client.close();
}
);
});
英文:
In elasticsearch, the documents have some meta information and also the source of the documents. So, the document structure is following :
{
"_index": "index-name",
"_id": "document-id-stored-by-elasticsearch",
"_source": {
// documents itself
}
}
So, inside the document, you can not use _id
field as follow because _id
field is reserved field :
POST myindex/_doc
{
"_id": "123123",
"field1": "value1"
}
So this request will give an error as follow :
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [_id] of type [_id] in document with id '16Y6koUBnxYxC21BP-tS'. Preview of field's value: '123123'",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
}
},
"status": 400
}
In your example, doc
variable which the value coming from Mongodb have a field as _id
. You need to change the name of that field as id
. You can put following lines inside of your anonymous function inside the flatMap function.
db.collection("article_beta")
.find()
.limit(100)
.toArray((err, docs) => {
if (err) throw err;
esClient.bulk(
{
body: docs.flatMap((doc) => {
doc.id = doc._id // <---- added
delete doc._id // <---- added
return [
{
index: {
_index: "impact-fulltext",
_id: doc.id.$oid,
},
},
doc,
]
}),
},
(err, resp) => {
if (err) throw err;
console.log(resp);
client.close();
}
);
});
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论