英文:
Why are some fields ignores despite clear mapping instructions?
问题
我有一个具有严格映射的索引。有一些字段定义如下：
"created_at": {
"type": "date",
"format": "MMM D, YYYY @ HH:mm:ss"
},
在 _reindex 脚本中，我将一些字段的值从源数值纳秒值（例如 1647846210839081472）转换为毫秒，以便将其识别为日期，像这样：
if (ctx._source.created_at != null) {
ctx._source.created_at = ctx._source.created_at / 1000000;
}
在 Kibana 中，这看起来大部分正常：类型设置为日期，它们在结果中正确显示，但是这些字段不能用于筛选！
查看文档 JSON，我注意到每个文档都有这个块：
"_ignored": [
"created_at",
"updated_at",
***其他类似的日期字段**
],
我该如何让 Elasticsearch 或 Kibana 将这些字段视为完整的成员？
英文:
I have an index with strict mapping. There are some fields defined like this:
"created_at": {
"type": "date",
"format": "MMM D, YYYY @ HH:mm:ss"
},
In the _reindex script, I convert some field values from the source numerical nanosecond values e.g. 1647846210839081472 into milliseconds so it can be recognized as a date, like this:
if (ctx._source.created_at != null) {
ctx._source.created_at = ctx._source.created_at / 1000000;
}
This looks mostly OK in Kibana: they type is set to Date, they are shown correctly in the results, except these fields can't be used for filters!
Looking at the document JSON, I notice this block on each:
"_ignored": [
"created_at",
"updated_at",
***OTHER SIMILAR DATE FIELDS**
],
How can I make ES or Kibana treat those fields as full members of the club?
答案1
得分: 1
字段格式错误（由于严格映射），并且ignore_malformed设置为true，因此被忽略。
我考虑了两种解决方案：
1- 更新您的映射以接受数值纳秒值。
您可以使用date_nanos类型来避免转换您的字段并接受原始值）。
2- 在创建索引时将ignore_malformed设置为false：不推荐，查看此处的文档和解释。
英文:
The field was malformed (due to the strict mapping) and ignore_malformed is set to true that's why it was ignored.
I'm thinking of two solutions:
1- update your mapping in order to accept the numerical nanosecond values.
you can use date_nanos type to avoid converting your field and accept the origin value).
2- set ignore_malformed to false when creating your index: not recommended, check the documentation and explanation here.
答案2
得分: 0
感谢@Val的建议，我发现ES和Kibana对映射中的"format": "MMM D, YYYY @ HH:mm:ss"感到困惑，这与来自_script的毫秒不匹配。
我期望ES会自动解释date并使用格式进行显示。我错了。在从映射中移除format后，ES和Kibana能够使用原始数据工作。
英文:
Thanks to @Val's suggestion, I figured ES and Kibana were confused by the "format": "MMM D, YYYY @ HH:mm:ss" in the mapping, which was not matching the milliseconds coming from the _script.
My expectation was that ES would interpret the date automatically and use the format for display. I was wrong. After removing the format from the mapping, ES and Kibana were able to work with the data as is.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论