英文:
Solr - "Error adding field … msg=Invalid Date String" when sending json data to core
问题
我对Solr还不熟悉。<br>我正在尝试将日志文件传递给Solr。为此，我使用日志文件 -> Filebeat -> Logstash -> Solr。<br><br>**问题**Logstash的输出是正常的，但是Solr保持为空。<br>因为Logstash输出的文本是"2020-01-01 00:00:00.000"，我尝试使用cURL执行Solr更新，例如以下命令。<br><br>第一个命令：（字段"my_datetime"的尾部是Z）curl -X POST -d '{ "add":{ "doc":{ "my_datetime":"2020-01-01T00:00:00.000Z" }}}' -H "Content-Type: application/json" http://localhost:8983/solr/Collection/update?commit=true第二个命令：（字段"my_datetime"的尾部不是Z）curl -X POST -d '{ "add":{ "doc":{ "my_datetime":"2020-01-01 00:00:00.000" }}}' -H "Content-Type: application/json" http://localhost:8983/solr/Collection/update?commit=true<br>**问题**第一个命令可以正常工作。但是第二个命令无法工作，我收到了异常：{"responseHeader":{"status":400,"QTime":13},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"ERROR: [doc=60146d1c-ed31-4dda-b90e-e93537b8a63a] Error adding field 'my_datetime'='2020-01-01 00:00:00.000' msg=Invalid Date String:'2020-01-01 00:00:00.000'","code":400}}<br>**环境**- 在Solr v8.3.0上测试过这是我的managed-schema<fieldType name="pdates" class="solr.DatePointField" docValues="true" multiValued="true"/><field name="my_datetime" type="pdate" indexed="true" stored="true"/>这是我的solrconfig.xml<updateProcessor class="solr.ParseDateFieldUpdateProcessorFactory" name="parse-date"><arr name="format"><str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str><str>yyyy-MM-dd'T'HH:mm:ss,SSS[Z</str><str>yyyy-MM-dd'T'HH:mm:ss.SSS</str>...（其他格式）...<str>[EEE, ]dd MMM yyyy HH:mm[:ss] z</str><str>EEEE, dd-MMM-yy HH:mm:ss z</str><str>EEE MMM ppd HH:mm:ss [z ]yyyy</str></arr></updateProcessor><updateRequestProcessorChain name="add-unknown-fields-to-the-schema" default="${update.autoCreateFields:true}"processor="uuid,remove-blank,field-name-mutating,parse-boolean,parse-long,parse-double,parse-date,add-schema-fields"><processor class="solr.LogUpdateProcessorFactory"/><processor class="solr.DistributedUpdateProcessorFactory"/><processor class="solr.RunUpdateProcessorFactory"/></updateRequestProcessorChain>谢谢您的帮助！
英文:
I am new to Solr.<br>
I am trying to pass log files to Solr. For that I use log file -> Filebeat -> Logstash -> Solr.
<br>
<br>
PROBLEM
Logstash output is fine, but Solr remain empty.<br>
Because Logstash output text is "2020-01-01 00:00:00.000", I have tried to use cURL to perform Solr updates, such as the following commands.
<br>
<br>
First Command: ( the trailing of field "my_datetime" is Z)
curl -X POST -d '{"add":{ "doc":{"my_datetime":"2020-01-01T00:00:00.000Z"}}}' -H "Content-Type: application/json" http://localhost:8983/solr/Collection/update?commit=true
Second Command: ( the trailing of field "my_datetime" is not Z)
curl -X POST -d '{"add":{ "doc":{"my_datetime":"2020-01-01 00:00:00.000"}}}' -H "Content-Type: application/json" http://localhost:8983/solr/Collection/update?commit=true
<br>
QUESTION
First command does work.
But second command does not work, I have received an exception:
{"responseHeader":{"status":400,"QTime":13},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"ERROR: [doc=60146d1c-ed31-4dda-b90e-e93537b8a63a] Error adding field 'my_datetime'='2020-01-01 00:00:00.000' msg=Invalid Date String:'2020-01-01 00:00:00.000'","code":400}}
<br>
ENVIRONMENT
- Tested on Solr v8.3.0
This is my managed-schema
<fieldType name="pdates" class="solr.DatePointField" docValues="true" multiValued="true"/><field name="my_datetime" type="pdate" indexed="true" stored="true"/>
This is my solrconfig.xml
<updateProcessor class="solr.ParseDateFieldUpdateProcessorFactory" name="parse-date"><arr name="format"><str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str><str>yyyy-MM-dd'T'HH:mm:ss,SSS[Z</str><str>yyyy-MM-dd'T'HH:mm:ss.SSS</str><str>yyyy-MM-dd' 'HH:mm:ss.SSS</str><str>yyyy-MM-dd'T'HH:mm:ssZ</str><str>yyyy-MM-dd'T'HH:mm:ss</str><str>yyyy-MM-dd'T'HH:mmZ</str><str>yyyy-MM-dd'T'HH:mm</str><str>yyyy-MM-dd HH:mm:ss.SSSZ</str><str>yyyy-MM-dd HH:mm:ss,SSSZ</str><str>yyyy-MM-dd HH:mm:ss.SSS</str><str>yyyy-MM-dd HH:mm:ss,SSS</str><str>yyyy-MM-dd HH:mm:ssZ</str><str>yyyy-MM-dd HH:mm:ss</str><str>yyyy-MM-dd HH:mmZ</str><str>yyyy-MM-dd HH:mm</str><str>yyyy-MM-dd</str><str>yyyy-MM-dd['T'[HH:mm[:ss[.SSS]]</str><str>yyyy-MM-dd['T'[HH:mm[:ss[.SSS]][z</str><str>yyyy-MM-dd['T'[HH:mm[:ss[.SSS]][z</str><str>yyyy-MM-dd['T'[HH:mm[:ss[,SSS]][z</str><str>yyyy-MM-dd HH:mm[:ss[.SSS]][z</str><str>yyyy-MM-dd HH:mm[:ss[.SSS]]</str><str>yyyy-MM-dd HH:mm[:ss[,SSS]][z</str><str>[EEE, ]dd MMM yyyy HH:mm[:ss] z</str><str>EEEE, dd-MMM-yy HH:mm:ss z</str><str>EEE MMM ppd HH:mm:ss [z ]yyyy</str></arr></updateProcessor><updateRequestProcessorChain name="add-unknown-fields-to-the-schema" default="${update.autoCreateFields:true}"processor="uuid,remove-blank,field-name-mutating,parse-boolean,parse-long,parse-double,parse-date,add-schema-fields"><processor class="solr.LogUpdateProcessorFactory"/><processor class="solr.DistributedUpdateProcessorFactory"/><processor class="solr.RunUpdateProcessorFactory"/></updateRequestProcessorChain>
Thanks for you help!
答案1
得分: 2
你应该以正确的格式传递日期值。
当日期时间字符串的格式无效时，SOLR会抛出错误。SOLR只允许使用 YYYY-MM-DDThh:mm:ssZ 格式的日期时间字符串。
正确的格式是 2020-10-01T00:00:00Z。在你分享的示例中也起作用了。
在你的logstash文件中添加适当的过滤器，将日期转换为上述期望的格式。
英文:
You should pass the date value in correct format.
The error is thrown by SOLR when a datetime string have an invalid format. SOLR only allows datetime strings in the format of YYYY-MM-DDThh:mm:ssZ.
The correct format is 2020-10-01T00:00:00Z. It also worked in the example shared by you.
Add appropriate filter in your logstash file to mutate the date in the above expected format.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论