Failed to parse mapping: Field [similarity] requires field [index] to be configured

huangapple go评论128阅读模式
英文:

Failed to parse mapping: Field [similarity] requires field [index] to be configured

问题

当我使用dense_vector并将相似度设置为余弦相似度时,启动应用程序会出现mapper_parsing_exception错误:字段[similarity]需要配置字段[index]。但是,@Field注解已经配置了index为true,所以我不知道为什么会报告这个错误。我是否错过了其他配置或注解?

依赖版本:org.springframework.boot:spring-boot-starter-data-elasticsearch:3.1.2

我的实体类:

@Data
@Document(indexName = "spare_parts")
public class SparePartsDoc {

    @Id
    @Field(name = "id", type = FieldType.Keyword)
    String id;

    @Field(name = "data_id", type = FieldType.Keyword)
    String dataId;

    @Field(name = "type", type = FieldType.Keyword)
    String type;

    @Field(name = "second_type", type = FieldType.Keyword)
    String secondType;

    @Field(name = "brand", type = FieldType.Text)
    String picture;

    @Field(name = "brand", type = FieldType.Keyword)
    String brand;

    @Field(name = "color", type = FieldType.Keyword)
    String color;

    @Field(name = "quality", type = FieldType.Keyword)
    String quality;

    @Field(name = "level", type = FieldType.Keyword)
    String level;

    @Field(name = "specs", type = FieldType.Object)
    Map<String, String> specs;

    @Field(name = "remark", type = FieldType.Object)
    Map<String, String> remark;

    @Field(name = "code", type = FieldType.Keyword)
    String code;

    @Field(name = "product_name", type = FieldType.Text)
    String productName;

    @Field(name = "embedding", type = FieldType.Dense_Vector, dims= 768, similarity = "cosine")
    double[] embedding;

    @Field(name = "product_code", type = FieldType.Keyword)
    String productCode;

    @Field(name = "binding_complete_machin_id", type = FieldType.Keyword)
    String bindingCompleteMachinId;

    @Field(name = "increment_code", type = FieldType.Keyword)
    Integer incrementCode;

    @Field(name = "is_valid", type = FieldType.Keyword)
    String isValid;

    @Field(name = "create_time", type = FieldType.Date)
    LocalDateTime createTime;

    @Field(name = "last_update_time", type = FieldType.Date)
    LocalDateTime lastUpdateTime;

    @Transient
    float score;

}

部分异常信息:

Caused by: org.springframework.data.elasticsearch.UncategorizedElasticsearchException: [es/indices.create] failed: [mapper_parsing_exception] Failed to parse mapping: Field [similarity] requires field [index] to be configured
...

我尝试使用IndexOperations手动配置索引和相似度而不会出现任何问题:

@Test
void test(){
    IndexOperations indexOperations = elasticsearchOperations.indexOps(RetrieverDoc.class);
    indexOperations.delete();
    indexOperations.create();
    Document mapping = indexOperations.createMapping(RetrieverDoc.class);
    if (mapping.get("properties") instanceof LinkedHashMap properties) {
        if (properties.get("embedding") instanceof LinkedHashMap embedding) {
            embedding.put("index", true);
            embedding.put("similarity", "dot_product");
            indexOperations.putMapping(mapping);
        }
    }
}
英文:

when I use dense_vecto and set cosine to similarity, start application will get a mapper_parsing_exception : Field [similarity] requires field [index] to be configured
But the @Field annotation has already configured the index to be true,
So I don't know why I reported this error. Did I miss any other configurations or annotations?

dependencies version: org.springframework.boot:spring-boot-starter-data-elasticsearch:3.1.2

my entity:

@Data
@Document(indexName = &quot;spare_parts&quot;)
public class SparePartsDoc {
@Id
@Field(name = &quot;id&quot;, type = FieldType.Keyword)
String id;
@Field(name = &quot;data_id&quot;, type = FieldType.Keyword)
String dataId;
@Field(name = &quot;type&quot;, type = FieldType.Keyword)
String type;
@Field(name = &quot;second_type&quot;, type = FieldType.Keyword)
String secondType;
@Field(name = &quot;brand&quot;, type = FieldType.Text)
String picture;
@Field(name = &quot;brand&quot;, type = FieldType.Keyword)
String brand;
@Field(name = &quot;color&quot;, type = FieldType.Keyword)
String color;
@Field(name = &quot;quality&quot;, type = FieldType.Keyword)
String quality;
@Field(name = &quot;level&quot;, type = FieldType.Keyword)
String level;
@Field(name = &quot;specs&quot;, type = FieldType.Object)
Map&lt;String, String&gt; specs;
@Field(name = &quot;remark&quot;, type = FieldType.Object)
Map&lt;String, String&gt; remark;
@Field(name = &quot;code&quot;, type = FieldType.Keyword)
String code;
@Field(name = &quot;product_name&quot;, type = FieldType.Text)
String productName;
@Field(name = &quot;embedding&quot;, type = FieldType.Dense_Vector, dims= 768, similarity = &quot;cosine&quot;)
double[] embedding;
@Field(name = &quot;product_code&quot;, type = FieldType.Keyword)
String productCode;
@Field(name = &quot;binding_complete_machin_id&quot;, type = FieldType.Keyword)
String bindingCompleteMachinId;
@Field(name = &quot;increment_code&quot;, type = FieldType.Keyword)
Integer incrementCode;
@Field(name = &quot;is_valid&quot;, type = FieldType.Keyword)
String isValid;
@Field(name = &quot;create_time&quot;, type = FieldType.Date)
LocalDateTime createTime;
@Field(name = &quot;last_update_time&quot;, type = FieldType.Date)
LocalDateTime lastUpdateTime;
@Transient
float score;
}

part of exception:

Caused by: org.springframework.data.elasticsearch.UncategorizedElasticsearchException: [es/indices.create] failed: [mapper_parsing_exception] Failed to parse mapping: Field [similarity] requires field [index] to be configured
at org.springframework.data.elasticsearch.client.elc.ElasticsearchExceptionTranslator.translateExceptionIfPossible(ElasticsearchExceptionTranslator.java:102)
at org.springframework.data.elasticsearch.client.elc.ElasticsearchExceptionTranslator.translateException(ElasticsearchExceptionTranslator.java:63)
at org.springframework.data.elasticsearch.client.elc.ChildTemplate.execute(ChildTemplate.java:73)
at org.springframework.data.elasticsearch.client.elc.IndicesTemplate.doCreate(IndicesTemplate.java:145)
at org.springframework.data.elasticsearch.client.elc.IndicesTemplate.createWithMapping(IndicesTemplate.java:135)
at org.springframework.data.elasticsearch.repository.support.SimpleElasticsearchRepository.&lt;init&gt;(SimpleElasticsearchRepository.java:84)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
... 106 more
Caused by: co.elastic.clients.elasticsearch._types.ElasticsearchException: [es/indices.create] failed: [mapper_parsing_exception] Failed to parse mapping: Field [similarity] requires field [index] to be configured
at co.elastic.clients.transport.rest_client.RestClientTransport.getHighLevelResponse(RestClientTransport.java:334)
at co.elastic.clients.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:154)
at co.elastic.clients.elasticsearch.indices.ElasticsearchIndicesClient.create(ElasticsearchIndicesClient.java:266)
at org.springframework.data.elasticsearch.client.elc.IndicesTemplate.lambda$doCreate$0(IndicesTemplate.java:145)
at org.springframework.data.elasticsearch.client.elc.ChildTemplate.execute(ChildTemplate.java:71)
... 115 more

I attempted to create an index using IndexOperations and manually configured the index and similarity without any issues.

@Test
void test(){
IndexOperations indexOperations = elasticsearchOperations.indexOps(RetrieverDoc.class);
indexOperations.delete();
indexOperations.create();
Document mapping = indexOperations.createMapping(RetrieverDoc.class);
if (mapping.get(&quot;properties&quot;) instanceof LinkedHashMap properties) {
if (properties.get(&quot;embedding&quot;) instanceof LinkedHashMap embedding) {
embedding.put(&quot;index&quot;, true);
embedding.pu(&quot;similarity&quot;, &quot;dot_product&quot;);
indexOperations.putMapping(mapping);
}
}
}

答案1

得分: 1

@Field注解中,index参数默认为_true_。当创建映射时,只有在它为_false_时才会写入映射,这是因为根据Elasticsearch文档 _true_是默认值,如果没有设置的话。所以让我感到惊讶的是,您需要在映射中明确设置它。

这可以通过在Spring Data Elasticsearch中始终设置index值为similarity类型来更改,我为此创建了一个问题

英文:

On the @Field annotation the index parameter defaults to true. When the mapping is created, the value is only written in the mapping when it is false. This is because according to the Elasticsearch documentation true is the default value if it is not set. So it astonishes me that you need to set it explicitly in the mapping.

This could be changed in Spring Data Elasticsearch by always setting the value for index when the type is similarity, I created an issue for that.

huangapple
  • 本文由 发表于 2023年8月10日 12:46:32
  • 转载请务必保留本文链接:https://go.coder-hub.com/76872712.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定