AWS批量索引使用时出现’非法参数异常’,’不允许在批量中使用显式索引’。

huangapple go评论68阅读模式
英文:

AWS bulk indexing using gives 'illegal_argument_exception', 'explicit index in bulk is not allowed')

问题

以下是您要翻译的内容:

"While I am trying to bulk index on AWS Opensearch Service (ElasticSearch V 10.1) using opensearch-py, I am getting below error

RequestError: RequestError(400, 'illegal_argument_exception', 'explicit index in bulk is not allowed')
from opensearchpy.helpers import bulk
bulk(client, format_embeddings_for_es_indexing(embd_data, titles_, _INDEX_))

format_embeddings_for_es_indexing() function yeilds

{
'_index': 'test_v1',
'_id': '208387',
'_source': {
    'article_id': '208387',
    'title': 'Battery and Performance',
    'title_vector': [ 1.77665558e-02,  1.95874255e-02,.....],
    ......
    }
}

I am able to index documents one by one using `open search.index()'

failed = {}
for document in format_embeddings_for_es_indexing(embd_data, titles_, _INDEX_):
    res = client.index(
        **document,
        refresh = True
    )
    if res['_shards']['failed'] > 0:
        failed[document["body"]["article_id"]] = res['_shards']

# document body for open search index
{
'index': 'test_v1',
'id': '208387',
'body': {
    'article_id': '208387',
    'title': 'Battery and Performance',
    'title_vector': [ 1.77665558e-02,  1.95874255e-02,.....],
    ......
    }
}
```"

<details>
<summary>英文:</summary>

While I am trying to bulk index on AWS Opensearch Service (ElasticSearch V 10.1) using opensearch-py, I am getting below error

RequestError: RequestError(400, 'illegal_argument_exception', 'explicit index in bulk is not allowed')

from opensearchpy.helpers import bulk
bulk(client, format_embeddings_for_es_indexing(embd_data, titles_, INDEX))

__format_embeddings_for_es_indexing()__ function yeilds

{
'_index': 'test_v1',
'_id': '208387',
'_source': {
'article_id': '208387',
'title': 'Battery and Performance',
'title_vector': [ 1.77665558e-02, 1.95874255e-02,.....],
......
}
}

I am able to index documents one by one using `open search.index()&#39;

failed = {}
for document in format_embeddings_for_es_indexing(embd_data, titles_, INDEX):
res = client.index(
**document,
refresh = True
)
if res['_shards']['failed'] > 0:
failed[document["body"]["article_id"]] = res['_shards']

document body for open search index

{
'index': 'test_v1',
'id': '208387',
'body': {
'article_id': '208387',
'title': 'Battery and Performance',
'title_vector': [ 1.77665558e-02, 1.95874255e-02,.....],
......
}
}


please help

</details>


# 答案1
**得分**: 1

这可能与这里记录的内容有关: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-advanced

请确保高级集群设置中的 `rest.action.multi.allow_explicit_index` 值为 true。

<details>
<summary>英文:</summary>

This may have something to do with what is documented here: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-advanced 

Please make sure that the value of `rest.action.multi.allow_explicit_index` in the Advanced cluster settings is true

</details>



huangapple
  • 本文由 发表于 2023年2月10日 15:58:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/75408317.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定