AWS批量索引使用时出现’非法参数异常’,’不允许在批量中使用显式索引’。

huangapple go评论93阅读模式
英文:

AWS bulk indexing using gives 'illegal_argument_exception', 'explicit index in bulk is not allowed')

问题

以下是您要翻译的内容:

"While I am trying to bulk index on AWS Opensearch Service (ElasticSearch V 10.1) using opensearch-py, I am getting below error

  1. RequestError: RequestError(400, 'illegal_argument_exception', 'explicit index in bulk is not allowed')
  1. from opensearchpy.helpers import bulk
  2. bulk(client, format_embeddings_for_es_indexing(embd_data, titles_, _INDEX_))

format_embeddings_for_es_indexing() function yeilds

  1. {
  2. '_index': 'test_v1',
  3. '_id': '208387',
  4. '_source': {
  5. 'article_id': '208387',
  6. 'title': 'Battery and Performance',
  7. 'title_vector': [ 1.77665558e-02, 1.95874255e-02,.....],
  8. ......
  9. }
  10. }

I am able to index documents one by one using `open search.index()'

  1. failed = {}
  2. for document in format_embeddings_for_es_indexing(embd_data, titles_, _INDEX_):
  3. res = client.index(
  4. **document,
  5. refresh = True
  6. )
  7. if res['_shards']['failed'] > 0:
  8. failed[document["body"]["article_id"]] = res['_shards']
  9. # document body for open search index
  10. {
  11. 'index': 'test_v1',
  12. 'id': '208387',
  13. 'body': {
  14. 'article_id': '208387',
  15. 'title': 'Battery and Performance',
  16. 'title_vector': [ 1.77665558e-02, 1.95874255e-02,.....],
  17. ......
  18. }
  19. }
  20. ```"
  21. <details>
  22. <summary>英文:</summary>
  23. While I am trying to bulk index on AWS Opensearch Service (ElasticSearch V 10.1) using opensearch-py, I am getting below error

RequestError: RequestError(400, 'illegal_argument_exception', 'explicit index in bulk is not allowed')

from opensearchpy.helpers import bulk
bulk(client, format_embeddings_for_es_indexing(embd_data, titles_, INDEX))

  1. __format_embeddings_for_es_indexing()__ function yeilds

{
'_index': 'test_v1',
'_id': '208387',
'_source': {
'article_id': '208387',
'title': 'Battery and Performance',
'title_vector': [ 1.77665558e-02, 1.95874255e-02,.....],
......
}
}

  1. I am able to index documents one by one using `open search.index()&#39;

failed = {}
for document in format_embeddings_for_es_indexing(embd_data, titles_, INDEX):
res = client.index(
**document,
refresh = True
)
if res['_shards']['failed'] > 0:
failed[document["body"]["article_id"]] = res['_shards']

document body for open search index

{
'index': 'test_v1',
'id': '208387',
'body': {
'article_id': '208387',
'title': 'Battery and Performance',
'title_vector': [ 1.77665558e-02, 1.95874255e-02,.....],
......
}
}

  1. please help
  2. </details>
  3. # 答案1
  4. **得分**: 1
  5. 这可能与这里记录的内容有关: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-advanced
  6. 请确保高级集群设置中的 `rest.action.multi.allow_explicit_index` 值为 true
  7. <details>
  8. <summary>英文:</summary>
  9. This may have something to do with what is documented here: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-advanced
  10. Please make sure that the value of `rest.action.multi.allow_explicit_index` in the Advanced cluster settings is true
  11. </details>

huangapple
  • 本文由 发表于 2023年2月10日 15:58:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/75408317.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定