AWS Opensearch是否支持ML Commons库?

huangapple go评论57阅读模式
英文:

Does AWS Opensearch Support ML Commons Library?

问题

I'm here to provide the translated content:

  1. 我正在尝试运行一个简单的代码示例,使用Opensearch ML Commons插件在AWS托管的Opensearch集群上训练和预测ML算法。该集群位于OS版本2.5上,明确支持该插件。该代码示例可以在这里找到。

  2. 当我在控制台中运行示例代码时,我收到以下错误消息:

"type": "m_l_resource_not_found_exception",
    "reason": "No eligible node found to execute this request. It's best practice to provision ML nodes to serve your models. You can disable this setting to serve the model on your data node for development purposes by disabling the \"plugins.ml_commons.only_run_on_ml_node\" configuration using the _cluster/setting api"
  1. 然而,尝试更新设置后,我得到了_cluster/settings payload not allowed,看起来AWS OS版本不支持cluster/settings API,除了以下几个属性:
/_cluster/settings for several properties:

action.auto_create_index

action.search.shard_count.limit

indices.breaker.fielddata.limit

indices.breaker.request.limit

indices.breaker.total.limit

cluster.max_shards_per_node

来源

一些问题:

  1. 我的假设是否正确,即我无法更新此集群设置,意味着我无法在AWS版本的Opensearch上运行ML Commons?
  2. 是否有一种方法可以在AWS OS上配置ML节点?所有看到的示例都要求您更新opensearch.yml文件,但我无法这样做,因为我使用的是托管版本。
  3. 是否有某些安全权限我没有,可以允许我更改这些集群设置?

谢谢!

英文:

I'm trying to run a simple code example to train and predict a ML algorithm using the Opensearch ML Commons plugin on a AWS managed Opensearch cluster. The cluster is on OS version 2.5, which definitely supports the plugin. The code example can be found here .

When I run the sample code in my console I get the following error message:

"type": "m_l_resource_not_found_exception",
    "reason": "No eligible node found to execute this request. It's best practice to provision ML nodes to serve your models. You can disable this setting to serve the model on your data node for development purposes by disabling the \"plugins.ml_commons.only_run_on_ml_node\" configuration using the _cluster/setting api"

However after trying to update the settings I get _cluster/settings payload not allowed, it doesn't seem like the AWS OS version supports the cluster/settings API except for these

/_cluster/settings for several properties:

action.auto_create_index

action.search.shard_count.limit

indices.breaker.fielddata.limit

indices.breaker.request.limit

indices.breaker.total.limit

cluster.max_shards_per_node

source

A few questions:

  1. Am I correct in my assumption that I can't update this cluster setting, meaning that I can't run ML Commons on an AWS version of Opensearch
  2. Is there a way to configure an ML node on a AWS OS. All examples seen require you to update the opensearch.yml file, which I can't do because I'm using a hosted version.
  3. Is there some security permission that I don't have that could allow me to change these cluster settings?

Thanks!

答案1

得分: 1

亚马逊 OpenSearch 服务 支持 ML Commons 版本 >= 1.3

这里可能发生了三种情况之一:

  1. 实例类型: 某些操作(例如异常检测、ML Commons)不支持所有实例。虽然 AWS 指定支持 ML Commons 的实例,但很可能 t2.microt2.small 节点不受支持 — 这在 异常检测 中是这样的。

  2. 访问控制: 您必须启用 细粒度访问控制(FGAC)以使用非标准插件。kNN、SQL 和其他 CRUD 导向的操作当然支持没有 FGAC。虽然没有文件确认此事,我想 ML Commons 和异常检测一样,可能需要 FGAC,因为该插件涉及对集群/节点配置的重大修改。

    • 此外,您需要将 ml_full_access 后端角色映射到您的用户,即使您已经有 all_access 角色。有关详细信息,请参阅 此处
  3. _settings API 调用的格式: 亚马逊的 OpenSearch 版本要求使用“PUT 请求到 _cluster/settings API,使用"flat" 设置形式[来源]

英文:

Amazon OpenSearch Service supports ML Commons for versions >= 1.3

One of three things is likely occurring here:

  1. Instance Types: certain operations (e.g. anomaly detection, ML Commons) are not supported on all instances. While AWS do not specify which instances are supported for ML Commons, it is likely that t2.micro and t2.small nodes will not be supported — this is the case with anomaly detection.

  2. Access Control: you must enable fine-grained access control (FGAC) in order to use non-standard plugins. kNN, SQL, and other CRUD-oriented actions, of course, are supported without FGAC. Although there is no documentation to confirm this, I imagine that ML Commons, as with Anomaly Detection, requires FGAC as the plugin involves significant modifications to cluster/node configuration.

    • Additionally, you’ll need to map the ml_full_access backend role to your user, even if you already have the all_access role. More on this here.
  3. Formatting of _settings API call: Amazon’s flavour of OpenSearch requires “PUT requests to the _cluster/settings API that use the "flat" settings form[source]

答案2

得分: 0

"Neural Search" 是一个实验性功能,您可以发送邮件至 semantic-search@amazon.com 以启用您账户的 Neural Search 功能,详细信息请参考此 链接

英文:

Notice Neural Search is an experimental feature, you can send an email to semantic-search@amazon.com to enable Neural Search feature for your account as explained in this link.

huangapple
  • 本文由 发表于 2023年5月18日 00:20:26
  • 转载请务必保留本文链接:https://go.coder-hub.com/76274221.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定