英文:
Does AWS Opensearch Support ML Commons Library?
问题
I'm here to provide the translated content:
-
我正在尝试运行一个简单的代码示例,使用Opensearch ML Commons插件在AWS托管的Opensearch集群上训练和预测ML算法。该集群位于OS版本2.5上,明确支持该插件。该代码示例可以在这里找到。
-
当我在控制台中运行示例代码时,我收到以下错误消息:
"type": "m_l_resource_not_found_exception",
"reason": "No eligible node found to execute this request. It's best practice to provision ML nodes to serve your models. You can disable this setting to serve the model on your data node for development purposes by disabling the \"plugins.ml_commons.only_run_on_ml_node\" configuration using the _cluster/setting api"
- 然而,尝试更新设置后,我得到了
_cluster/settings payload not allowed
,看起来AWS OS版本不支持cluster/settings API,除了以下几个属性:
/_cluster/settings for several properties:
action.auto_create_index
action.search.shard_count.limit
indices.breaker.fielddata.limit
indices.breaker.request.limit
indices.breaker.total.limit
cluster.max_shards_per_node
一些问题:
- 我的假设是否正确,即我无法更新此集群设置,意味着我无法在AWS版本的Opensearch上运行ML Commons?
- 是否有一种方法可以在AWS OS上配置ML节点?所有看到的示例都要求您更新opensearch.yml文件,但我无法这样做,因为我使用的是托管版本。
- 是否有某些安全权限我没有,可以允许我更改这些集群设置?
谢谢!
英文:
I'm trying to run a simple code example to train and predict a ML algorithm using the Opensearch ML Commons plugin on a AWS managed Opensearch cluster. The cluster is on OS version 2.5, which definitely supports the plugin. The code example can be found here .
When I run the sample code in my console I get the following error message:
"type": "m_l_resource_not_found_exception",
"reason": "No eligible node found to execute this request. It's best practice to provision ML nodes to serve your models. You can disable this setting to serve the model on your data node for development purposes by disabling the \"plugins.ml_commons.only_run_on_ml_node\" configuration using the _cluster/setting api"
However after trying to update the settings I get _cluster/settings payload not allowed
, it doesn't seem like the AWS OS version supports the cluster/settings API except for these
/_cluster/settings for several properties:
action.auto_create_index
action.search.shard_count.limit
indices.breaker.fielddata.limit
indices.breaker.request.limit
indices.breaker.total.limit
cluster.max_shards_per_node
A few questions:
- Am I correct in my assumption that I can't update this cluster setting, meaning that I can't run ML Commons on an AWS version of Opensearch
- Is there a way to configure an ML node on a AWS OS. All examples seen require you to update the opensearch.yml file, which I can't do because I'm using a hosted version.
- Is there some security permission that I don't have that could allow me to change these cluster settings?
Thanks!
答案1
得分: 1
亚马逊 OpenSearch 服务 支持 ML Commons 版本 >= 1.3
这里可能发生了三种情况之一:
-
实例类型: 某些操作(例如异常检测、ML Commons)不支持所有实例。虽然 AWS 未 指定支持 ML Commons 的实例,但很可能
t2.micro
和t2.small
节点不受支持 — 这在 异常检测 中是这样的。 -
访问控制: 您必须启用 细粒度访问控制(FGAC)以使用非标准插件。kNN、SQL 和其他 CRUD 导向的操作当然支持没有 FGAC。虽然没有文件确认此事,我想 ML Commons 和异常检测一样,可能需要 FGAC,因为该插件涉及对集群/节点配置的重大修改。
- 此外,您需要将
ml_full_access
后端角色映射到您的用户,即使您已经有all_access
角色。有关详细信息,请参阅 此处。
- 此外,您需要将
-
_settings
API 调用的格式: 亚马逊的 OpenSearch 版本要求使用“PUT 请求到 _cluster/settings API,使用"flat" 设置形式” [来源]。
英文:
Amazon OpenSearch Service supports ML Commons for versions >= 1.3
One of three things is likely occurring here:
-
Instance Types: certain operations (e.g. anomaly detection, ML Commons) are not supported on all instances. While AWS do not specify which instances are supported for ML Commons, it is likely that
t2.micro
andt2.small
nodes will not be supported — this is the case with anomaly detection. -
Access Control: you must enable fine-grained access control (FGAC) in order to use non-standard plugins. kNN, SQL, and other CRUD-oriented actions, of course, are supported without FGAC. Although there is no documentation to confirm this, I imagine that ML Commons, as with Anomaly Detection, requires FGAC as the plugin involves significant modifications to cluster/node configuration.
- Additionally, you’ll need to map the
ml_full_access
backend role to your user, even if you already have theall_access
role. More on this here.
- Additionally, you’ll need to map the
-
Formatting of
_settings
API call: Amazon’s flavour of OpenSearch requires “PUT requests to the _cluster/settings API that use the "flat" settings form” [source]
答案2
得分: 0
"Neural Search" 是一个实验性功能,您可以发送邮件至 semantic-search@amazon.com 以启用您账户的 Neural Search 功能,详细信息请参考此 链接。
英文:
Notice Neural Search is an experimental feature, you can send an email to semantic-search@amazon.com to enable Neural Search feature for your account as explained in this link.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论