Hibernate Search AWS OpenSearch Serverless connection problem

I am using Hibernate Search v. 6.1.8.Final within my Spring Boot application. My implementation works for OpenSearch v. 2.5 which runs locally and in Kubernetes.

Besides the basis libraries I include org.hibernate.search:hibernate-search-backend-elasticsearch-aws AWS adapter.

When I install it in AWS EKS I configure my Spring properties like

spring:
  jpa:
    properties:
      hibernate:
        search:
          backend:
            type: elasticsearch
            hosts: xxxxxxxx.<region>.aoss.amazonaws.com
            version: opensearch:2.5
            version_check:
              enabled: false
            aws:
              region: <region>
              signing:
                enabled: true

I do not put my AWS credentials in this configuration since I hope I can authenticate with Service Account my pod is running with.

The service account eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxx:role/<role-name> defines the role name which has following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "aoss:*"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:aoss:<region>:xxxxxxxxx:collection/<collection-id>"
        }
    ]
}

I have connection to the OpenSearch Serverless Collection (with curl). Also in the IAM Policy Simulator I can verify that my role has APIAccessAll for aoss.

Now when I start my application I get following error:

2023-06-05T08:51:22,164+0000  WARN [,] --- [main] dialect.impl.ElasticsearchDialectFactory : HSEARCH400085: Unknown Elasticsearch version running on the cluster: 'opensearch:2.5'. Hibernate Search may not work correctly. Consider updating to a newer version of Hibernate Search, if any.
2023-06-05T08:51:22,367+0000  WARN [,] --- [main] als.internal.WebIdentityCredentialsUtils : To use web identity tokens, the 'sts' service module must be on the class path.
2023-06-05T08:51:22,863+0000 ERROR [,] --- [Hibernate Search - default backend - Transport thread - 2] ngine.reporting.spi.RootFailureCollector : HSEARCH000521: Hibernate Search encountered a failure during bootstrap; continuing for now to list all problems, but the process will ultimately be aborted.
Context: Hibernate ORM mapping, type 'com.example.MyEntity'
Failure:
org.hibernate.search.util.common.SearchException: HSEARCH400034: Unable to retrieve index metadata from Elasticsearch: HSEARCH400007: Elasticsearch request failed: HSEARCH400090: Elasticsearch response indicates a failure.
Request: GET /myentity-write,myentity-read with parameters {ignore_unavailable=true, allow_no_indices=true}
Response: 403 'Forbidden' from 'https://xxxxxx.<region>.aoss.amazonaws.com' with body
{
  "status": 403,
  "request-id": "<request-id>",
  "error": {
    "reason": "Credential should be scoped to correct service: \u0027aoss\u0027, not \u0027es\u0027.",
    "type": "AccessDenied"
  }
}

Does anyone know this issue? For me it seems that the policy is correct, also without Resource restriction.

Hibernate Search AWS integration is not tested against Amazon OpenSearch Serverless. From the message you get back from AWS:

"Credential should be scoped to correct service: \u0027aoss\u0027, not \u0027es\u0027."

I would assume that the problem comes from the fact that hibernate-search-backend-elasticsearch-aws is signing the request for the es service rather than the aoss that you need...

I'd suggest starting a discussion here https://discourse.hibernate.org/ or open a request at https://hibernate.atlassian.net/