AWS Sagemaker model creation failing with internal error

I am trying to create a sagemaker model in multimodel mode. For this, I use terraform and I have the following resources to be created:

module.main.module.toing.aws_sagemaker_model.sagemaker_multimodel will be created
  + resource "aws_sagemaker_model" "sagemaker_multimodel" {
      + arn                = (known after apply)
      + execution_role_arn = "arn:aws:iam::XXXXX:role/us-west-1-toing_role"
      + id                 = (known after apply)
      + name               = "dev-us-west-test-model"
      + primary_container {
          + image          = "us-west-1-toing_image"
          + mode           = "MultiModel"
          + model_data_url = "s3://toing_bucket/"
        }
    }

The resource creation of this model fails the first time with the error:

Error: creating SageMaker model: ValidationException: The execution role ARN "arn:aws:iam::XXXXX:role/us-west-1-toing_role" is invalid. Please ensure that the role exists and that its trust relationship policy allows the action "sts:AssumeRole" for the service principal "sagemaker.amazonaws.com".
│       status code: 400, request id: XXX-XXX-XXX-XXX-XXX

However, when I check the toing_role role on iam it has the neccesary permissions and trust relationships.

Trust Relationships

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "sagemaker.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Permissions policies

AmazonS3FullAccess
AmazonSageMakerFullAccess

When I re-apply, the creation of the model hangs on terraform. I checked on cloud trails, and it has the following error response to the CreateModel request from terraform. This happens repeatedly.

...
"errorCode": "InternalFailure",
"errorMessage": "An unknown error occurred",
...

When I try to create the model manually using the GUI with the same role, I get a ThrottlingException. When I wait for some time and re-run the request, I get an InternalFailure agian.

I do not understand if this is problem with my execution role, since the same code works perfectly in eu-west-1, but not in us-west-1. And the roles have exactly the same permissions as above.

The S3 bucket is also created in their right respective regions. Hence I do not understand where this failure comes from, since the S3 buckets too have exactly the same permissions.

PS1: I have also verified that the container toing_image exists separately in the ECR repositories of their respective regions.

PS2: The role is attached to the model as follows:

# Defining the SageMaker "Assume Role" policy
data "aws_iam_policy_document" "sm_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["sagemaker.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "sagemaker_inferencer_iam_role" {
  name               = "${var.app_environment}-inferencer-sm-${var.aws_region}-iam-role-${var.endpoint_postfix}"
  assume_role_policy = data.aws_iam_policy_document.sm_assume_role_policy.json
}

# Attaching the AWS default policy, "AmazonSageMakerFullAccess"
data "aws_iam_policy" "sm_required_policy" {
  name = "AmazonSageMakerFullAccess"
}

resource "aws_iam_role_policy_attachment" "sm_full_access_attach" {
  role       = aws_iam_role.sagemaker_inferencer_iam_role.name
  policy_arn = data.aws_iam_policy.sm_required_policy.arn
}

# Attaching the AWS default policy, "AmazonSageMakerFullAccess"
data "aws_iam_policy" "s3_required_policy" {
  name = "AmazonS3FullAccess"
}

resource "aws_iam_role_policy_attachment" "s3_full_access_attach" {
  role       = aws_iam_role.sagemaker_inferencer_iam_role.name
  policy_arn = data.aws_iam_policy.s3_required_policy.arn
}

resource "aws_sagemaker_model" "sagemaker_multimodel" {
  name               = "${var.app_environment}-${var.endpoint_postfix}-model"
  execution_role_arn = aws_iam_role.sagemaker_inferencer_iam_role.arn

  primary_container {
    image          = local.multi_model_inferencer_container_name
    mode           = "MultiModel"
    model_data_url = "s3://${local.model_bucket_name}/"
  }

  tags = var.default_tags
}

What am I doing wrong?

This was caused by STS (AWS Security Token Service) not being enabled for the specific regions that I was trying to run my infrastructure. Apparently in some regions, STS is not e,nabled by default, and has to be enabled through IAM -> Account Settings -> STS -> Endpoints as explained here.