Terraform: aws_lambda_permission 达到最终策略大小(20718),超出限制(20480)

huangapple go评论63阅读模式
英文:

Terraform: aws_lambda_permission reaching The final policy size (20718) is bigger than the limit (20480)

问题

Context:

我有一个名为console的Lambda函数,它允许我使用PHP Symfony的控制台命令运行所有类似于cron的任务。由于我的代码库是一个庞大的单体应用,所以我只有一个Lambda函数(为每个命令创建一个Lambda函数仍然需要99.9%的代码,而且部署会成为一场噩梦)

这个Lambda函数由CloudWatch事件调用,基本上是1个CloudWatch规则 => 1个任务

由于我们还有无关的CloudWatch规则,我希望只有适当的规则才能调用这个Lambda函数。换句话说,我不想允许所有CloudWatch规则。

用Terraform来管理它,像这样:

resource "aws_lambda_function" "console" {
   # 不相关的部分
}

# 对于每个“task”,这3个资源都会重复N次
resource "aws_cloudwatch_event_rule" "run_task_foobar" {
  # 不相关的部分
}

resource "aws_cloudwatch_event_target" "run_task_foobar" {
  rule      = aws_cloudwatch_event_rule.run_task_foobar.name
  target_id = "foobar"
  arn       = aws_lambda_function.console.arn
  input     = "\" console run foobar \""
}

resource "aws_lambda_permission" "run_task_foobar" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.console.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.run_task_foobar.arn

  # 不与标签兼容
}

问题

由于我们有数十个任务,确切地说是53个,当我们尝试为第54个任务申请时,我们现在遇到了这个错误:

PolicyLengthExceededException: 最终策略大小(20701)大于限制(20480)。

是的,当我在AWS控制台中检查时,我可以看到生成的IAM策略非常庞大。

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "foo",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-west-1:xxxx:function:console",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:events:eu-west-1:xxxxx:rule/foo"
        }
      }
    },
    {
      "Sid": "bar",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-west-1:xxxx:function:console",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:events:eu-west-1:xxxxx:rule/bar"
        }
      }
    },
    ... # 重复了53
  ]
}

当然,很明显,“Condition”块应该被因子化成伪代码,像这样:

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "bar",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-west-1:xxxx:function:console",
      "Condition": {
        "InArray": {
          "AWS:SourceArn": [
               "arn:aws:events:eu-west-1:xxxxx:rule/bar",
               "arn:aws:events:eu-west-1:xxxxx:rule/foo",
               ... # 重复53
           ]
        }
      }
    },

然而,没有对“Condition”进行精细的控制,它是由Terraform和AWS控制台自动生成的。

问题:

  1. 是否有实际的方法来因子化这个?
  2. 是否可以通过直接创建IAM策略来找到替代解决方案?

我的当前解决方法是建立一个约定,即所有的CloudWatch事件规则都应以“console-”开头,以便我可以使用通配符,但这似乎很脆弱?

英文:

Context:

I have a lambda named console that allows me to run all cron-like tasks using PHP Symfony's console command. As my codebase is a big monolith that's why I have only one lambda (having one lambda by command would still require 99.9% of the code and would be a nightmare to deploy)

This lambda is invoked by cloudwatch events , with basically 1 cloudwatch rule => 1 task

As we also have unrelated cloudwatch rules, I want to be sure that only the appropriate rules can invoke this lambda. I.e I don't want to just allow all cloudwatch rules.

With terraform we manage it like that


resource "aws_lambda_function" "console" {
   # not relevant
}

# These 3 resources  are repeated  N times for each "task"
resource "aws_cloudwatch_event_rule" "run_task_foobar" {
  # not relevant
}

resource "aws_cloudwatch_event_target" "run_task_foobar" {
  rule      = aws_cloudwatch_event_rule.run_task_foobar.name
  target_id = "foobar"
  arn       = aws_lambda_function.console.arn
  input     = "\" console run foobar \""
}

resource "aws_lambda_permission" "run_task_foobar" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.console.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.run_task_foobar.arn

  # not compatible with tags
}

The problem

As we have several dozen of task , 53 to be exact , we now have this error when we try to apply for a 54th task

> PolicyLengthExceededException: The final policy size (20701) is bigger than the limit (20480).

And yes when I check in the AWS Console , I can see the resulting IAM Policy is HUGE

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "foo",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-west-1:xxxx:function:console",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:events:eu-west-1:xxxxx:rule/foo"
        }
      }
    },
    {
      "Sid": "bar",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-west-1:xxxx:function:console",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:events:eu-west-1:xxxxx:rule/bar"
        }
      }
    },
    ... # repeated 53 times 

Of course it becomes clearly obvious that the Condition block should be factorized into the pseudo code

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "bar",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:eu-west-1:xxxx:function:console",
      "Condition": {
        "InArray": {  # I don't know if inarray exists but it does not matter see below
          "AWS:SourceArn": [
               "arn:aws:events:eu-west-1:xxxxx:rule/bar",
               "arn:aws:events:eu-west-1:xxxxx:rule/foo",
               ... , # 53 times
           ]
        }
      }
    },

However there's no fine grain control on "Condition" , it's autogenerated by both terraform and the AWS console

Question(s) :

  1. Is there an actual way to factorize this ?
  2. Is there an alternate solution by creating directly an IAM policy ?

My current workaround would be to setup a convention that all the cloudwatch event rule should start with "console-" so that I can use a wildcard , but that seems fragile ?

答案1

得分: 1

根据您当前的上下文,您唯一的选项就是如您已经提到的,在策略构建中使用通配符,正如这篇AWS文章也提倡的 https://repost.aws/knowledge-center/lambda-resource-based-policy-size-error

关于您对使用通配符可能导致的安全问题的担忧,如果您在这个AWS账户中有适当的安全控制,也就是说,IAM写权限只对一组受限制的用户/自动化可访问,那么您将是安全的。

英文:

Given your current context your only option is as you already stated the use wild cards in the policy construction as this AWS article also advocates https://repost.aws/knowledge-center/lambda-resource-based-policy-size-error

Regarding your security concerns around the fragility related to the usage of wildcards, if you have proper security controls in this AWS account, meaning IAM write permissions are only accessible to a restricted set of users/automations you will be fine.

huangapple
  • 本文由 发表于 2023年5月18日 00:38:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/76274352.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定