英文:
AWS SSM error while targets.1.member.values failed to satisfy constraint: Member must have length less than or equal to 50
问题
我正在尝试在我的 EC2 实例群中运行 SSM 命令,超过 50 个实例。使用 AWS boto3 的 SSM 客户端,我在我的节点上运行特定命令。我的代码如下。运行代码后,出现了意外错误。
# 运行 ec2 实例
instances = client.describe_instances()
instance_ids = [inst["InstanceId"] for inst in instances] # 可能包含超过 50 个实例
# 运行命令
run_cmd_resp = ssm_client.send_command(
Targets=[
{"Key": "InstanceIds", "Values": inst_ids_all},
],
DocumentName="AWS-RunShellScript",
DocumentVersion="1",
Parameters={
"commands": ["#!/bin/bash", "ls -ltrh", "# 一些命令"]
}
)
执行此操作后,出现以下错误
调用 SendCommand 操作时发生错误 (ValidationException):检测到 1 个验证错误:值 '[...91 个实例 ID...]' 在 'targets.1.member.values' 处未能满足约束:成员必须具有小于或等于 50 的长度。
如何运行 SSM 命令在我的整个实例群中?
英文:
I am trying to run a SSM command on more than 50 EC2 instances of my fleet. By using AWS boto3's SSM client, I am running a specific command on my nodes. My code is given below. After running the code, an unexpected error is showing up.
# running ec2 instances
instances = client.describe_instances()
instance_ids = [inst["InstanceId"] for inst in instances] # might contain more than 50 instances
# run command
run_cmd_resp = ssm_client.send_command(
Targets=[
{"Key": "InstanceIds", "Values": inst_ids_all},
],
DocumentName="AWS-RunShellScript",
DocumentVersion="1",
Parameters={
"commands": ["#!/bin/bash", "ls -ltrh", "# some commands"]
}
)
On executing this, getting below error
An error occurred (ValidationException) when calling the SendCommand operation: 1 validation error detected: Value '[...91 instance IDs...]' at 'targets.1.member.values' failed to satisfy constraint: Member must have length less than or equal to 50.
How do I run the SSM command my whole fleet?
答案1
得分: 1
根据错误消息和boto3文档 (链接) 显示,每个 send_command
调用中的实例数量限制为最多50个。要为所有实例运行SSM命令,将原始列表分成每个50个可能是一个解决方案。
FYI: 如果您的帐户有大量实例,describe_instances()
无法在一个API调用中检索所有实例信息,因此最好检查响应中是否存在 NextToken
。
参考: https://stackoverflow.com/questions/52068066/how-do-you-use-nexttoken-in-aws-api-calls
# 运行EC2实例
instances = client.describe_instances()
instance_ids = [inst["InstanceId"] for inst in instances]
while "NextToken" in instances:
instances = client.describe_instances(NextToken=instances["NextToken"])
instance_ids += [inst["InstanceId"] for inst in instances]
# 运行命令
for i in range(0, len(instance_ids), 50):
target_instances = instance_ids[i : i + 50]
run_cmd_resp = ssm_client.send_command(
Targets=[
{"Key": "InstanceIds", "Values": target_instances},
],
DocumentName="AWS-RunShellScript",
DocumentVersion="1",
Parameters={
"commands": ["#!/bin/bash", "ls -ltrh", "# 一些命令"]
}
)
英文:
As shown in the error message and boto3 documentation (link), the number of instances in one send_command
call is limited up to 50. To run the SSM command for all instances, splitting the original list into 50 each could be a solution.
FYI: If your account has a fair amount of instances, describe_instances()
can't retrieve all instance info in one api call, so it would be better to check whether NextToken
is in response.
ref: https://stackoverflow.com/questions/52068066/how-do-you-use-nexttoken-in-aws-api-calls
# running ec2 instances
instances = client.describe_instances()
instance_ids = [inst["InstanceId"] for inst in instances]
while "NextToken" in instances:
instances = client.describe_instances(NextToken=instances["NextToken"])
instance_ids += [inst["InstanceId"] for inst in instances]
# run command
for i in range(0, len(instance_ids), 50):
target_instances = instance_ids[i : i + 50]
run_cmd_resp = ssm_client.send_command(
Targets=[
{"Key": "InstanceIds", "Values": inst_ids_all},
],
DocumentName="AWS-RunShellScript",
DocumentVersion="1",
Parameters={
"commands": ["#!/bin/bash", "ls -ltrh", "# some commands"]
}
)
答案2
得分: 0
在@Rohan Kishibe的回答后,我尝试实现了以下批量执行SSM runShellScript的方法。
import math
ec2_ids_all = [...] # 通过分页获取的所有实例ID。
PG_START, PG_STOP = 0, 50
PG_SIZE = 50
PG_COUNT = math.ceil(len(ec2_ids_all) / PG_SIZE)
for page in range(PG_COUNT):
cmd = ssm.send_command(
Targets=[{"Key": "InstanceIds", "Values": ec2_ids_all[PG_START:PG_STOP]}],
DocumentVersion="AWS-RunShellScript",
Parameters={"commands": ["ls -ltrh", "# other commands"]}
}
PG_START += PG_SIZE
PG_STOP += PG_SIZE
通过以上方式,实例ID的总数将被分批处理,然后相应地执行。还可以保存命令ID和批量实例ID的映射以供将来使用。
英文:
Finally after @Rohan Kishibe's answer, I tried to implement below batched execution for the SSM runShellScript.
import math
ec2_ids_all = [...] # all instance IDs fetched by pagination.
PG_START, PG_STOP = 0, 50
PG_SIZE = 50
PG_COUNT = math.ceil(len(ec2_ids_all) / PG_SIZE)
for page in range(PG_COUNT):
cmd = ssm.send_command(
Targets=[{"Key": "InstanceIds", "Values": ec2_ids_all[PG_START:PG_STOP]}],
DocumentVersion="AWS-RunShellScript",
Parameters={"commands": ["ls -ltrh", "# other commands"]}
}
PG_START += PG_SIZE
PG_STOP += PG_SIZE
In above way, the total number of instance IDs will be distributed in batches and then executed accordingly. One can also save the Command IDs and batch instance IDs in a mapping for future usage.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论