英文:
How to resume a step function with aws batch
问题
我有一个包含2个状态的步骤函数，假设这两个状态为state1和state2。state1触发一个Lambda函数，然后返回到state1，步骤函数进入等待状态。Lambda触发AWS Batch，当AWS Batch执行完成后，应该恢复步骤函数，并且步骤函数应该进入下一个状态，即state2。
那么，如何在AWS Batch作业完成时恢复步骤函数呢？这是一个无服务器架构。
我已经尝试了CloudWatch事件。是否有其他替代方法？
注意：AWS Batch执行需要超过20分钟。
英文:
I have a step function which has 2 states, lets suppose state1 and state2, state1 triggers a lambda and returns a wait state to state1, step function waits.
Lambda triggers aws batch, on completion of aws batch, it should resume the step function.
And step function should go to the next state i.e. state2.
So How to resume step function on aws batch job completion.
It's a serverless architecture.
I have tried cloudwatch event.
Is there any alternate way?
Note: AWS Batch takes more than 20 minutes for execution.
答案1
得分: 1
我怀疑你想要使用 AWS Step Functions 的 优化服务集成 来与 AWS Batch 集成。此集成支持 .sync / 运行作业集成模式，其中 Step Functions 将管理监视批处理作业并在完成时继续工作流执行。
你可以在 Workflow Studio 中将此拖放到工作流定义中，如下图所示。
这个任务的 Amazon States 语言如下所示：
"Manage Batch task": {
"Type": "Task",
"Resource": "arn:aws:states:::batch:submitJob.sync",
"Parameters": {
"JobDefinition": "arn:aws:batch:us-east-2:123456789012:job-definition/testJobDefinition",
"JobName": "testJob",
"JobQueue": "arn:aws:batch:us-east-2:123456789012:job-queue/testQueue"
},
"Next": "NEXT_STATE"
}
英文:
I suspect you want to use the AWS Step Functions Optimized Service Integration for AWS Batch. This integration supports the .sync / Run a Job Integration Pattern where Step Functions will manage monitoring your Batch Job and continuing the workflow execution when it completes.
You can do drag and drop this into your workflow definition using Workflow Studio, as shown in this screen shot.
The Amazon States Language for this will follow this form:
"Manage Batch task": {
"Type": "Task",
"Resource": "arn:aws:states:::batch:submitJob.sync",
"Parameters": {
"JobDefinition": "arn:aws:batch:us-east-2:123456789012:job-definition/testJobDefinition",
"JobName": "testJob",
"JobQueue": "arn:aws:batch:us-east-2:123456789012:job-queue/testQueue"
},
"Next": "NEXT_STATE"
}
答案2
得分: 0
启用步骤函数第一个 Lambda 调用中的等待回调选项。
您可以像下面这样添加有效载荷以使用令牌，根据您的输入数据。
{
"token.$": "$$.Task.Token",
"input.$": "$",
"callback": "true"
}
最后，在 AWS 批处理完成后，通过使用 send_task_token_success 返回任务令牌的 Lambda。
import boto3
import json
client = boto3.client('stepfunctions')
client.send_task_success(
taskToken=event['token'],
output=json.dumps({"key" : "value"})
)
英文:
Note: This will only work if your batches in aws lambda take less than 15 minutes.
Enable wait for callback option in first lambda invoke of step function.
you can add payload something like below to use pass token, as per your input data.
{
"token.$": "$$.Task.Token",
"input.$": "$",
"callback": "true"
}
Finally in lambda return task token on completion of aws batch by using send task token success.
client = boto3.client('stepfunctions')
client.send_task_success(
taskToken=event['token'],
output=json.dumps({"key" : "value"})
)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论