英文:
How can you wait for an end-to-end Step Function test to finish when triggered by EventBridge?
问题
我们目前正在使用基于事件的方法来处理加载到S3的文件。当上传文件时，它会触发一个EventBridge S3事件，然后启动一个Step Function。当数据移动到目标位置时，Step Function 完成。
我们有兴趣在我们的Python测试环境（本地和CICD）中实施自动化的端到端测试。
在检查输出位置之前，如何正确等待Step Function完成？
我考虑了一些方法，每种方法都有一些缺点：
- 设置一个定时器，等待n分钟以等待作业完成 - 效率低下
- 向Step Function添加最终步骤，将数据写入SNS，不断轮询SNS以获取传入消息 - 需要专门用于测试的新基础设施，在生产环境中会浪费资源，仍然效率低下
- 向Step Function添加标签或参数，以便我们可以搜索它并找到其ARN，然后使用该ARN轮询其完成状态 - 我们无法在Step Function被EventBridge触发时附加标签，仍然有些低效
- 不断检查我们的输出位置 - 效率低下
- 使用Step Functions Local测试Step Function - 仍然无法复制EventBridge触发，因此我们无法完全测试端到端
是否有一种方法可以找到由您有意发起的EventBridge事件触发的Step Function的ARN？是否有其他有效的方法？
英文:
We are currently using an event-based approach to process files loaded into S3. When a file is uploaded it triggers an EventBridge S3 event, which kicks off a Step Function. When the data is moved to the target the Step Function finishes.
We are interested in implementing an automated end-to-end test in our Python testing environments (both local and CICD) for this.
How can we properly wait for the Step Function to finish before checking our output location?
There are a few approaches I've considered, each with a drawback:
- Set a timer, wait n minutes for job to finish - inefficient
- Add final step to Step Function which writes to SNS, constantly poll SNS for incoming messages - requires new infrastructure just for testing, would be a waste in Production, still inefficient
- Add tags or parameters to Step Function so we can search for it and find its ARN, then use that to poll for when it finishes - we can't find a way to attach tags when the Step Function is triggered by EventBridge, still somewhat inefficient
- Constantly check our output location - inefficient
- Test Step Function using Step Functions Local - still doesn't duplicate the EventBridge trigger, so we're not exactly testing the full end-to-end
Is there a way to find the ARN of the Step Function which has been triggered by an EventBridge event that you intentionally initiated? Is there another effective approach?
答案1
得分: 1
您可以使用AWS Step Functions的同步表达式工作流。这种类型的步骤函数允许您快速接收工作流响应（类似于同步的Lambda函数）。
请注意，此类函数的最长持续时间为5分钟，如果整个流程适用于此场景，则这是一个不错的选择。
英文:
You can use Synchronous Express Workflows for AWS Step Functions.
This type of Step functions allows you to quickly receive the workflow response (like a synchronous lambda function).
Keep in mind that the max duration for this kind of functions is 5 minutes, it's a good option if your entire process fits in this scenario.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论