AWS Step functions embedded workflows appear to have 2-3 second overhead

I'm creating a Step function workflow with multiple embedded workflows (using startExecution.sync:2). I noticed that all of the embedded workflows appear to have a 2-3 second overhead. This appears to be the same across all of my embedded workflows. Is this normal or am I doing something wrong?

Looking at the runtime for the logic in one of the embedded workflows, the time spent was milliseconds, however, when looking at the parent the time spent on that execution task was over 2 seconds. I would have expected it to only tike milliseconds at the parent level as well.

2-3 second latency between completion of a job executed using the Run a Job (.sync) integration pattern and continuation of the Step Functions Workflow is not uncommon.

With these optimized integrations, Step Functions makes a call to the target service (in this case to itself) to start an asynchronous process (in this case a Step Functions Workflow Execution). The target service returns an identifier for the asynchronous process to Step Functions which pauses calling workflow execution.

Step Functions then polls the target service (typically using a Describe* API Action) at a regular interval to determine when the job completes. For serivces which support Amazon EventBridge events, Step Functions will create a Managed Rule in your account so that it can learn of job completion more quickly (Step Functions is one such service). When the calling workflow execution learns of completion via an event or a poll response, it continues the workflow execution.

Given these asynchronous interactions, there will be latency between completion of the job and resumption of the calling workflow execution.

If Express Workflows will meet the needs for your embedded workflows, you can call them using the StartSyncExecution API Action with AWS SDK Service Integrations. Step Functions will then call these synchronously in the same way it does with AWS Lambda which will reduce the latency.

Update: I contacted AWS support and they responded with the following:

> Hello,
>
> From your query, I understand that you have created a Step Function
> workflow with multiple embedded workflows i.e. a nested workflow using
> startExecution.sync:2 and observed a delay of 2-3 seconds in the
> parent workflow while executing the task. However, the time spent in
> executing the logic in each of the embedded/nested workflow is in
> milli seconds. You wanted to understand if this is the expected
> behavior or not.
>
> Here, I would like to inform you that when an embedded/nested workflow
> execution is started using .sync integration pattern 1, there is a
> 2-3 seconds latency observed in parent StateMachine becoming aware of
> nested StateMachine execution completion status. Kindly note that the
> internal team is aware of this issue and identified it as an expected
> behavior.
>
> They are working to improve this latency in case of a .sync
> integration with nested StateMachines. We do not have the visibility
> regarding the feature's roadmap and backlog and unfortunately, we will
> not be able to provide an ETA on when this will be implemented.
> Therefore I would kindly request you to keep an eye on the what’s new
> page or the AWS Blogs as all new feature requests and enhancements are
> posted there.
>
> [+] AWS blogs: https://blogs.amazon.com/ [+] What’s New:
> https://aws.amazon.com/new/ [+] AWS Release Notes:
> https://aws.amazon.com/releasenotes/ [+] AWS Step Functions Compute
> Blog: https://aws.amazon.com/blogs/compute/tag/aws-step-functions/
>
> As a workaround to avoid this 2-3 seconds delay caused by .sync
> integration, we would request you to leverage .waitForTaskToken
> integration pattern instead of .sync integration pattern. Using
> .waitForTaskToken 2 integration pattern with nested StateMachine
> execution, eliminates the 2-3 seconds delay.
>
> 1
> https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-sync
> 2
> https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-wait-token

Switching to the waitForTaskToken worked for me.