How to monitor progress of "startpipelineexecution" for sagemaker on stepfunctions?

I currently have a sagemaker pipeline that is executed using step functions. However, while I am able to start the execution, I am unable to allow the step to wait before moving on to the next step. Hence I should I set it up within step function such that it waits for the pipeline to be executed completely before executing the next step?

StartPipelineExecution is an asynchronous API that returns immediately with the execution ARN. You have several options to wait for pipeline completion with a Step Function. Here are two:

Option 1: Polling

One option is to poll for execution completion within a single State Machine. Polling could be implemented with a Wait - Lambda - Choice task loop. Or by a single Lambda function with a callback pattern (in which case the looping is the Lambda's job). In both cases, the Lambda checks the status of the pipeline execution with an SDK call.

Visually, the State Machine looks like this:

 SM #1
[x x x S P x x x x]

where S = Sagemaker StartPieplineExecution task, P = Poller tasks, x = Other tasks.

Option #2: Event-driven

A second option avoids polling. Instead, split your State Machine in two, the first one ending with the StartPieplineExecution task. Add an EventBridge rule that triggers the second half of your tasks when a Pipeline execution state change event with "currentPipelineExecutionStatus": "Succeeded" is emitted.

 SM #1                                  SM #2
[x x x S] -> pipeline success event -> [x x x x]

These patterns apply more generally to orchestratiing asynchronous tasks. See this related question for another example.