Azure Data Factory - Copy data REST API => ADLS

I'm strugnling with an issue on Azure Data Facotry

I an extracting data from a REST API from Salesforce Marketing cloud.
The first page is goes well but when I want to use the pagination it doesen't work anymore.

In fact the base URL is like https://mcXXXXXXXXXXXXXXX.rest.marketingcloudapis.com

On my pipeline I use a copy data:

The method id a GET and I send the API path with the dataset properties : /data/v1/customobjectdata/key/GMC_Historique_ADF/rowset

My first response is like that :

So on pagination rules I use this :

The issue is that on the base URL /data/ is missing in links.next value.

Is there any solution to add /data/ in base url from links.next to have something like the following link : https://mcXXXXXXXXXXXXXXX.rest.marketingcloudapis.com/data/v1/customobjectdata/token/xxxxxxxxxx-xxxxxxx-xxxxxx/rowset?$page=2

>The next link itself is not valid, i would like to setup manually the base URL and only take the next.link value to build a custom URL

If the next link itself is not valid, then you cannot use pagination in ADF. Pagination requires all the next page links to be there in the current page.
You need to add the next link to your API page to use the pagination.

>Is there any solution to add /data/ in base url from links.next to have something like the following link : https://mcXXXXXXXXXXXXXXX.rest.marketingcloudapis.com/data/v1/customobjectdata/token/xxxxxxxxxx-xxxxxxx-xxxxxx/rowset?$page=2

If all of your URL are same and only difference between among them is the page number, then you can try the below workaround to achieve your requirement.

NOTE: This method only works if you know the total numbers of pages.

Build an array of length of total number of pages and give it to ForEach activity.

@range(1,<total_number_of_pages>).

Inside ForEach, use copy activity with REST dataset as source and temporary location of ADLS as sink.

For demo I have used a sample REST API. For you, you need to give your URL in the base and use the dataset parameters for the Relative URL.

Give this rowset?$page=@{item()} as the value for it in your source.

Use the dataset parameters for JSON sink dataset(temporary location) filename also and give the dynamic content as file@{item()}.json for it.

Outside the ForEach, use another copy activity. This is for merging all individual page response JSONs from temporary location to a single JSON file in target location.

In the source of this copy activity, give the temporary location and * in the wild card path.

In the sink dataset, give your target file location and set the Copy behavior as Merge files. Also, set the File pattern to Array of objects. This will give you the final JSON file after Execution.