Continuous integration and delivery for an Azure Synapse Analytics workspace using DevOps

I am using Azure DevOps for continuous integration and delivery for an Azure Synapse Analytics workspace where I have deploy the workspace to Development, Test, PreProduction and Production environments. The pipelines use parametrized LinkedServices, Datasets and Trigger. I am following these Microsoft documents document1, document2.

I have these different artifacts that where I need to pass parameters from the yaml file using the TemplateParametersForWorkspace.json.

{
	"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
	"contentVersion": "1.0.0.0",
	"parameters": {
		"workspaceName": {
			"value": "devSynapse"
		},
		"LinkedService_connectionString": {
			"value": "Integrated Security=False;Encrypt=True;Connection Timeout=30;Data Source=;Initial Catalog="
		},
		"Trigger_Test_parameters_server": {
			"value": "devServer"
		}
	}
}

I need you to point me in right direction on how to pass variables from DevOps CI/CD to change the values according to the environment the Synpase workspace is deployed to?

Approach 1)

You can add Environment variable in the ARM template and use it for different environments in your pipeline like below:-

"connectionstring": {

"type": "string"

},

"environment": {

"type": "string"

}

},

"variables": {

"storageBlobDataContributorRoleID": "ba92f5b4-2d11-453d-a403-e96b0029c9fe",

"defaultDataLakeStorageAccountUrl": "[concat('https://', parameters('defaultDataLakeStorageAccountName'), '.dfs.core.windows.net')]",

"connectionString": "[if(equals(parameters('environment'), 'Production'), parameters('connectionStringProd'), parameters('connectionStringNonProd'))]"

Output:-

The above parameter checks, If the environment variable is set to Production, then it uses Production connection string or else other environment.

Approach 2)

You can also mention each production in the Azure DevOps Pipeline variables yaml script and use it in the Azure Pipelines like below:-

trigger:
- main

variables:
  - name: DevEnvironment
    value: "env-url"
  - name: TestEnvironment
    value: "env-url"
  - name: PreProdEnvironment
    value: "env-url"
  - name: ProdEnvironment
    value: "env-url"

stages:
- stage: Build
  displayName: 'Build stage'
  jobs:
  - job: Build
    displayName: 'Build job'
    steps:
    - script: echo 'Building the Synapse Analytics workspace...'
      displayName: 'Build step'

- stage: DeployToDev
  displayName: 'Deploy to Development'
  dependsOn: Build
  condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
  jobs:
  - job: Deploy
    displayName: 'Deploy job'
    steps:
    - task: AzureResourceGroupDeployment@2
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: 'MyAzureResourceManagerConnection'
        subscriptionId: '<subscription-id>'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '<resource-group-name>'
        location: '<resource-group-location>'
        templateLocation: 'Linked artifact'
        csmFile: '<path-to-arm-template>'
        csmParametersFile: '<path-to-arm-template-parameters>'
        overrideParameters: '-environment $(DevEnvironment)'

- stage: DeployToTest
  displayName: 'Deploy to Test'
  dependsOn: Build
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/main'), eq(variables['Build.Reason'], 'Schedule'))
  jobs:
  - job: Deploy
    displayName: 'Deploy job'
    steps:
    - task: AzureResourceGroupDeployment@2
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: 'MyAzureResourceManagerConnection'
        subscriptionId: '<subscription-id>'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '<resource-group-name>'
        location: '<resource-group-location>'
        templateLocation: 'Linked artifact'
        csmFile: '<path-to-arm-template>'
        csmParametersFile: '<path-to-arm-template-parameters>'
        overrideParameters: '-environment $(TestEnvironment)'

- stage: DeployToPreProd
  displayName: 'Deploy to Pre-Production'
  dependsOn: Build
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/main'), eq(variables['Build.Reason'], 'Manual'))
  jobs:
  - job: Deploy
    displayName: 'Deploy job'
    steps:
    - task: AzureResourceGroupDeployment@2
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: 'MyAzureResourceManagerConnection'
        subscriptionId: '<subscription-id>'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '<resource-group-name>'
        location: '<resource-group-location>'
        templateLocation: 'Linked artifact'
        csmFile: '<path-to-arm-template>'
        csmParametersFile: '<path-to-arm-template-parameters>'
        overrideParameters: '-environment $(PreProdEnvironment)'

- stage: DeployToProd
  displayName: 'Deploy to Production'
  dependsOn: Build
  condition: and(eq(variables['Build.SourceBranch'], 'refs/heads/main'), eq(variables['Build.Reason'], 'Manual'))
  jobs:
  - job: Deploy
    displayName: 'Deploy job'

Approach 3)

Alternatively you can also configure different stages Dev, Test, Prod, Pre-Prod to deploy your Synapse workspace in specific environment like below:-