如何从ADF更新SQL Server表?

huangapple go评论65阅读模式
英文:

How to update SQL Server table from ADF?

问题

I need to update the Azure SQL Server table if another SQL Server table is updated.

For example: I have one Azure SQL table dbo.Azuretable, and I also have a SQL Server table dbo.sqlservertable. Both these tables are part of two different SQL Server instances, but the table schema is exactly the same.

What happens is dbo.sqlservertable updates on a daily basis. Let's say every day one row inserts in dbo.sqlservertable, I want the same row to be inserted into dbo.Azuretable as soon as dbo.sqlservertable gets updated. I want to do it in Azure if possible, or I am open to any other method as well.

What I have done so far:

I created one copy activity which replicates the changes to dbo.Azuretable whatever happened in dbo.sqlservertable. But I need to manually trigger it because I don't know when my dbo.sqlservertable gets updated. I want to automate this process.

英文:

I need to update the Azure SQL Server table if another SQL Server tables is updated.

For example: I have one Azure SQL table dbo.Azuretable and I also have a SQL Server table dbo.sqlservertable. Both these tables are part of two different SQL Server instances, but the table schema is exactly the same.

What happens is dbo.sqlservertable updates on daily basis, let's say every day one row inserts in dbo.sqlservertable, I want the same row to be inserted into dbo.Azuretable as soon as dbo.sqlservertable gets updated. I want to do it Azure if possible or I am open to any other method as well.

What I have done so far:

I created one copy activity which replicates the changes to dbo.Azuretable what ever happened in dbo.sqlservertable. But I need to manually trigger it because I don't know when my dbo.sqlservertable gets updated. I want to automate this process.

答案1

得分: 2

First, you require to find inserted rows. Use except functions.

首先,您需要找到插入的行。使用 except 函数。

select * from dbo.sqlservertable
except
select * from dbo.Azuretable

然后添加到 "dbo.Azuretable"

然后添加到 "dbo.Azuretable"

WITH new_records AS (
select * from dbo.sqlservertable
except
select * from dbo.Azuretable
)

INSERT INTO dbo.Azuretable
SELECT * FROM new_records

You can use data factory or synapse pipeline for it. Pipeline->Activities->General-> Script meets your demand. After that you'll see a lightning icon named "add trigger," click and choose "new/edit." Bring your mouse to choose a trigger and click. Now, click "new." There are all schedule settings. I hope this solution helps you.

您可以使用数据工厂或 Synapse 管道来完成这项任务。管道->活动->常规-> 脚本 符合您的需求。之后,您将看到一个名为 "add trigger" 的闪电图标,点击并选择 "new/edit"。将鼠标移到触发器上并点击。现在,点击 "new"。这里有所有的调度设置。希望这个解决方案对您有帮助。

英文:

First, you require to find inserted rows. Use except functions.

select * from dbo.sqlservertable
except
select * from dbo.Azuretable

Then add to "dbo.Azuretable"

WITH new_records AS(
    select * from dbo.sqlservertable
    except
    select * from dbo.Azuretable
)

INSERT INTO dbo.Azuretable
SELECT * FROM new_records 

You can use data factory or synapse pipeline for it. Pipeline->Activities->General-> Script meets your demand. After that you'll see lightning icon names add trigger, click and new/edit. Bring your mouse to choose trigger and click. Now, click new. There are all schedule settings. I hope this solution helps you.

如何从ADF更新SQL Server表?

答案2

得分: 0

以下是翻译好的部分:

  1. 使用 Azure SQL 数据库中的数据同步功能,从 SQL Server 同步数据到 Azure SQL 数据库。
    https://learn.microsoft.com/en-us/azure/azure-sql/database/sql-data-sync-data-sql-server-sql-database?view=azuresql

  2. 您可以在 SQL Server 2017 中的表上创建触发器,在插入、更新、删除时触发,并通过触发器调用存储过程,将数据通过 PolyBase 导出到 Blob 存储。利用 ADF 管道中的 Blob 触发器来将数据从 SQL Server 同步到 Azure SQL 数据库。
    https://learn.microsoft.com/en-us/sql/relational-databases/polybase/polybase-versioned-feature-summary?view=sql-server-ver16

或者,您还可以触发一个 SQL Server 作业,其中可以包含用于生成 Blob 存储中文件的 PowerShell 逻辑,或者触发 ADF REST API 来触发管道。
示例参考:
https://datasharkx.wordpress.com/2022/10/01/event-trigger-data-sync-from-sql-server-to-synapse-via-azure-data-factory-synapse-pipeline/

英文:

There are multiple ways to sync the data :

  1. use data sync in azure sql database to sync from sql server to Azure SQL database
    https://learn.microsoft.com/en-us/azure/azure-sql/database/sql-data-sync-data-sql-server-sql-database?view=azuresql

  2. You can create a trigger on the table in SQL server 2017 on insert,update,delete

and via trigger, call a stored procedure which would export data into blob via polybase.
leverage blob trigger in adf pipeline to sync data from sql server to Azure SQL database

https://learn.microsoft.com/en-us/sql/relational-databases/polybase/polybase-versioned-feature-summary?view=sql-server-ver16

Else you can also trigger a SQL server job wherein you can have a powershell logic to generate a file in blob or trigger ADF REST API for pipeline trigger

sample reference:
https://datasharkx.wordpress.com/2022/10/01/event-trigger-data-sync-from-sql-server-to-synapse-via-azure-data-factory-synapse-pipeline/

huangapple
  • 本文由 发表于 2023年4月13日 17:20:50
  • 转载请务必保留本文链接:https://go.coder-hub.com/76003781.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定