英文:
How to update SQL Server table from ADF?
问题
I need to update the Azure SQL Server table if another SQL Server table is updated.
For example: I have one Azure SQL table dbo.Azuretable
, and I also have a SQL Server table dbo.sqlservertable
. Both these tables are part of two different SQL Server instances, but the table schema is exactly the same.
What happens is dbo.sqlservertable
updates on a daily basis. Let's say every day one row inserts in dbo.sqlservertable
, I want the same row to be inserted into dbo.Azuretable
as soon as dbo.sqlservertable
gets updated. I want to do it in Azure if possible, or I am open to any other method as well.
What I have done so far:
I created one copy activity which replicates the changes to dbo.Azuretable
whatever happened in dbo.sqlservertable
. But I need to manually trigger it because I don't know when my dbo.sqlservertable
gets updated. I want to automate this process.
英文:
I need to update the Azure SQL Server table if another SQL Server tables is updated.
For example: I have one Azure SQL table dbo.Azuretable
and I also have a SQL Server table dbo.sqlservertable
. Both these tables are part of two different SQL Server instances, but the table schema is exactly the same.
What happens is dbo.sqlservertable
updates on daily basis, let's say every day one row inserts in dbo.sqlservertable
, I want the same row to be inserted into dbo.Azuretable
as soon as dbo.sqlservertable
gets updated. I want to do it Azure if possible or I am open to any other method as well.
What I have done so far:
I created one copy activity which replicates the changes to dbo.Azuretable
what ever happened in dbo.sqlservertable
. But I need to manually trigger it because I don't know when my dbo.sqlservertable
gets updated. I want to automate this process.
答案1
得分: 2
First, you require to find inserted rows. Use except functions.
首先,您需要找到插入的行。使用 except 函数。
select * from dbo.sqlservertable
except
select * from dbo.Azuretable
然后添加到 "dbo.Azuretable"
然后添加到 "dbo.Azuretable"
WITH new_records AS (
select * from dbo.sqlservertable
except
select * from dbo.Azuretable
)
INSERT INTO dbo.Azuretable
SELECT * FROM new_records
You can use data factory or synapse pipeline for it. Pipeline->Activities->General-> Script meets your demand. After that you'll see a lightning icon named "add trigger," click and choose "new/edit." Bring your mouse to choose a trigger and click. Now, click "new." There are all schedule settings. I hope this solution helps you.
您可以使用数据工厂或 Synapse 管道来完成这项任务。管道->活动->常规-> 脚本 符合您的需求。之后,您将看到一个名为 "add trigger" 的闪电图标,点击并选择 "new/edit"。将鼠标移到触发器上并点击。现在,点击 "new"。这里有所有的调度设置。希望这个解决方案对您有帮助。
英文:
First, you require to find inserted rows. Use except functions.
select * from dbo.sqlservertable
except
select * from dbo.Azuretable
Then add to "dbo.Azuretable"
WITH new_records AS(
select * from dbo.sqlservertable
except
select * from dbo.Azuretable
)
INSERT INTO dbo.Azuretable
SELECT * FROM new_records
You can use data factory or synapse pipeline for it. Pipeline->Activities->General-> Script meets your demand. After that you'll see lightning icon names add trigger, click and new/edit. Bring your mouse to choose trigger and click. Now, click new. There are all schedule settings. I hope this solution helps you.
答案2
得分: 0
以下是翻译好的部分:
-
使用 Azure SQL 数据库中的数据同步功能,从 SQL Server 同步数据到 Azure SQL 数据库。
https://learn.microsoft.com/en-us/azure/azure-sql/database/sql-data-sync-data-sql-server-sql-database?view=azuresql -
您可以在 SQL Server 2017 中的表上创建触发器,在插入、更新、删除时触发,并通过触发器调用存储过程,将数据通过 PolyBase 导出到 Blob 存储。利用 ADF 管道中的 Blob 触发器来将数据从 SQL Server 同步到 Azure SQL 数据库。
https://learn.microsoft.com/en-us/sql/relational-databases/polybase/polybase-versioned-feature-summary?view=sql-server-ver16
或者,您还可以触发一个 SQL Server 作业,其中可以包含用于生成 Blob 存储中文件的 PowerShell 逻辑,或者触发 ADF REST API 来触发管道。
示例参考:
https://datasharkx.wordpress.com/2022/10/01/event-trigger-data-sync-from-sql-server-to-synapse-via-azure-data-factory-synapse-pipeline/
英文:
There are multiple ways to sync the data :
-
use data sync in azure sql database to sync from sql server to Azure SQL database
https://learn.microsoft.com/en-us/azure/azure-sql/database/sql-data-sync-data-sql-server-sql-database?view=azuresql -
You can create a trigger on the table in SQL server 2017 on insert,update,delete
and via trigger, call a stored procedure which would export data into blob via polybase.
leverage blob trigger in adf pipeline to sync data from sql server to Azure SQL database
Else you can also trigger a SQL server job wherein you can have a powershell logic to generate a file in blob or trigger ADF REST API for pipeline trigger
sample reference:
https://datasharkx.wordpress.com/2022/10/01/event-trigger-data-sync-from-sql-server-to-synapse-via-azure-data-factory-synapse-pipeline/
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论