英文:
Is it possible to build an Azure pipeline from blob to on-premises SFTP relying only on AZIR?
问题
我们有一个混合网络，使用MS ExpressRoute连接我们的本地环境与Azure云环境。
供应商的SFTP服务器端点在本地可用，通过IPsec隧道与供应商之间暴露。我们将我们的私有IP地址映射到在隧道上暴露的供应商私有IP地址。
是否可以配置一个数据工厂管道，用于在本地可用的端点（例如SFTP服务器）与Azure Blob存储之间执行数据传输，其中IR进程仅在Azure计算环境中运行？
因此，管道复制活动的源和接收端都将依赖于配置了AZIR而不是SHIR的链接服务。
如果这是可能的，是否需要在本地SFTP端点和数据工厂之间建立网络路由？还是在SFTP和Blob存储之间建立路由？
英文:
We have hybrid network with MS ExpressRoute connecting our on-prem environment to our Azure cloud environment.
A vendor SFTP server endpoint is available on-prem exposed via an IPsec tunnel between us and the vendor. We NAT our private IPs to the vendor private IP exposed on the tunnel.
Is it possible to configure a data-factory pipeline that performs data-transfer between an endpoint available on-premises ( such as a SFTP server ) and Azure blob storage, where the IR process runs exclusively in the Azure compute fabric ?
So the pipeline copy-activity source & sink would both rely on linked-services that are configured with AZIR, not SHIR.
If this is possible, would it entail establishing a network route between on-premises SFTP endpoint and data-factory ? Or between SFTP and blob storage ?
答案1
得分: 1
是可能的吗？是的。
有几种选择。这些选项主要与SFTP方面有关。
- 公共路由
仅用于上下文提及。不会（也不应该）被安全性批准。
网络团队需要为一系列已知的公共互联网地址（如前面的问题中提到的）打开一个入站端口，以连接到您的SFTP服务器。这允许您的AZITR进行连接。
- 托管的虚拟网络
请参阅此处https://learn.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint
您使用托管的虚拟网络为您的ADF进行配置。现在，您的AZIR有一个_私有_地址，您可以通过现有的“混合网络”路由到您的SFTP服务器。通常情况下，这是通过创建您的本地网络（托管SFTP的网络）与Azure网络枢纽之间的VPN链接，然后将ADF端点链接到Azure网络枢纽来完成的。
- 假装Blob是SFTP
不要将SFTP用作文件暂存区，而是更改您的流程，直接将文件放入Blob存储中。您应该能够无缝地执行此操作，即使用现有的SFTP脚本直接连接到Blob，就像它是SFTP一样。
https://learn.microsoft.com/en-us/azure/storage/blobs/secure-file-transfer-protocol-support
英文:
Is it possible? Yes.
There are a few options. These ones refer primarly to the SFTP side.
<h3>1. Public route </h3>
Only mentioned for context. Won't (and shouldn't) be approved by security
Network team needs to open an incoming port for a range of known public internet addresses (as mentioned in the prior question) to your SFTP server. This allows your AZITR to connect.
<h3> 2. Managed VNET</h3>
See here https://learn.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint
You provision your ADF with a managed VNET. Now your AZIR has a private address that you can route to your SFTP server over your existing "Hybrid netowrk". This would normally by done by creating a VPN link between your on prem network (which hosts SFTP) and your Azure network hub then link the ADF endpoint to the Azure network hub.
<h3>3. Pretend blob is SFTP</h3>
Instead of using SFTP as a file staging area, change your processes to drop the files directly in blob storage. You should be able to do this transparently, i.e. use your existing SFTP scripts to connect directly to blob as though it is SFTP
https://learn.microsoft.com/en-us/azure/storage/blobs/secure-file-transfer-protocol-support
答案2
得分: 0
ADF根据其托管的地区拥有一组静态IP地址列表。
因此，您可以在SFTP中将静态IP地址范围添加到白名单中，之后您可以从ADF访问SFTP（无需任何SHIR的需求）。
https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses
英文:
ADF has a list of static IPs based on the region in which it is hosted.
So you can whitelist the range of static IPs within the SFTP, post which you can access the SFTP from ADF (without the need of any SHIR)
https://learn.microsoft.com/en-us/azure/data-factory/azure-integration-runtime-ip-addresses
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论