在Azure WebJob中存储数据

huangapple go评论53阅读模式
英文:

Store data in a Azure WebJob

问题

I have created a gRPC client that receives data from a server.
The client runs as an Azure WebJob.

The received data looks like there is a replay id for each message.
I currently store this replay id locally as a file. The received replay id is appended to the file.
The Replay Id is needed so that if there is a problem it can continue from that ID.

If I run the client locally, this is not a problem, but I think that as a WebJob is not ideal.
My problem currently is that the files that are created/modified at runtime are stored here:

C:\local\Temp\jobs\continuous\SalesforceIngestor.App\ev4ygzfw.ezh

If the WebJob restarts for any reason, the data in the replayIdStorage.txt file will be lost.

Is there any way to safely store data in a WebJob so that it can be retrieved after a restart?

Or would Azure Blob Storage be a better option?

英文:

I have created a gRPC client that receives data from a server.
The client runs as a Azure WebJob.

The received data looks like that there is a replay id for each message.
I currently store this replay id locally as a file. The received replay id is appended to the file.
The Replay Id is needed so that if there is a problem it can continue from that ID.

If I run the client locally this is not a problem, but I think that as a WebJob is not ideal.
My problem currently is that the files that are created/modified at runtime are stored here:

C:\local\Temp\jobs\continuous\SalesforceIngestor.App\ev4ygzfw.ezh

If the WebJob restarts for any reason, the data in the replayIdStorage.txt file will be lost.

Is there any way to safely store data in a WebJob so that it can be retrieved after a restart?

Or would Azure Blob Storage be a better option?

答案1

得分: 1

我尝试了以下的.NET Web Job代码,可以成功地将Web应用程序的Web Jobs数据存储到我的存储账户容器中。

代码:

Program.cs:

using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Configuration;
using System;
using System.IO;

namespace WebJob1
{
    internal class Program
    {
        static void Main()
        {
            var config = new JobHostConfiguration();

            if (config.IsDevelopment)
            {
                config.UseDevelopmentSettings();
            }

            var host = new JobHost(config);
            host.RunAndBlock();
        }
    }
}

Function.cs:

using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.IO;

namespace WebJob1
{
    public class Functions
    {
        private const string ConnectionStringName = "AzureWebJobsStorage"; 

        public static void ProcessQueueMessage([QueueTrigger("queue")] string message, TextWriter log)
        {
            log.WriteLine(message);
            string replayId = GenerateReplayIdFromMessage(message);
            StoreReplayIdInBlobStorage(replayId);
        }

        private static string GenerateReplayIdFromMessage(string message)
        {
            return Guid.NewGuid().ToString();
        }

        private static void StoreReplayIdInBlobStorage(string replayId)
        {
            var storageAccount = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable(ConnectionStringName));
            var blobClient = storageAccount.CreateCloudBlobClient();
            var containerName = "replayids"; 
            var container = blobClient.GetContainerReference(containerName);
            container.CreateIfNotExists();

            var blobName = "replayIdStorage.txt"; 
            var blob = container.GetBlockBlobReference(blobName);
            blob.UploadText(replayId);
        }
    }
}

app.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<connectionStrings>
		<!-- 连接字符串的格式为 "DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" -->
		<!-- 对于本地执行,可以在此配置文件中设置值,也可以通过环境变量设置 -->
		<add name="AzureWebJobsDashboard" connectionString="<storage_connec_string>" />
		<add name="AzureWebJobsStorage" connectionString="<storage_connec_string>" />
	</connectionStrings>
	<startup>
		<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.8" />
	</startup>
	<runtime>
		<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
			<dependentAssembly>
				<assemblyIdentity name="Microsoft.WindowsAzure.Storage" publicKeyToken="31bf3856ad364e35" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
			</dependentAssembly>
			<dependentAssembly>
				<assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-12.0.0.0" newVersion="12.0.0.0" />
			</dependentAssembly>
		</assemblyBinding>
	</runtime>
</configuration>

输出:

它成功运行如下,

在Azure WebJob中存储数据

在我的Azure门户中,我创建了两个容器,如下所示,

在Azure WebJob中存储数据

我在azure-webjobs-hosts容器中得到了Id,如下所示,

在Azure WebJob中存储数据

并在azure-jobs-host-output容器中得到以下结果,

在Azure WebJob中存储数据

在Azure WebJob中存储数据

然后,我将我的代码发布到了web app,如下所示,

在Azure WebJob中存储数据

它成功地发布到了Azure门户中的web jobs。然后,我点击Run以开始运行Web Job,如下所示,

在Azure WebJob中存储数据

Web Job的运行成功开始,如下所示,

在Azure WebJob中存储数据

在Azure WebJob中存储数据

然后,我在我的存储账户的azure-jobs-host-output容器中得到了web app的详细信息,如下所示,

在Azure WebJob中存储数据

英文:

I tried below .Net Web Job code and can able to store the Web Jobs data of the web app in my storage account container.

Code:

Program.cs:

using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Configuration;
using System;
using System.IO;

namespace WebJob1
{
    internal class Program
    {
        static void Main()
        {
            var config = new JobHostConfiguration();

            if (config.IsDevelopment)
            {
                config.UseDevelopmentSettings();
            }

            var host = new JobHost(config);
            host.RunAndBlock();
        }
    }
}

Function.cs:

using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.IO;

namespace WebJob1
{
    public class Functions
    {
        private const string ConnectionStringName = "AzureWebJobsStorage"; 

        public static void ProcessQueueMessage([QueueTrigger("queue")] string message, TextWriter log)
        {
            log.WriteLine(message);
            string replayId = GenerateReplayIdFromMessage(message);
            StoreReplayIdInBlobStorage(replayId);
        }

        private static string GenerateReplayIdFromMessage(string message)
        {
            return Guid.NewGuid().ToString();
        }

        private static void StoreReplayIdInBlobStorage(string replayId)
        {
            var storageAccount = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable(ConnectionStringName));
            var blobClient = storageAccount.CreateCloudBlobClient();
            var containerName = "replayids"; 
            var container = blobClient.GetContainerReference(containerName);
            container.CreateIfNotExists();

            var blobName = "replayIdStorage.txt"; 
            var blob = container.GetBlockBlobReference(blobName);
            blob.UploadText(replayId);
        }
    }
}

app.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<connectionStrings>
		<!-- The format of the connection string is "DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" -->
		<!-- For local execution, the value can be set either in this config file or through environment variables -->
		<add name="AzureWebJobsDashboard" connectionString="<storage_connec_string>" />
		<add name="AzureWebJobsStorage" connectionString="<storage_connec_string>" />
	</connectionStrings>
	<startup>
		<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.8" />
	</startup>
	<runtime>
		<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
			<dependentAssembly>
				<assemblyIdentity name="Microsoft.WindowsAzure.Storage" publicKeyToken="31bf3856ad364e35" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
			</dependentAssembly>
			<dependentAssembly>
				<assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
				<bindingRedirect oldVersion="0.0.0.0-12.0.0.0" newVersion="12.0.0.0" />
			</dependentAssembly>
		</assemblyBinding>
	</runtime>
</configuration>

Output:

It runs successfully as below,

在Azure WebJob中存储数据

I got two containers created in my storage account at Azure portal as below,

在Azure WebJob中存储数据

I got Id in azure-webjobs-hosts container as below,

在Azure WebJob中存储数据

And got below in azure-jobs-host-output container,

在Azure WebJob中存储数据

在Azure WebJob中存储数据

Then, I published my code to web app as below,

在Azure WebJob中存储数据

It successfully published to web jobs in web app at Azure Portal. Then, I click on Run to start running the web job as below,

在Azure WebJob中存储数据

Web Job running started successfully as below,

在Azure WebJob中存储数据

在Azure WebJob中存储数据

Then, I got web app details stored im my storage account azure-jobs-host-output container as below,

在Azure WebJob中存储数据

huangapple
  • 本文由 发表于 2023年7月4日 22:27:16
  • 转载请务必保留本文链接:https://go.coder-hub.com/76613629.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定