英文:
How do I create bulk upserts for azure tables with Azure.Data.Tables
问题
我目前正在尝试使用Azure表存储。
我正在使用Azure.Data.Tables
而不是Microsoft.WindowsAzure.Storage.Table
,因为我了解到它是更新版本,将得到维护。
所以现在我有这个类,它有两个方法,一个用于插入更新,另一个用于插入更新多个实体:
public class TableRepository
{
private TableServiceClient _tableServiceClient;
private TableClient _tableClient;
public TableRepository(string key, string table)
{
_tableServiceClient = new TableServiceClient(key);
_tableClient = _tableServiceClient.GetTableClient(tableName: table);
}
public async Task AddAsync(TableEntity mapping)
{
await _tableClient.UpsertEntityAsync(mapping);
}
public async Task AddEntitiesToTableAsync(IEnumerable<TableEntity> mappings)
{
foreach (var m in mappings)
{
await _tableClient.UpsertEntityAsync(m);
}
}
}
对于我的批量插入,虽然它有效,但对于大量插入来说速度较慢,我想看看是否可以批量插入实体。
我发现对于Microsoft.WindowsAzure.Storage.Table
,我可以执行一批操作,做类似这样的事情:
public async Task AddEntitiesToTableAsync(IEnumerable<EmailToIdMapping> mappings)
{
// 获取表示Azure存储表的CloudTable实例
CloudTable table = GetCloudTable(); // 用自己的逻辑替换以获取CloudTable
// 创建TableOperation的列表以保存批处理操作
List<TableOperation> batchOperations = new List<TableOperation>();
foreach (var m in mappings)
{
var tableEntity = new DynamicTableEntity(m.PartitionKey, m.RowKey)
{
Properties = new Dictionary<string, EntityProperty>
{
{ "Email", new EntityProperty(m.Email) },
{ "Id", new EntityProperty(m.Id) }
}
};
// 为实体创建InsertOrReplace操作
TableOperation upsertOperation = TableOperation.InsertOrReplace(tableEntity);
batchOperations.Add(upsertOperation);
}
// 使用批处理操作创建TableBatchOperation
TableBatchOperation batchOperation = new TableBatchOperation();
batchOperation.AddRange(batchOperations);
// 执行批处理操作
await table.ExecuteBatchAsync(batchOperation);
}
但我在Azure.Data.Tables
中找不到类似的东西,我唯一能想到的是以更智能的方式并行执行我已经在做的事情,以提高效率。
我可以做什么?
英文:
I am currently dipping my toes in the water with azure table storage.
I am using Azure.Data.Tables
rather than Microsoft.WindowsAzure.Storage.Table
, since I could read that it was the newer version, which would be maintained.
So now I have this class, which has to methods, one for upserting, and one for upserting many entities:
public class TableRepository
{
private TableServiceClient _tableServiceClient;
private TableClient _tableClient;
public TableRepository(string key, string table)
{
_tableServiceClient = new TableServiceClient(key);
_tableClient = _tableServiceClient.GetTableClient(tableName: table);
}
public async Task AddAsync(TableEntity mapping)
{
await _tableClient.UpsertEntityAsync(mapping);
}
public async Task AddEntitiesToTableAsync(IEnumerable<TableEntity> mappings)
{
foreach (var m in mappings)
{
await _tableClient.UpsertEntityAsync(m);
}
}
}
And for my bulk upsert, this works, but is slow for many upserts, I wanted to see If I could upsert entities in bulk.
I figured out that for Microsoft.WindowsAzure.Storage.Table
, I could execute a batch of operations, and do something like this:
public async Task AddEntitiesToTableAsync(IEnumerable<EmailToIdMapping> mappings)
{
// Retrieve the CloudTable instance representing your Azure Storage table
CloudTable table = GetCloudTable(); // Replace with your own logic to get the CloudTable
// Create a List of TableOperation to hold the batch operations
List<TableOperation> batchOperations = new List<TableOperation>();
foreach (var m in mappings)
{
var tableEntity = new DynamicTableEntity(m.PartitionKey, m.RowKey)
{
Properties = new Dictionary<string, EntityProperty>
{
{ "Email", new EntityProperty(m.Email) },
{ "Id", new EntityProperty(m.Id) }
}
};
// Create an InsertOrReplace operation for the entity
TableOperation upsertOperation = TableOperation.InsertOrReplace(tableEntity);
batchOperations.Add(upsertOperation);
}
// Create a TableBatchOperation with the batch operations
TableBatchOperation batchOperation = new TableBatchOperation();
batchOperation.AddRange(batchOperations);
// Execute the batch operation
await table.ExecuteBatchAsync(batchOperation);
}
But I cannot find anything similar for Azure.Data.Tables
, the only thing I can think of is to find a smarter way to do the thing I am already doing more effeciently in parralel
What can I do?
答案1
得分: 1
尝试使用Azure.Data.Tables
中的SubmitTransactionAsync
方法:
try
{
var actions = new List<TableTransactionAction>();
// 遍历实体并累加值
foreach (var entity in entities)
{
var upsertAction = new UpsertEntityAction(entity);
actions.Add(upsertAction);
}
var transaction = new TableTransaction(actions);
var tableClient = client.GetTableClient("<pass-your-table-name-here>");
// 作为批处理事务执行
await tableClient.SubmitTransactionAsync(transaction);
}
catch (Exception e)
{
throw;
}
请注意,为了成功,您应该具有相同的分区键。否则,根据我的了解,您无法实现批处理事务。
英文:
Try out the SubmitTransactionAsync
method in Azure.Data.Tables
try
{
var actions = new List<TableTransactionAction>();
// Iterate over the entities and add up values
foreach (var entity in entities)
{
var upsertAction = new UpsertEntityAction(entity);
actions.Add(upsertAction);
}
var transaction = new TableTransaction(actions);
var tableClient = client.GetTableClient("<pass-your-table-name-here>");
// Execute as a batch transaction
await tableClient.SubmitTransactionAsync(transaction);
}
catch (Exception e)
{
throw;
}
Note that in order to you to succeed you should have the same partitioning key. Otherwise, as of my knowledge, there is no you can achieve batch transactions
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论