英文:

Struggling with batch processing in Azure Table Storage locally

问题

我有这些包含超过100个实体的批次，我试图将它们拆分并发送到我的本地Azure存储账户（Azurite）：

```py
def save_results_in_database(self, operations, table_name):
        table = self.table_service_client.get_table_client(table_name)
        if table_name is None:
            raise Exception(f"表格 {table_name} 不存在！")
        try:
            table.submit_transaction(operations)
        except Exception as e:
            logging.exception(f"在表格 {table_name} 中保存结果到数据库时发生错误")

if 0 < len(results) < 100:
        helpers.save_results_in_database(results, action)
    else:
        # 如果结果数大于100，我们需要将它们分成每批50个的批次，并分批保存
        批次大小 = 50
        for i in range(0, len(results), 批次大小):
            批次 = results[i : i + 批次大小]
            helpers.save_results_in_database(批次, action)
            sleep(10)

由于某种原因，有些批次通过了，有些没有。例如，如果我有107个实体要推送，前50个通过了，接下来的50个没有，剩下的7个通过了。

我尝试加了一些延迟，但我不确定它是否真的有帮助。
这是错误消息：

在表格<表格名称>中保存结果到数据库时发生错误
文件“/usr/local/lib/python3.10/site-packages/azure/data/tables/_table_client.py”，第734行，在submit_transaction中
return self._batch_send(self.table_name, *batched_requests.requests, **kwargs)  # type: ignore
 文件“/usr/local/lib/python3.10/site-packages/azure/data/tables/_base_client.py”，第334行，在_batch_send中
    raise decoded
azure.data.tables._error.TableTransactionError: 2:在处理此请求时发生错误。
错误代码：InvalidInput
内容：{"odata.error":{"code":"InvalidInput","message":{"lang":"en-US","value":"2:在处理此请求时发生错误。\nRequestId:994b1eca-1109-4ff9-aa56-0dc73ae97a18\nTime:2023-04-06T13:08:29.278Z"}}}

我是否漏掉了什么？因为我相当确定输入在模式上是有效的。
提前致谢。

<details>
<summary>英文:</summary>

I have these batches that contain more than 100 entities that I&#39;m trying to split to send in my local Azure storage account (Azurite):

```py
def save_results_in_database(self, operations, table_name):
        table = self.table_service_client.get_table_client(table_name)
        if table_name is None:
            raise Exception(f&quot;Table {table_name} does not exist !&quot;)
        try:
            table.submit_transaction(operations)
        except Exception as e:
            logging.exception(
                f&quot;Error while saving results in database for table {table_name}&quot;
            )

if 0 &lt; len(results) &lt; 100:
        helpers.save_results_in_database(results, action)
    else:
        # if the number of results is greater than 100, we need to split them in batches of 50
        # and save them in batches
        batch_size = 50
        for i in range(0, len(results), batch_size):
            batch = results[i : i + batch_size]
            helpers.save_results_in_database(batch, action)
            sleep(10)

And for some reason somes batches pass and some don't. For example if I have 107 entities to push, the first 50 pass, the second 50 doesn't and the remaining 7 pass.

I tried putting some sleep but I'm not sure it actually helps.
Here's the error message:

Error while saving results in database for table <table name>
File "/usr/local/lib/python3.10/site-packages/azure/data/tables/_table_client.py", line 734, in submit_transaction
return self._batch_send(self.table_name, *batched_requests.requests, **kwargs)  # type: ignore
 File "/usr/local/lib/python3.10/site-packages/azure/data/tables/_base_client.py", line 334, in _batch_send
    raise decoded
azure.data.tables._error.TableTransactionError: 2:An error occurred while processing this request.
ErrorCode:InvalidInput
Content: {"odata.error":{"code":"InvalidInput","message":{"lang":"en-US","value":"2:An error occurred while processing this request.\nRequestId:994b1eca-1109-4ff9-aa56-0dc73ae97a18\nTime:2023-04-06T13:08:29.278Z"}}}

Am I missing something? Because I'm fairly certain that the input is valid schema-wise.
TIA

答案1

得分: 0

原来我错了，其中一个 RowKeys 中有一个"/"，导致操作失败。请确保检查您的数据！

英文:

Turns out I was wrong, one of the RowKeys had a "/" in it, breaking the operation.
Make sure to check your data!