英文:
Ingest multiple files from storage using Kusto
问题
我使用.export
命令 导出了约 6000 万行到存储,如下所示:
.export async to json (h@"https://contoso.blob.core.windows.net/oversub/adx/VmCounterFiveMinutes/2023-05-20;storage-account-access-key") <|
contosoEntity
我从查询中删除了查询的一部分,但导出正常。导出了 192 个文件,共计 62374652 行。
我还需要将数据导入到另一个 Kusto 群集,如何使用.ingest
命令从存储中摄取所有存储中特定文件夹中的 .json
文件呢?
如果有其他更好的方案用于将数据摄取回另一个 Kusto 群集,我将很高兴知道。
我尝试过但失败了的方法:
1-
.ingest into table VmCounterFiveMinuteRoleInstanceCentralBondTable_2 (h@'https://contoso.blob.core.windows.net/oversub/adx/VmCounterFiveMinutes/2023-05-20/*.json;storage-account-access-key')
2-
.ingest into table VmCounterFiveMinuteRoleInstanceCentralBondTable_2 (h@'https://contoso.blob.core.windows.net/oversub/adx/VmCounterFiveMinutes/2023-05-20/*.ext;storage-account-access-key')
with (format='json')
英文:
I exported about 60 millions rows using the .export
command to a storage as my KQL script below:
.export async to json (h@"https://contoso.blob.core.windows.net/oversub/adx/VmCounterFiveMinutes/2023-05-20;storage-account-access-key") <|
contosoEntity
I removed sentity parts from my query, but the exportation works fine. It exported 192 file(s), total of 62374652 rows.
I also need to import into another Kusto cluster, how can I use the .ingest
command from storage to ingest all .json
files from a specific folder from storage?
If there any other better solution to ingest back to another Kusto cluster, I will be glad to know.
What I tried and failed:
1-
.ingest into table VmCounterFiveMinuteRoleInstanceCentralBondTable_2 (h@'https://contoso.blob.core.windows.net/oversub/adx/VmCounterFiveMinutes/2023-05-20/*.json;storage-account-access-key')
2-
.ingest into table VmCounterFiveMinuteRoleInstanceCentralBondTable_2 (h@'https://contoso.blob.core.windows.net/oversub/adx/VmCounterFiveMinutes/2023-05-20/*.ext;storage-account-access-key')
with (format='json')
答案1
得分: 1
你可以使用名为"lightingest"的命令行工具,或在Kusto Web资源管理器中使用OneClick向导工具。
英文:
You can use the command line tool called "lightingest" or use the OneClick wizard tool in Kusto Web explorer
答案2
得分: 0
你在运行(不建议使用的)直接导入命令(.ingest
)时无法指定通配符(*
)路径。
你可以使用LightIngest或导入向导来导入已导出的文件(如果需要,可以筛选 blob 名称前缀)。
英文:
you can't specify a wildcarded (*
) path when running the (unrecommended) direct ingestion command (.ingest
).
you could use LightIngest or the Ingestion wizard to ingest the files you've exported (with filters on blob name prefix, if required)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论