如何使用Google.Cloud.Storage在.NET中下载多个文件从GCP?

huangapple go评论54阅读模式
英文:

How to download multiple files from GCP using Google.Cloud.Storage in .net?

问题

今天图像保存在GCP的单独文件夹中,因此需要执行多个查询来搜索它们。为了优化查询,理想情况是将图像保存在一个文件夹中,并在单个查询中获取所有图像。Google.Cloud.Storage软件包似乎不提供下载文件列表的选项,是否有办法实现这个或者有其他替代方案?

英文:

Today images are saved in separate folders in GCP, so it is necessary to do more than one query to search for them. To optimize the query, the ideal would be to save the images in a single folder and bring them all in a single query. The Google.Cloud.Storage package doesn't seem to provide the option to download a list of files, is there a way to do this or an alternative?

答案1

得分: 1

The Google.Cloud.Storage package doesn't seem to provide the option to download a list of files, is there a way to do this or an alternative?

我们的库中没有提供此选项,因为底层 API 也没有提供。如果我们尝试在底层 API 之上添加某种并行化,很可能会对某些甚至大多数用户不适用。

客户端是线程安全的,因此可以完全在并行中启动多个下载操作,无论是在单独的线程中使用同步调用,或者(可能更好的是)只是使用 Task.WhenAll 启动多个异步调用,以确定它们何时全部完成。

请注意,GCS 中对象的位置不会对此产生影响 - 目录结构实际上是合成的;下载两个具有共同前缀名称的对象与下载两个完全不同名称的对象一样快。使用单个文件夹可以更容易找到要下载的所有对象,但是可以使用带有适当前缀的 ListObjects 调用来实现相同的效果。

英文:

> The Google.Cloud.Storage package doesn't seem to provide the option to download a list of files, is there a way to do this or an alternative?

We don't provide it in the library because it's not provided in the underlying API. If we were to try to add some sort of parallelization over the top of the underlying API, chances are it would be a bad fit for some or even most users.

The client is thread-safe, so it's absolutely fine to start multiple downloads in parallel, whether using synchronous calls in separate threads, or (probably better) just multiple asynchronous calls using Task.WhenAll to determine when they're all complete.

Note that the location of the objects within GCS won't affect this at all - the directory structure is effectively synthetic; downloading two objects with names that happen to have a common prefix is no quicker than downloading two objects with entirely different names. Using a single folder would make it easier to find all the objects to download, however, using a ListObjects call with a suitable prefix.

huangapple
  • 本文由 发表于 2023年5月22日 22:58:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/76307455.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定