英文:
How to send large file from controller endpoint in ASP.NET Core
问题
我正在尝试从我的 asp.net api 控制器发送一个相当大的 .zip
存档文件。我的设置如下所示:
我有一个 Angular 前端,它发起了一个用于获取存档文件的 GET
请求:
public getArchive(archiveName: string): Observable<any> {
return this.http.get(
`${this.runtimeConfig.backendApiPath}api/tenants/${this.auth.profile.tid}/objects/archives/${archiveName}`,
{
headers: new HttpHeaders({ Authorization: `Bearer ${this.auth.accessToken}` }),
});
}
然后,这个请求在后端的 BFF (Backend For Frontend) api 中处理:
[HttpGet("archives/{archiveName}")]
public async Task<IActionResult> GetArchive(
[FromRoute] string archiveName)
{
try
{
var result = await this._objectsServiceClient.GetArchive(archiveName);
return Ok(result);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error while retrieving archive.");
return BadRequest(ex.Message);
}
}
该服务返回一个来自后端 Api 的 byte[]
,这是一个 GET
请求的结果:
public Task<byte[]> GetArchive(string archiveName)
{
var url = $"{this.options.BackendUrl}"
.AppendPathSegments("api", "tenants", $"{this.tenantInfo.TenantId}", "objects", "archives", archiveName)
.SetQueryParam("api-version", "1.0")
.ToString();
var res = this.Get<byte[]>(url);
return res;
}
我有一个名为 this.Get<byte[]>
的方法:
protected async Task<T> Get<T>(string url)
{
var response = await _httpClient.GetAsync(new Uri(url));
await response.EnsureSuccessStatusCodeExtendedAsync();
var retVal = await response.Content.ReadAsAsync<T>();
return retVal;
}
后端控制器的端点代码如下:
[HttpGet("objects/archives/{archiveName}")]
public async Task<IActionResult> GetArchive(Guid tenantId, [FromRoute] string archiveName)
{
using (Infrastructure.Extensions.LoggerExtensions.BeginScope(
this.logger,
LoggingScopeField.TenantId(tenantId)))
{
try
{
var path = new BlobStorageRepoPathBuilder()
.WithId(tenantId)
.WithPath("archives")
.WithId(this.userInfo.UserId)
.WithPath($"{archiveName}.zip")
.Build();
var blob = await this.repo.DownloadAsync(path);
using (MemoryStream ms = new MemoryStream())
{
await blob.Stream.CopyToAsync(ms);
ms.Position = 0;
return Ok(ms.ToArray());
}
}
catch (Exception ex)
{
this.logger.LogError(ex, "Error occurred during report generation.");
return this.BadRequest("Error occurred during report generation. Please check the logs.");
}
}
}
基本上,我从文件存储中下载文件,将其加载到 MemoryStream
中,然后将其作为 byte[]
发送。对于小文件来说这是可以的(我知道使用 MemoryStream
不是最佳决策,这就是我想要更改它的原因),但是当一个大文件出现(例如大于 2GB)时,我会收到 OutOfMemory
异常。我找到了一篇文章,描述了如何接收大文件,而不是将其作为响应发送。因此,我的问题是,如何处理大于 2GB 的文件,并从中删除 MemoryStream
。感谢您提前的回答!
英文:
I'm trying to send a quite large .zip
archive from my asp.net api controller.
My setup is something like this:
I have an Angular frontend that makes a GET
request for an archive file.
public getArchive(archiveName: string): Observable<any> {
return this.http.get(
`${this.runtimeConfig.backendApiPath}api/tenants/${this.auth.profile.tid}/objects/archives/${archiveName}`,
{
headers: new HttpHeaders({ Authorization: `Bearer ${this.auth.accessToken}` }),
});
}
Then this request is handled in a Backend For Frontend (BFF) api:
[HttpGet("archives/{archiveName}")]
public async Task<IActionResult> GetArchive(
[FromRoute] string archiveName)
{
try
{
var result = await this._objectsServiceClient.GetArchive(archiveName);
return Ok(result);
}
catch (Exception ex)
{
_logger.LogError(ex, $"Error while retrieving archive.");
return BadRequest(ex.Message);
}
}
the service returns a byte[]
that is coming from a GET
request from the backend Api:
public Task<byte[]> GetArchive(string archiveName)
{
var url = $@"{this.options.BackendUrl}"
.AppendPathSegments("api", "tenants", $"{this.tenantInfo.TenantId}", "objects", "archives", archiveName)
.SetQueryParam("api-version", "1.0")
.ToString();
var res = this.Get<byte[]>(url);
return res;
}
and the this.Get<byte[]>
method I have:
protected async Task<T> Get<T>(string url)
{
var response = await _httpClient.GetAsync(new Uri(url));
await response.EnsureSuccessStatusCodeExtendedAsync();
var retVal = await response.Content.ReadAsAsync<T>();
return retVal;
}
The backend controller's endpoint has the following code:
[HttpGet("objects/archives/{archiveName}")]
public async Task<IActionResult> GetArchive(Guid tenantId, [FromRoute] string archiveName)
{
using (Infrastructure.Extensions.LoggerExtensions.BeginScope(
this.logger,
LoggingScopeField.TenantId(tenantId)))
{
try
{
var path = new BlobStorageRepoPathBuilder()
.WithId(tenantId)
.WithPath("archives")
.WithId(this.userInfo.UserId)
.WithPath($"{archiveName}.zip")
.Build();
var blob = await this.repo.DownloadAsync(path);
using (MemoryStream ms = new MemoryStream())
{
await blob.Stream.CopyToAsync(ms);
ms.Position = 0;
return Ok(ms.ToArray());
}
}
catch (Exception ex)
{
this.logger.LogError(ex, "Error occurred during report generation.");
return this.BadRequest("Error occurred during report generation. Please check the logs.");
}
}
}
Basically I download the file from a file storage, load it in a MemoryStream
and send it as a byte[]
.This works fine with small files (I know that using MemoryStream
is not the best decision, that's why I want to change that), but when a large file comes (like more than 2GB) I got an OutOfMemory
exceptions. I found out an article that describes how to receive a large file, not to send one as a response.
So my question is, how can I handle large (>2GB) files, and remove the MemoryStream
from the equation
Thanks in advance!
答案1
得分: 1
我不完全确定this.repo.DownloadAsync
的内部工作原理,因为它没有包含在您的问题中,但似乎它只是提供了文件源的直接IO响应流(看起来像Azure Blob Storage)。如果是这样的话,为什么不直接将文件流传输到客户端呢?:
var blob = await this.repo.DownloadAsync(path);
return File(blob.Stream, "application/zip", fileDownloadName: $"{archiveName}.zip");
根据您对这个答案的评论,似乎this.repo.DownloadAsync
是基于HttpClient
的,您可以使用以下代码直接从HttpClient
获取流:
HttpClient client; // 您已经在使用的HttpClient实例
var stream = await client.GetStreamAsync(theUrl);
流将在接收到HTTP响应读取器后返回给您,然后您可以将其返回给您的控制器方法,然后使用return File
直接通过您的服务器将结果流式传输给客户端,而不需要先下载它。
英文:
I'm not 100% sure how the internals of this.repo.DownloadAsync
work since it's not included in your question, but it appears that it just provides the direct IO response stream from the file source (which looks like Azure Blob Storage). If that's the case, why not just stream the file to the client?:
var blob = await this.repo.DownloadAsync(path);
return File(blob.Stream, "application/zip", fileDownloadName: $"{archiveName}.zip");
From your comment on this answer, it seems like this.repo.DownloadAsync
is using HttpClient
based on your comment. To directly get the stream from HttpClient
you can use the following code:
HttpClient client; // your HttpClient instance that you're already using
var stream = await client.GetStreamAsync(theUrl);
The stream will be returned to you after the HTTP response readers have been received, and then you can return this to your controller method, and then to the client using return File
to directly stream the result through your server without first downloading it.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论