如何异步(或并行)复制文件?

huangapple go评论53阅读模式
英文:

How to copy files async (or parallel)?

问题

I'm here to assist with the translation. Here's the translated code portion:

我正在尝试使用C#中的async/await方法来学习。我有以下简单的任务:
从一个Windows-PC复制2个文件到另一个Windows-PC(都在同一个本地网络上)。实际上,可能有大约1000个文件,但为简单起见,我将其减少到两个。
我的代码如下:

using System.Diagnostics;
public static class Program
{
    public static async Task Main()
    {
        var destinationPath = @"路径\到\目标\文件夹";
        List<string> filePaths = new()
        {
            @"\\远程-pc\c$\文件\文件1",
            @"\\远程-pc\c$\文件\文件2",
        };
        
        var watch = Stopwatch.StartNew();
        List<Task> tasks = new List<Task>();
        foreach (string path in filePaths)
        {
            Console.WriteLine($"开始复制 {Path.GetFileName(path)}");
            using var sourceStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 4096, useAsync: true);
            using var destinationStream = new FileStream(Path.Combine(destinationPath, Path.GetFileName(path)), FileMode.CreateNew, FileAccess.Write, FileShare.None, bufferSize: 4096, useAsync: true);
            var task = sourceStream.CopyToAsync(destinationStream).ContinueWith(_ => {
                Console.WriteLine($"结束复制 {Path.GetFileName(path)}");
            });
            tasks.Add(task);
        }
        
        await Task.WhenAll(tasks);
        watch.Stop();
        var elapsedMs = watch.ElapsedMilliseconds;
        Console.WriteLine(elapsedMs);
    }
}

If you have any further questions or need additional translations, feel free to ask.

英文:

I'm trying to learn using async/await methods in C#. I have the following simple task:
Copy 2 files from one Windows-PC to another (both one the same local network). In reality, there can be about 1000 files, but for simplicity, I reduce it to two.
My code:

using System.Diagnostics;
public static class Program
{
    public static async Task Main()
    {
        var destinationPath = @&quot;path\to\destination\folder&quot;;
        List&lt;string&gt; filePaths = new()
        {
            @&quot;\\remote-pc\c$\files\file1&quot;,
            @&quot;\\remote-pc\c$\files\file2&quot;,
        };
        
        var watch = Stopwatch.StartNew();
        List&lt;Task&gt; tasks = new List&lt;Task&gt;();
        foreach (string path in filePaths)
        {
            Console.WriteLine($&quot;Start copy {Path.GetFileName(path)}&quot;);
            using var sourceStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 4096, useAsync: true);
            using var destinationStream = new FileStream(Path.Combine(destinationPath, Path.GetFileName(path)), FileMode.CreateNew, FileAccess.Write, FileShare.None, bufferSize: 4096, useAsync: true);
            var task = sourceStream.CopyToAsync(destinationStream).ContinueWith(_ =&gt; {
                Console.WriteLine($&quot;End copy {Path.GetFileName(path)}&quot;);
            });
            tasks.Add(task);
        }
        
        await Task.WhenAll(tasks);
        watch.Stop();
        var elapsedMs = watch.ElapsedMilliseconds;
        Console.WriteLine(elapsedMs);
    }
}

If I understand correctly, async / await must be used in I / O-bound operations - just this case (or am I mistaken?). When executing the

sourceStream.CopyToAsync(destinationStream)

line, the calling thread will be freed to perform the next operation (apparently this is true, so when running at the same time
there are two WriteLine's about the beginning of copying files).
However, by analyzing the code execution time (elapsedMs), which is ~30 s. for two files, I conclude that the files are not copied in parallel at all. When you start copying each file separately, the execution time is ~20 s. and ~6 s. for each file respectively. Therefore, with "parallel copying" I expect the total execution time = the time of copying the largest file.

Please help me understand my reasoning.

答案1

得分: 0

确实不准确,因为在大多数情况下,总耗时不会等于最大文件的大小,因为通常会受到读取或写入磁盘的限制。可能会提高性能,但可能需要限制最大并行拷贝的数量。

代码部分已更正,不要使用 ContinueWith,而是使用 await。不要在任务外部处置流,否则拷贝可能失败,而是创建一个 async lambda,在其中进行处置。

foreach (string path in filePaths)
{
    Console.WriteLine($"开始拷贝 {Path.GetFileName(path)}");
    tasks.Add(Task.Run(async () =>
    {
        using var sourceStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 4096, useAsync: true);
        using var destinationStream = new FileStream(Path.Combine(destinationPath, Path.GetFileName(path)), FileMode.CreateNew, FileAccess.Write, FileShare.None, bufferSize: 4096, useAsync: true);
        await sourceStream.CopyToAsync(destinationStream);
        Console.WriteLine($"结束拷贝 {Path.GetFileName(path)}");
    }));
}

await Task.WhenAll(tasks);
英文:

It is not really true that the total time taken will be the size of the largest file, because in most cases you will bottleneck the reading or writing disks. You may still get better performance, but you might want to limit the maxmimium number of parallel copies.

Either way, your code is not correct. Do not use ContinueWith, use await instead. And don't dispose the streams outside the task, otherwise the copying may fail, instead make an async lambda and dispose inside it.

foreach (string path in filePaths)
{
    Console.WriteLine($&quot;Start copy {Path.GetFileName(path)}&quot;);
    tasks.Add(Task.Run(async () =&gt;
    {
        using var sourceStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 4096, useAsync: true);
        using var destinationStream = new FileStream(Path.Combine(destinationPath, Path.GetFileName(path)), FileMode.CreateNew, FileAccess.Write, FileShare.None, bufferSize: 4096, useAsync: true);
        await sourceStream.CopyToAsync(destinationStream);
        Console.WriteLine($&quot;End copy {Path.GetFileName(path)}&quot;);
    });
}

await Task.WhenAll(tasks);

huangapple
  • 本文由 发表于 2023年5月10日 20:54:32
  • 转载请务必保留本文链接:https://go.coder-hub.com/76218672.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定