The Enumerable not taking the next batch, instead always taking the first 3 items.

huangapple go评论98阅读模式
英文:

Why is the Enumerable not taking the next batch, instead always taking the first 3 items

问题

  1. 我正在进行一个将字符串列表分成批次并异步处理每个批次的POC
  2. 但当我运行程序时,它总是取第一组项目(批次大小为3)。
  3. 所以请问有谁能帮我解决如何移动到列表的下一组项目。
  4. `Take` 是我写的一个扩展方法。我尝试使用了 `async/await` 模式。
  5. 提前感谢
  6. public class Program
  7. {
  8. public static async Task Main(string[] args)
  9. {
  10. var obj = new Class1();
  11. List<string> fruits = new()
  12. {
  13. "1",
  14. "2",
  15. "3",
  16. "4",
  17. "5",
  18. "6",
  19. "7",
  20. "8",
  21. "9",
  22. "10"
  23. };
  24. await Class1.Start(fruits);
  25. Console.ReadLine();
  26. }
  27. }
  28. public class Class1
  29. {
  30. private const int batchSize = 3;
  31. public static async Task Start(List<string> fruits)
  32. {
  33. if (fruits == null)
  34. return;
  35. var e = fruits.GetEnumerator();
  36. while (true)
  37. {
  38. var batch = e.Take(3); // 总是取前3个项目,而不移动到列表的下一个项目
  39. if (batch.Count == 0)
  40. {
  41. break;
  42. }
  43. await StartProcessing(batch);
  44. }
  45. }
  46. public static async Task StartProcessing(List<string> batch)
  47. {
  48. await Parallel.ForEachAsync(batch, async (item, CancellationToken) =>
  49. {
  50. var list = new List<string>();
  51. await Task.Delay(1000);
  52. Console.WriteLine($"水果名称:{item}");
  53. list.Add(item);
  54. });
  55. }
  56. }
  57. *****Extension.cs*****
  58. public static class Extensions
  59. {
  60. public static List<T> Take<T>(this IEnumerator<T> e, int num)
  61. {
  62. List<T> list = new List<T>(num);
  63. int taken = 0;
  64. while (taken < num && e.MoveNext())
  65. {
  66. list.Add(e.Current);
  67. taken++;
  68. }
  69. return list;
  70. }
  71. }
英文:

I'm doing a POC to split a List of strings into batches and process each batch asynchronously.
But when I run the program, it always takes the first set of items (that's 3 as per the batch size). So could anyone please help me how to move to the next set of items.
Take is an extension method that I have written. And I tried using async/await pattern for it.

Thanks in advance

  1. public class Program
  2. {
  3. public static async Task Main(string[] args)
  4. {
  5. var obj = new Class1();
  6. List<string> fruits = new()
  7. {
  8. "1",
  9. "2",
  10. "3",
  11. "4",
  12. "5",
  13. "6",
  14. "7",
  15. "8",
  16. "9",
  17. "10"
  18. };
  19. await Class1.Start(fruits);
  20. Console.ReadLine();
  21. }
  22. }
  23. public class Class1
  24. {
  25. private const int batchSize = 3;
  26. public static async Task Start(List<string> fruits)
  27. {
  28. if (fruits == null)
  29. return;
  30. var e = fruits.GetEnumerator();
  31. while (true)
  32. {
  33. var batch = e.Take(3); // always taking the first 3 items and not moving to the next items of the list
  34. if (batch.Count == 0)
  35. {
  36. break;
  37. }
  38. await StartProcessing(batch);
  39. }
  40. }
  41. public static async Task StartProcessing(List<string> batch)
  42. {
  43. await Parallel.ForEachAsync(batch, async (item, CancellationToken) =>
  44. {
  45. var list = new List<string>();
  46. await Task.Delay(1000);
  47. Console.WriteLine($"Fruit Name: {item}");
  48. list.Add(item);
  49. });
  50. }
  51. }

Extension.cs

  1. public static class Extensions
  2. {
  3. public static List<T> Take<T>(this IEnumerator<T> e, int num)
  4. {
  5. List<T> list = new List<T>(num);
  6. int taken = 0;
  7. while (taken < num && e.MoveNext())
  8. {
  9. list.Add(e.Current);
  10. taken++;
  11. }
  12. return list;
  13. }
  14. }

答案1

得分: 6

List<T>.Enumerator 是一个结构体。因此,在你的 Take 扩展方法中修改的是枚举器的副本。以下是使用你的扩展方法的一个更简单的示例(fiddle):

  1. using System;
  2. using System.Collections.Generic;
  3. public class Program
  4. {
  5. public static void Main()
  6. {
  7. List<string> fruits = new() { "1", "2", "3", "4", "5", "6", "7", "8", "9", "10" };
  8. var e = fruits.GetEnumerator();
  9. var firstThree = e.Take(3);
  10. var nextThree = e.Take(3);
  11. // 打印出 1, 2, 3
  12. foreach (var x in firstThree)
  13. Console.WriteLine(x);
  14. // 也会打印出 1, 2, 3
  15. foreach (var x in nextThree)
  16. Console.WriteLine(x);
  17. }
  18. }
  19. public static class Extensions
  20. {
  21. public static List<T> Take<T>(this IEnumerator<T> e, int num)
  22. {
  23. List<T> list = new List<T>(num);
  24. int taken = 0;
  25. while (taken < num && e.MoveNext())
  26. {
  27. list.Add(e.Current);
  28. taken++;
  29. }
  30. return list;
  31. }
  32. }

你可以通过确保 e 包含一个装箱的枚举器来解决这个问题,将

  1. var e = fruits.GetEnumerator();

替换为

  1. IEnumerable<string> e = fruits.GetEnumerator();

(fiddle)

另外,新版本的 C# 允许你使用 ref 扩展方法,这将使你能够像这样做(fiddle):

  1. var e = fruits.GetEnumerator();
  2. // 由于某种原因,泛型类型推断在这里无法工作
  3. var firstThree = e.Take<string, List<string>.Enumerator>(3);
  4. var nextThree = e.Take<string, List<string>.Enumerator>(3);
  5. ...
  6. public static class Extensions
  7. {
  8. public static List<T> Take<T, TEnum>(ref this TEnum e, int num)
  9. where TEnum : struct, IEnumerator<T>
  10. {
  11. ...
  12. }
  13. }

但是,老实说,你的代码之所以不能工作是因为枚举器并不是用来这样使用的。内置的 Enumerable.Take 方法适用于可枚举对象,而不是枚举器,这是在 .NET 中进行此类操作的惯用方式。

对于你的用例,Enumerable.Chunk 是最合适的内置方法。如果你想了解它如何为教育目的从头实现,请参考以下相关问题:

英文:

List<T>.Enumerator is a struct. Thus, a copy of your enumerator is modified in your Take extension method. Here is a simpler example using your extension method (fiddle):

  1. using System;
  2. using System.Collections.Generic;
  3. public class Program
  4. {
  5. public static void Main()
  6. {
  7. List<string> fruits = new() { "1", "2", "3", "4", "5", "6", "7", "8", "9", "10" };
  8. var e = fruits.GetEnumerator();
  9. var firstThree = e.Take(3);
  10. var nextThree = e.Take(3);
  11. // prints 1, 2, 3
  12. foreach (var x in firstThree)
  13. Console.WriteLine(x);
  14. // also prints 1, 2, 3
  15. foreach (var x in nextThree)
  16. Console.WriteLine(x);
  17. }
  18. }
  19. public static class Extensions
  20. {
  21. public static List<T> Take<T>(this IEnumerator<T> e, int num)
  22. {
  23. List<T> list = new List<T>(num);
  24. int taken = 0;
  25. while (taken < num && e.MoveNext())
  26. {
  27. list.Add(e.Current);
  28. taken++;
  29. }
  30. return list;
  31. }
  32. }

You can fix this by making sure that e contains a boxed enumerator by replacing

  1. var e = fruits.GetEnumerator();

with

  1. IEnumerable<string> e = fruits.GetEnumerator();

(fiddle)


Alternatively, newer versions of C# allow you to use ref extension methods, which would enable you to do something like this (fiddle):

  1. var e = fruits.GetEnumerator();
  2. // For some reason generic type inference won't work here
  3. var firstThree = e.Take<string, List<string>.Enumerator>(3);
  4. var nextThree = e.Take<string, List<string>.Enumerator>(3);
  5. ...
  6. public static class Extensions
  7. {
  8. public static List<T> Take<T, TEnum>(ref this TEnum e, int num)
  9. where TEnum : struct, IEnumerator<T>
  10. {
  11. ...
  12. }
  13. }

But, honestly, the real reason why your code does not work is because enumerators aren't meant to be used like this. The built-in Enumerable.Take method works on Enumerables, not on Enumerators, and that's the idiomatic way to do those things in .NET.

For your use case, Enumerable.Chunk is the most appropriate built-in method. If you want to see how it could be implemented from scratch for educational purposes, have a look at these related questions:

huangapple
  • 本文由 发表于 2023年4月19日 14:38:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/76051418.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定