EnterUpgradeableReadlock and EnterWriteLock cannot be paired with Exit lock and throw The upgradeable lock is being released without being held

huangapple
go评论101阅读模式
英文:

EnterUpgradeableReadlock and EnterWriteLock cannot be paired with Exit lock and throw The upgradeable lock is being released without being held

问题

目前我正在处理一个使用.NET 6.0编写的服务器程序上的多线程问题。存在问题的方法如下：

  1. private async Task<Cell> LoadCell()
  2. {
  3.     if (_loadingLock.TryEnterUpgradeableReadLock(new TimeSpan(0, 0, 0, 0, 20)))
  4.     {
  5.         _globalReadlockCountEnter++;
  6.         Log.Info("_loadingLock Enter upgrade readlock, count: " + _globalReadlockCountEnter);
  7.         try
  8.         {
  9.             if (_mapLock.TryEnterReadLock(new TimeSpan(0, 0, 0, 0, 20)))
  10.             {
  11.                 try
  12.                 {
  13.                     //尝试获取值
  14.                 }
  15.                 finally
  16.                 {
  17.                     _mapLock.ExitReadLock();
  18.                 }
  19.             }
  20.             else
  21.             {
  22.                 throw new TimeoutException("Timeout trying to get read lock in level cell manager.");
  23.             }
  25.             bool acquired = false;
  26.             try
  27.             {
  28.                 if (!_loadingLock.TryEnterWriteLock(new TimeSpan(0, 0, 0, 20)))
  29.                 {
  30.                     throw new TimeoutException("Timeout trying to get write lock in level cell manager.");
  31.                 }
  32.                 else
  33.                 {
  34.                     acquired = true;
  35.                     _globalWritelockCountEnter++;
  36.                     Log.Info("_loadingLock Enter writelock, count: " + _globalWritelockCountEnter);
  37.                     //处理某个值
  38.                 }
  39.             }
  40.             catch (Exception e)
  41.             {
  42.                 throw new Exception("Unknown exception error.");
  43.             }
  44.             finally
  45.             {
  46.                 if (acquired)
  47.                 {
  48.                     if (!_loadingLock.IsWriteLockHeld)
  49.                     {
  50.                         Log.Warn("write lock has not been held, check is everything alright?");
  51.                     }
  53.                     _loadingLock.ExitWriteLock();
  54.                     _globalWritelockCountExit++;
  55.                     Log.Info("_loadingLock exit writelock, count: " + _globalWritelockCountExit);
  56.                 }
  57.                 else
  58.                 {
  59.                     Log.Warn("acquire is false some how it failed to exit the write lock.");
  60.                 }
  61.             }
  62.         }
  63.         finally
  64.         {
  65.             if (!_loadingLock.IsUpgradeableReadLockHeld)
  66.             {
  67.                 Log.Warn("upgrade read lock has not been held, check is everything alright?");
  68.             }
  70.             _loadingLock.ExitUpgradeableReadLock();
  71.             _globalReadlockCountExit++;
  72.             Log.Info("_loadingLock Exit upgrade readlock, count: " + _globalReadlockCountExit);
  73.         }
  74.     }
  75.     else
  76.     {
  77.         throw new TimeoutException("Timeout trying to get upgrade read lock in level cell manager.");
  78.     }
  79. }

日志和错误消息如下：

  1. 2023-07-19 07:43:48,601 INFO _loadingLock Exit upgrade readlock, count: 1060
  2. 2023-07-19 07:43:48,601 INFO _loadingLock Enter upgrade readlock, count: 1061
  3. 2023-07-19 07:43:48,601 INFO _loadingLock Enter writelock, count: 358
  4. 2023-07-19 07:43:48,602 INFO _loadingLock exit writelock, count: 358
  5. 2023-07-19 07:43:48,602 INFO _loadingLock Exit upgrade readlock, count: 1061
  6. 2023-07-19 07:43:48,602 INFO _loadingLock Enter upgrade readlock, count: 1062
  7. 2023-07-19 07:43:48,602 INFO _loadingLock Enter writelock, count: 359
  8. 2023-07-19 07:43:48,603 INFO _loadingLock exit writelock, count: 359
  9. 2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1062
  10. 2023-07-19 07:43:48,603 INFO _loadingLock Enter upgrade readlock, count: 1063
  11. 2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1063
  12. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1064
  13. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1064
  14. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1065
  15. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1065
  16. ...
  17. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1080
  18. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1080
  19. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1081
  20. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1081
  21. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1082
  22. 2023-07-19 07:43:48,604 INFO _loadingLock Enter writelock, count: 360
  23. 2023-07-19 07:43:48,623 WARN write lock has not been held, check is everything alright?
  24. 2023-07-19 07:43:48,623 WARN upgrade read lock has not been held, check is everything alright?
  25. Unhandled exception. Unhandled exception. System.Threading.SynchronizationLockException: The upgradeable lock is being released without being held.

问题似乎在调用该方法时偶尔发生，有时可以连续两天没有任何问题，有时在服务器启动后不久就会发生，而且每次在发生之前，所有的进入/退出锁都能成对出现。我尝试大量记录日志来查看发生了什么，正如您在上面看到的那样，我最终发现总是写锁没有被退出（或者至少我没有看到它应该退出的日志），然后当执行退出可升级读取锁时，会抛出异常。

从代码中我认为工作流程应该是正确的，即进入可升级读取锁 -> 进入写锁 -> 退出写锁 -> 退出可升级读取锁，不应该引发任何问题，就像官方文档中提到的示例一样，但实际情况下，我不确定是我做错了什

英文:

Currently I'm dealing with a multi thread issue on a server program with .NET 6.0. The method with issue likes below:

  1. private async Task&lt;Cell&gt; LoadCell()
  2. {
  3. if (_loadingLock.TryEnterUpgradeableReadLock(new TimeSpan(0, 0, 0, 0, 20)))
  4. {
  5. _globalReadlockCountEnter++;
  6. Log.Info(&quot;_loadingLock Enter upgrade readlock, count: &quot; + _globalReadlockCountEnter);
  7. try
  8. {
  9. if (_mapLock.TryEnterReadLock(new TimeSpan(0, 0, 0, 0, 20)))
  10. {
  11. try
  12. {
  13. //Try get value
  14. }
  15. finally
  16. {
  17. _mapLock.ExitReadLock();
  18. }
  19. }
  20. else
  21. {
  22. throw new TimeoutException(&quot;Timeout trying to get read lock in level cell manager.&quot;);
  23. }
  24. bool acquired = false;
  25. try
  26. {
  27. if (!_loadingLock.TryEnterWriteLock(new TimeSpan(0, 0, 0, 20)))
  28. {
  29. throw new TimeoutException(&quot;Timeout trying to get write lock in level cell manager.&quot;);
  30. }
  31. else
  32. {
  33. acquired = true;
  34. _globalWritelockCountEnter++;
  35. Log.Info(&quot;_loadingLock Enter writelock, count: &quot; + _globalWritelockCountEnter);
  36. //deal with certain value
  37. }
  38. }
  39. catch (Exception e)
  40. {
  41. throw new Exception(&quot;Unknown exception error.&quot;);
  42. }
  43. finally
  44. {
  45. if (acquired)
  46. {
  47. if (!_loadingLock.IsWriteLockHeld)
  48. {
  49. Log.Warn(&quot;write lock has not been held, check is everything alright?&quot;);
  50. }
  51. _loadingLock.ExitWriteLock();
  52. _globalWritelockCountExit++;
  53. Log.Info(&quot;_loadingLock exit writelock, count: &quot; + _globalWritelockCountExit);
  54. }
  55. else
  56. {
  57. Log.Warn(&quot;acquire is false some how it failed to exit the write lock.&quot;);
  58. }
  59. }
  60. }
  61. finally
  62. {
  63. if (!_loadingLock.IsUpgradeableReadLockHeld)
  64. {
  65. Log.Warn(&quot;upgrade read lock has not been held, check is everything alright?&quot;);
  66. }
  67. _loadingLock.ExitUpgradeableReadLock();
  68. _globalReadlockCountExit++;
  69. Log.Info(&quot;_loadingLock Exit upgrade readlock, count: &quot; + _globalReadlockCountExit);
  70. }
  71. }
  72. else
  73. {
  74. throw new TimeoutException(&quot;Timeout trying to get upgrade read lock in level cell manager.&quot;);
  75. }
  76. }

The logs and error messages are like:

  1. 2023-07-19 07:43:48,601 INFO _loadingLock Exit upgrade readlock, count: 1060
  2. 2023-07-19 07:43:48,601 INFO _loadingLock Enter upgrade readlock, count: 1061
  3. 2023-07-19 07:43:48,601 INFO _loadingLock Enter writelock, count: 358
  4. 2023-07-19 07:43:48,602 INFO _loadingLock exit writelock, count: 358
  5. 2023-07-19 07:43:48,602 INFO _loadingLock Exit upgrade readlock, count: 1061
  6. 2023-07-19 07:43:48,602 INFO _loadingLock Enter upgrade readlock, count: 1062
  7. 2023-07-19 07:43:48,602 INFO _loadingLock Enter writelock, count: 359
  8. 2023-07-19 07:43:48,603 INFO _loadingLock exit writelock, count: 359
  9. 2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1062
  10. 2023-07-19 07:43:48,603 INFO _loadingLock Enter upgrade readlock, count: 1063
  11. 2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1063
  12. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1064
  13. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1064
  14. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1065
  15. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1065
  16. ...
  17. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1080
  18. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1080
  19. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1081
  20. 2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1081
  21. 2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1082
  22. 2023-07-19 07:43:48,604 INFO _loadingLock Enter writelock, count: 360
  23. 2023-07-19 07:43:48,623 WARN write lock has not been held, check is everything alright?
  24. 2023-07-19 07:43:48,623 WARN upgrade read lock has not been held, check is everything alright?
  25. Unhandled exception. Unhandled exception. System.Threading.SynchronizationLockException: The upgradeable lock is being released without being held.

The issue seems happened occasionally when the method is being called, sometimes it can keep two days without any issue and sometimes it happens just after the server start for a while, and every time before it happens, all enter/exit lock could be paired. I tried spamming the logs to check what happened, and as you can see above, I finally found it's always the write lock doesn't get exit(or at least I didn't see the log when it should exit) and then when it goes the exit upgradeable read lock, exception throws.

I thought from the code the workflow should be alright that enter upgradeable readlock -> enter writelock -> exit writelock -> exit upgradeable readlock and it shouldn't cause any trouble as the example mentioned in official documentation, but in actual case, I'm not sure whether it's something I did wrong or it concerns something of base level which I'm not familiar with. As I understood, the upgradeable readlock should be something that deals with the race condition, so the lock shouldn't be released by another thread. Also I know that for IsXXXLockHeld property, as mentioned in official docs: This property is intended for use in asserts or for other debugging purposes. Do not use it to control the flow of program execution(Mention this here as I've seen that in some simular issues that some one recommand using this for flow control).

I've tried using both try or none-try enter lock method but the result is usually same, well, if uses none-try enter lock it sometimes throw exceptions at the enterUpgradeableReadlock step, give me a feeling that previous lock has not exited.

Not sure what to do know and any tip or help is appreciated.

答案1

得分: 1

升级读/写锁和普通读/写锁都不支持异步（尽管我认为升级锁应该是解决此问题的方法）。经过一些研究，我认为SemaphoreSlim 是解决这个问题的正确方向，但这需要重新设计。如果有人遇到相同的问题并且不想重新设计，可以尝试在Github上找到的AsyncReaderWriterLockSlim，它内部使用SemaphoreSlim来实现异步读/写锁。

英文:

So it turns out that both upgrade read/write lock and the normal read/write lock are not async-affine(though I thought the upgrade locks should be the solution of this issue). After some researches I think the SemaphoreSlim is the right direction to solve this, however, that will need a rework for me. If any one who met same issue and don't want a rework on this, you can try AsyncReaderWriterLockSlim which I found on Github, internally using SemaphoreSlim to make async read/write locks.

huangapple
  • 本文由 发表于 2023年7月20日 11:29:40
  • 转载请务必保留本文链接：https://go.coder-hub.com/76726501.html
  • .net
  • c#
  • multithreading
  • server
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定