EnterUpgradeableReadlock and EnterWriteLock cannot be paired with Exit lock and throw The upgradeable lock is being released without being held

huangapple go评论72阅读模式
英文:

EnterUpgradeableReadlock and EnterWriteLock cannot be paired with Exit lock and throw The upgradeable lock is being released without being held

问题

目前我正在处理一个使用.NET 6.0编写的服务器程序上的多线程问题。存在问题的方法如下:

private async Task<Cell> LoadCell()
{
    if (_loadingLock.TryEnterUpgradeableReadLock(new TimeSpan(0, 0, 0, 0, 20)))
    {
        _globalReadlockCountEnter++;
        Log.Info("_loadingLock Enter upgrade readlock, count: " + _globalReadlockCountEnter);
        try
        {
            if (_mapLock.TryEnterReadLock(new TimeSpan(0, 0, 0, 0, 20)))
            {
                try
                {
                    //尝试获取值
                }
                finally
                {
                    _mapLock.ExitReadLock();
                }
            }
            else
            {
                throw new TimeoutException("Timeout trying to get read lock in level cell manager.");
            }

            bool acquired = false;
            try
            {
                if (!_loadingLock.TryEnterWriteLock(new TimeSpan(0, 0, 0, 20)))
                {
                    throw new TimeoutException("Timeout trying to get write lock in level cell manager.");
                }
                else
                {
                    acquired = true;
                    _globalWritelockCountEnter++;
                    Log.Info("_loadingLock Enter writelock, count: " + _globalWritelockCountEnter);
                    //处理某个值
                }
            }
            catch (Exception e)
            {
                throw new Exception("Unknown exception error.");
            }
            finally
            {
                if (acquired)
                {
                    if (!_loadingLock.IsWriteLockHeld)
                    {
                        Log.Warn("write lock has not been held, check is everything alright?");
                    }

                    _loadingLock.ExitWriteLock();
                    _globalWritelockCountExit++;
                    Log.Info("_loadingLock exit writelock, count: " + _globalWritelockCountExit);
                }
                else
                {
                    Log.Warn("acquire is false some how it failed to exit the write lock.");
                }
            }
        }
        finally
        {
            if (!_loadingLock.IsUpgradeableReadLockHeld)
            {
                Log.Warn("upgrade read lock has not been held, check is everything alright?");
            }

            _loadingLock.ExitUpgradeableReadLock();
            _globalReadlockCountExit++;
            Log.Info("_loadingLock Exit upgrade readlock, count: " + _globalReadlockCountExit);
        }
    }
    else
    {
        throw new TimeoutException("Timeout trying to get upgrade read lock in level cell manager.");
    }
}

日志和错误消息如下:

2023-07-19 07:43:48,601 INFO _loadingLock Exit upgrade readlock, count: 1060
2023-07-19 07:43:48,601 INFO _loadingLock Enter upgrade readlock, count: 1061
2023-07-19 07:43:48,601 INFO _loadingLock Enter writelock, count: 358
2023-07-19 07:43:48,602 INFO _loadingLock exit writelock, count: 358
2023-07-19 07:43:48,602 INFO _loadingLock Exit upgrade readlock, count: 1061
2023-07-19 07:43:48,602 INFO _loadingLock Enter upgrade readlock, count: 1062
2023-07-19 07:43:48,602 INFO _loadingLock Enter writelock, count: 359
2023-07-19 07:43:48,603 INFO _loadingLock exit writelock, count: 359
2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1062
2023-07-19 07:43:48,603 INFO _loadingLock Enter upgrade readlock, count: 1063
2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1063
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1064
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1064
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1065
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1065
...
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1080
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1080
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1081
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1081
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1082
2023-07-19 07:43:48,604 INFO _loadingLock Enter writelock, count: 360
2023-07-19 07:43:48,623 WARN write lock has not been held, check is everything alright?
2023-07-19 07:43:48,623 WARN upgrade read lock has not been held, check is everything alright?
Unhandled exception. Unhandled exception. System.Threading.SynchronizationLockException: The upgradeable lock is being released without being held.

问题似乎在调用该方法时偶尔发生,有时可以连续两天没有任何问题,有时在服务器启动后不久就会发生,而且每次在发生之前,所有的进入/退出锁都能成对出现。我尝试大量记录日志来查看发生了什么,正如您在上面看到的那样,我最终发现总是写锁没有被退出(或者至少我没有看到它应该退出的日志),然后当执行退出可升级读取锁时,会抛出异常。

从代码中我认为工作流程应该是正确的,即进入可升级读取锁 -> 进入写锁 -> 退出写锁 -> 退出可升级读取锁,不应该引发任何问题,就像官方文档中提到的示例一样,但实际情况下,我不确定是我做错了什

英文:

Currently I'm dealing with a multi thread issue on a server program with .NET 6.0. The method with issue likes below:

private async Task&lt;Cell&gt; LoadCell()
{
if (_loadingLock.TryEnterUpgradeableReadLock(new TimeSpan(0, 0, 0, 0, 20)))
{
_globalReadlockCountEnter++;
Log.Info(&quot;_loadingLock Enter upgrade readlock, count: &quot; + _globalReadlockCountEnter);
try
{
if (_mapLock.TryEnterReadLock(new TimeSpan(0, 0, 0, 0, 20)))
{
try
{
//Try get value
}
finally
{
_mapLock.ExitReadLock();
}
}
else
{
throw new TimeoutException(&quot;Timeout trying to get read lock in level cell manager.&quot;);
}
bool acquired = false;
try
{
if (!_loadingLock.TryEnterWriteLock(new TimeSpan(0, 0, 0, 20)))
{
throw new TimeoutException(&quot;Timeout trying to get write lock in level cell manager.&quot;);
}
else
{
acquired = true;
_globalWritelockCountEnter++;
Log.Info(&quot;_loadingLock Enter writelock, count: &quot; + _globalWritelockCountEnter);
//deal with certain value
}
}
catch (Exception e)
{
throw new Exception(&quot;Unknown exception error.&quot;);
}
finally
{
if (acquired)
{
if (!_loadingLock.IsWriteLockHeld)
{
Log.Warn(&quot;write lock has not been held, check is everything alright?&quot;);
}
_loadingLock.ExitWriteLock();
_globalWritelockCountExit++;
Log.Info(&quot;_loadingLock exit writelock, count: &quot; + _globalWritelockCountExit);
}
else
{
Log.Warn(&quot;acquire is false some how it failed to exit the write lock.&quot;);
}
}
}
finally
{
if (!_loadingLock.IsUpgradeableReadLockHeld)
{
Log.Warn(&quot;upgrade read lock has not been held, check is everything alright?&quot;);
}
_loadingLock.ExitUpgradeableReadLock();
_globalReadlockCountExit++;
Log.Info(&quot;_loadingLock Exit upgrade readlock, count: &quot; + _globalReadlockCountExit);
}
}
else
{
throw new TimeoutException(&quot;Timeout trying to get upgrade read lock in level cell manager.&quot;);
}
}

The logs and error messages are like:

2023-07-19 07:43:48,601 INFO _loadingLock Exit upgrade readlock, count: 1060
2023-07-19 07:43:48,601 INFO _loadingLock Enter upgrade readlock, count: 1061
2023-07-19 07:43:48,601 INFO _loadingLock Enter writelock, count: 358
2023-07-19 07:43:48,602 INFO _loadingLock exit writelock, count: 358
2023-07-19 07:43:48,602 INFO _loadingLock Exit upgrade readlock, count: 1061
2023-07-19 07:43:48,602 INFO _loadingLock Enter upgrade readlock, count: 1062
2023-07-19 07:43:48,602 INFO _loadingLock Enter writelock, count: 359
2023-07-19 07:43:48,603 INFO _loadingLock exit writelock, count: 359
2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1062
2023-07-19 07:43:48,603 INFO _loadingLock Enter upgrade readlock, count: 1063
2023-07-19 07:43:48,603 INFO _loadingLock Exit upgrade readlock, count: 1063
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1064
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1064
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1065
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1065
...
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1080
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1080
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1081
2023-07-19 07:43:48,604 INFO _loadingLock Exit upgrade readlock, count: 1081
2023-07-19 07:43:48,604 INFO _loadingLock Enter upgrade readlock, count: 1082
2023-07-19 07:43:48,604 INFO _loadingLock Enter writelock, count: 360
2023-07-19 07:43:48,623 WARN write lock has not been held, check is everything alright?
2023-07-19 07:43:48,623 WARN upgrade read lock has not been held, check is everything alright?
Unhandled exception. Unhandled exception. System.Threading.SynchronizationLockException: The upgradeable lock is being released without being held.

The issue seems happened occasionally when the method is being called, sometimes it can keep two days without any issue and sometimes it happens just after the server start for a while, and every time before it happens, all enter/exit lock could be paired. I tried spamming the logs to check what happened, and as you can see above, I finally found it's always the write lock doesn't get exit(or at least I didn't see the log when it should exit) and then when it goes the exit upgradeable read lock, exception throws.

I thought from the code the workflow should be alright that enter upgradeable readlock -> enter writelock -> exit writelock -> exit upgradeable readlock and it shouldn't cause any trouble as the example mentioned in official documentation, but in actual case, I'm not sure whether it's something I did wrong or it concerns something of base level which I'm not familiar with. As I understood, the upgradeable readlock should be something that deals with the race condition, so the lock shouldn't be released by another thread. Also I know that for IsXXXLockHeld property, as mentioned in official docs: This property is intended for use in asserts or for other debugging purposes. Do not use it to control the flow of program execution(Mention this here as I've seen that in some simular issues that some one recommand using this for flow control).

I've tried using both try or none-try enter lock method but the result is usually same, well, if uses none-try enter lock it sometimes throw exceptions at the enterUpgradeableReadlock step, give me a feeling that previous lock has not exited.

Not sure what to do know and any tip or help is appreciated.

答案1

得分: 1

升级读/写锁和普通读/写锁都不支持异步(尽管我认为升级锁应该是解决此问题的方法)。经过一些研究,我认为SemaphoreSlim 是解决这个问题的正确方向,但这需要重新设计。如果有人遇到相同的问题并且不想重新设计,可以尝试在Github上找到的AsyncReaderWriterLockSlim,它内部使用SemaphoreSlim来实现异步读/写锁。

英文:

So it turns out that both upgrade read/write lock and the normal read/write lock are not async-affine(though I thought the upgrade locks should be the solution of this issue). After some researches I think the SemaphoreSlim is the right direction to solve this, however, that will need a rework for me. If any one who met same issue and don't want a rework on this, you can try AsyncReaderWriterLockSlim which I found on Github, internally using SemaphoreSlim to make async read/write locks.

huangapple
  • 本文由 发表于 2023年7月20日 11:29:40
  • 转载请务必保留本文链接:https://go.coder-hub.com/76726501.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定