英文:
How to avoid and handle Neo.TransientError.Transaction.DeadlockDetected?
问题
在进行负载测试时,我偶尔会遇到死锁错误,我添加了重试机制,效果还不错。但是,我该如何改进呢?错误信息相当模糊,所以我无法确定是哪个查询导致了这个问题。
以下是代码部分,不做翻译:
{code: Neo.TransientError.Transaction.DeadlockDetected} {message: ForsetiClient[transactionId=75637, clientId=2431] 无法获得 ExclusiveLock{owner=ForsetiClient[transactionId=75634, clientId=2450]} 在 NODE_RELATIONSHIP_GROUP_DELETE(116) 上,因为持有该锁的人正在等待 ForsetiClient[transactionId=75637, clientId=2431]。
奇怪的是,我认为我没有删除关系的代码。实际上,我是在执行 MERGE 操作以添加节点和关系,类似于以下操作:
MERGE (o:Org {id: $orgId})
SET o.name=$name
MERGE (a:Item {id: $itemId})
SET a.orgId=$orgId,
a.name=$itemName
MERGE (s:SubItem {id: $subItemId})
SET s.name=$subItemName
MERGE (o)-[:HAS_ITEM]->(a)
MERGE (a)-[:HAS_SUB_ITEM]->(s)
更新:
我成功找出了导致死锁的部分。我将标签建模为节点,我怀疑当许多项使用相同的标签时,当我尝试合并关系时,会导致死锁。
MATCH (t:Tag {name: "a"})
MATCH (i:Item {name: "x"})
MERGE (i)-[:TAGGED]->(t)
我认为这部分经常导致死锁,我应该如何避免这种情况?
英文:
When doing load testing, I am seeing Deadlock errors happening occasionally, I added retries and its been working well. However, how do I know how to improve? The errors are quite vague so I cannot tell which query is causing this
> {code: Neo.TransientError.Transaction.DeadlockDetected} {message: ForsetiClient[transactionId=75637, clientId=2431] can't acquire ExclusiveLock{owner=ForsetiClient[transactionId=75634, clientId=2450]} on NODE_RELATIONSHIP_GROUP_DELETE(116) because holders of that lock are waiting for ForsetiClient[transactionId=75637, clientId=2431].
The weird thing is I dont think I have code that does a relationship delete. I am actually doing MERGE to add nodes and relationships along the lines of:
MERGE (o:Org {id: $orgId})
SET o.name=$name
MERGE (a:Item {id: $itemId})
SET a.orgId=$orgId,
a.name=$itemName
MERGE (s:SubItem {id: $subItemId})
SET s.name=$subItemName
MERGE (o)-[:HAS_ITEM]->(a)
MERGE (a)-[:HAS_SUB_ITEM]->(s)
UPDATE:
Managed to identity the part causing the deadlock. I am modelling tags as a node and I suspect when many items use the same tag, and I try to merge the relationship, it causes a deadlock
MATCH (t:Tag {name: "a"})
MATCH (i:Item {name: "x"})
MERGE (i)-[:TAGGED]->(t)
I think this is the part causing deadlock often, how might I avoid this?
答案1
得分: 0
死锁通常不是由单个查询引起的,而是由并行更新相同节点或关系的查询引起的。为了确保数据的有效性,Neo4j在单个事务中更新节点时会创建死锁,以防止两个事务同时更新相同的节点。似乎delete
操作是一个底层操作,不一定会删除关系。请查看文档以获取更多信息。
英文:
Deadlocks are usually not caused by a single query, but by queries that update the same node or relationship in parallel. To ensure valid data, Neo4j creates deadlocks on a node if it is being updated in a single transaction, so that two transactions cannot update the same node at the same time. It seems that the delete
operation is an underlying operation that doesn't necessarily delete relationships. Check the documentation for more information.
答案2
得分: 0
找到了频繁发生死锁的原因。同时也要感谢@Tomaž Bratanič,他的文档链接提供了帮助。具体来说,以下部分是重要的:
在事务创建阶段对节点采取的锁,以防止删除该节点和/或关系组。这与节点锁不同,允许同时进行标签和属性更改以及关系修改。
以及
表2. 图修改的获取锁
创建关系*
如果节点稀疏:节点锁。
如果节点密集:节点删除预防锁。
重要的是与密集节点相关的部分。因为我正在将:Category
建模为一个节点,一个类别可以有许多边,从而创建一个“密集节点”。
通过更多的搜索,我找到了https://medium.com/neo4j/graph-modeling-all-about-super-nodes-d6ad7e11015b,并开始尝试通过一些权衡使节点变得不那么密集。基本上,我将其作为与“用户”特定相关的类别,而不是全局性的。因此,与其按名称唯一标识类别,我还添加了一种类似的userId,以便它成为一个“复合键”。我还在名称上添加了一个TEXT索引,以便可以快速查询所有用户的类别。
英文:
Found the reason why deadlocks happen very frequently already. Also thanks to @Tomaž Bratanič's whose link to the docs helped. Specifically this part
> Lock taken on a node during the transaction creation phase to prevent deletion of that node and/or relationship group. This is different from the NODE lock in order to allow concurrent label and property changes together with relationship modifications.
And
> Table 2. Obtained locks for graph modifications
> Creating a relationship*
> If the node is sparse: NODE lock.
> If a node is dense: NODE DELETE prevention lock.
Important part is the part about dense node. Because I am modelling :Category
as a node, 1 category can have many edges creating a "dense node".
Googling more about it, I found https://medium.com/neo4j/graph-modeling-all-about-super-nodes-d6ad7e11015b and started attempting to make the node less dense with some trade offs. Basically I made it a category specific to the "user" rather than globally. So instead of the category being uniqued by name, I also added a userId of sort so its a "composite key". I also added a TEXT index on name such that I can query easily for all category regardless of user quickly
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论