2023年5月10日 22:48:07go评论66阅读模式

英文:

Why Is JanusGraph Finding Expired KeyColumn Claims?

问题

是什么导致 JanusGraph 抛出如此多的警告，从而导致完全的重试？
JanusGraph 似乎找到了应该被丢弃的顶点，如果有延迟的话，那是可以接受的；
我只想在丢弃顶点之后添加一个新的顶点。

[WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 在 KeyColumn 上丢弃了过期的声明

在后端操作期间出现临时异常 [CacheMutation]。正在尝试回退重试

复现

步骤

创建并启动 Cassandra [Docker 容器]

docker run --name jg-cassandra -d -e CASSANDRA_START_RPC=true -p 9160:9160 -p 9042:9042 -p 7199:7199 -p 7001:7001 -p 7000:7000 cassandra:3.11

将大型（或至少是非平凡的）数据集加载到 JanusGraph 中
1. 一个示例的“中等大小”数据集是 ICIJ Offshore Dataleaks
2. 中等大小意味着一个比某人能够硬编码的不同和“随机”数据更大的数据集
创建并运行这个 Java + Maven 项目（下面是代码）
结果
1. 期望的是每个实例每次只有1个或没有错误或警告，然后继续
2. 实际上：抛出了一大堆警告，每次增加k值100

在 KeyColumn 上丢弃了过期的声明 [k=0x0x7800000000008080, c=0x0x02]，时间戳为 2023-05-09T20:16:08.988543Z

日志

2023-05-10 09:23:19,141 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::	 联系点 localhost:9042 解析为多个地址，将使用它们全部（[localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]）
2023-05-10 09:23:19,226 [INFO] [c.d.o.d.i.c.DefaultMavenCoordinates.main] ::	 Apache Cassandra(R) 的 DataStax Java 驱动程序（com.datastax.oss:java-driver-core）版本 4.15.0
2023-05-10 09:23:19,726 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::	 使用微秒精度的本地时钟
2023-05-10 09:23:19,987 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::	 [JanusGraph Session|default] 您指定了 datacenter1 作为本地 DC，但一些联系点来自不同的 DC：Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=37423099)=null；请提供正确的本地 DC，或检查您的联系点
2023-05-10 09:23:20,231 [INFO] [o.j.g.i.UniqueInstanceIdRetriever.main] ::	 生成的唯一实例 ID=c0a8563c404-rmt-lap-win201
2023-05-10 09:23:20,252 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::	 联系点 localhost:9042 解析为多个地址，将使用它们全部（[localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]）
2023-05-10 09:23:20,297 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::	 使用微秒精度的本地时钟
2023-05-10 09:23:20,346 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::	 [JanusGraph Session|default] 您指定了 datacenter1 作为本地 DC，但一些联系点来自不同的 DC：Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=1ec16fcd)=null；请提供正确的本地 DC，或检查您的联系点
2023-05-10 09:23:20,362 [INFO] [o.j.d.c.ExecutorServiceBuilder.main] ::	 启动固定大小为 40 的线程池
2023-05-10 09:23:20,469 [INFO] [o.j.g.d.StandardJanusGraph.main] ::	 禁用 Gremlin 脚本评估
2023-05-10 09:23:20,494 [INFO] [o.j.d.l.k.KCVSLog.main] ::	 将未识别的 ReadMarker 启动时间 2023-05-10T14:23:20.494021Z 加载到 org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@f1d0004
2023-05-10 09:23:20,549 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::	 查询需要遍历所有顶点[[]]。为了获得更好的性能，请使用索引
2023-05-10 09:23:21,835 [INFO] [Main.main] ::	 g.V().count().next():	70005
2023-05-10 09:23:21,839 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::	 查询需要遍历所有顶点[[]]。为了获得更好的性能，请使用索引
2023-

<details>
<summary>英文:</summary>

# Problem 

What is causing JanusGraph to throw-out so many of this warnings going into full-blown retries? 
JanusGraph seems to find supposedly dropped vertexes, which is fine if there&#39;s a delay;  
I just want to add a new vertex after vertex-dropping.

```json 
[WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn

Temporary exception during backend operation [CacheMutation]. Attempting backoff retry

Reproduction

Steps

Create-and-Startup a Cassandra [Docker container]

docker run --name jg-cassandra -d -e CASSANDRA_START_RPC=true -p 9160:9160 -p 9042:9042 -p 7199:7199 -p 7001:7001 -p 7000:7000 cassandra:3.11

Load a large (or at least non-trival) dataset into JanusGraph
1. A sample "medium-sized" dataset is ICIJ Offshore Dataleaks
2. Medium meaning a larger dataset than someone could hard-code distinct and "random" data
Create-and-Run this Java + Maven Project (code below)
Results
1. Expected: little-to-no errors-or-warnings with limit-of-1 per instance and then move on
2. Actually: thrown a wall of warnings that increment k-value up by 100 each time
```
Discarded expired claim on KeyColumn [k=0x0x7800000000008080, c=0x0x02] with timestamp 2023-05-09T20:16:08.988543Z
```

Logs

2023-05-10 09:23:19,141 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::	 Contact point localhost:9042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-10 09:23:19,226 [INFO] [c.d.o.d.i.c.DefaultMavenCoordinates.main] ::	 DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.15.0
2023-05-10 09:23:19,726 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::	 Using native clock for microsecond precision
2023-05-10 09:23:19,987 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::	 [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=37423099)=null; please provide the correct local DC, or check your contact points
2023-05-10 09:23:20,231 [INFO] [o.j.g.i.UniqueInstanceIdRetriever.main] ::	 Generated unique-instance-id=c0a8563c404-rmt-lap-win201
2023-05-10 09:23:20,252 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::	 Contact point localhost:9042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-10 09:23:20,297 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::	 Using native clock for microsecond precision
2023-05-10 09:23:20,346 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::	 [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=1ec16fcd)=null; please provide the correct local DC, or check your contact points
2023-05-10 09:23:20,362 [INFO] [o.j.d.c.ExecutorServiceBuilder.main] ::	 Initiated fixed thread pool of size 40
2023-05-10 09:23:20,469 [INFO] [o.j.g.d.StandardJanusGraph.main] ::	 Gremlin script evaluation is disabled
2023-05-10 09:23:20,494 [INFO] [o.j.d.l.k.KCVSLog.main] ::	 Loaded unidentified ReadMarker start time 2023-05-10T14:23:20.494021Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@f1d0004
2023-05-10 09:23:20,549 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::	 Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:23:21,835 [INFO] [Main.main] ::	 g.V().count().next():	70005
2023-05-10 09:23:21,839 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::	 Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:26:13,733 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::	 Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:26:15,427 [INFO] [Main.main] ::	 g.V().count().next():	0
2023-05-10 09:33:40,131 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Lock write succeeded but took too long: duration PT0.197258S exceeded limit PT0.1S
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000008080, c=0x0x02] with timestamp 2023-05-09T20:16:07.308532Z
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000008180, c=0x0x02] with timestamp 2023-05-09T20:16:08.146591Z
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000008280, c=0x0x02] with timestamp 2023-05-09T20:16:08.988543Z

...

2023-05-10 09:38:03,876 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-10T14:28:09.380600Z
2023-05-10 09:38:03,876 [INFO] [o.j.d.u.BackendOperation.main] ::	 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry.
org.janusgraph.diskstorage.locking.consistentkey.ExpiredLockException: Expired lock on KeyColumn [k=0x0x7800000000018080, c=0x0x02]: lock timestamp 2023-05-10T14:28:09.380600Z Micros is older than storage.lock.expiry-time=PT5M
	at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:472) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:131) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:348) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:178) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:157) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:77) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:99) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:96) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:66) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:52) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:96) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:147) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:210) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:147) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:917) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1558) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:322) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) [gremlin-core-3.6.2.jar:3.6.2]
	at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.commit(JanusGraphBlueprintsGraph.java:300) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
	at Test6.main(Test6.java:28) [test-classes/:?]
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:09:05.345276Z
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:14:34.917340Z
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:21:07.864503Z
2023-05-10 09:38:03,933 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::	 Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-10T14:28:09.380600Z

...

2023-05-10 09:38:45,394 [INFO] [o.j.d.u.BackendOperation.main] ::	 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. ...

Process finished with exit code -1

Code

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.janusgraph.core.JanusGraph;
import org.janusgraph.core.JanusGraphFactory;
import org.janusgraph.core.JanusGraphVertex;
import org.janusgraph.core.PropertyKey;
import org.janusgraph.core.schema.JanusGraphManagement;

public class Test6 {
    private static final Logger logger = LogManager.getLogger(Main.class);

    public static void main(String[] args) {
        JanusGraph janusGraph = JanusGraphFactory.build().set(&quot;storage.backend&quot;, &quot;cql&quot;).set(&quot;storage.hostname&quot;, &quot;localhost:9042&quot;).open();
        GraphTraversalSource g = janusGraph.traversal();
        logger.info(&quot;g.V().count().next():\t&quot; + g.V().count().next());
        g.V().drop().iterate();
        logger.info(&quot;g.V().count().next():\t&quot; + g.V().count().next());
        JanusGraphManagement janusGraphManagement = janusGraph.openManagement();
        PropertyKey propertyKey = janusGraphManagement.getOrCreatePropertyKey(&quot;_id&quot;);
        if (!janusGraphManagement.containsGraphIndex(&quot;_id&quot;))
            janusGraphManagement.buildIndex(&quot;_id&quot;, Vertex.class).addKey(propertyKey).buildCompositeIndex();
        janusGraphManagement.commit();
        JanusGraphVertex janusGraphVertex = janusGraph.addVertex();
        janusGraphVertex.property(&quot;test&quot;, &quot;test1&quot;);
        janusGraph.tx().commit();
        janusGraph.close();
    }
}

Resources

答案1

得分: 0

JanusGraph并没有将它们删除。

因为JanusGraph的服务器是Cassandra；这个实例不是一个“JanusGraph服务器”，
in-memory或local-distribution，正如Stephen Mallette所明确指出的。

> Gremlin Server确实会自动提交事务，但您没有连接到Gremlin Server。您直接连接到JanusGraph实例，因此Gremlin Server不会对事务进行任何管理。鉴于您似乎正在实时工作并在进行过程中提出问题，stackoverflow并不适合进行对话式帮助。我强烈建议您将对话带到JanusGraph和/或TinkerPop Discord服务器以获取此类支持。您可以在项目主页上找到Discord的链接。 - Stephen M.

英文:

JanusGraph isn't dropping them.

Because JanusGraph's server is Cassandra; this instance is not a "JanusGraph Server"
in-memory or local-distribution as made clear by thee Stephen Mallette

> it's true that Gremlin Server will do auto-commits on transactions but your'e not connected to Gremlin Server. you connected to JanusGraph instance directly so Gremlin Server isn't doing anything to manage transactions. Just a suggestion as you seem to be working in a realtime way and asking questions as you go, stackoverflow really isn't meant for conversational help. i'd highly recommend you bring your conversations to the JanusGraph and/or TinkerPop discord servers for this sort of support. you can find the link to discord on the project home pages. - Stephen M.

通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。

JanusGraph为什么找到了过期的KeyColumn声明？

问题

问题

复现

步骤

日志

Reproduction

Steps

Logs

Code

Resources

答案1

你可以使用Angular登录代替Spring Security的登录来配置你的OAuth2服务器。

获取特定标签的子节点列表，深度为n级。

Cassandra中保留数据更长时间的最有效方法是什么？

Java会自动将接口转换为子类型。

What's the correct way to type hint an empty list as a literal in python?

如何在Highcharts Gantt中更改本地化的星期名称

如何在同一个流中使用多个过滤器和映射函数？

如何使用Map/Set来将代码优化到O(n)？

.NET MAUI Android在GitHub Actions上构建失败，错误代码为1。

如何在Playwright视觉比较中屏蔽多个定位器？

在C++中，可以使用可变模板参数来检索类型的内部类型。

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found

Creating and opening a URL to log in to Website via Basic Auth with Robot Framework/Selenium (Python)

AG Grid 在上下文菜单中以大文本形式打开

发表评论