Solr java.lang.ClassNotFoundException: net.minidev.json.writer.JsonReaderI when executing DataImportHandler

huangapple go评论86阅读模式
英文:

Solr java.lang.ClassNotFoundException: net.minidev.json.writer.JsonReaderI when executing DataImportHandler

问题

我目前正在为 Solr 8.4.1 开发自定义的 DataImportHandler

我编写了自己的实体处理器类,在其中使用了 json-path 库。尽管 Solr 的数据导入处理程序(我必须在自定义类中导入)似乎已经在自身上具有了 json-path 的依赖,但为了安全起见,我还是将所有必要的类编译到了 OparlEntityProccessor.jar 中。

然而,当我通过 Web 接口执行数据导入时,我遇到了这个错误:

Exception in thread "Thread-16" java.lang.NoClassDefFoundError: net/minidev/json/writer/JsonReaderI
            at com.jayway.jsonpath.internal.DefaultsImpl.<init>(DefaultsImpl.java:17)
            at com.jayway.jsonpath.internal.DefaultsImpl.<clinit>(DefaultsImpl.java:15)
            at com.jayway.jsonpath.Configuration.getEffectiveDefaults(Configuration.java:48)
            at com.jayway.jsonpath.Configuration.defaultConfiguration(Configuration.java:173)
            at com.jayway.jsonpath.internal.ParseContextImpl.<init>(ParseContextImpl.java:21)
            at com.jayway.jsonpath.JsonPath.parse(JsonPath.java:599)
            at com.github.fbecker97.OparlEntityProcessor.initOparlObjects(OparlEntityProcessor.java:55)
            at com.github.fbecker97.OparlEntityProcessor.init(OparlEntityProcessor.java:33)
            at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:77)
            at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:434)
            at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:415)
            at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:330)
            at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:233)
            at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:424)
            at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
            at org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:466)
            at java.base/java.lang.Thread.run(Thread.java:830)
    Caused by: java.lang.ClassNotFoundException: net.minidev.json.writer.JsonReaderI
            at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
            at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
            at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
            at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:555)
            at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
            ... 17 more

该 JAR 文件位于 C:...\solr-8.4.1\server\solr\maincore\lib,我的 solrconfig.xml 的重要部分如下:

  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" />
  
  <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
    <lst name="defaults">
      <str name="config">custom-import-config.xml</str>
    </lst>
  </requestHandler>

其中 custom-import-config.xml 如下:

<dataConfig>
	<dataSource type="URLDataSource"
			connectionTimeout="3000"
			readTimeout="5000"
			baseUrl="..."/>
			
	<document>
		<entity name="oparl"
			processor="com.github.fbecker97.OparlEntityProcessor"
			transformer="LogTransformer"
			logTemplate="TEST: ${oparl.id}" logLevel="error">
		</entity>
	</document>
</dataConfig>

我目前所知...

... 是 Solr 明确地找到了 JAR 文件并执行了类,否则错误信息会不同。甚至日志文件中还显示添加了 1 个库到类加载器。
我还检查了 JAR 文件,确保它包含了 "缺失" 的 net.minidev.json.writer.JsonReaderI 类,而它确实包含在其中。那么,为什么 Solr 在执行导入程序时找不到它呢?

我还发现了另一个奇怪的事情,即使从技术上讲它不应该有影响:
如我之前所说,Solr 和 solr-dataimporthandler 已经具有了 com.jayway.jsonpath 的依赖,这让我想知道为什么我首先需要在我的 JAR 文件中包含这些依赖项。然而,我在 solr/core/build.gradle 中找到了这个:

implementation('com.jayway.jsonpath:json-path', {
    exclude group: "net.minidev", module: "json-smart"
})

我对 Gradle 不是很了解,但他们似乎从依赖中排除了我所需的关键类。不过,再次强调,我明确地将所有必需的类编译到了我的 JAR 文件中。

有没有人熟悉这种问题?

编辑:

我现在为了测试目的编写了另一个小型处理器类,并包含了另一个名为 ejml 的外部库的一些内容。Solr 本身没有对这个库有任何依赖,所以就像之前一样,我将所有必要的类都编译到了 JAR 文件中。

这次 Solr 能够无缺地执行该处理器。

这让我认为确实存在着一些与 json-path 依赖以及上述排除 json-smart 的奇怪问题。

英文:

I am currently working on a custom DataImportHandler for Solr 8.4.1.

I wrote my own entity processor class where I used the json-path library. And even though the solr-dataimporthandler (which I have to import in my custom class) seems to already have a json-path dependency on itself, I wanted to go the safe way and compiled all necessary classes into the OparlEntityProccessor.jar.

However, when I execute the dataimporter via the web interface, I get this Error:

Exception in thread &quot;Thread-16&quot; java.lang.NoClassDefFoundError: net/minidev/json/writer/JsonReaderI
            at com.jayway.jsonpath.internal.DefaultsImpl.&lt;init&gt;(DefaultsImpl.java:17)
            at com.jayway.jsonpath.internal.DefaultsImpl.&lt;clinit&gt;(DefaultsImpl.java:15)
            at com.jayway.jsonpath.Configuration.getEffectiveDefaults(Configuration.java:48)
            at com.jayway.jsonpath.Configuration.defaultConfiguration(Configuration.java:173)
            at com.jayway.jsonpath.internal.ParseContextImpl.&lt;init&gt;(ParseContextImpl.java:21)
            at com.jayway.jsonpath.JsonPath.parse(JsonPath.java:599)
            at com.github.fbecker97.OparlEntityProcessor.initOparlObjects(OparlEntityProcessor.java:55)
            at com.github.fbecker97.OparlEntityProcessor.init(OparlEntityProcessor.java:33)
            at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:77)
            at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:434)
            at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:415)
            at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:330)
            at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:233)
            at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:424)
            at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
            at org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:466)
            at java.base/java.lang.Thread.run(Thread.java:830)
    Caused by: java.lang.ClassNotFoundException: net.minidev.json.writer.JsonReaderI
            at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
            at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
            at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
            at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:555)
            at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
            ... 17 more

The jar is located at C:...\solr-8.4.1\server\solr\maincore\lib and the important part of my solrconfig.xml looks like this:

  &lt;lib dir=&quot;${solr.install.dir:../../../..}/dist/&quot; regex=&quot;solr-dataimporthandler-.*\.jar&quot; /&gt;
  
  &lt;requestHandler name=&quot;/dataimport&quot; class=&quot;org.apache.solr.handler.dataimport.DataImportHandler&quot;&gt;
    &lt;lst name=&quot;defaults&quot;&gt;
	  &lt;str name=&quot;config&quot;&gt;custom-import-config.xml&lt;/str&gt;
    &lt;/lst&gt;
  &lt;/requestHandler&gt;

with this custom-import-config.xml:

&lt;dataConfig&gt;
	&lt;dataSource type=&quot;URLDataSource&quot;
			connectionTimeout=&quot;3000&quot;
			readTimeout=&quot;5000&quot;
			baseUrl=&quot;...&quot;/&gt;
			
	&lt;document&gt;
		&lt;entity name=&quot;oparl&quot;
			processor=&quot;com.github.fbecker97.OparlEntityProcessor&quot;
			transformer=&quot;LogTransformer&quot;
			logTemplate=&quot;TEST: ${oparl.id}&quot; logLevel=&quot;error&quot; &gt;
			
			
		&lt;/entity&gt;
	&lt;/document&gt;
&lt;/dataConfig&gt;

What I know so far...

... is that solr definitely finds the jar and executes the class, otherwise the error would be different. It even says in the log file that 1 lib has been added to the classloader.
I also looked into the jar to make sure it includes the "missing" net.minidev.json.writer.JsonReaderI class, and it does. So.. why doesn't solr find it when executing the importer?

There's another strange thing I found, even though it technically shouldn't matter:
Like I said before, solr and the solr-dataimporthandler already have a com.jayway.jsonpath dependency, which made me wonder why I even had to include these dependencies in my jar in the first place.
But then I found this in solr/core/build.gradle:

implementation(&#39;com.jayway.jsonpath:json-path&#39;, {
    exclude group: &quot;net.minidev&quot;, module: &quot;json-smart&quot;

I don't know much about gradle, but they seem to exclude the crucial classes I need from their dependencies.
Then again, Im not sure how this would matter as I explicitly compiled all needed classes into my jar.

Is anyone familiar with this kind of problem ?

EDIT:

I now wrote another small proccessor class for testing purposes and I included some stuff from another external library called ejml. Solr itself doesn't have any dependencies on this library so just like before I compiled all neccessary classes into the jar.

This time solr was able to execute the proccessor flawlessly.

This makes me think that there really is some weird stuff going on with the json-path dependency and the above mentioned exclusion of json-smart.

答案1

得分: 2

问题已解决!

我从...\solr-8.4.1\server\solr-webapp\webapp\WEB-INF\lib中删除了json-path-2.4.0.jar。这样solr除了在我的jar中搜索缺失的类之外别无选择...我猜是这样吧?
我仍然不太清楚那里发生了什么,但是它能正常工作,所以...

英文:

Solved it!

I deleted the json-path-2.4.0.jar from ...\solr-8.4.1\server\solr-webapp\webapp\WEB-INF\lib. That way solr has no other option than searching for the missing classes in my jar.. i guess ?
I still don't really know what's going on there, but it works so...

huangapple
  • 本文由 发表于 2020年3月16日 03:42:21
  • 转载请务必保留本文链接:https://go.coder-hub.com/60696899.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定