java.lang.ClassNotFoundException: com.google.cloud.sql.mysql.SocketFactory

huangapple go评论142阅读模式
英文:

java.lang.ClassNotFoundException: com.google.cloud.sql.mysql.SocketFactory

问题

我需要在从Dataproc连接到Cloud SQL时识别问题。我需要从spark.read.jdbc读取Cloud SQL实例中的表格。我参考了这个链接:https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory。

添加了Maven依赖项:

<dependency>
    <groupId>com.google.cloud.sql</groupId>
    <artifactId>mysql-socket-factory-connector-j-8</artifactId>
    <version>1.0.15</version>
</dependency>

JDBC URL => "jdbc:mysql://${OJDBCConfig.getString("database_name")}... ... ...

我还为此创建了一个"fat jar",并且也检查过我的Jar文件中是否包含SocketFactory类。

尽管如此,我仍然遇到以下异常:

Exception in thread "main" com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: 由于底层异常无法创建套接字工厂 'com.google.cloud.sql.mysql.SocketFactory':
...
...
...
Caused by: java.lang.ClassNotFoundException: com.google.cloud.sql.mysql.SocketFactory
...
...
英文:

I need help on identifying the issue when connecting to cloud sql from dataproc.
I need to read the table from cloud sql instance from spark.read.jdbc.
I am referencing this: https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory.

Added maven dependency:

 &lt;dependency&gt;
        &lt;groupId&gt;com.google.cloud.sql&lt;/groupId&gt;
        &lt;artifactId&gt;mysql-socket-factory-connector-j-8&lt;/artifactId&gt;
        &lt;version&gt;1.0.15&lt;/version&gt;
    &lt;/dependency&gt;

JDBC URL =&gt; s&quot;jdbc:mysql://${OJDBCConfig.getString(&quot;database_name&quot;)}?cloudSqlInstance=${OJDBCConfig.getString(&quot;instance_name&quot;)}&amp;socketFactory=com.google.cloud.sql.mysql.SocketFactory&amp;useSSL=false&amp;user=${OJDBCConfig.getString(&quot;user&quot;)}&amp;password=${OJDBCConfig.getString(&quot;password&quot;)}&quot;

I am also creating the fat jar for this, and also checked that my Jar contains the SocketFactory class.

Still I'm getting below exception:

Exception in thread &quot;main&quot; com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create socket factory &#39;com.google.cloud.sql.mysql.SocketFactory&#39; due to underlying exception:
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.Util.getInstance(Util.java:408)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
        at com.mysql.jdbc.MysqlIO.createSocketFactory(MysqlIO.java:3330)
        at com.mysql.jdbc.MysqlIO.&lt;init&gt;(MysqlIO.java:296)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2193)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2225)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2024)
        at com.mysql.jdbc.ConnectionImpl.&lt;init&gt;(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.&lt;init&gt;(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
        at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
        at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
        at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
        at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
        at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
        at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
        at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:238)
        at com.express.processing.enrichment.TestMysqlConnection$.main(TestMysqlConnection.scala:16)
        at com.express.processing.enrichment.TestMysqlConnection.main(TestMysqlConnection.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: com.google.cloud.sql.mysql.SocketFactory
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at com.mysql.jdbc.MysqlIO.createSocketFactory(MysqlIO.java:3328)
        ... 37 more

答案1

得分: 1

好的,以下是翻译好的部分:

似乎您没有正确构建uber-jar。

从v1.0.15版本开始,包含驱动程序和依赖项的uber-jars已在发布页面上提供。尝试使用其中之一,看看是否有效。

英文:

It sounds like you aren't building the uber-jar correctly.

As of v1.0.15, uber-jars with the driver and dependencies are provided on the Releases page. Try giving one of those a shot and see if it works.

huangapple
  • 本文由 发表于 2020年1月30日 19:14:03
  • 转载请务必保留本文链接:https://go.coder-hub.com/59984734.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定