Strimzi Kafka – Kafka Connect 未能安装

huangapple
go评论100阅读模式
英文:

Strimzi kafka - kafka connect not getting installed

问题

根据提供的日志，看起来你的Kafka Connect 在与 Kafka 集群建立连接时遇到了SSL认证问题，导致连接失败。错误消息中提到了"Failed to process post-handshake messages"和"Tag mismatch"，这通常表示SSL握手过程中发生了问题。

为了解决这个问题，你可以采取以下步骤：

  1. 检查证书配置：确保Kafka Connect 的TLS/SSL证书配置正确。检查证书文件的位置、密码和类型是否都正确。

  2. 检查Kafka集群配置：确保Kafka Connect 的配置与Kafka集群的配置匹配，包括安全协议、信任库等。确保Kafka集群的连接信息正确，如bootstrap.servers。

  3. 检查证书匹配：确保Kafka Connect 使用的证书与Kafka集群期望的证书匹配。这包括证书的有效性和相关密钥的匹配。

  4. 检查网络连接：确保Kafka Connect 能够与Kafka集群建立网络连接，确保防火墙或网络策略没有阻止连接。

  5. 详细日志：检查Kafka Connect 的详细日志以获取更多信息，有助于确定SSL握手中的问题。

如果你能提供更多有关证书、Kafka集群配置和Kafka Connect 配置的信息，将更容易帮助你解决这个问题。同时，你可以考虑查看Kafka Connect的官方文档以获取更多关于TLS/SSL配置的信息。

英文:

I've Strimzi Kafka setup on GKE, and it is working fine.
I've a requirement to setup MirrorMaker2 to push data from source kafka topic to target Kafka topic.
From what i understand, MirrorMaker2 requires KafkaConnect.

I'm trying to install KafkaConnect on the GKE cluster in namespace kafka-connect, the yaml used is the following.

  1. apiVersion: kafka.strimzi.io/v1beta2
  2. kind: KafkaConnect
  3. metadata:
  4.   name: my-connect-cluster
  5. #  annotations:
  6. #    strimzi.io/use-connector-resources: "true"
  7. spec:
  8.   version: 3.0.0
  9.   replicas: 1
  10.   bootstrapServers: versa-kafka-gke-kafka-bootstrap:9093
  11.   tls:
  12.     trustedCertificates:
  13.       - secretName: versa-kafka-gke-cluster-ca-cert
  14.         certificate: ca.crt
  15.   config:
  16.     group.id: connect-cluster
  17.     offset.storage.topic: connect-cluster-offsets
  18.     config.storage.topic: connect-cluster-configs
  19.     status.storage.topic: connect-cluster-status
  20.     # -1 means it will use the default replication factor configured in the broker
  21.     config.storage.replication.factor: -1
  22.     offset.storage.replication.factor: -1
  23.     status.storage.replication.factor: -1

Command run to install:

  1. kubectl apply -f kafka-connect.yaml -n kafka-connect

Note : Strimzi kafka is installed in namespace 'kafka', version - 3.0.0

I was expecting the KafkaConnect to get installed & the pods to be installed as well, however KafkaConnect resource is created, but not showing as Ready.
Also, the pods are not getting created.

  1. (base) Karans-MacBook-Pro:kafkaConnect karanalang$ kc get kafkaconnect my-connect-cluster -n kafka-connect
  2. NAME                 DESIRED REPLICAS   READY
  3. my-connect-cluster   1

On describing the my-connect-cluster, here is the output (not showing any errors)

  1. (base) Karans-MacBook-Pro:kafkaConnect karanalang$ kc describe kafkaconnect my-connect-cluster -n kafka-connect
  2. Name:         my-connect-cluster
  3. Namespace:    kafka-connect
  4. Labels:       <none>
  5. Annotations:  <none>
  6. API Version:  kafka.strimzi.io/v1beta2
  7. Kind:         KafkaConnect
  8. Metadata:
  9.   Creation Timestamp:  2023-02-16T06:38:41Z
  10.   Generation:          1
  11.   Managed Fields:
  12.     API Version:  kafka.strimzi.io/v1beta2
  13.     Fields Type:  FieldsV1
  14.     fieldsV1:
  15.       f:metadata:
  16.         f:annotations:
  17.           .:
  18.           f:kubectl.kubernetes.io/last-applied-configuration:
  19.       f:spec:
  20.         .:
  21.         f:bootstrapServers:
  22.         f:config:
  23.           .:
  24.           f:config.storage.replication.factor:
  25.           f:config.storage.topic:
  26.           f:group.id:
  27.           f:offset.storage.replication.factor:
  28.           f:offset.storage.topic:
  29.           f:status.storage.replication.factor:
  30.           f:status.storage.topic:
  31.         f:replicas:
  32.         f:tls:
  33.           .:
  34.           f:trustedCertificates:
  35.         f:version:
  36.     Manager:         kubectl-client-side-apply
  37.     Operation:       Update
  38.     Time:            2023-02-16T06:38:41Z
  39.   Resource Version:  266457732
  40.   UID:               bdd697f8-e38a-466b-8ddf-81ed1ae54efe
  41. Spec:
  42.   Bootstrap Servers:  versa-kafka-gke-kafka-bootstrap:9093
  43.   Config:
  44.     config.storage.replication.factor:  -1
  45.     config.storage.topic:               connect-cluster-configs
  46.     group.id:                           connect-cluster
  47.     offset.storage.replication.factor:  -1
  48.     offset.storage.topic:               connect-cluster-offsets
  49.     status.storage.replication.factor:  -1
  50.     status.storage.topic:               connect-cluster-status
  51.   Replicas:                             1
  52.   Tls:
  53.     Trusted Certificates:
  54.       Certificate:  ca.crt
  55.       Secret Name:  versa-kafka-gke-cluster-ca-cert
  56.   Version:          3.0.0
  57. Events:             <none>

How do i debug/fix this ?

tia!

Update :
Based on note from Jakub, i re-installed kafkaconnect in the same namespace as Strimzi kafka (i.e. namespace - kafka), and pods are coming up now. However the logs show error as shown below :

  1. (base) Karans-MacBook-Pro:kafkaConnect karanalang$ kc logs -f pod/my-connect-cluster-connect-67f76f5d89-nv9sj -n kafka
  2. Preparing truststore
  3. Certificate was added to keystore
  4. Preparing truststore is complete
  5. Starting Kafka Connect with configuration:
  6. # Bootstrap servers
  7. bootstrap.servers=versa-kafka-gke-w-kafka-bootstrap:9093
  8. # REST Listeners
  9. rest.port=8083
  10. rest.advertised.host.name=10.6.0.199 
  11. rest.advertised.port=8083
  12. # Plugins
  13. plugin.path=/opt/kafka/plugins
  14. # Provided configuration
  15. offset.storage.topic=connect-cluster-offsets
  16. value.converter=org.apache.kafka.connect.json.JsonConverter
  17. config.storage.topic=connect-cluster-configs
  18. key.converter=org.apache.kafka.connect.json.JsonConverter
  19. group.id=connect-cluster
  20. status.storage.topic=connect-cluster-status
  21. config.storage.replication.factor=-1
  22. offset.storage.replication.factor=-1
  23. status.storage.replication.factor=-1
  24. security.protocol=SSL
  25. producer.security.protocol=SSL
  26. consumer.security.protocol=SSL
  27. admin.security.protocol=SSL
  28. # TLS / SSL
  29. ssl.truststore.location=/tmp/kafka/cluster.truststore.p12
  30. ssl.truststore.password=[hidden]
  31. ssl.truststore.type=PKCS12
  32. producer.ssl.truststore.location=/tmp/kafka/cluster.truststore.p12
  33. producer.ssl.truststore.password=[hidden]
  34. consumer.ssl.truststore.location=/tmp/kafka/cluster.truststore.p12
  35. consumer.ssl.truststore.password=[hidden]
  36. admin.ssl.truststore.location=/tmp/kafka/cluster.truststore.p12
  37. admin.ssl.truststore.password=[hidden]
  38. # Additional configuration
  39. client.rack=
  40. 2023-02-17 06:43:08,952 INFO WorkerInfo values: 
  41. jvm.args = -Xms128M, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -XX:MaxInlineLevel=15, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/opt/kafka, -Dlog4j.configuration=file:/opt/kafka/custom-config/log4j.properties
  42. jvm.spec = Red Hat, Inc., OpenJDK 64-Bit Server VM, 11.0.12, 11.0.12+7-LTS
  43. jvm.classpath = /opt/kafka/bin/../libs/accessors-smart-2.4.7.jar:/opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/annotations-13.0.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/bin/../libs/automaton-1.11-8.jar:/opt/kafka/bin/../libs/checker-qual-3.5.0.jar:/opt/kafka/bin/../libs/commons-cli-1.4.jar:/opt/kafka/bin/../libs/commons-lang-2.6.jar:/opt/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/bin/../libs/connect-api-3.0.0.jar:/opt/kafka/bin/../libs/connect-basic-auth-extension-3.0.0.jar:/opt/kafka/bin/../libs/connect-file-3.0.0.jar:/opt/kafka/bin/../libs/connect-json-3.0.0.jar:/opt/kafka/bin/../libs/connect-mirror-3.0.0.jar:/opt/kafka/bin/../libs/connect-mirror-client-3.0.0.jar:/opt/kafka/bin/../libs/connect-runtime-3.0.0.jar:/opt/kafka/bin/../libs/connect-transforms-
  44. ......
  45. 2023-02-17 06:43:24,944 INFO Added alias 'InsertHeader' to plugin 'org.apache.kafka.connect.transforms.InsertHeader' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  46. 2023-02-17 06:43:24,945 INFO Added alias 'RegexRouter' to plugin 'org.apache.kafka.connect.transforms.RegexRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  47. 2023-02-17 06:43:24,945 INFO Added alias 'TimestampRouter' to plugin 'org.apache.kafka.connect.transforms.TimestampRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  48. 2023-02-17 06:43:24,945 INFO Added alias 'ValueToKey' to plugin 'org.apache.kafka.connect.transforms.ValueToKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  49. 2023-02-17 06:43:24,945 INFO Added alias 'HasHeaderKey' to plugin 'org.apache.kafka.connect.transforms.predicates.HasHeaderKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  50. 2023-02-17 06:43:24,945 INFO Added alias 'RecordIsTombstone' to plugin 'org.apache.kafka.connect.transforms.predicates.RecordIsTombstone' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  51. 2023-02-17 06:43:24,945 INFO Added alias 'TopicNameMatches' to plugin 'org.apache.kafka.connect.transforms.predicates.TopicNameMatches' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  52. 2023-02-17 06:43:24,945 INFO Added alias 'BasicAuthSecurityRestExtension' to plugin 'org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  53. 2023-02-17 06:43:24,945 INFO Added aliases 'AllConnectorClientConfigOverridePolicy' and 'All' to plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  54. 2023-02-17 06:43:24,945 INFO Added aliases 'NoneConnectorClientConfigOverridePolicy' and 'None' to plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  55. 2023-02-17 06:43:24,945 INFO Added aliases 'PrincipalConnectorClientConfigOverridePolicy' and 'Principal' to plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
  56. 2023-02-17 06:43:25,347 INFO DistributedConfig values: 
  57. access.control.allow.methods = 
  58. access.control.allow.origin = 
  59. admin.listeners = null
  60. bootstrap.servers = [versa-kafka-gke-w-kafka-bootstrap:9093]
  61. client.dns.lookup = use_all_dns_ips
  62. client.id = 
  63. config.providers = []
  64. config.storage.replication.factor = -1
  65. config.storage.topic = connect-cluster-configs
  66. connect.protocol = sessioned
  67. connections.max.idle.ms = 540000
  68. connector.client.config.override.policy = All
  69. group.id = connect-cluster
  70. header.converter = class org.apache.kafka.connect.storage.SimpleHeaderConverter
  71. heartbeat.interval.ms = 3000
  72. inter.worker.key.generation.algorithm = HmacSHA256
  73. inter.worker.key.size = null
  74. inter.worker.key.ttl.ms = 3600000
  75. inter.worker.signature.algorithm = HmacSHA256
  76. inter.worker.verification.algorithms = [HmacSHA256]
  77. key.converter = class org.apache.kafka.connect.json.JsonConverter
  78. listeners = [http://:8083]
  79. metadata.max.age.ms = 300000
  80. metric.reporters = []
  81. metrics.num.samples = 2
  82. metrics.recording.level = INFO
  83. metrics.sample.window.ms = 30000
  84. offset.flush.interval.ms = 60000
  85. offset.flush.timeout.ms = 5000
  86. offset.storage.partitions = 25
  87. offset.storage.replication.factor = -1
  88. offset.storage.topic = connect-cluster-offsets
  89. plugin.path = [/opt/kafka/plugins]
  90. rebalance.timeout.ms = 60000
  91. receive.buffer.bytes = 32768
  92. reconnect.backoff.max.ms = 1000
  93. reconnect.backoff.ms = 50
  94. request.timeout.ms = 40000
  95. response.http.headers.config = 
  96. rest.advertised.host.name = 10.6.0.199
  97. rest.advertised.listener = null
  98. rest.advertised.port = 8083
  99. rest.extension.classes = []
  100. retry.backoff.ms = 100
  101. sasl.client.callback.handler.class = null
  102. sasl.jaas.config = null
  103. sasl.kerberos.kinit.cmd = /usr/bin/kinit
  104. sasl.kerberos.min.time.before.relogin = 60000
  105. sasl.kerberos.service.name = null
  106. sasl.kerberos.ticket.renew.jitter = 0.05
  107. sasl.kerberos.ticket.renew.window.factor = 0.8
  108. sasl.login.callback.handler.class = null
  109. sasl.login.class = null
  110. sasl.login.refresh.buffer.seconds = 300
  111. sasl.login.refresh.min.period.seconds = 60
  112. sasl.login.refresh.window.factor = 0.8
  113. sasl.login.refresh.window.jitter = 0.05
  114. sasl.mechanism = GSSAPI
  115. scheduled.rebalance.max.delay.ms = 300000
  116. security.protocol = SSL
  117. send.buffer.bytes = 131072
  118. session.timeout.ms = 10000
  119. socket.connection.setup.timeout.max.ms = 30000
  120. socket.connection.setup.timeout.ms = 10000
  121. ssl.cipher.suites = null
  122. ssl.client.auth = none
  123. ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
  124. ssl.endpoint.identification.algorithm = https
  125. ssl.engine.factory.class = null
  126. ssl.key.password = null
  127. ssl.keymanager.algorithm = SunX509
  128. ssl.keystore.certificate.chain = null
  129. ssl.keystore.key = null
  130. ssl.keystore.location = null
  131. ssl.keystore.password = null
  132. ssl.keystore.type = JKS
  133. ssl.protocol = TLSv1.3
  134. ssl.provider = null
  135. ssl.secure.random.implementation = null
  136. ssl.trustmanager.algorithm = PKIX
  137. ssl.truststore.certificates = null
  138. ssl.truststore.location = /tmp/kafka/cluster.truststore.p12
  139. ssl.truststore.password = [hidden]
  140. ssl.truststore.type = PKCS12
  141. status.storage.partitions = 5
  142. status.storage.replication.factor = -1
  143. status.storage.topic = connect-cluster-status
  144. task.shutdown.graceful.timeout.ms = 5000
  145. topic.creation.enable = true
  146. topic.tracking.allow.reset = true
  147. topic.tracking.enable = true
  148. value.converter = class org.apache.kafka.connect.json.JsonConverter
  149. worker.sync.timeout.ms = 3000
  150. worker.unsync.backoff.ms = 300000
  151. (org.apache.kafka.connect.runtime.distributed.DistributedConfig) [main]
  152. 2023-02-17 06:43:25,355 INFO Creating Kafka admin client (org.apache.kafka.connect.util.ConnectUtils) [main]
  153. 2023-02-17 06:43:25,359 INFO AdminClientConfig values: 
  154. bootstrap.servers = [versa-kafka-gke-w-kafka-bootstrap:9093]
  155. client.dns.lookup = use_all_dns_ips
  156. client.id = 
  157. connections.max.idle.ms = 300000
  158. default.api.timeout.ms = 60000
  159. metadata.max.age.ms = 300000
  160. metric.reporters = []
  161. metrics.num.samples = 2
  162. metrics.recording.level = INFO
  163. metrics.sample.window.ms = 30000
  164. receive.buffer.bytes = 65536
  165. reconnect.backoff.max.ms = 1000
  166. reconnect.backoff.ms = 50
  167. request.timeout.ms = 30000
  168. retries = 2147483647
  169. retry.backoff.ms = 100
  170. sasl.client.callback.handler.class = null
  171. sasl.jaas.config = null
  172. sasl.kerberos.kinit.cmd = /usr/bin/kinit
  173. sasl.kerberos.min.time.before.relogin = 60000
  174. sasl.kerberos.service.name = null
  175. sasl.kerberos.ticket.renew.jitter = 0.05
  176. sasl.kerberos.ticket.renew.window.factor = 0.8
  177. sasl.login.callback.handler.class = null
  178. sasl.login.class = null
  179. sasl.login.refresh.buffer.seconds = 300
  180. sasl.login.refresh.min.period.seconds = 60
  181. sasl.login.refresh.window.factor = 0.8
  182. sasl.login.refresh.window.jitter = 0.05
  183. sasl.mechanism = GSSAPI
  184. security.protocol = SSL
  185. security.providers = null
  186. send.buffer.bytes = 131072
  187. socket.connection.setup.timeout.max.ms = 30000
  188. socket.connection.setup.timeout.ms = 10000
  189. ssl.cipher.suites = null
  190. ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
  191. ssl.endpoint.identification.algorithm = https
  192. ssl.engine.factory.class = null
  193. ssl.key.password = null
  194. ssl.keymanager.algorithm = SunX509
  195. ssl.keystore.certificate.chain = null
  196. ssl.keystore.key = null
  197. ssl.keystore.location = null
  198. ssl.keystore.password = null
  199. ssl.keystore.type = JKS
  200. ssl.protocol = TLSv1.3
  201. ssl.provider = null
  202. ssl.secure.random.implementation = null
  203. ssl.trustmanager.algorithm = PKIX
  204. ssl.truststore.certificates = null
  205. ssl.truststore.location = /tmp/kafka/cluster.truststore.p12
  206. ssl.truststore.password = [hidden]
  207. ssl.truststore.type = PKCS12
  208. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  209. 2023-02-17 06:43:27,848 WARN The configuration 'producer.ssl.truststore.password' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  210. 2023-02-17 06:43:27,849 WARN The configuration 'group.id' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  211. 2023-02-17 06:43:27,849 WARN The configuration 'rest.advertised.port' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  212. 2023-02-17 06:43:27,849 WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  213. 2023-02-17 06:43:27,850 WARN The configuration 'admin.security.protocol' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  214. 2023-02-17 06:43:27,850 WARN The configuration 'consumer.ssl.truststore.location' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  215. 2023-02-17 06:43:27,850 WARN The configuration 'producer.ssl.truststore.location' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  216. 2023-02-17 06:43:27,850 WARN The configuration 'status.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  217. 2023-02-17 06:43:27,850 WARN The configuration 'offset.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  218. 2023-02-17 06:43:27,850 WARN The configuration 'consumer.security.protocol' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  219. 2023-02-17 06:43:27,851 WARN The configuration 'value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  220. 2023-02-17 06:43:27,851 WARN The configuration 'key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  221. 2023-02-17 06:43:27,851 WARN The configuration 'admin.ssl.truststore.password' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  222. 2023-02-17 06:43:27,851 WARN The configuration 'consumer.ssl.truststore.password' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  223. 2023-02-17 06:43:27,851 WARN The configuration 'config.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  224. 2023-02-17 06:43:27,851 WARN The configuration 'producer.security.protocol' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  225. 2023-02-17 06:43:27,852 WARN The configuration 'rest.advertised.host.name' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  226. 2023-02-17 06:43:27,852 WARN The configuration 'status.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  227. 2023-02-17 06:43:27,852 WARN The configuration 'client.rack' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  228. 2023-02-17 06:43:27,852 WARN The configuration 'rest.port' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  229. 2023-02-17 06:43:27,853 WARN The configuration 'config.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  230. 2023-02-17 06:43:27,853 WARN The configuration 'offset.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  231. 2023-02-17 06:43:27,855 WARN The configuration 'admin.ssl.truststore.location' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [main]
  232. 2023-02-17 06:43:27,856 INFO Kafka version: 3.0.0 (org.apache.kafka.common.utils.AppInfoParser) [main]
  233. 2023-02-17 06:43:27,857 INFO Kafka commitId: 8cb0a5e9d3441962 (org.apache.kafka.common.utils.AppInfoParser) [main]
  234. 2023-02-17 06:43:27,857 INFO Kafka startTimeMs: 1676616207856 (org.apache.kafka.common.utils.AppInfoParser) [main]
  235. 2023-02-17 06:43:30,857 INFO [AdminClient clientId=adminclient-1] Failed re-authentication with versa-kafka-gke-w-kafka-bootstrap/10.6.131.57 (Failed to process post-handshake messages) (org.apache.kafka.common.network.Selector) [kafka-admin-client-thread | adminclient-1]
  236. 2023-02-17 06:43:30,867 ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (versa-kafka-gke-w-kafka-bootstrap/10.6.131.57:9093) failed authentication due to: Failed to process post-handshake messages (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-1]
  237. 2023-02-17 06:43:30,870 WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-1]
  238. org.apache.kafka.common.errors.SslAuthenticationException: Failed to process post-handshake messages
  239. Caused by: javax.net.ssl.SSLException: Tag mismatch!
  240. at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133)
  241. at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
  242. at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
  243. at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
  244. at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:123)
  245. at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681)
  246. at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636)
  247. at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454)
  248. at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433)
  249. at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637)
  250. at org.apache.kafka.common.network.SslTransportLayer.read(SslTransportLayer.java:567)
  251. at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:95)
  252. at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)
  253. at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)
  254. at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)
  255. at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)
  256. at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
  257. at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:551)
  258. at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1389)
  259. at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1320)
  260. at java.base/java.lang.Thread.run(Thread.java:829)
  261. Caused by: javax.crypto.AEADBadTagException: Tag mismatch!
  262. at java.base/com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:623)
  263. at java.base/com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1116)
  264. at java.base/com.sun.crypto.provider.CipherCore.fillOutputBuffer(CipherCore.java:1053)
  265. at java.base/com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:941)
  266. at java.base/com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:491)
  267. at java.base/javax.crypto.CipherSpi.bufferCrypt(CipherSpi.java:779)
  268. at java.base/javax.crypto.CipherSpi.engineDoFinal(CipherSpi.java:730)
  269. at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2497)
  270. at java.base/sun.security.ssl.SSLCipher$T13GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1903)
  271. at java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:240)
  272. at java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:197)
  273. at java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:160)
  274. at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
  275. ... 16 more
  276. 2023-02-17 06:43:30,946 INFO App info kafka.admin.client for adminclient-1 unregistered (org.apache.kafka.common.utils.AppInfoParser) [kafka-admin-client-thread | adminclient-1]
  277. 2023-02-17 06:43:30,947 INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-1]
  278. org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: fetchMetadata
  279. 2023-02-17 06:43:30,948 INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-1]
  280. org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: fetchMetadata
  281. 2023-02-17 06:43:30,949 INFO [AdminClient clientId=adminclient-1] Timed out 2 remaining operation(s) during close. (org.apache.kafka.clients.admin.KafkaAdminClient) [kafka-admin-client-thread | adminclient-1]
  282. 2023-02-17 06:43:30,959 INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics) [kafka-admin-client-thread | adminclient-1]
  283. 2023-02-17 06:43:30,960 INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics) [kafka-admin-client-thread | adminclient-1]
  284. 2023-02-17 06:43:30,961 INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics) [kafka-admin-client-thread | adminclient-1]
  285. 2023-02-17 06:43:30,961 ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed) [main]
  286. org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check worker's broker connection and security properties.
  287. at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:70)
  288. at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:51)
  289. at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:97)
  290. at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)
  291. Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SslAuthenticationException: Failed to process post-handshake messages
  292. at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
  293. at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
  294. at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
  295. at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:64)
  296. ... 3 more
  297. Caused by: org.apache.kafka.common.errors.SslAuthenticationException: Failed to process post-handshake messages
  298. Caused by: javax.net.ssl.SSLException: Tag mismatch!
  299. at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133)
  300. at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
  301. at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
  302. at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
  303. at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:123)
  304. at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681)
  305. at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636)
  306. at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454)
  307. at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433)
  308. at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637)
  309. at org.apache.kafka.common.network.SslTransportLayer.read(SslTransportLayer.java:567)
  310. at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:95)
  311. at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)
  312. at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)
  313. at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)
  314. at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)
  315. at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
  316. at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:551)
  317. at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1389)
  318. at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1320)
  319. at java.base/java.lang.Thread.run(Thread.java:829)
  320. Caused by: javax.crypto.AEADBadTagException: Tag mismatch!
  321. at java.base/com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:623)
  322. at java.base/com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1116)
  323. at java.base/com.sun.crypto.provider.CipherCore.fillOutputBuffer(CipherCore.java:1053)
  324. at java.base/com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:941)
  325. at java.base/com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:491)
  326. at java.base/javax.crypto.CipherSpi.bufferCrypt(CipherSpi.java:779)
  327. at java.base/javax.crypto.CipherSpi.engineDoFinal(CipherSpi.java:730)
  328. at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2497)
  329. at java.base/sun.security.ssl.SSLCipher$T13GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1903)
  330. at java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:240)
  331. at java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:197)
  332. at java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:160)
  333. at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
  334. ... 16 more

Pls note :
Strimzi Kafka version - 3.0.0, hence I've changed the version of kafkaconnect to - 3.0.0 as well.

答案1

得分: 1

你说你想使用MirrorMaker2吗？那么你应该使用那个种类，而不是KafkaConnect。详见https://strimzi.io/blog/2020/03/30/introducing-mirrormaker2/

另外，正如评论所提到的，确保操作员正在监视你安装资源的命名空间。仅仅因为你可以获取/描述资源并不意味着操作员知道它，或正在处理它（查看它的日志）。

英文:

You say you want to use MirrorMaker2? Then you should be using that kind, not KafkaConnect.
https://strimzi.io/blog/2020/03/30/introducing-mirrormaker2/

And, as commented, ensure the operator is watching the namespace where you install any resources. Just because you can get/describe the resource doesn't mean the operator knows about it, or is processing it (look at its logs)

huangapple
  • 本文由 发表于 2023年2月16日 14:54:17
  • 转载请务必保留本文链接：https://go.coder-hub.com/75468758.html
  • apache-kafka
  • apache-kafka-connect
  • strimzi
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定