如何停止 Dapr Kafka 组件连接到已关闭的消费者组？

huangapple
go评论115阅读模式
英文:

How can I stop Dapr Kafka Component from connecting to closed consumer groups?

问题

我在Kubernetes中有以下组件：

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: kafka-pubsub
  5.   namespace: default
  6. spec:
  7.   type: pubsub.kafka
  8.   version: v1
  9.   metadata:
  10.     # Kafka broker连接设置
  11.     - name: brokers
  12.       value: "kafka-0.kafka-headless.default.svc.cluster.local:9092, kafka.default.svc.cluster.local:9092"
  13.     - name: authRequired
  14.       value: "false"
  15.     - name: saslUsername
  16.       value: ""
  17.     - name: saslPassword
  18.       value: ""
  19. scopes:
  20.   - signalrconnectorinterface
  21.   - cardhandler
  22.   - eventtrigger

我有许多微服务，它们都正确订阅，但订阅两个主题（可能更多）的微服务在daprd日志中订阅时会收到以下错误消息：

  1. time="2023-08-04T13:41:55.200932919Z" level=error msg="Error consuming [receive-command-message]. Retrying...: kafka: tried to use a consumer group that was closed" app_id=cardhandler component="kafka-pubsub (pubsub.kafka/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2

它注意到有两个订阅，尝试分配它们然后失败。有什么建议吗？

英文:

I have the following component in k8s:

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: kafka-pubsub
  5.   namespace: default
  6. spec:
  7.   type: pubsub.kafka
  8.   version: v1
  9.   metadata:
  10.       # Kafka broker connection setting
  11.     - name: brokers
  12.       value: "kafka-0.kafka-headless.default.svc.cluster.local:9092, 
  13. kafka.default.svc.cluster.local:9092"
  14.     - name: authRequired
  15.       value: "false"
  16.     - name: saslUsername
  17.       value: ""
  18.     - name: saslPassword
  19.       value: ""
  20. scopes:
  21.   - signalrconnectorinterface
  22.   - cardhandler
  23.   - eventtrigger

I have a number of microservices which all subscribe properly, however the ones which subscribe to two topics (and more presumably) get the following error message upon subscription in the daprd logs:

  1. time="2023-08-04T13:41:54.24563974Z" level=info msg="initializing Dapr workflow component" app_id=cardhandler component="dapr (workflow.dapr/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2
  2. time="2023-08-04T13:41:54.263847422Z" level=info msg="app is subscribed to the following topics: [send-card receive-command-message] through pubsub=kafka-pubsub" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime type=log ver=1.11.2
  3. time="2023-08-04T13:41:54.267674781Z" level=info msg="placement tables updated, version: 0" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime.actor.internal.placement type=log ver=1.11.2
  4. time="2023-08-04T13:41:55.200932919Z" level=error msg="Error consuming [receive-command-message]. Retrying...: kafka: tried to use a consumer group that was closed" app_id=cardhandler component="kafka-pubsub (pubsub.kafka/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2
  5. time="2023-08-04T13:41:58.812244394Z" level=info msg="dapr initialized. Status: Running. Init Elapsed 4843ms" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime type=log ver=1.11.2

It notices there are two subscriptions, tries to allocate them then fails.
Any advice?

答案1

得分: 0

根据OneCricketeer的说法，这实际上没有问题，这是预期的行为。由于某种原因，当我使用新的Apache Kafka的helm安装时，这种情况突然发生。

最终发现是代码库的问题，所以重置了一切，一切都正常 如何停止 Dapr Kafka 组件连接到已关闭的消费者组？

英文:

As per OneCricketeer, there is actually no issue here, this is intended behaviour. For some reason this just started happening when I was using a new helm install of Apache Kafka.

Ended up being a codebase issue so reset and all fine 如何停止 Dapr Kafka 组件连接到已关闭的消费者组？

huangapple
  • 本文由 发表于 2023年8月4日 22:04:40
  • 转载请务必保留本文链接：https://go.coder-hub.com/76836678.html
  • apache-kafka
  • dapr
  • dapr-sidekick
  • sarama
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定