Karafka 服务器为什么会抛出错误的查询水印偏移量?

huangapple go评论54阅读模式
英文:

Why Karafka Server Throw error query watermark offset?

问题

我正在为我的Ruby on Rails应用程序使用这个宝石来生成和消费来自Kafka服务器的消息。 https://karafka.io/docs/
这是我的karafka.rb文件

# frozen_string_literal: true

class KarafkaApp < Karafka::App
  setup do |config|
    config.kafka = { "bootstrap.servers": ENV["KAFKA_BOOTSTRAP_SERVERS"] }
    config.client_id = "store"
    config.consumer_persistence = !Rails.env.development?
  end

  Karafka.monitor.subscribe(Karafka::Instrumentation::LoggerListener.new)

  Karafka.producer.monitor.subscribe(
    WaterDrop::Instrumentation::LoggerListener.new(Karafka.logger)
  )

  Karafka.monitor.subscribe "error.occurred" do |event|
    type = event[:type]
    error = event[:error]
    details = (error.backtrace || []).join("\n")

    puts "Oh no! An error: #{error} of type: #{type} occurred!"
    puts details
    puts "=" * 100
    NewRelic::Agent.notice_error(error)
  end

  routes.draw do
    topic "payment-order" do
      consumer PaymentOrderConsumer
    end

    topic "payment-method-config" do
      consumer PaymentMethodConsumer
    end

    topic "order-refund" do
      consumer OrderRefundConsumer
    end
  end
end

Karafka::Web.enable!

相当长一段时间后,我遇到了这个错误
“查询karafka_consumers_states的分区0的水印偏移错误 - 本地:未知分区(unknown_partition)

消费者仍然正常消费新消息,但我定期收到上面的错误。

我在Google上没有找到其他错误。

我尝试在服务器上安装bash和librdkafka。
只使用一个Kafka引导服务器。

这是Kafka服务器上已经存在的主题。
Karafka 服务器为什么会抛出错误的查询水印偏移量?
1: https://i.stack.imgur.com/a7zPr.jpg

英文:

I'm using this gem for my Ruby on Rails Apps to produce and consume message from kafka server.
https://karafka.io/docs/

and this is my karafka.rb file

# frozen_string_literal: true

class KarafkaApp &lt; Karafka::App
  setup do |config|
    config.kafka = {&quot;bootstrap.servers&quot;: ENV[&quot;KAFKA_BOOTSTRAP_SERVERS&quot;]}
    config.client_id = &quot;store&quot;
    config.consumer_persistence = !Rails.env.development?
  end

  Karafka.monitor.subscribe(Karafka::Instrumentation::LoggerListener.new)
  
  Karafka.producer.monitor.subscribe(
    WaterDrop::Instrumentation::LoggerListener.new(Karafka.logger)
  )

  Karafka.monitor.subscribe &quot;error.occurred&quot; do |event|
    type = event[:type]
    error = event[:error]
    details = (error.backtrace || []).join(&quot;\n&quot;)

    puts &quot;Oh no! An error: #{error} of type: #{type} occurred!&quot;
    puts details
    puts &quot;=&quot; * 100
    NewRelic::Agent.notice_error(error)
  end

  routes.draw do
    topic &quot;payment-order&quot; do
      consumer PaymentOrderConsumer
    end

    topic &quot;payment-method-config&quot; do
      consumer PaymentMethodConsumer
    end

    topic &quot;order-refund&quot; do
      consumer OrderRefundConsumer
    end
  end
end

Karafka::Web.enable!

After quite sometimes, i got this error
Error querying watermark offsets for partition 0 of karafka_consumers_states - Local: Unknown partition (unknown_partition)

The consumer still consuming new message without problem, but i got error above periodically.

I'm not found another error on google.

I've tried install bash and librdkafka on server.
Only using 1 kafka bootstrap server.

This is the topic already present on the kafka server.
Karafka 服务器为什么会抛出错误的查询水印偏移量?

答案1

得分: 1

我是Karafka的作者。

你没有仔细遵循说明。Web界面需要进行引导:https://karafka.io/docs/Web-UI-Getting-Started/

第三步:

> 运行以下命令以在您的项目中安装karafka-web:

bundle exec karafka-web install

如果没有这个步骤,Karafka Web-UI将无法获取所需的数据来运行。

您可以选择运行安装或禁用Web界面。选择权在您手中。

Karafka Web-UI提供了一个状态页面,可以帮助您识别和减轻导致Web UI出现故障或不正常工作的问题。如果您看到404页面或在Karafka Web UI上遇到问题,值得访问此页面。

Karafka 服务器为什么会抛出错误的查询水印偏移量?

英文:

I'm the Karafka author.

You did not follow the instructions thoughtfully. The web-ui requires a bootstrap: https://karafka.io/docs/Web-UI-Getting-Started/

Point 3:

> Run the following command to install the karafka-web in your project:

bundle exec karafka-web install

Without that, Karafka Web-UI cannot get the expected data to operate.

You can either run the install or you can disable the web-ui. The choice is yours.

Karafka Web-UI provides a status page that can help you identify and mitigate problems that would cause the Web UI to malfunction or misbehave. If you see the 404 page or have issues with Karafka Web UI, this page is worth visiting.

Karafka 服务器为什么会抛出错误的查询水印偏移量?

huangapple
  • 本文由 发表于 2023年6月15日 05:08:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/76477548.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定