如何防止Rails并行测试和Sidekiq调用Redis。

huangapple go评论68阅读模式
英文:

How to prevent Rails parallel tests and Sidekiq from calling Redis

问题

我有一个使用Sidekiq进行异步作业的Rails应用程序。
我设置了并行测试来加速我的测试流程。 这个方法效果很好,但我有一个主要问题:Redis在试图将Sidekiq作业排队时成为瓶颈。

我一直看到这些错误:

Redis::CannotConnectError:
无法连接到127.0.0.1:6379上的Redis (Errno::ECONNREFUSED)

我正在运行一些旧版本的gems,还使用了Sidekiq debounce(不确定是否是原因)

sidekiq(5.2.1)
sidekiq-debounce(1.1.0)
rspec-sidekiq(3.0.3)

我原本以为rspec-sidekiq能够通过使用Sidekiq::Testing.inline!Sidekiq::Testing.fake!来将作业内联运行或将它们推送到数组中来模拟对Redis的调用。
然而,这两个选项都不起作用。 我漏掉了什么?

我尝试设置测试环境以伪造或内联作业,如下所示:

require "sidekiq/testing"

Sidekiq::Testing.inline!

require "sidekiq/testing"

Sidekiq::Testing.fake!

然后,我甚至在一些测试中检查以确保这是被设置的:

puts "SIDEKIQ MODE ENABLED: #{Sidekiq::Testing.enabled?}"
puts "SIDEKIQ MODE FAKE: #{Sidekiq::Testing.fake?}"
puts "SIDEKIQ MODE INLINE: #{Sidekiq::Testing.inline?}"
puts "SIDEKIQ MODE DISABLED: #{Sidekiq::Testing.disabled?}"

并且返回我期望的结果,但我仍然得到了Redis连接被拒绝的错误。

我还尝试将各个部分包装在块中,如下所示:

Sidekiq::Testing.fake! do
 # 一些代码
end
英文:

I have a Rails application that is using Sidekiq for asynchronous jobs.
I setup parallel testing to speed up my testing pipeline. This is working great but I have one major problem: Redis is bottlenecking with Sidekiq jobs trying to enqueue work there.

I keep seeing these errors:

Redis::CannotConnectError:
Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED)

I am running some old versions of gems and also using Sidekiq debounce (Not sure if this is the reason)

sidekiq (5.2.1)
sidekiq-debounce (1.1.0)
rspec-sidekiq (3.0.3)

I was under the impression that rspec-sidekiq would be able to either stub the calls to Redis by using either: Sidekiq::Testing.inline! or Sidekiq::Testing.fake! to run the jobs inline or push them to an array respectively.
However neither of these options are working. What am I missing?

I tried setting up the test environment to fake or inline the jobs like:

require "sidekiq/testing"

Sidekiq::Testing.inline!

OR

require "sidekiq/testing"

Sidekiq::Testing.fake!

Then I even checked in some of the tests to make sure this was being set by:

puts "SIDEKIQ MODE ENABLED: #{Sidekiq::Testing.enabled?}"
puts "SIDEKIQ MODE FAKE: #{Sidekiq::Testing.fake?}"
puts "SIDEKIQ MODE INLINE: #{Sidekiq::Testing.inline?}"
puts "SIDEKIQ MODE DISABLED: #{Sidekiq::Testing.disabled?}"

and it returned what I expected but I still got the Redis connections refused.

I also tried wrapping the individual parts in blocks like:

Sidekiq::Testing.fake! do
 # some code
end

答案1

得分: 1

我认为我已经找到了问题所在或需要进行的更改。
我们使用了Sidekiq以及Sidekiq-debounce
我认为实际上它截获了基本的Sidekiq异步调用,因此即使我将它们设置为Sidekiq::Testing.fake!Sidekiq::Testing.inline!,这也不会影响使用SomeWorker.perform_in()调用的作业。

我最终在我们的sidekiq.rb初始化文件中为测试环境禁用了防抖中间件:

unless Rails.env.test?
  Sidekiq.configure_client do |config|
    config.client_middleware do |chain|
      chain.add Sidekiq::Debounce
    end
  end
end

if Rails.env.test?
  require 'sidekiq/testing'
  Sidekiq::Testing.fake!
end

再次运行管道,不再看到Redis连接问题。

英文:

So I think I figured out what was wrong or what to change.
We were using Sidekiq along with Sidekiq-debounce
I think this was actually intercepting the basic Sidekiq async calls so even though I set them to Sidekiq::Testing.fake! or Sidekiq::Testing.inline!, this did not affect jobs being called using SomeWorker.perform_in()

I ended disabling the debounce middleware for the test environment in our sidekiq.rb initializer:

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-html -->

unless Rails.env.test?
  Sidekiq.configure_client do |config|
    config.client_middleware do |chain|
      chain.add Sidekiq::Debounce
    end
  end
end

if Rails.env.test?
  require &#39;sidekiq/testing&#39;
  Sidekiq::Testing.fake!
end

<!-- end snippet -->

Ran the pipeline again and no longer saw the Redis connection problems.

huangapple
  • 本文由 发表于 2023年6月5日 16:25:03
  • 转载请务必保留本文链接:https://go.coder-hub.com/76404650.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定