英文:
How to prevent Rails parallel tests and Sidekiq from calling Redis
问题
我有一个使用Sidekiq进行异步作业的Rails应用程序。
我设置了并行测试来加速我的测试流程。 这个方法效果很好,但我有一个主要问题:Redis在试图将Sidekiq作业排队时成为瓶颈。
我一直看到这些错误:
Redis::CannotConnectError:
无法连接到127.0.0.1:6379上的Redis (Errno::ECONNREFUSED)
我正在运行一些旧版本的gems,还使用了Sidekiq debounce(不确定是否是原因)
sidekiq(5.2.1)
sidekiq-debounce(1.1.0)
rspec-sidekiq(3.0.3)
我原本以为rspec-sidekiq能够通过使用Sidekiq::Testing.inline!
或Sidekiq::Testing.fake!
来将作业内联运行或将它们推送到数组中来模拟对Redis的调用。
然而,这两个选项都不起作用。 我漏掉了什么?
我尝试设置测试环境以伪造或内联作业,如下所示:
require "sidekiq/testing"
Sidekiq::Testing.inline!
或
require "sidekiq/testing"
Sidekiq::Testing.fake!
然后,我甚至在一些测试中检查以确保这是被设置的:
puts "SIDEKIQ MODE ENABLED: #{Sidekiq::Testing.enabled?}"
puts "SIDEKIQ MODE FAKE: #{Sidekiq::Testing.fake?}"
puts "SIDEKIQ MODE INLINE: #{Sidekiq::Testing.inline?}"
puts "SIDEKIQ MODE DISABLED: #{Sidekiq::Testing.disabled?}"
并且返回我期望的结果,但我仍然得到了Redis连接被拒绝的错误。
我还尝试将各个部分包装在块中,如下所示:
Sidekiq::Testing.fake! do
# 一些代码
end
英文:
I have a Rails application that is using Sidekiq for asynchronous jobs.
I setup parallel testing to speed up my testing pipeline. This is working great but I have one major problem: Redis is bottlenecking with Sidekiq jobs trying to enqueue work there.
I keep seeing these errors:
Redis::CannotConnectError:
Error connecting to Redis on 127.0.0.1:6379 (Errno::ECONNREFUSED)
I am running some old versions of gems and also using Sidekiq debounce (Not sure if this is the reason)
sidekiq (5.2.1)
sidekiq-debounce (1.1.0)
rspec-sidekiq (3.0.3)
I was under the impression that rspec-sidekiq would be able to either stub the calls to Redis by using either: Sidekiq::Testing.inline!
or Sidekiq::Testing.fake!
to run the jobs inline or push them to an array respectively.
However neither of these options are working. What am I missing?
I tried setting up the test environment to fake or inline the jobs like:
require "sidekiq/testing"
Sidekiq::Testing.inline!
OR
require "sidekiq/testing"
Sidekiq::Testing.fake!
Then I even checked in some of the tests to make sure this was being set by:
puts "SIDEKIQ MODE ENABLED: #{Sidekiq::Testing.enabled?}"
puts "SIDEKIQ MODE FAKE: #{Sidekiq::Testing.fake?}"
puts "SIDEKIQ MODE INLINE: #{Sidekiq::Testing.inline?}"
puts "SIDEKIQ MODE DISABLED: #{Sidekiq::Testing.disabled?}"
and it returned what I expected but I still got the Redis connections refused.
I also tried wrapping the individual parts in blocks like:
Sidekiq::Testing.fake! do
# some code
end
答案1
得分: 1
我认为我已经找到了问题所在或需要进行的更改。
我们使用了Sidekiq以及Sidekiq-debounce。
我认为实际上它截获了基本的Sidekiq异步调用,因此即使我将它们设置为Sidekiq::Testing.fake!
或Sidekiq::Testing.inline!
,这也不会影响使用SomeWorker.perform_in()
调用的作业。
我最终在我们的sidekiq.rb
初始化文件中为测试环境禁用了防抖中间件:
unless Rails.env.test?
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add Sidekiq::Debounce
end
end
end
if Rails.env.test?
require 'sidekiq/testing'
Sidekiq::Testing.fake!
end
再次运行管道,不再看到Redis连接问题。
英文:
So I think I figured out what was wrong or what to change.
We were using Sidekiq along with Sidekiq-debounce
I think this was actually intercepting the basic Sidekiq async calls so even though I set them to Sidekiq::Testing.fake!
or Sidekiq::Testing.inline!
, this did not affect jobs being called using SomeWorker.perform_in()
I ended disabling the debounce middleware for the test environment in our sidekiq.rb
initializer:
<!-- begin snippet: js hide: false console: true babel: false -->
<!-- language: lang-html -->
unless Rails.env.test?
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add Sidekiq::Debounce
end
end
end
if Rails.env.test?
require 'sidekiq/testing'
Sidekiq::Testing.fake!
end
<!-- end snippet -->
Ran the pipeline again and no longer saw the Redis connection problems.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论