如何在Spring Boot中防止外部连接失败（如Redis服务器）时出现快速失败？

huangapple
go评论85阅读模式
英文:

How to prevent fail fast on external connection failure like Redis server in Spring boot?

问题

有没有办法防止spring-boot应用程序在启动时因外部连接失败而失败？我已经找到了其他类似的问题，建议使用@Lazy注解来防止@Configurationbean的初始化，但是这个解决方案对我使用Spring Data Redis和Jedis客户端不起作用。

另外，其他解决方案，比如这个，是特定于应用程序中使用的依赖项。例如，Spring Cloud有以下属性来控制故障快速行为 -

spring.cloud.config.fail-fast=true

您可以使用我为我的问题创建的这个项目来重现通过关闭redis服务器来重现。

以下是我的代码示例 -

@Lazy
@Configuration
public class RedisConfiguration {
    @Value("${spring.redis.sentinel.master}")
    private String SENTINEL_MASTER;

    @Value("${spring.redis.sentinel.nodes}")
    private String SENTINEL_NODES;

    @Value("${spring.redis.security.enabled:false}")
    private boolean REDIS_SECURITY_ENABLED;

    @Value("${spring.redis.security.password:}")
    private String REDIS_PASSWORD;

    @Lazy
    @Bean // somehow this always gets initialized
    public RedisConnectionFactory jedisConnectionFactory() { 
        // create set of sentinel nodes
        System.out.println(SENTINEL_NODES);
        Set<String> sentinelNodesSet = new HashSet<>(5);
        StringTokenizer st = new StringTokenizer(SENTINEL_NODES, ",");
        while (st.hasMoreTokens())
            sentinelNodesSet.add(st.nextToken());
        RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration(SENTINEL_MASTER, sentinelNodesSet);
        if (REDIS_SECURITY_ENABLED) {
            sentinelConfig.setPassword(REDIS_PASSWORD);
        }
        JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(sentinelConfig);
        return jedisConnectionFactory;
    }
}

以下是异常跟踪 -

org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'stringRedisTemplate' defined in class
path resource
[org/springframework/boot/autoconfigure/data/redis/RedisAutoConfiguration.class]:
Unsatisfied dependency expressed through method 'stringRedisTemplate'
parameter 0; nested exception is
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'jedisConnectionFactory' defined in class path
resource [com/springboot/redisintegration/RedisConfiguration.class]:
Invocation of init method failed; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: All sentinels
down, cannot determine where is mysentinel master is running...

简而言之：

  1. @Lazy注解对RedisStandaloneConfiguration有效，但对RedisSentinelConfiguration无效，原因不明？
  2. 使用@Lazy注解是有风险的，因为您需要确保使用Redis的所有服务也是懒加载的。
  3. 寻找类似spring.cloud.config.fail-fast=true为Spring Cloud提供的解决方案。

更新：

我已经为此功能创建了以下Jira问题 -

https://jira.spring.io/browse/DATAREDIS-1208

英文:

Is there a way to prevent spring-boot application from failing on startup due to external connection failures? I have found other similar questions that suggest using @Lazy annotation to prevent @Configuration beans initialisation but this solution did not work for me with Spring Data Redis using Jedis client.

Also, other solutions like this one are specific to dependencies being used in the app. For example, Spring Cloud has below property to control failfast behaviour -

spring.cloud.config.fail-fast=true

You can use this project that I created for my problem to reproduce by shutting down the redis server.

Below is how my code looks like -

@Lazy
@Configuration
public class RedisConfiguration {
	@Value(&quot;${spring.redis.sentinel.master}&quot;)
	private String SENTINEL_MASTER;

	@Value(&quot;${spring.redis.sentinel.nodes}&quot;)
	private String SENTINEL_NODES;

	@Value(&quot;${spring.redis.security.enabled:false}&quot;)
	private boolean REDIS_SECURITY_ENABLED;

	@Value(&quot;${spring.redis.security.password:}&quot;)
	private String REDIS_PASSWORD;

	@Lazy
	@Bean // somehow this always gets initialized
	public RedisConnectionFactory jedisConnectionFactory() { 
		// create set of sentinel nodes
		System.out.println(SENTINEL_NODES);
		Set&lt;String&gt; sentinelNodesSet = new HashSet&lt;&gt;(5);
		StringTokenizer st = new StringTokenizer(SENTINEL_NODES, &quot;,&quot;);
		while (st.hasMoreTokens())
			sentinelNodesSet.add(st.nextToken());
		RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration(SENTINEL_MASTER, sentinelNodesSet);
		if (REDIS_SECURITY_ENABLED) {
			sentinelConfig.setPassword(REDIS_PASSWORD);
		}
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(sentinelConfig);
		return jedisConnectionFactory;
	}

Below is the exception trace -

> org.springframework.beans.factory.UnsatisfiedDependencyException:
> Error creating bean with name 'stringRedisTemplate' defined in class
> path resource
> [org/springframework/boot/autoconfigure/data/redis/RedisAutoConfiguration.class]:
> Unsatisfied dependency expressed through method 'stringRedisTemplate'
> parameter 0; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'jedisConnectionFactory' defined in class path
> resource [com/springboot/redisintegration/RedisConfiguration.class]:
> Invocation of init method failed; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: All sentinels
> down, cannot determine where is mysentinel master is running... at
> org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:797)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:538)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1176)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879)
> ~[spring-context-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551)
> ~[spring-context-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
> ~[spring-boot-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> com.springboot.redisintegration.RedisIntegrationApplication.main(RedisIntegrationApplication.java:21)
> ~[classes/:na] Caused by:
> org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'jedisConnectionFactory' defined in class path
> resource [com/springboot/redisintegration/RedisConfiguration.class]:
> Invocation of init method failed; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: All sentinels
> down, cannot determine where is mysentinel master is running... at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1794)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1307)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1227)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:884)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] ... 20 common frames
> omitted Caused by:
> redis.clients.jedis.exceptions.JedisConnectionException: All sentinels
> down, cannot determine where is mysentinel master is running... at
> redis.clients.jedis.JedisSentinelPool.initSentinels(JedisSentinelPool.java:249)
> ~[jedis-3.3.0.jar:na] at
> redis.clients.jedis.JedisSentinelPool.<init>(JedisSentinelPool.java:154)
> ~[jedis-3.3.0.jar:na] at
> redis.clients.jedis.JedisSentinelPool.<init>(JedisSentinelPool.java:122)
> ~[jedis-3.3.0.jar:na] at
> redis.clients.jedis.JedisSentinelPool.<init>(JedisSentinelPool.java:116)
> ~[jedis-3.3.0.jar:na] at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createRedisSentinelPool(JedisConnectionFactory.java:374)
> ~[spring-data-redis-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createPool(JedisConnectionFactory.java:358)
> ~[spring-data-redis-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.afterPropertiesSet(JedisConnectionFactory.java:342)
> ~[spring-data-redis-2.3.3.RELEASE.jar:2.3.3.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1853)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1790)
> ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE] ... 31 common frames
> omitted

In short:

  1. @Lazy annotation works for RedisStandaloneConfiguration but not RedisSentinelConfiguration, not sure why?
  2. Using @Lazy annotation is risky because you need to make sure all your services which are using Redis are loaded lazily too.
  3. Looking for a solution like spring.cloud.config.fail-fast=true provided for spring cloud.

Update:

I've created below Jira issue for this feature -

https://jira.spring.io/browse/DATAREDIS-1208

答案1

得分: 1

Spring Cloud Config在底层使用了spring-retry和AOP（spring-boot-starter-aop）来配置重试机制。

这个过程是在ConfigServiceBootstrapConfiguration中实现的。

代码的相关部分如下：

/* @ConditionalOnProperty(&quot;spring.cloud.config.fail-fast&quot;) */
@ConditionalOnClass({ Retryable.class, Aspect.class, AopAutoConfiguration.class })
@Configuration(proxyBeanMethods = false)
@EnableRetry(proxyTargetClass = true)
@Import(AopAutoConfiguration.class)
@EnableConfigurationProperties(RetryProperties.class)
protected static class RetryConfiguration {

  @Bean
  @ConditionalOnMissingBean(name = &quot;configServerRetryInterceptor&quot;)
  public RetryOperationsInterceptor configServerRetryInterceptor(
      RetryProperties properties) {
    return RetryInterceptorBuilder.stateless()
        .backOffOptions(properties.getInitialInterval(),
            properties.getMultiplier(), properties.getMaxInterval())
        .maxAttempts(properties.getMaxAttempts()).build();
  }

}

正如你所看到的，基本思路是提供一个RetryConfiguration，在考虑应用程序失败之前处理一定数量的重试。

Spring Cloud客户端的文档提供了有关配置此机制的不同属性的更多信息。你还可以在RetryProperties类的源代码中查看默认值。

请尝试将两个必需的依赖项spring-retry和spring-boot-starter-aop包含在你的主配置的子配置中，为重试机制的配置属性提供一些合理的默认值，然后查看发生了什么。

你可以将解决方案推到极限，并尝试在非常多的情况下重新连接，也许增加它们之间的速度，等待服务器可用。

我认为@Lazy注解将不再需要，并且可以安全地删除。

编辑

查看你的错误堆栈跟踪时，你还可以尝试禁用Spring Boot Redis自动配置类。

你可以在你的注解中这样做：

@SpringBootApplication(
  exclude = { RedisAutoConfiguration.class, RedisRepositoriesAutoConfiguration.class }
)

或者在你的属性文件中：

spring.autoconfigure.exclude= \
org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration, \
org.springframework.boot.autoconfigure.data.redis.RedisRepositoriesAutoConfiguration

你还可以使用以下属性禁用Redis存储库配置：

spring.data.redis.repositories.enabled: false

一旦禁用了Redis自动配置，你就可以在适当的时候自由地实例化RedisTemplate或与Redis交互所需的其他组件。

例如，你可以根据需要按需初始化它，尝试建立与Redis的连接，并在实际需要时初始化所有必需的工厂。你可以用try和catch包围初始化Redis连接的逻辑，并只有在有可用连接时才初始化RedisTemplate，类似以下方式。

一方面：

public RedisConnectionFactory jedisConnectionFactory() { 
    try {
      // create set of sentinel nodes
      System.out.println(SENTINEL_NODES);
      Set<String> sentinelNodesSet = new HashSet<>(5);
      StringTokenizer st = new StringTokenizer(SENTINEL_NODES, ",");
      while (st.hasMoreTokens())
          sentinelNodesSet.add(st.nextToken());
      RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration(SENTINEL_MASTER, sentinelNodesSet);
      if (REDIS_SECURITY_ENABLED) {
          sentinelConfig.setPassword(REDIS_PASSWORD);
      }
      JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(sentinelConfig);
      return jedisConnectionFactory;
    } catch (redis.clients.jedis.exceptions.JedisConnectionException re) {
      logger.error("Unable to initialize Redis connection factory", re);
      return null;
    }
}

另一方面：

public RedisTemplate getRedisTemplate() {
  // 我们可以假设这两个方法都在同一个类中定义，尽管这不是必需的
  final RedisConnectionFactory redisConnectionFactory = this.jedisConnectionFactory();
  if (redisConnectionFactory == null) {
    return null;
  }

  final RedisTemplate redisTemplate = new StringRedisTemplate(redisConnectionFactory);
  return redisTemplate;
}

你可以根据需要以适当的方式使用这个RedisTemplate，当然，根据需要进行缓存和重用。

这些方法可以在专门为此任务创建的服务或助手类中定义，并且当然不要在你的配置类中定义。

英文:

Under the hood Spring Cloud Config uses spring-retry and AOP (spring-boot-starter-aop) to configure a retry mechanism.

This process is implemented in ConfigServiceBootstrapConfiguration.

The relevant part of the code is this:

/* @ConditionalOnProperty(&quot;spring.cloud.config.fail-fast&quot;) */
@ConditionalOnClass({ Retryable.class, Aspect.class, AopAutoConfiguration.class })
@Configuration(proxyBeanMethods = false)
@EnableRetry(proxyTargetClass = true)
@Import(AopAutoConfiguration.class)
@EnableConfigurationProperties(RetryProperties.class)
protected static class RetryConfiguration {

  @Bean
  @ConditionalOnMissingBean(name = &quot;configServerRetryInterceptor&quot;)
  public RetryOperationsInterceptor configServerRetryInterceptor(
      RetryProperties properties) {
    return RetryInterceptorBuilder.stateless()
        .backOffOptions(properties.getInitialInterval(),
            properties.getMultiplier(), properties.getMaxInterval())
        .maxAttempts(properties.getMaxAttempts()).build();
  }

}

As you can see, the basic idea is to provide a RetryConfiguration that handles a certain number of retries before considering the application failed.

The documentation of Spring Cloud Client provide more information about the different properties used for configuring this mechanism. You can see the default values also in the source code of the RetryProperties class.

Please, try and include the two required dependencies, spring-retry and spring-boot-starter-aop, and the RetryConfiguration as a child of your main configuration, provide some reasonable defaults for the configuration properties of the retry mechanism, and see what happens.

You can push the solution to the limit and try to reconnect on a very large number of occasions, perhaps increasing the cadence between them, waiting for the server to be available.

I think that the @Lazy annotations will no be longer necessary and could be safely removed.

EDIT

Reviewing your error stack trace, one think you can also try is to disable String Boot Redis auto-configuration classes.

You can do it in your annotations:

@SpringBootApplication(
  exclude = { RedisAutoConfiguration.class, RedisRepositoriesAutoConfiguration.class }
)

Or in your properties files:

spring.autoconfigure.exclude= \
org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration, \
org.springframework.boot.autoconfigure.data.redis.RedisRepositoriesAutoConfiguration

You can disable Redis repositories configuration with this property also:

spring.data.redis.repositories.enabled: false

Once you disable the Redis auto-configuration, you will be free to instantiate the RedisTemplate or the stuff you need to interact with Redis when you consider appropriate.

You can, for instance, initialize it on demand, trying to establish a connection to Redis, by initializing all the required factories, when it is actually required. You can surround with try and catch the logic required to initialize your Redis connection and only initialize the RedisTemplate if a connection is available, something like the following.

On one hand:

public RedisConnectionFactory jedisConnectionFactory() { 
    try {
      // create set of sentinel nodes
      System.out.println(SENTINEL_NODES);
      Set&lt;String&gt; sentinelNodesSet = new HashSet&lt;&gt;(5);
      StringTokenizer st = new StringTokenizer(SENTINEL_NODES, &quot;,&quot;);
      while (st.hasMoreTokens())
          sentinelNodesSet.add(st.nextToken());
      RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration(SENTINEL_MASTER, sentinelNodesSet);
      if (REDIS_SECURITY_ENABLED) {
          sentinelConfig.setPassword(REDIS_PASSWORD);
      }
      JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(sentinelConfig);
      return jedisConnectionFactory;
    } catch (redis.clients.jedis.exceptions.JedisConnectionException re) {
      logger.error(&quot;Unable to initialize Redis connection factory&quot;, re);
      return null;
    }
}

On the other:

public RedisTemplate getRedisTemplate() {
  // We can assume that both methods are defined in the same class, 
  // although it is not necessary
  final RedisConnectionFactory redisConnectionFactory = this.jedisConnectionFactory();
  if (redisConnectionFactory == null) {
    return null;
  }

  final RedisTemplate redisTemplate = new StringRedisTemplate(redisConnectionFactory);
  return redisTemplate;
}

You can use this RedisTemplate in the way you consider appropriate and, of course, cache and reuse it as necessary.

These methods can be defined in a service or helper class created for this task and, of course, not in your configuration classes.

huangapple
  • 本文由 发表于 2020年8月11日 19:38:10
  • 转载请务必保留本文链接：https://go.coder-hub.com/63357343.html
  • java
  • spring
  • spring-boot
  • spring-data-redis
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定