“keycloak spring boot starter 内存泄漏”

huangapple go评论81阅读模式
英文:

keycloak spring boot starter memory leak

问题

我的后端微服务在生产环境中出现了OutOfMemory异常。

更新:看起来问题来自嵌入式的Tomcat,我找到了这个配置:

server:
  port: 38083
  compression:
    enabled: true
    mime-types: text/html, text/xml, text/plain, text/css, application/javascript, application/json
    min-response-size: 1024

spring:
  session:
    store-type: none
    timeout: 1

我仍然不确定,但问题似乎来自org.keycloak.representations.AccessToken.roles哈希映射。我将微服务简化为尽可能简单,只有这个控制器:

@RestController
@RequestMapping("/nurse")
public class NurseController {

  List<NurseDto> cache = null;

  public NurseController() {
    cache = new ArrayList<>();
    int i = 5000;
    while (i-- > 0) {
      NurseDto n = new NurseDto();
      n.setId("123");
      n.setInternalId("234234");
      n.setName("asdfasfasoinasdf");
      n.setType("NURSE");
      cache.add(n);
    }
  }

  @GetMapping
  public ResponseEntity<List<NurseDto>> findAllNurses() {
    return ResponseEntity.ok(cache);
  }
}

我从20个线程中调用此控制器,没有任何暂停。几分钟后,应用程序因内存泄漏异常而失败。与此同时,我从堆转储中获得了以下信息:

堆转储1:dump1

其中对象的数量为:dump2

其中99%的哈希映射节点指向org.keycloak.representations.AccessToken.roles,而99%的字符串指向访问令牌的权限。之前,应用程序使用以下依赖项正常运行:

<dependency>
    <groupId>org.keycloak</groupId>
    <artifactId>keycloak-spring-boot-starter</artifactId>
    <version>6.0.1</version>
</dependency>

但在将其更新为11.0.2后问题仍然存在:

<dependency>
    <groupId>org.keycloak</groupId>
    <artifactId>keycloak-spring-boot-starter</artifactId>
    <version>11.0.2</version>
</dependency>

有什么想法是为什么会发生这种情况吗?这是预期的行为吗?我是否应该设置类似“缓存”限制?

这是我的安全配置,也许有所帮助:

@KeycloakConfiguration
@EnableGlobalMethodSecurity(prePostEnabled = true, securedEnabled = true)
@Profile("!test")
public class WebSecurityConfig extends KeycloakWebSecurityConfigurerAdapter {

  @Autowired
  public void configure(AuthenticationManagerBuilder auth) {
    auth.authenticationProvider(keycloakAuthenticationProvider());
  }

  @Bean
  public KeycloakSpringBootConfigResolver KeycloakConfigResolver() {
    return new KeycloakSpringBootConfigResolver();
  }

  @Bean
  @Override
  protected SessionAuthenticationStrategy sessionAuthenticationStrategy() {
    return new NullAuthenticatedSessionStrategy();
  }

  @Override
  protected void configure(HttpSecurity http) throws Exception {
    super.configure(http);
    http.authorizeRequests().anyRequest().permitAll().and().cors().and().csrf().disable();
  }

  @Bean
  public CorsFilter corsFilter() {
    final UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
    final CorsConfiguration config = new CorsConfiguration();
    config.setAllowCredentials(true);
    config.setAllowedOrigins(Collections.singletonList("*"));
    config.setAllowedHeaders(Collections.singletonList("*"));
    config.setAllowedMethods(
        Arrays.stream(HttpMethod.values()).map(HttpMethod::name).collect(Collectors.toList()));
    source.registerCorsConfiguration("/**", config);
    return new CorsFilter(source);
  }
}
英文:

My backend microservice fall down with OutOfMemory exception in production environment.
////////////////////////////////

UPDATE: look like problem come from embedded tomcat, I found this
“keycloak spring boot starter 内存泄漏”
It store sessions for each requests. Currently Have no idea how to clean it. I have this config:

server:
port: 38083
compression:
enabled: true
mime-types: text/html, text/xml, text/plain, text/css, application/javascript, application/json
min-response-size: 1024
spring:
session:
store-type: none
timeout: 1

////////////////////////

I'm still not sure, but look like problem come from
org.keycloak.representations.AccessToken.roles hash map. Currently I reduce microservice to simpliest as it possible, I have only this controller:

@RestController
@RequestMapping(&quot;/nurse&quot;)
public class NurseController {
List&lt;NurseDto&gt; cache = null;
public NurseController() {
cache = new ArrayList&lt;&gt;();
int i = 5000;
while (i-- &gt; 0) {
NurseDto n = new NurseDto();
n.setId(&quot;123&quot;);
n.setInternalId(&quot;234234&quot;);
n.setName(&quot;asdfasfasoinasdf&quot;);
n.setType(&quot;NURSE&quot;);
cache.add(n);
}
}
@GetMapping
public ResponseEntity&lt;List&lt;NurseDto&gt;&gt; findAllNurses() {
return ResponseEntity.ok(cache);
}
}

and I call this controller from 20 threads without any pause. After couple minutes application fail with memory leak exception. In same time, I have this information from heap dump:
“keycloak spring boot starter 内存泄漏”
with this amount of objects:
“keycloak spring boot starter 内存泄漏”

Where 99% of hash map nodes refer to org.keycloak.representations.AccessToken.roles
and 99% of string refer to privileges from access token.
Previously, application work with

&lt;dependency&gt;
&lt;groupId&gt;org.keycloak&lt;/groupId&gt;
&lt;artifactId&gt;keycloak-spring-boot-starter&lt;/artifactId&gt;
&lt;version&gt;6.0.1&lt;/version&gt;
&lt;/dependency&gt;

but after I update it to 11.0.2 issue remain

&lt;dependency&gt;
&lt;groupId&gt;org.keycloak&lt;/groupId&gt;
&lt;artifactId&gt;keycloak-spring-boot-starter&lt;/artifactId&gt;
&lt;version&gt;11.0.2&lt;/version&gt;
&lt;/dependency&gt;

Any thoughts why it happen? Is it desired behavior, and I should set something like a "cache" limit?

Here is my security config, may be it can help:

@KeycloakConfiguration
@EnableGlobalMethodSecurity(prePostEnabled = true, securedEnabled = true)
@Profile(&quot;!test&quot;)
public class WebSecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
@Autowired
public void configure(AuthenticationManagerBuilder auth) {
auth.authenticationProvider(keycloakAuthenticationProvider());
}
@Bean
public KeycloakSpringBootConfigResolver KeycloakConfigResolver() {
return new KeycloakSpringBootConfigResolver();
}
@Bean
@Override
protected SessionAuthenticationStrategy sessionAuthenticationStrategy() {
return new NullAuthenticatedSessionStrategy();
}
@Override
protected void configure(HttpSecurity http) throws Exception {
super.configure(http);
http.authorizeRequests().anyRequest().permitAll().and().cors().and().csrf().disable();
}
@Bean
public CorsFilter corsFilter() {
final UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
final CorsConfiguration config = new CorsConfiguration();
config.setAllowCredentials(true);
config.setAllowedOrigins(Collections.singletonList(&quot;*&quot;));
config.setAllowedHeaders(Collections.singletonList(&quot;*&quot;));
config.setAllowedMethods(
Arrays.stream(HttpMethod.values()).map(HttpMethod::name).collect(Collectors.toList()));
source.registerCorsConfiguration(&quot;/**&quot;, config);
return new CorsFilter(source);
}
}

答案1

得分: 2

因为Tomcat会话的原因发生了这种情况。Tomcat为每个请求创建会话,并在一个小时内将它们存储起来,之后可能会清除内存。我找到了这个解决方案:

http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);

举个例子,Spring Security配置应该如下所示:

@KeycloakConfiguration
public class WebSecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
  @Override
  protected void configure(HttpSecurity http) throws Exception {
    super.configure(http);
    http.authorizeRequests().anyRequest().permitAll().and().cors().and().csrf().disable();
    http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);
  }
}

常识告诉我们这可能是不够的,因为会话是在Tomcat容器的级别上创建的,我正试图修改Servlet配置,而Servlet配置是在容器内部启动的,但它却完美地工作。我还尝试设置Spring属性,类似于这样:

servlet:
  session:
    cookie:
      http-only: true
      secure: true
      max-age: 1
    timeout: 1s

结果是类似的,但当然它仍然会创建1秒钟的会话。我现在记不清楚当我将超时设置为0时会发生什么,但我在以下设置上停止了我的探索:

http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);

没有任何其他的Spring配置。

英文:

It happen because of tomcat sessions. Tomcat create session for each request, and store them for an hour, after it memory could be cleaned. I found this solution:

http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);

For an example, spring security configuration should look like that:

@KeycloakConfiguration
public class WebSecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
@Override
protected void configure(HttpSecurity http) throws Exception {
super.configure(http);
http.authorizeRequests().anyRequest().permitAll().and().cors().and().csrf().disable();
http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);
}

Common sense tell that it should be not enough, because session created on the level of tomcat container, and I'm trying to play with servlet configuration, which is started inside of container, but it works perfectly. I also try to set spring properties, something like this:

  servlet:
session:
cookie:
http-only: true
secure: true
max-age: 1
timeout: 1s

Result was similar, but of course it still create sessions for 1 sec. Currently I can't remember how it behave when I set timeouts to 0, but I stop my discovery on

http.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS);

without any other spring configurations.

huangapple
  • 本文由 发表于 2020年10月23日 22:18:16
  • 转载请务必保留本文链接:https://go.coder-hub.com/64501743.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定