spring-rabbitmq发现内存泄漏,但我不知道原因,有人可以帮助我吗?

huangapple go评论81阅读模式
英文:

spring-rabbitmq find a memory leak, but i don't know the reason, can someone help me

问题

我的应用程序使用了 RabbitMQ,版本如下:

> spring-amqp:2.2.17.RELEASE<br/>
> spring-rabbit:2.2.17.RELEASE<br/>
> spring-boot:2.3.11.RELEASE<br/>

@Primary
@Bean(name = "firstConnectionFactory")
public ConnectionFactory firstConnectionFactory() {
    CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
    connectionFactory.setHost(host);
    connectionFactory.setPort(port);
    connectionFactory.setUsername(username);
    connectionFactory.setPassword(password);
    connectionFactory.setPublisherConfirms(true);
    return connectionFactory;
}

@Primary
@Bean(name = "firstRabbitTemplate")
@Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public RabbitTemplate firstRabbitTemplate() {
    return new RabbitTemplate(firstConnectionFactory());
}

// 发送消息
private void prepareForString(String exchange, String queueName, JSONObject message) {
    try {
        RabbitTemplate template = rabbitConfig.firstRabbitTemplate();
        template.convertAndSend(exchange, queueName, JSON.toJSONStringWithDateFormat(message, DateUtils.YYYYMMDDHHMMSS, SerializerFeature.WriteMapNullValue));
    } catch (Exception ex) {
        log.error(ex.getMessage());
    }
}

应用程序运行了大约一年后,我发现应用程序运行非常缓慢,导出堆,然后我看到

[堆内存显示][2]
> 由 "org.springframework.amqp.rabbit.connection.PublisherCallbackChannelImpl" 加载的一个实例,由 "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c001a6d0" 占用了 405.62 MB(12.57%)字节。内存积累在一个实例中,由 "org.springframework.amqp.rabbit.connection.PublisherCallbackChannelImpl" 加载,由 "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c001a6d0" 加载。

关键词
org.springframework.amqp.rabbit.connection.PublisherCallbackChannelImpl
org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c001a6d0

看起来 PublisherCallbackChannelImpl.java 有一个监听器,而且监听器非常庞大。监听器定义如下:

// PublisherCallbackChannelImpl.java 第 59 行
private final ConcurrentMap<String, PublisherCallbackChannel.Listener> listeners = new ConcurrentHashMap();


以下是将元素放入映射的代码:

// PublisherCallbackChannelImpl.java 第 549 行
public void addListener(PublisherCallbackChannel.Listener listener) {
Assert.notNull(listener, "Listener cannot be null");
if (this.listeners.size() == 0) {
this.delegate.addConfirmListener(this);
this.delegate.addReturnListener(this);
}
// 关键代码,下面的 'listener' 是 RabbitTemplate 对象
if (this.listeners.putIfAbsent(listener.getUUID(), listener) == null) {
this.pendingConfirms.put(listener, new ConcurrentSkipListMap());
if (this.logger.isDebugEnabled()) {
this.logger.debug("Added listener " + listener);
}
}
}


我想知道为什么会发生这种情况,以及 listeners.remove(XXX) 在哪里,或者它是否根本没有 listeners.remove()?

我尝试查找 PublisherCallbackChannelImpl.java,但在任何地方都找不到 listeners.remove()。

所以我想知道,这是否是内存泄漏的原因?
英文:

my application use rabbit mq, the versions are below

> spring-amqp : 2.2.17.RELEASE<br/>
> spring-rabbit : 2.2.17.RELEASE<br/>
> spring-boot: 2.3.11.RELEASE<br/>

    @Primary
    @Bean(name = &quot;firstConnectionFactory&quot;)
    public ConnectionFactory firstConnectionFactory() {
        CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
        connectionFactory.setHost(host);
        connectionFactory.setPort(port);
        connectionFactory.setUsername(username);
        connectionFactory.setPassword(password);
        connectionFactory.setPublisherConfirms(true); 
        return connectionFactory;
    }

    @Primary
    @Bean(name = &quot;firstRabbitTemplate&quot;)
    @Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
    public RabbitTemplate firstRabbitTemplate() {
        return new RabbitTemplate(firstConnectionFactory());
    }

    //send a message
    private void prepareForString(String exchange, String queueName, JSONObject message) {
        try {
            RabbitTemplate template = rabbitConfig.firstRabbitTemplate();
            template.convertAndSend(exchange, queueName, JSON.toJSONStringWithDateFormat(message,   DateUtils.YYYYMMDDHHMMSS, SerializerFeature.WriteMapNullValue));
        } catch (Exception ex) {
            log.error(ex.getMessage());
        }
    }

after the application run about 1 year, i find the application run very slow, dump the heap, then i see

heap show
> One instance of "org.springframework.amqp.rabbit.connection.PublisherCallbackChannelImpl" loaded by "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c001a6d0" occupies 405.62 MB (12.57%) bytes. The memory is accumulated in one instance of "org.springframework.amqp.rabbit.connection.PublisherCallbackChannelImpl" loaded by "org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c001a6d0".
Keywords
org.springframework.amqp.rabbit.connection.PublisherCallbackChannelImpl
org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c001a6d0

it looks like the PublisherCallbackChannelImpl.java has a listeners, and the listeners is very huge. the listeners is defined below

//PublisherCallbackChannelImpl.java line 59
private final ConcurrentMap&lt;String, PublisherCallbackChannel.Listener&gt; listeners = new ConcurrentHashMap(); 

the below is the code of put elements in map

    //PublisherCallbackChannelImpl.java line 549
    public void addListener(PublisherCallbackChannel.Listener listener) {
        Assert.notNull(listener, &quot;Listener cannot be null&quot;);
        if (this.listeners.size() == 0) {
            this.delegate.addConfirmListener(this);
            this.delegate.addReturnListener(this);
        }
        //the key code, below &#39;listener&#39; is RabbitTemplate object
        if (this.listeners.putIfAbsent(listener.getUUID(), listener) == null) {
            this.pendingConfirms.put(listener, new ConcurrentSkipListMap());
            if (this.logger.isDebugEnabled()) {
                this.logger.debug(&quot;Added listener &quot; + listener);
            }
        }

    }

i want know why this happens, and where is the listeners.remove(XXX), or it has no listeners.remove()?

i try look for the PublisherCallbackChannelImpl.java , but there is no listeners.remove() anywhere

so i what know, is this the reason of memory leak?

答案1

得分: 1

泄漏的原因在这里,SCOPE_PROTOTYPE和RabbitTemplate template = rabbitConfig.firstRabbitTemplate(); 太多的rabbitTemplate对象,并且这些rabbit template存储在map中,而且没有被jvm垃圾回收,因为通道没有关闭,map与通道相关联。所以map不会被jvm垃圾回收。我认为spring-rabbit的设计者没有考虑在该map中有太多的值。设计者可能只考虑到一个或几个rabbitTemplate对象,所以map不会太大。但是当rabbitTemplate使用SCOPE_PROTOTYPE或每次新建时,map中会生成太多的rabbitTemplate对象,导致内存泄漏。我已经更改为以下代码。

//移除@Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
//默认是单例
@Primary
@Bean(name = &quot;firstRabbitTemplate&quot;)
public RabbitTemplate firstRabbitTemplate() {
    return new RabbitTemplate(firstConnectionFactory());
}

//我使用spring的自动装配来导入RabbitTemplate对象
@Autowired
RabbitTemplate firstRabbitTemplate;

/**
 * 发送消息;
 */
private void prepareForString(String exchange, String queueName, JSONObject message) {
    try {
       //只使用一个firstRabbitTemplate对象
       firstRabbitTemplate.convertAndSend(exchange, queueName, JSON.toJSONStringWithDateFormat(message,
                    DateUtils.YYYYMMDDHHMMSS, SerializerFeature.WriteMapNullValue));
    } catch (Exception ex) {
        log.error(ex.getMessage());
    }

}
英文:

the reason of leak is here, SCOPE_PROTOTYPE and RabbitTemplate template = rabbitConfig.firstRabbitTemplate(); too many rabbitTemplate objects, and these rabbit template is in the map, and not gc by jvm, because the channel is not close, the map is related to the channel. so the map not gc by jvm. i think the designer of spring-rabbit not consider too many values in that map. the designer might think only one or serval rabbitTemplate objects, so the map will not too big. but when rabbitTemplate use SCOPE_PROTOTYPE or new every time, that will generate too many rabbitTemplate in the map, that cause the memory leak.
I had changed to the below code.

//remove the @Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
//default singleton
@Primary
@Bean(name = &quot;firstRabbitTemplate&quot;)
public RabbitTemplate firstRabbitTemplate() {
    return new RabbitTemplate(firstConnectionFactory());
}

//I use spring autowired to import RabbitTemplate Object 
@Autowired
RabbitTemplate firstRabbitTemplate;

/**
 * send message;
 */
private void prepareForString(String exchange, String queueName, JSONObject message) {
    try {
       //use just one firstRabbitTemplate obejct
       firstRabbitTemplate.convertAndSend(exchange, queueName, JSON.toJSONStringWithDateFormat(message,
                    DateUtils.YYYYMMDDHHMMSS, SerializerFeature.WriteMapNullValue));
    } catch (Exception ex) {
        log.error(ex.getMessage());
    }

}

huangapple
  • 本文由 发表于 2023年6月16日 16:06:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/76488166.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定