Java内存泄漏与数据源

huangapple go评论100阅读模式
英文:

Java memory leak with a data source

问题

我听说很多次不关闭数据库连接可能会导致内存泄漏。(例如在这篇文章中)

我尝试通过从org.apache.commons.dbcp2.BasicDataSource获取连接并不关闭它来复现相同的问题。

这是我的代码:

//Repo类
private final BasicDataSource ds;
public Repo() {
    ds = new BasicDataSource();

    ds.setDriverClassName("org.postgresql.Driver");
    ds.setUrl("jdbc:postgresql://localhost/postgres");
    ds.setUsername("postgres");
    ds.setPassword("postgres");
    ds.setMaxOpenPreparedStatements(10000);
    ds.setMaxTotal(10000);
}

public PreparedStatement prepStatement(String sql) throws SQLException {
    return this.ds.getConnection().prepareStatement(sql);
}

//Logic类
public JsonNode logic(String name) {
    PreparedStatement ps = this.repo.prepStatement("select data from public.users where name = ? ");
    ps.setString(1, name);
    //其余逻辑 - 没有关闭方法或使用try-with-resources
}

我已经重复了近400次相同的过程,还降低了初始和最大堆大小。然而,没有出现内存泄漏的迹象。即使在VisualVM监控中,堆图看起来非常正常:

Java内存泄漏与数据源

有关如何复现此问题的任何想法吗?

英文:

I have heard many times that not closing database connections might lead to a memory leak.
(For example in this article)

I tried reproducing the same issue by getting a connection from org.apache.commons.dbcp2.BasicDataSource
and not closing it.

This is my code:

//Repo class
private final BasicDataSource ds;
public Repo() {
    ds = new BasicDataSource();

    ds.setDriverClassName("org.postgresql.Driver");
    ds.setUrl("jdbc:postgresql://localhost/postgres");
    ds.setUsername("postgres");
    ds.setPassword("postgres");
    ds.setMaxOpenPreparedStatements(10000);
    ds.setMaxTotal(10000);


}
public PreparedStatement prepStatement(String sql) throws SQLException {
    return this.ds.getConnection().prepareStatement(sql);
}

//Logic class
public JsonNode logic(String name) {
    PreparedStatement ps = this.repo.prepStatement("select data from public.users where name = ? ");
    ps.setString(1, name);
    //The rest of the logic - no close method or try with resource
}

I have repeated the same process almost 400 times and also decreased the initial and max heap sizes.
Still, there was no sign of a memory leak. Even in the VisualVM monitoring, the heap graph seems pretty normal:

Java内存泄漏与数据源

Any ideas on how to reproduce this issue?

答案1

得分: 3

资源未关闭的问题不在于潜在的内存泄漏,而在于潜在的资源泄漏。我们谈论的是文件句柄、网络连接,甚至是在数据库服务器端潜在分配的资源,在你的 JVM 中根本不可见。

实际的 PreparedStatement 实现是否具有像终结器或清理器这样的保护取决于特定的数据库驱动程序,它将在对象变为垃圾回收时关闭资源。但是即使有这样的机制,这意味着资源会一直保持,直到随后的垃圾回收周期识别出不可达对象并触发终结操作。

在你的特定设置中,似乎大约每一分钟会进行一次垃圾回收。也许,在这些时间点上,关键的非内存资源会被清理掉,你甚至没有检查过它们。

但是即使对于这个设置,如果这些资源被清理掉了,你必须注意以下几点:

  • 并非每个数据库驱动程序都可能以这种方式工作。
  • 在真实的生产环境中,持有关键的非内存资源(如锁、文件句柄、数据库连接等)比必要时间长一分钟,已经可能成为一个巨大的问题。
  • 不能保证每一分钟都会进行垃圾回收。系统可能连续运行数小时甚至数天而没有垃圾回收。
  • 不能保证垃圾回收会识别出特定的不可达对象。这在简单的设置中可能会顺利进行,其中对象属于年轻代,当下次收集发生时,但是现代并发收集器愿意在短时间内回收大量内存,而不急于回收每个对象。
  • 就内存而言,每个对象的内存都是相同的,因此忽视这种“投入产出最大”的收集中忽略的对象是无关紧要的。因此,PreparedStatement 实例可能是每次收集时都被忽视的不幸的不可达对象之一。对于少数几个字节来说,无所谓,它所阻塞的内容,这就是为什么允许采取这种策略的原因。正如前面所说,问题在于它可能无限期地持有的非内存资源。
英文:

The problem with not closing resources is not the potential memory leak, but the potential resource leak. We are talking about file handles, network connections, but even resources potentially allocated on the database server side, not visible within your JVM at all.

It depends on the particular database driver whether the actual PreparedStatement implementation has a safeguard like a finalizer or cleaner that will close the resources when the object becomes garbage collected. But even if it has, this would imply that the resources are held until a subsequent garbage collection cycle identifies the unreachable object and triggers the finalization.

In your particular setup, there seems to be a garbage collection about every minute. Perhaps, the critical non-memory resources got cleaned up at those points; you didn’t even check for them.

But even if those resources got cleaned up for this setup, you have to be aware of the following points:

  • Not every database driver might work this way

  • In real production environments, holding critical non-memory resources, like locks, file handles, database connections, etc. for one minute longer than necessary, can already be a huge problem.

  • There is no guaranty that you have a garbage collection every minute. A system may run for hours or even days without a garbage collection.

  • There is no guaranty that garbage collection identifies a particular unreachable object. This may work smoothly in a simple setup where the object belongs to the Young Generation when the next collection happens, but modern concurrent collectors are happy with reclaiming a large amount of memory in a short, configurable time limit, without being eager to collect every object.

    As far as memory is concerned, every object’s memory is equal, hence, it is irrelevant, which objects are overlooked by such “biggest bang for the bucks” collection. So the PreparedStatement instance might be one of the unlucky unreachable objects that are overlooked on every collection. It doesn’t matter for the few bytes, it blocks, that’s why this strategy is permitted. As said, the problems are the non-memory resources that it may hold for an indefinite time.

huangapple
  • 本文由 发表于 2020年10月6日 16:13:43
  • 转载请务必保留本文链接:https://go.coder-hub.com/64221853.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定