如何理解频繁垃圾回收的原因?

huangapple go评论79阅读模式
英文:

How do I understand the reason for frequent garbage collection?

问题

jstat -gc 27539
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
901632.0 468480.0 0.0 0.0 911360.0 911360.0 5464064.0 5463748.3 21632.0 20948.0 2944.0 2777.7 153 33.727 401 782.598 816.325

jstat -gccapacity 27539
NGCMN NGCMX NGC S0C S1C EC OGCMN OGCMX OGC OC MCMN MCMX MC CCSMN CCSMX CCSC YGC FGC
171008.0 2732032.0 2714624.0 901632.0 468480.0 911360.0 343040.0 5464064.0 5464064.0 5464064.0 0.0 1069056.0 21632.0 0.0 1048576.0 2944.0 153 404

I added EU and OU to find the total heap used. That looks 6GB is used. I referred this

But there are 400+ FGC happened. It has reached 700+ now. After sometime, it just performs GC. It is 850+ now.

My job:

It is multi threading. 100 reader, 100 writer threads. Each one has it's own connection to the database. Each reader thread reads 100000 records and stores in a LinkedList and sends to writer thread. Writer writes the data to another collection in the same database. LinkedList is not reused means each 1L creates a new LinkedList.

It is akka based multi-threading. So I don't handle thread failure, thread spawning i.e thread management.

But my doubt is that why such huge FGC happening when I have 32gb ram? any pointers to look further? It ran into GC Overhead LIMIt exceeded error sometimes. I didn't set any explicit min, max memory for the job.

EDIT:

As per my analysis, it has fixed some EU and OU. It is full, hence it is keep on performing GC. Is it possible and am I correct?

Edit2

Thanks @emotionlessbanans, @Cascader. I have the below.

uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 526385152 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 8392802304 {product}
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)

Any specific reason to stop at only 6GB when I have 8gb ? Or am I missing something?

英文:
jstat -gc 27539
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
901632.0 468480.0  0.0    0.0   911360.0 911360.0 5464064.0  5463748.3  21632.0 20948.0 2944.0 2777.7    153   33.727  401   782.598  816.325

jstat -gccapacity 27539
 NGCMN    NGCMX     NGC     S0C   S1C       EC      OGCMN      OGCMX       OGC         OC       MCMN     MCMX      MC     CCSMN    CCSMX     CCSC    YGC    FGC 
171008.0 2732032.0 2714624.0 901632.0 468480.0 911360.0   343040.0  5464064.0  5464064.0  5464064.0      0.0 1069056.0  21632.0      0.0 1048576.0   2944.0    153   404

I added EU and OU to find the total heap used. That looks 6GB is used. I referred this

But there are 400+ FGC happened. It has reached 700+ now. After sometime, it just performs GC. It is 850+ now.

My job:

It is multi threading. 100 reader, 100 writer threads. Each one has it's own connection to the database. Each reader thread reads 100000 records and stores in a LinkedList and sends to writer thread. Writer writes the data to another collection in the same database. LinkedList is not reused means each 1L creates a new LinkedList.

It is akka based multi-threading. So I don't handle thread failure, thread spawning i.e thread management.

But my doubt is that why such huge FGC happening when I have 32gb ram? any pointers to look further?

It ran into GC Overhead LIMIt exceeded error sometimes.

I didn't set any explicit min, max memory for the job.

EDIT:

As per my analysis, it has fixed some EU and OU. It is full, hence it is keep on performing GC. Is it possible and am I correct?

Edit2

Thanks @emotionlessbanans, @Cascader. I have the below.

    uintx ErgoHeapSizeLimit                         = 0                                   {product}
    uintx HeapSizePerGCThread                       = 87241520                            {product}
    uintx InitialHeapSize                          := 526385152                           {product}
    uintx LargePageHeapSizeThreshold                = 134217728                           {product}
    uintx MaxHeapSize                              := 8392802304                          {product}
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)

Any specific reason to stop at only 6GB when I have 8gb ? Or am I missing something?

答案1

得分: 3

当没有使用“-Xmx”开关指定最大堆大小时,JVM会选择默认值。这个值是JVM供应商特定的,通常取决于体系结构(32/64位)和总可用内存。

似乎您的应用程序使用了分配的内存(由您的JVM确定),从某一点开始,它在GC上花费了太多时间,直到出现“GC超过限制”错误。

您应该明确设置最大堆大小(例如“-Xmx 10g”),以确保充分利用所有可用内存。

英文:

When no max heap size is specified using -Xmx switch, the JVM chooses a default value. This value is JVM vendor specific and it usually depends on architecture (32/64bit) and total available memory.

It seems like your application uses all allocated memory (the one determined by your JVM) and from some point on it spends too much time on GC until you get "GC Overhead Limit Exceeded" error.

You should explicitly set a maximum heap size (i.e. -Xmx 10g) in order to be sure you utilize all of your available memory.

答案2

得分: 0

默认最小内存未指定,最大内存为256Mb。可能是堆分配的内存不足的问题。(这是我的怀疑)

另外,您可以使用VisualVM或任何其他工具,可能会显示更多关于发生的情况的信息。

英文:

Default min memory is not specified and max is 256Mb. It may be problem with not sufficient memory assigned to heap. (It is my suspicion)

Also you could use VisualVM or any other tool that might show more info about what happens there.

huangapple
  • 本文由 发表于 2020年8月6日 15:10:40
  • 转载请务必保留本文链接:https://go.coder-hub.com/63278516.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定