英文:
Java app is using <1% cpu after upgrade from Ubuntu 18.04 to 20.04
问题
我正在处理一个Java/MySQL应用程序,批处理的测试运行在Ubuntu 18.04上需要3-4秒。
我昨晚升级到了Ubuntu 20.04,保留了相同的mysql配置文件,现在相同的应用程序需要1分45-47秒!
6.27用户 0.41系统 1:45.53经过的时间 6%CPU (0平均文本+0平均数据 168724最大驻留)k
0输入+384输出 (0主要+42056次要)页面错误 0交换
我注意到通过lscpu,服务器现在平均以最低频率运行。我尝试将"acpi=ht"添加到/etc/default/grub中,并重新启动机器,以关闭电源管理,但处理器仍然以低频率运行,很可能频率提升也不起作用。
CPU(s): 16
在线CPU(s)列表: 0-15
每核线程数: 2
每个插槽核心数: 4
插槽数: 2
NUMA节点数: 2
厂商ID: GenuineIntel
CPU系列: 6
型号: 44
型号名称: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
步进: 2
频率提升: 启用
CPU MHz: 1599.592
CPU最大MHz: 2401.0000
CPU最小MHz: 1600.0000
BogoMIPS: 4799.75
虚拟化: VT-x
L1数据缓存: 256 KiB
L1指令缓存: 256 KiB
L2缓存: 2 MiB
L3缓存: 24 MiB
使用mysqltuner检查服务器的InnoDB指标,事情看起来还不错,所以这似乎与CPU设置/操作系统限制或Java配置/代码问题有关。
-------- InnoDB指标 ----------------------------------------------------------------------------
[--] 已启用InnoDB。
[--] InnoDB线程并发性: 8
[OK] 已激活每表一个InnoDB文件
[OK] InnoDB缓冲池/数据大小: 9.0G/2.6G
[OK] InnoDB日志文件大小/InnoDB缓冲池大小比例: 1.0G * 2/9.0G 应该等于 25%
[OK] InnoDB缓冲池实例数: 9
[--] InnoDB缓冲池块数: 72 用于 9 个缓冲池实例
[OK] Innodb_buffer_pool_size与Innodb_buffer_pool_chunk_size和Innodb_buffer_pool_instances对齐
[OK] InnoDB读缓冲效率: 99.90% (2901594命中/ 2904524总数)
[!!] InnoDB写日志效率: 75.47% (19033命中/ 25219总数)
[OK] InnoDB日志等待: 0.00% (0等待 / 6186写入)
在应用程序运行时运行top命令,显示Java在启动后仅使用<1%的CPU使用率。对于批处理,理想情况下应该使用所有可用的处理能力。
2113 mysql 20 0 25.7g 2.7g 36264 S 4.3 11.6 0:12.35 mysqld
353 root 20 0 0 0 0 S 2.7 0.0 0:02.99 md0_raid5
244 root 0 -20 0 0 0 I 0.7 0.0 0:00.98 kworker/10:1H-kblockd
248 root 0 -20 0 0 0 I 0.7 0.0 0:00.07 kworker/8:1H-kblockd
399 root 20 0 0 0 0 D 0.7 0.0 0:00.61 jbd2/md0-8
2372 bias 20 0 5597164 180592 27784 S 0.7 0.7 0:07.54 java
运行stress命令,我可以获得完全的CPU使用率。
4654 user 20 0 3856 100 0 R 100.0 0.0 0:13.88 stress
4643 user 20 0 3856 100 0 R 100.0 0.0 0:13.82 stress
4645 user 20 0 134932 3272 272 R 100.0 0.0 0:13.84 stress
4646 user 20 0 3856 100 0 R 100.0 0.0 0:13.80 stress
maven的pom.xml文件使用了与操作系统安装的mysql服务器版本相同的mysql连接器版本8.0.21。
39 <dependency>
40 <groupId>mysql</groupId>
41 <artifactId>mysql-connector-java</artifactId>
42 <version>8.0.21</version>
43 </dependency>
这个Java应用程序是被操作系统还是被JVM限制的?
在20.04和mysql或java之间有已知的问题吗?
英文:
I'm working on a Java / MySQL app and test runs of batch processing were taking 3-4seconds on Ubuntu 18.04.
I made the upgrade to Ubuntu 20.04 last night, and kept the same mysql conf file, and now the same app is taking 1min and 45-47 seconds!!!
6.27user 0.41system 1:45.53elapsed 6%CPU (0avgtext+0avgdata 168724maxresident)k
0inputs+384outputs (0major+42056minor)pagefaults 0swaps
I noticed with lscpu that the server is now running in min freq on average. I tried adding "acpi=ht" to /etc/default/grub, and restarted the machine, to turn off power management but the processors is still running at low frequency and presumably frequency boost isn't working.
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 44
Model name: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
Stepping: 2
Frequency boost: enabled
CPU MHz: 1599.592
CPU max MHz: 2401.0000
CPU min MHz: 1600.0000
BogoMIPS: 4799.75
Virtualization: VT-x
L1d cache: 256 KiB
L1i cache: 256 KiB
L2 cache: 2 MiB
L3 cache: 24 MiB
Using mysqltuner to check the server's InnoDB metrics things look OK, so it seems like this is related to the CPU settings / OS throttling or a Java configuration / code issue.
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 8
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 9.0G/2.6G
[OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 1.0G * 2/9.0G should be equal 25%
[OK] InnoDB buffer pool instances: 9
[--] Number of InnoDB Buffer Pool Chunk : 72 for 9 Buffer Pool Instance(s)
[OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances
[OK] InnoDB Read buffer efficiency: 99.90% (2901594 hits/ 2904524 total)
[!!] InnoDB Write Log efficiency: 75.47% (19033 hits/ 25219 total)
[OK] InnoDB log waits: 0.00% (0 waits / 6186 writes)
Running top while the app is running shows that Java is only using <1%cpu usage after startup. For a batch process ideally it'd use all available processing power.
2113 mysql 20 0 25.7g 2.7g 36264 S 4.3 11.6 0:12.35 mysqld
353 root 20 0 0 0 0 S 2.7 0.0 0:02.99 md0_raid5
244 root 0 -20 0 0 0 I 0.7 0.0 0:00.98 kworker/10:1H-kblockd
248 root 0 -20 0 0 0 I 0.7 0.0 0:00.07 kworker/8:1H-kblockd
399 root 20 0 0 0 0 D 0.7 0.0 0:00.61 jbd2/md0-8
2372 bias 20 0 5597164 180592 27784 S 0.7 0.7 0:07.54 java
Running stress I get full cpu usage
4654 user 20 0 3856 100 0 R 100.0 0.0 0:13.88 stress
4643 user 20 0 3856 100 0 R 100.0 0.0 0:13.82 stress
4645 user 20 0 134932 3272 272 R 100.0 0.0 0:13.84 stress
4646 user 20 0 3856 100 0 R 100.0 0.0 0:13.80 stress
The maven pom.xml file is using the same mysql connector version that the OS has installed for the mysql server 8.0.21
39 <dependency>
40 <groupId>mysql</groupId>
41 <artifactId>mysql-connector-java</artifactId>
42 <version>8.0.21</version>
43 </dependency>
Is the java app being throttled by the OS or by the JVM?
Are there known issues with 20.04 and mysql or java?
答案1
得分: 2
以下是翻译好的内容:
MySQL JDBC驱动程序的默认行为在更新操作系统时显然发生了变化。
以前的驱动程序关闭了Java批量更新的事务自动提交 - 在数据连接上手动添加/运行setAutoCommit(false)
将运行时间恢复到正常(显然它正在为1000多次插入运行原子事务)。
这是一个很好的例子,说明为什么人们应该进行防御性编码,即使默认的行为/配置是代码中使用的内容 - 默认值可以并且会发生变化!
链接:https://coderanch.com/t/299833/databases/Batch-update-setAutoCommit-false
英文:
The MySQL JDBC driver default behavior apparently changed when updating the OS.
The previous driver turned off transaction auto commits for Java batch updates - manually adding/running setAutoCommit(false)
on the data connection dropped the run time back to normal (apparently it was running atomic transactions for 1000 some inserts).
This is a good example of why people should defensively code and add intentional code even if the default behavior/configuration is what's used in code - the default can and will change!
https://coderanch.com/t/299833/databases/Batch-update-setAutoCommit-false
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论