英文:

Why doesn't psutil accurately represent available memory after deleting a class instance in Python?

问题

我目前在设备上删除类的实例后，正在努力查找实际剩余多少内存。我尝试使用`psutil`库，但它没有正确显示我实际可用的内存。特别是：

```python
import psutil

x = torch.randint(0, 255, size=(300, 3, 200, 200), dtype=torch.float32)
conv = Modules.Conv2D(3, 3, 2, 10, save_flag=False)
print("已用内存：", psutil.virtual_memory().percent, "% 可用内存：", round(psutil.virtual_memory().available * 100 /
      psutil.virtual_memory().total), "%")
# 已用内存: 74.0% 可用内存: 26.0 %

conv是我自定义类的对象，它有不同的属性占用大量内存（正如您从psutil看到的）。现在，如果我尝试以下操作：

import gc

del conv
gc.collect()
print("已用内存：", psutil.virtual_memory().percent, "% 可用内存：", round(psutil.virtual_memory().available * 100 /
      psutil.virtual_memory().total), "%")
# 已用内存: 74.0% 可用内存: 26.0 %

即使我删除了实例并显式调用了Python的垃圾收集器，内存似乎也没有被释放。但是，如果我使用Windows资源管理器检查，内存实际上是空的：

# 在执行程序之前 --> 正在使用: 6025 MB
exec_program()
# 程序正在执行，分配了conv实例 --> 正在使用: 6899 Mb
# 调用gc.collect --> 正在使用: 6058 Mb
end_of_program()

所以问题是psutil没有正确表示这种情况。我猜测尽管gc.collect释放了内存，但虚拟内存地址仍然标记为已使用，即使它实际上是空的，因此导致了错误的表示。是否有其他方法/库可以正确实现我试图实现的目标？谢谢！

<details>
<summary>英文:</summary>

I&#39;m currently struggling to find out how much memory I actually have left on my device after I have deleted an instance of a class. I have tried to use the library `psutil` but it doesn&#39;t correctly show the memory I actually have available. In particular:

import psutil

x = torch.randint(0,255,size=(300,3,200,200),dtype=torch.float32)
conv = Modules.Conv2D(3,3,2,10,save_flag=False)

print("Used memory:", psutil.virtual_memory().percent,"% Free memory:", round(psutil.virtual_memory().available * 100 /
psutil.virtual_memory().total),"%")
#Used memory: 74.0% Free memory: 26.0 %

`conv` is an object of my custom class and It has different attributes that take up a lot of memory(as you can see from psutil). Now If I try the following:

import gc

del conv
gc.collect()
print("Used memory:", psutil.virtual_memory().percent,"% Free memory:", round(psutil.virtual_memory().available * 100 /
psutil.virtual_memory().total),"%")
#Used memory: 74.0% Free memory: 26.0 %

Even if I delete the instance and I explicitly call python's garbage collector, the memory doesn't seem to be freed up. Although if I check with Windows' resource manager the memory is actually emptied:

#Before executing the program --> In use: 6025 MB
exec_program()
#Program is executing and conv instance is allocated --> In use: 6899 Mb

gc.collect is called --> In use: 6058 Mb

end_of_program()

So the problem is that psutil doesn't represent correctly this situation. My guess is that although the memory is freed up by `gc.collect` the virtual memory address is still marked as used even if it is actually empty, hence the misrepresentation. Is there any other way/library that can do correctly what I'm trying to achieve?
Thanks in advance!

</details>


# 答案1
**得分**: 0

哎呀，内存分配和释放通常是一个灰色区域，很少有事情可以在Python层面或甚至在操作系统层面得到保证。具体而言，你无法知道在Python层面删除某些内容是否会将释放的内存放入（进程）本地空闲列表，还是将其归还给操作系统。

这个本地空闲列表的存在是因为当程序进行大量内存分配/释放时，将内存保持在本地比将其返回到操作系统再请求更为高效。换句话说，psutil完成了它的工作，告诉你给定进程使用了多少内存。但它不知道（因此不能告诉你）进程如何使用内存…

<details>
<summary>英文:</summary>

Oops, memory allocation and deallocation is often a grey zone, where few things are guaranteed, either at Python level, or even at OS one. Specifically you cannot know whether deleting something at Python level will put the freed memory into a (process) local free list or will return it to the OS.

That local free list exists because when a program does a lot of memory allocation/deallocation, it is far more efficient to keep the memory local than to return it back to the OS to later ask for it again. Said differently, psutil does its job and tells you how much memory is given to a specific process. But it does not know (hence cannot tell you) how the memory is used by the process...

</details>