英文:
A conceptual question on python GIL, what does it actually mean
问题
我怀疑类似的问题可能已经被问过多次。
但是很难为我得到一个答案来解决我的问题。。
我理解GIL只允许单个Python线程同时执行。
(我的理解来自https://dabeaz.com/python/UnderstandingGIL.pdf)
但是如果我考虑一下,拥有GIL实质上与单核环境有着相同的效果。
有了GIL,Python多线程在单个核心上运行。
作为一个程序员,我仍然必须处理可能发生的所有竞态条件。
或者目的不是为了我吗?而是为了使Python解释器更安全?
我猜这个问题可以重新表述为,如果Python移除了GIL,用户程序会受到什么影响?
我的理解是,
如果没有了GIL,
它将使Python多线程:
单核线程编程 -> 多核线程编程
或者还有其他事情发生吗?
(如果我的理解是正确的,GIL实际上是我们可以去掉的..我的意思是如果可以做到,那就可以在不影响用户程序的情况下完成)
我想我必须强调在这里我只关心用户程序的角度。(GIL如何影响Python运行时(解释器)不是我在这里问的问题)
英文:
I suspect similar questions might have been asked multiple times.
But it's hard for me to get an answer for my question..
I understand GIL makes only single python thread can execute at a time.
(My understanding comes from https://dabeaz.com/python/UnderstandingGIL.pdf)
But if I think about it, having GIL has essentially the same effect of a single core environment.
With Gil, python multiple threads run in a single core.
As a programmer, I still have to deal with all the race conditions that might occur.
Or is the purpose not about me? it's about python interpreter to be safe?
I guess the question can be rephrased as, if python removes GIL, how is the user program affected?
My understanding is that,
if GIL is gone,
it will make python multithreading:
single core thread programming -> multi core thread programming
Or is there something else going on?
(If my understanding is correct, GIL is something we can actually take out.. I mean if it can be done, it can be done without affecting user programs)
I guess I have to emphasize that here I'm only interested in the user program perspective. (How GIL affects python runtime (interpreter) is not something I ask here)
答案1
得分: 2
首先,要理解并不是所有的Python实现都有全局解释器锁(GIL),比如Jython和IronPython就没有。但是,大多数人使用的主要Python实现——CPython,是有GIL的。
需要注意的一点是,CPython曾试图移除GIL,但由于很多代码都是基于GIL存在的前提构建的,所以这非常困难。关于移除GIL所涉及的一些问题的简要概述可以在PyPy FAQ中找到(PyPy也有GIL):
是的,PyPy有GIL。移除GIL非常困难。在CPython的基础上,你面临两个问题:(1) 垃圾回收,这里是指引用计数;(2) 整个Python语言。
对于PyPy来说,难点在于问题(2):我指的是如果一个可变对象在一个线程中被修改,同时在另一个线程中被读取会发生什么。这对于任何可变类型都是一个问题:需要仔细审查并修复整个Python解释器中的问题(主要是细粒度的锁)。这是一项重大工作,虽然不是完全不可能的,正如Jython/IronPython所展示的那样。这还包括有关某些效果对于用户(即Python程序员)是否可以接受的微妙决策。
CPython还有问题(1)的引用计数。对于PyPy来说,这个子问题较为简单:我们需要使我们的垃圾回收器支持多线程。这在PyPy中相对于CPython来说更容易高效地实现。但它并没有解决问题(2)。
注意,曾经有关于支持一种软件事务内存(STM)版本的PyPy的工作。这应该提供一个不需要GIL的替代PyPy,同时继续给Python程序员提供完整的GIL幻觉。由于其自身的技术难题,这项工作目前有点停滞不前。
而所有这些问题都来自一组开发人员,他们正在构建自己的全新编译器。想象一下,CPython团队更大,还需要确保向后兼容性,所以需要处理的问题更多。
所以,你的猜测是正确的:这不仅仅是关于你的问题。如果有人能够找出如何移除GIL而不会破坏其他大量功能的方法,那么它可能会被合并到CPython中。这将使Python代码更快,也更符合人们对线程通常运行方式的期望。但是迄今为止还没有人能够做到足够好。根据我的了解,它逐渐变得不再是一个“我们应该解决的问题”,而更像是一个“我们必须应对的问题”。
英文:
First, understand that not all Python implementations have a GIL, like Jython and IronPython. However, CPython, the big main one that most people use, does have one.
One thing that's important to note is that CPython has tried to remove the GIL, but it's really hard due to all the stuff built assuming the GIL is there. One quick and easy overview of just some of the problems involved in removing the GIL is described in the PyPy FAQ (PyPy also has a GIL):
> Yes, PyPy has a GIL. Removing the GIL is very hard. On top of CPython, you have two problems: (1) GC, in this case reference counting; (2) the whole Python language.
>
> For PyPy, the hard issue is (2): by that I mean issues like what occurs if a mutable object is changed from one thread and read from another concurrently. This is a problem for any mutable type: it needs careful review and fixes (fine-grained locks, mostly) through the whole Python interpreter. It is a major effort, although not completely impossible, as Jython/IronPython showed. This includes subtle decisions about whether some effects are ok or not for the user (i.e. the Python programmer).
>
> CPython has additionally the problem (1) of reference counting. With PyPy, this sub-problem is simpler: we need to make our GC multithread-aware. This is easier to do efficiently in PyPy than in CPython. It doesn’t solve the issue (2), though.
>
> Note that there was work to support a Software Transactional Memory (STM) version of PyPy. This should give an alternative PyPy which works without a GIL, while at the same time continuing to give the Python programmer the complete illusion of having one. This work is currently a bit stalled because of its own technical difficulties.
And all of that's coming from a group of developers that are building their whole new own compiler. Imagine all the issues the CPython group, which is much bigger and needs to ensure a level of backward compatibility too, needs to deal with.
So yes, your guess is right: it's not all about you. If someone can figure out how to take out the GIL without messing up a bunch of other stuff, then it'd probably be merged by CPython. It'd make Python code faster and less confusing to people expecting how threads usually work. But no one's been able to do that good enough yet. From what I've read, it's slowly become less of a "problem we should fix" and more of a "problem we have to deal with".
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论