Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Notifications
Mark all as read
Q&A

What is the Python Global Interpreter Lock?

+4
−0

What exactly is the Python Global Interpreter Lock (GIL)? As someone who is relatively new to Python, is this something I need to be aware of, or is this just some implementation detail of the interpreter which I shouldn't concern myself with?

Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

1 answer

+8
−0

The Python Global Interpreter Lock (GIL) is a mutex in the primary Python implementation (CPython) that is acquired whenever Python (byte)code is executing. This means within a single (OS) process only one (OS) thread can be executing Python code at any point in time.

This is absolutely an implementation detail as other implementations of Python do not have this restriction.

Practically speaking, this only comes up if you want to write multi-threaded code, particularly to improve performance. Single threaded code doesn't need to worry about the GIL at all. For I/O-bound code, the typical high-performance solution is to use asynchronous I/O APIs. The performance of these are not significantly impacted by the GIL. The issue is for CPU-bound code, particularly when you'd like to exploit multiple cores. Here the GIL is an almost complete showstopper.

As a concrete example, in a simple ray tracer, each pixel of the output image can be computed independently of the others. Therefore you'd expect a speed-up nearly linear in the number of cores for a multithreaded implementation that split the image over multiple cores. If you implemented this ray tracer in (C)Python, however, the GIL would mean you'd see no speed-up.

In the context of CPython, the main way to address this is via the multiprocessing library. This will spawn multiple processes rather than a single process with multiple threads. As such each process will have its own copy of the Python interpreter with its own GIL. However, having multiple processes talk to each other is a much heavier weight and clumsier thing to do. Aspects of how the OS handles interprocess communication become relevant. For example, the Queue API in the multiprocessing module needs to pickle the data sent through the queue since pointers don't make sense across processes. A shared memory interface is provided as well, but this is a much lower-level experience as compared to threads which could pass around Python objects.

The benefits of the GIL are mostly for the implementors. The GIL is a very simple and straightforward solution to having a thread-safe runtime. More importantly for its longevity, it allows incorporating non-thread-safe (C) extensions with little trouble. Implementations like IronPython on the CLR tend not to have a GIL as the underlying platform has a better story for multithreading and would be less dependent on non-thread-safe extensions.

Why does this post require moderator attention?
You might want to add some details to your flag.

0 comment threads

Sign up to answer this question »

This community is part of the Codidact network. We have other communities too — take a look!

You can also join us in chat!

Want to advertise this community? Use our templates!

Like what we're doing? Support us! Donate