Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
1 answer
The Python Global Interpreter Lock (GIL) is a mutex in the primary Python implementation (CPython) that is acquired whenever Python (byte)code is executing. This means within a single (OS) process only one (OS) thread can be executing Python code at any point in time.
This is absolutely an implementation detail as other implementations of Python do not have this restriction.
Practically speaking, this only comes up if you want to write multi-threaded code, particularly to improve performance. Single threaded code doesn't need to worry about the GIL at all. For I/O-bound code, the typical high-performance solution is to use asynchronous I/O APIs. The performance of these are not significantly impacted by the GIL. The issue is for CPU-bound code, particularly when you'd like to exploit multiple cores. Here the GIL is an almost complete showstopper.
As a concrete example, in a simple ray tracer, each pixel of the output image can be computed independently of the others. Therefore you'd expect a speed-up nearly linear in the number of cores for a multithreaded implementation that split the image over multiple cores. If you implemented this ray tracer in (C)Python, however, the GIL would mean you'd see no speed-up.
In the context of CPython, the main way to address this is via the multiprocessing library. This will spawn multiple processes rather than a single process with multiple threads. As such each process will have its own copy of the Python interpreter with its own GIL. However, having multiple processes talk to each other is a much heavier weight and clumsier thing to do. Aspects of how the OS handles interprocess communication become relevant. For example, the Queue API in the multiprocessing module needs to pickle the data sent through the queue since pointers don't make sense across processes. A shared memory interface is provided as well, but this is a much lower-level experience as compared to threads which could pass around Python objects.
The benefits of the GIL are mostly for the implementors. The GIL is a very simple and straightforward solution to having a thread-safe runtime. More importantly for its longevity, it allows incorporating non-thread-safe (C) extensions with little trouble. Implementations like IronPython on the CLR tend not to have a GIL as the underlying platform has a better story for multithreading and would be less dependent on non-thread-safe extensions.
0 comment threads
0 comment threads