Post History

## Understanding distutils and setuptools: the history

The short version of "why does installing Setuptools fix the problem?" is that Setuptools provides its own `distutils` package, overriding anything the standard library might or might not have. When you `import setuptools` explicitly, it furthermore ensures that a patched version of `distutils` is imported.

[Way back in the prehistory of Python](https://wiki.python.org/moin/Distutils), the Distutils package was created so that there would be *some* standard way of distributing and installing each others' Python code - including compiling C extensions, with some nice wrappers to deal with other computers having different C compilers available. It was made available when Python 1.5 was current, and became part of the standard library in 1.6.

However, over time a lot of weaknesses were exposed in this system, and people also started to suspect that relying on the standard library in this particular way might be a mistake. [Multiple third-party alternatives popped up, and Setuptools won out](https://stackoverflow.com/questions/25337706/). To this day, Setuptools is still technically third-party.

For backwards-compatibility reasons, Setuptools worked by patching the standard library `distutils` package, in addition to providing its own modules.

Starting around Python 3.4, using Distutils directly became "informally deprecated"; the Python documentation started to recommend using Setuptools instead, and moved the actual information about how to use Distutils into a separate legacy documentation section. In Python 3.10 [the deprecation became official](https://peps.python.org/pep-0632/); and following the usual deprecation policy, the `distutils` package was removed from the standard library in Python 3.12. Since Setuptools can no longer depend on the standard library including `distutils` at all, it now includes its own "vendored" version of the old `distutils`, and applies its own modifications on top of that.

## Fixing problems with others' code, in Python 3.12

To fix problems running code that relies on `distutils`, install Setuptools in the environment where the code will run.

If you have a problem *installing* code on 3.12 because of `distutils`, the situation is a lot more complicated.

*Usually* there is no problem with installing a library that relies on `distutils`. Ideally, the library was pre-built into a wheel, so all the parts that rely on `distutils` have been done ahead of time. If that didn't happen (which means Pip will try to install from source, unless told to give up), the library is supposed to be responsible for specifying build requirements. Newer projects that use a `pyproject.toml` file can explicitly say that the build environment should contain `setuptools`, and that will make Pip install Setuptools first. When the `pyproject.toml` doesn't say what "build system" to use, [Setuptools is supposed to be the default anyway](https://packaging.python.org/en/latest/specifications/pyproject-toml/#declaring-build-system-dependencies-the-build-system-table).

However, in Python 3.12, circumstances can combine to create a very annoying problem. Pip tries to use [build isolation](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation) when it installs from source, meaning that it creates a separate virtual environment for the installation process (so that, for example, the setup process can use specific versions of Setuptools and other such tools, even if the code will be installed into an environment with a different version of those tools). Starting with Python 3.12, virtual environments created using the `venv` standard library (like how Pip does it) [don't include Setuptools by default](https://github.com/python/cpython/issues/95299). This sometimes causes installations to fail: even though Setuptools is installed in the Python where we want to *install* the new library, it isn't installed in the Python where it will be *built*.

In these cases, you can try passing the `--no-build-isolation` flag to Pip, to make it use the Setuptools (and potentially any other previously installed libraries!) from the existing environment while trying to install the new library. If that doesn't work, the only remaining option is to file a bug for that library (or check if the author has already said anything about it).

## Fixing problems with others' code, in earlier Python versions

If `distutils.core`, `distutils.util` etc. seem to be missing on your system, especially on Linux, this is probably because you are using a copy of Python that came with the system that has these components disabled. Your best bet is to use a virtual environment and make sure Setuptools is installed in that virtual environment (this should happen automatically by default). Otherwise, you can check with your distro to see if there is a *system-level* package you can install to add Distutils to your system Python.

<section class="notice is-danger">

Be aware of the usual risks involved in modifying a program that came with your operating system - especially since this is taking a step towards making it possible to add other third-party code to your system Python directly, bypassing the system package manager. (For the same reason, your system Python might not include Pip.) If someone managed to get malware onto PyPI and you inadvertently installed it this way, it could result in serious damage. Even ordinary third-party packages could potentially interfere with critical system Python scripts.
</section>

## Fixing problems with your own code

If you must maintain code that depends on `distutils` functionality and want it to work in Python 3.12 (especially if you are distributing it to others), [consider replacing](https://peps.python.org/pep-0632/#migration-advice) uses of `distutils` that can be covered by other standard library modules. If you still have parts for which "`setuptools` is the best substitute":

* Install Setuptools locally

* If your code uses the functionality *at runtime*, make sure that Setuptools is listed as a runtime dependency - typically, by mentioning it [in the `project.dependencies` in `pyproject.toml`](https://packaging.python.org/en/latest/specifications/pyproject-toml/#pyproject-toml-spec).

* If at all possible, build wheels locally and distribute them as well, [even if your project uses only Python source code](https://pradyunsg.me/blog/2022/12/31/wheels-are-faster-pure-python/).

* If you must distribute source that must use Distutils during the installation process, make sure your project has a `pyproject.toml` (add one, if it didn't already) that explicitly lists Setuptools as a *build-time* dependency, by describing it in the `[build-tool]` table. If applicable, modify your `setup.py` so that `setup` gets imported from Setuptools rather than Distutils (i.e., replace `from distutils import setup` with `from setuptools import setup`).