Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Post History

71%
+3 −0
Q&A Why does `pip` seem to be missing or broken? Isn't it part of the standard library?

Contrary to popular belief, Pip is not part of the Python standard library - although it comes as part of a standard distribution of Python. Pip is developed and maintained by the arms-length Pytho...

posted 6mo ago by Karl Knechtel‭  ·  edited 6mo ago by Karl Knechtel‭

Answer
#2: Post edited by user avatar Karl Knechtel‭ · 2024-06-18T10:17:57Z (6 months ago)
  • Contrary to popular belief, Pip is **not** part of the Python standard library - although it comes as part of a standard *distribution* of Python. Pip is developed and maintained by the arms-length [Python Packaging Authority](https://www.pypa.io/en/latest/), which allows it to be versioned independently of Python (developed by the core Python development team, under the umbrella of the [Python Software Foundation](https://www.python.org/psf-landing/)).
  • Typically, when Python is installed, Pip is *bootstrapped* into the distribution's `site-packages` third-party library. This is done by running the [`ensurepip`](https://github.com/python/cpython/tree/main/Lib/ensurepip) standard library package after the main part of the installation. Basically, the Python source code distribution contains a specific, pre-made "wheel" for Pip (a specific version corresponding to each Python version), which `ensurepip` copies into place. `ensurepip` will also create and install the expected wrapper executables so that the `pip` command can work.
  • So, ordinarily, if Pip is missing, `python -m ensurepip` would be enough to restore it. **However, Python provided as part of Debian Linux (and downstream distros) deliberately disables this:**
  • <details><summary>Important notes for Debian-based Linux distros</summary>
  • The included `ensurepip` has been modified to remove the "vendored" wheel (although there may be a Pip wheel in `/usr/share/python-wheels`, to support virtual environment creation) and so that a diagnostic message is displayed instead if you try to use it outside of a virtual environment:
  • ```
  • $ python -m ensurepip
  • ensurepip is disabled in Debian/Ubuntu for the system python.
  • Python modules for the system python are usually handled by dpkg and apt-get.
  • apt install python3-<module name>
  • Install the python3-pip package to use pip itself. Using pip together
  • with the system python might have unexpected results for any system installed
  • module, so use it on your own risk, or make sure to only use it in virtual
  • environments.
  • ```
  • <section class="notice is-warning">
  • You can use the system package manager to add Pip to the system Python by following the instructions from the diagnostic message. However, **this is discouraged, because of the risks involved**.
  • Installing packages for the system Python in the default location (i.e. the system Python's own `site-packages` folder) requires root privileges. **Please never use `sudo` to run Pip. Pip installation supports running arbitrary code from `setup.py` which would run with root privileges in this situation.** Newer versions of Pip will default to a "user" install in this situation, putting the installed library in a version-specific subdirectory within `$(HOME)/.local/lib`. (With older Pip you may need to specify `--user` explicitly.) Packages installed here will still be recognized when running the system Python, so they could still theoretically interfere with critical system scripts; but at least the root filesystem will be protected during installation.
  • It's **not necessary, anyway** to install Pip for the system Python in order to do Python development. Instead, just create a virtual environment (you *may* have to install *`venv`* for the system Python). By default, `ensurepip` will run in the new virtual environment and bootstrap Pip into that environment.
  • </section></details>
  • The basic idea here is that Debian wants you to use the system package manager (`apt`) to install third-party libraries for the system Python, rather than Pip. That way, they can explicitly vet specific packages for installation in the system Python and add their own patches.
  • This is called "external management", and newer versions of Pip (such as included with Python 3.11) can [cooperate with Debian to recognize this "externally managed environment"](https://packaging.python.org/en/latest/specifications/externally-managed-environments/). Thus, newer versions of Pip will report an error when you try to use it with the system Python, unless you explicitly pass `--break-system-packages` as a confirmation. Other distros may use this marker system even if they provide Pip by default.
  • Contrary to popular belief, Pip is **not** part of the Python standard library - although it comes as part of a standard *distribution* of Python. Pip is developed and maintained by the arms-length [Python Packaging Authority](https://www.pypa.io/en/latest/), which allows it to be versioned independently of Python (developed by the core Python development team, under the umbrella of the [Python Software Foundation](https://www.python.org/psf-landing/)).
  • Typically, when Python is installed, Pip is *bootstrapped* into the distribution's `site-packages` third-party library. This is done by running the [`ensurepip`](https://github.com/python/cpython/tree/main/Lib/ensurepip) standard library package after the main part of the installation. Basically, the Python source code distribution contains a specific, pre-made "wheel" for Pip (a specific version corresponding to each Python version), which `ensurepip` copies into place. `ensurepip` will also create and install the expected wrapper executables so that the `pip` command can work.
  • So, ordinarily, if Pip is missing, `python -m ensurepip` would be enough to restore it. **However, Python provided as part of Debian Linux (and downstream distros) deliberately disables this:**
  • <details><summary>Important notes for Debian-based Linux distros</summary>
  • The included `ensurepip` has been modified to remove the "vendored" wheel (although there may be a Pip wheel in `/usr/share/python-wheels`, to support virtual environment creation) and so that a diagnostic message is displayed instead if you try to use it outside of a virtual environment:
  • ```bash
  • $ python -m ensurepip
  • ensurepip is disabled in Debian/Ubuntu for the system python.
  • Python modules for the system python are usually handled by dpkg and apt-get.
  • apt install python3-<module name>
  • Install the python3-pip package to use pip itself. Using pip together
  • with the system python might have unexpected results for any system installed
  • module, so use it on your own risk, or make sure to only use it in virtual
  • environments.
  • ```
  • <section class="notice is-warning">
  • You can use the system package manager to add Pip to the system Python by following the instructions from the diagnostic message. However, **this is discouraged, because of the risks involved**.
  • Installing packages for the system Python in the default location (i.e. the system Python's own `site-packages` folder) requires root privileges. **Please never use `sudo` to run Pip. Pip installation supports running arbitrary code from `setup.py` which would run with root privileges in this situation.** Newer versions of Pip will default to a "user" install in this situation, putting the installed library in a version-specific subdirectory within `$(HOME)/.local/lib`. (With older Pip you may need to specify `--user` explicitly.) Packages installed here will still be recognized when running the system Python, so they could still theoretically interfere with critical system scripts; but at least the root filesystem will be protected during installation.
  • It's **not necessary, anyway** to install Pip for the system Python in order to do Python development. Instead, just create a virtual environment (you *may* have to install *`venv`* for the system Python). By default, `ensurepip` will run in the new virtual environment and bootstrap Pip into that environment.
  • </section></details>
  • The basic idea here is that Debian wants you to use the system package manager (`apt`) to install third-party libraries for the system Python, rather than Pip. That way, they can explicitly vet specific packages for installation in the system Python and add their own patches.
  • This is called "external management", and newer versions of Pip (such as included with Python 3.11) can [cooperate with Debian to recognize this "externally managed environment"](https://packaging.python.org/en/latest/specifications/externally-managed-environments/). Thus, newer versions of Pip will report an error when you try to use it with the system Python, unless you explicitly pass `--break-system-packages` as a confirmation. Other distros may use this marker system even if they provide Pip by default.
#1: Initial revision by user avatar Karl Knechtel‭ · 2024-06-18T10:16:48Z (6 months ago)
Contrary to popular belief, Pip is **not** part of the Python standard library - although it comes as part of a standard *distribution* of Python. Pip is developed and maintained by the arms-length [Python Packaging Authority](https://www.pypa.io/en/latest/), which allows it to be versioned independently of Python (developed by the core Python development team, under the umbrella of the [Python Software Foundation](https://www.python.org/psf-landing/)).

Typically, when Python is installed, Pip is *bootstrapped* into the distribution's `site-packages` third-party library. This is done by running the [`ensurepip`](https://github.com/python/cpython/tree/main/Lib/ensurepip) standard library package after the main part of the installation. Basically, the Python source code distribution contains a specific, pre-made "wheel" for Pip (a specific version corresponding to each Python version), which `ensurepip` copies into place. `ensurepip` will also create and install the expected wrapper executables so that the `pip` command can work.

So, ordinarily, if Pip is missing, `python -m ensurepip` would be enough to restore it. **However, Python provided as part of Debian Linux (and downstream distros) deliberately disables this:**

<details><summary>Important notes for Debian-based Linux distros</summary>

The included `ensurepip` has been modified to remove the "vendored" wheel (although there may be a Pip wheel in `/usr/share/python-wheels`, to support virtual environment creation) and so that a diagnostic message is displayed instead if you try to use it outside of a virtual environment:

```
$ python -m ensurepip
ensurepip is disabled in Debian/Ubuntu for the system python.

Python modules for the system python are usually handled by dpkg and apt-get.

    apt install python3-<module name>

Install the python3-pip package to use pip itself.  Using pip together
with the system python might have unexpected results for any system installed
module, so use it on your own risk, or make sure to only use it in virtual
environments.
```

<section class="notice is-warning">

You can use the system package manager to add Pip to the system Python by following the instructions from the diagnostic message. However, **this is discouraged, because of the risks involved**.

Installing packages for the system Python in the default location (i.e. the system Python's own `site-packages` folder) requires root privileges. **Please never use `sudo` to run Pip. Pip installation supports running arbitrary code from `setup.py` which would run with root privileges in this situation.** Newer versions of Pip will default to a "user" install in this situation, putting the installed library in a version-specific subdirectory within `$(HOME)/.local/lib`. (With older Pip you may need to specify `--user` explicitly.) Packages installed here will still be recognized when running the system Python, so they could still theoretically interfere with critical system scripts; but at least the root filesystem will be protected during installation.



It's **not necessary, anyway** to install Pip for the system Python in order to do Python development. Instead, just create a virtual environment (you *may* have to install *`venv`* for the system Python). By default, `ensurepip` will run in the new virtual environment and bootstrap Pip into that environment.
</section></details>

The basic idea here is that Debian wants you to use the system package manager (`apt`) to install third-party libraries for the system Python, rather than Pip. That way, they can explicitly vet specific packages for installation in the system Python and add their own patches.

This is called "external management", and newer versions of Pip (such as included with Python 3.11) can [cooperate with Debian to recognize this "externally managed environment"](https://packaging.python.org/en/latest/specifications/externally-managed-environments/). Thus, newer versions of Pip will report an error when you try to use it with the system Python, unless you explicitly pass `--break-system-packages` as a confirmation. Other distros may use this marker system even if they provide Pip by default.