Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Why can't I use a library I just installed with Pip?
I tried installing package-installation-test
[1] using Pip, and it appeared to be successful. But I can't use it as advertised, either by itself or by importing it from my code:
Failed attempts
$ pip install package-installation-test
Collecting package-installation-test
Using cached package_installation_test-1.0.0-py3-none-any.whl (3.1 kB)
Installing collected packages: package-installation-test
Successfully installed package-installation-test-1.0.0
$ demo-example-package
demo-example-package: command not found
$ python -m example_package
/path/to/python: No module named example_package
$ python -q
>>> import example_package
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'example_package'
>>>
Also, demo-example-package
doesn't tab-complete in my shell.
According to the documentation, I should get results like this instead:
Expected results
$ demo-example-package
Version 1.0.0 of example_package successfully installed.
The source code is in /path/to/example_package.
$ python -m example_package
Version 1.0.0 of example_package successfully installed.
The source code is in /path/to/example_package.
$ python -q
>>> import example_package # no error
>>> example_package.home()
PosixPath('/path/to/example_package')
Attempting to reinstall doesn't work either:
$ pip install package-installation-test
Requirement already satisfied: package-installation-test in /path/to/site-packages (1.0.0)
What gives?
-
Full disclosure: I am the author of this package. I created it specifically to demonstrate the problem for the purpose of a canonical Q&A. Whatever package you're actually trying to install and use, the fundamentally problem is still the same. ↩︎
1 answer
The following users marked this post as Works for me:
User | Comment | Date |
---|---|---|
Karl Knechtel | (no comment) | Jun 27, 2024 at 19:07 |
Problems like this occur, fundamentally, because there is more than one Python installation on the system, and the system is configured in a way that pip
installs for a different Python than the one that python
runs.
This problem can be addressed by either:
- using a virtual environment
-
running Pip indirectly through Python -
python -m pip
orpy -m pip
.
In detail, the problem occurs because:
-
Pip is itself implemented in Python, and even when the command you use is
pip
, a Python interpreter needs to run. (On Mac and Linux,pip
is a Python script that has a shebang line and executable bit set; on Windows, it's a very small program that spawns a Python process to run the Pip code - sort of like a bundled-interpreter application, but without actually bundling the interpreter.) -
Pip generally installs packages according to the path where that Python interpreter was found. This means you don't have to tell it every time, and is convenient for some systems that are carefully configured to make it work properly. (Activating a virtual environment could be considered as this kind of configuration.)
-
There can only be one file (executable or not) with the same name in the same folder; and when you run a command at the command line, it can only find one of those programs. So
pip
means whichever program (if any) is found first by looking at entries in thePATH
environment variable, one by one - once Pip is found, the search stops. Similarly forpython
(orpython3
etc.) -
What happens next depends on the exact system configuration.
-
When you install Python on Windows, each Python minor version gets a separate sub-folder within
Program Files
or a similarly named folder. (Long ago, Python would put its install folders directly underC:\
.) By default, these folders are not added toPATH
, in part because over time that could make thePATH
really long (long enough to bump into system-imposed limits). Instead, you're expected to use the Python launcher for Windows, calledpy
(which is common to all versions, and sites directly inC:\Windows
). But there is no equivalent "launcher" for Pip (of course,py -m pip
solves the problem anyway).If
pip
is runnable as a command, it's because some Python folder was added toPATH
- and it might not be the one you want. Normally, ifpip
andpython
both works as commands, then they would find the same installation folder. But this wouldn't work if, for example, Pip were not installed for first Python listed: thenpython
would find that Python, butpip
wouldn't (and could find a different Python). -
When you install Python on Linux, typically it puts versioned
pip
andpython
executables (for example, they might be namedpip3.12
andpython3.12
) side by side right in/usr/bin
, and sets up symlinks forpython
,python3
etc. But those symlinks can only mean one version at a time, and it's possible for them to get out of sync. Sopip
andpython
can use different Python installations even though they're in the same folder.Also, keep in mind that it's common for a Python installation that comes with Linux, to have several features disabled - especially Pip - typically for security reasons. (It would be bad if, for example, you could easily use Pip to upgrade a third-party library that came pre-installed in your system Python, which a critical system script depends on, and the update broke that script. Even worse, PyPI can host malware - anyone can offer a package, and not everyone is reading the source code for every version of every upload. Never use
sudo
to run Pip.)
-
There is nothing wrong with having multiple Python installations on your system.
Many developers will find it necessary to have multiple versions installed for their day-to-day work. Beyond that, virtual environments (which are essentially separate installations, for these purposes) are very useful for organizing projects.
Finally, if you're on Linux and Python came with your system, it may not be very suitable for development work because of the modifications made by the distro. It's possible to work around these issues (by using the system package manager to add missing pieces, and passing the --user
flag to Pip); but just using a virtual environment ensures that everything "just works" and that you can follow tutorials and guides without issue. It's also better for development: you won't accidentally write code that depends on some third-party package that your distro provided, mistakenly thinking that it's in the standard library.
However, in order to use third-party libraries successfully, you need to understand these basic principles.
1 comment thread