Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Reinstall old Python libraries after update

+4
−0

Recently, Python updated from 3.11 to 3.12 and now all my libraries are gone. Actually they're in the old 3.11 site-packages, but now that python points to 3.12 those are not active. Every time I run a program I'm used to, I get a bunch of import errors, and have to reinstall the dependencies.

I know how to do it, but it's tedious. Isn't there a way to carry over the 3rd party packages that were installed on the old Python, but not "internal" stuff like setuptools and pip? Those are usually part of the Python distribution and I don't want nor need to mess with them.

History
Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

1 comment thread

How do you manage your Python installation? (1 comment)

2 answers

+4
−0

It depends.

Pure Python is usually forward-compatible with newer minor versions, but this is not always the case. Features/modules that were available in a previous minor version may no longer be available in the next minor version. Some features may not behave in the same way as in a previous minor version.

Packages that make use of C extensions, such as numpy, are even more likely to have compatibility issues between minor versions (and even micro/patch versions). Cython makes for a good example, here. Cython takes Python code and converts it to C code. If you use Cython to compile Python 3.11 code, it will work in all micro/patch versions of 3.11 (meaning 3.11.0, 3.11.1, 3.11.2, etc), but not for any other minor version. Then again, the same Python code that was compiled with Cython may be valid in multiple minor versions (in other words, the underlying Python 3.11 code may also be valid 3.12 code and compile with Cython just fine).

In short, it's difficult to know which specific packages will be forward-compatible with newer minor versions, meaning that simply copying the package directories from one installation of Python to another is risky, at best. Unless one is willing to scour the source code of each package in order to verify compatibility, then your best bet is to just use the traditional/conventional methods of reinstalling the packages.


I'm not sure how your Python is installed, but it sounds like it might be your system Python, since that is usually when a Python installation gets replaced with a newer version. If that's the case, and you installed your Python packages through your system's package manager (I would avoid doing this in the future), then your distro's package repos should already contain the updated packages to go along with the new Python version. If you are instead using pip's "user install" feature or a venv (created by python -m venv somevenv), then I'm afraid you'll have to use pip to reinstall the packages.

A quick and easy way to reinstall a list of specific packages and their versions is to use pip freeze to output a list of the installed packages and their versions:

pip freeze > /tmp/requirements.txt

And then use that file to install the same packages and the same versions of said packages:

pip install -r /tmp/requirements.txt

If you would like to avoid having your Python installation replaced/upgraded like this, you could always build one from source. Once built, it will never change. Personally, I keep multiple Python installations under ~/.pythons/, but I keep every version isolated from each other (even micro/patch versions).

Going into detail about the different methods of maintaining a Python built from source (or multiple) is beyond the scope of this post, though, so that's where I'll leave this answer.

History
Why does this post require moderator attention?
You might want to add some details to your flag.

1 comment thread

Works for me (1 comment)
+1
−0

The pip freeze command may output individual dependencies with format

zipp @ file:///Users/abcxyz/work/recipes/ci_py311/zipp_1677907997878/work

Using this output will not work (unless you happen to be that user abcxyz). To generate a requirements file that does not refer to local, non-existing or no-longer existing cache directories, you can instead use

pip list --format=freeze > requirements.txt

which will generate a requirements.txt file that only lists the specific versions, in the format zipp==3.11.0.

If you then run pip install -r requirements.txt for a newer version of Python, you may sometimes run into version conflicts. You can try to fix those by not pinning the dependency to a specific version (so simply by removing the ==3.11.0 in the example of zipp==3.11.0), but in some cases there may be no fix (if a dependency has no release yet for the newer Python and the older version somehow cannot be installed); in the worst case you'll have to temporarily remove the 3d party library.

History
Why does this post require moderator attention?
You might want to add some details to your flag.

0 comment threads

Sign up to answer this question »