Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Comments on What is the point of pipx?

Parent

What is the point of pipx?

+9
−0

Background

Many Python programs now recommend installing with pipx, and there is a sense that you shouldn't install with pip anymore, you should use tools like pipx.

Main Question

However, what does pipx actually do that makes it such a preferred alternative to pip?

Thoughts

I checked their docs and what I found doesn't really make sense. In sum:

  • pipx is described as a package manager, but package managers install files in system locations whereas pipx installs them in user locations. Also, package managers already have python-... packages where it makes sense.
  • It mentions that unlike pip, it is specifically for CLI apps. But what exactly does pip not do? AFAIK executable packages just have a wrapper script in ~/.local/bin/ that calls them. This doesn't seem worth a whole program.
  • It talks about PyPi as an "app store", which sounds weird. Yes, people can and do distribute on PyPi, but there are major differences which pipx hardly closes.

I do see that it mentions isolating envs. I can see how it is not straightforward with pip to install each CLI app in a venv, but also make it available in PATH. So is that all pipx is, CLI apps in venv? This seems like a rather inefficient way to handle packaging (see also "static link everything").

History
Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

1 comment thread

"package managers install files in system locations" (3 comments)
Post
+9
−0

Well, to start, it is not an alternative to pip. It's built on top of pip and exclusively deals with applications. pip is more of a development tool, while pipx is aimed at end-users (who may also be developers).

The bulk of your question seems to be complaining that loose analogies aren't tight analogies. I don't know why you're taking these analogies so strongly. Here is the relevant text from that page: "It's roughly similar to macOS's brew, JavaScript's npx, and Linux's apt." and "In a way, it turns Python Package Index (PyPI) into a big app store for Python applications." (emphasis mine)

As a final note before turning to more substantive topics, an application does not need to be large and complicated to be valuable. In this case, the main value proposition is ergonomics/ease-of-use. A secondary benefit is that it provides an abstraction layer. You don't need to know how it works to use it, and that means how it works can change without changing how you use it.

So, why use this instead of pip?

Mainly, pip provides no isolation. This makes it a headache to deal with different libraries and applications that have dependencies on different versions of the same libraries. It also means installing or upgrading a package might unwittingly break something else. Common practice is to use tools like virtualenv, Conda, venv, and others to get isolation, but these are much more oriented to developers. Using these together to run an application with isolation would require several commands and manually keeping track of various "environments". This "environment" concept is great for developers who want to install several libraries within a single environment, but for applications there's (usually) no reason to have different applications "see" the same environment. This "environment" concept just complicates the flow that pipx is aiming for.

Okay, why use this instead of a system package manager?

Mainly, most package managers need someone else to package things for you. If they haven't, then oh well. Package managers also usually don't provide isolation and make it very difficult to have multiple versions of the same "package". Their ethos is usually provide a "known good" selection of packages, but this makes them slow to change and often incomplete and out-of-date. A small but real annoyance is often system packages have slightly different names than the corresponding Python packages meaning you need an extra "look up the corresponding system package" step to installing something you see on PyPI or GitHub.

What does pipx provide?

  • Easy Python application installation with no need to worry about conflicting versions of things.
  • Addition of such applications to the PATH allowing them to be used seamlessly.
  • Easy uninstallation and upgrade.
  • Use of the PyPI namespace.
  • Ephemeral environments for one-off executions.
  • For developers of Python applications, less need to deal with unknown mixtures of libraries.

How does pipx work?

I refer you to How pipx works, but, as a quick summary as of August 2023, it does the "obvious" thing. It installs each application in its own virtual environment via venv. It then adds symbolic links to ~/.local/bin. It is very slightly cleverer than that and will reuse a virtual environment for the packaging tools themselves.

This does mean you will have duplication of library code. This is a common problem with many language-specific package managers, not just Python's. It is a real problem as it is very easy to have many gigabytes of wasted space in duplicated dependencies. There's also a lot of time wasted installing duplicate dependencies. The best solution I've seen to this problem is to adopt Nix's approach. The key to this approach is to allow not just installing different versions of the same library side-by-side, but also the same library version with different dependencies side-by-side. This allows a single shared environment where dependencies are never duplicated, though you can definitely have multiple very similar libraries installed. This requires all dependencies to be easily identifiable and builds to be deterministic which are properties that are hard to ensure in the Python ecosystem. The only language-specific package manager I'm aware of that uses this approach is Haskell's cabal-install.

As I mentioned before, pipx could, in theory, transparently switch to such an approach if it ever became available.

Why not use pipx?

There are plenty of use-cases pipx is not aimed at where it would be unnecessary or irrelevant, e.g. system images in an immutable infrastructure setup. For the use-case it is targeted at, i.e. personal machines, the only good options are, if possible, restrict yourself to "standard" system packages, or use pipx. The other options are to manually do what pipx does or forgo isolation and enter dependency hell.

History
Why does this post require moderator attention?
You might want to add some details to your flag.

1 comment thread

Biased? (4 comments)
Biased?
matthewsnyder‭ wrote 9 months ago

This has some information but it seems a bit too biased in favor of pipx. The first three paragraphs don't add anything. There's some speculation about features that are not actually part of pipx. In the bulleted list, IMO 3, 4 and 6 are already things you get in pip. That said, there are some reasonable points raised here (managing PATH, isolation), so I'd suggest trimming out some of the unnecessary stuff to improve the answer (which is a bit long in any case).

I think paragraphs 4 and some of 6 (the bulleted list) are the most useful with respect to addressing the question.

Derek Elkins‭ wrote 9 months ago

Feel free to remove the parts those first paragraphs address, and I will remove those paragraphs.

As for "bias", why would you expect a tool purpose-built to do a specific task to not be better at that task than tools not purpose-built for that task? The "problem" with pipx is that it is not able to or trying to solve the same problems that pip or apt are trying to solve.

jmathew‭ wrote 9 months ago

The first few paragraphs are not biased and were useful to me: a non-biased observer with knowledge of python and pip, but new to pipx. It puts the statements in your question into context. For example, I thought it was indeed strange that they talked about PyPi being an "app store" as suggested in your question, but the quote from the answer clarified it was a loose analogy. I could've read the links but its good to have that in the answer as it is directly relevant to the question.

The only potentially unnecessary portion of this answer is "how pipx works". That wasnt asked directly but you did wonder whether the system was inefficient which can only be answered by talking about the internals. Regardless, that section was useful to me as a reader.

matthewsnyder‭ wrote 9 months ago

I agree on that as well. "How pipx works" is interesting and useful, but it's not pertinent information. It should be in a separate post.