What does Nixpkgs' `callPackage` do?

^{For the record, this write-up is not better than any of the resources listed in the question; writing it simply helped me better understand callPackage.}

Level 0. Poor, but short

callPackage f attr_set will

• call function f with inputs automatically supplied from attr_set and Nixpkgs
• make the resulting attribute set overridable.

callPackage is used overwhelmingly with functions that produce a derivation. Building a project can require a lot of dependencies that won't have to be specified manually this way, and callPackage also makes it possible to configure "Nix derivation expressions" after callPackage has been called using overrides. (This email goes has the full rationale.)

^{"Nix derivation expression" is just a personal term, not an official one, that I use to refer to an attribute set that represents a store derivation in terms of the Nix language before instantiation.}

Level 1. Wordier and still inexact

callPackage accepts 2 inputs:

1. a function that takes as input an attribute set
   ^{e.g., { stdenv, miez, lofa ? 27 }: { inherit stdenv miez lofa; }}

2. an attribute set that must be a subset of the input function's input attribute set
   ^{e.g., in the case of the example in item 1., this would be ok: { miez = 7; lofa = 9; }}

callPackage will

1. call the input function with an attribute set whose attributes are supplied from

   • the attribute set in callPackage's 2^nd argument
   • a subset of the Nixpkgs package (attribute) set¹

2. make the resulting attribute set overridable.

Level 2. More precise but also more abstruse

callPackage ::    ( attr_set_function | path_to_file_with_attr_set_function )
               -> constrained_attr_set
               -> ( overridable_attr_set | arbitrary_nix_value )

attr_set_function :: input_attr_set -> attr_set

^{The Nix language is dynamically typed, so this is just a personal notation.}

where

• input_attr_set is a regular attribute set with caveats depending on whether its attributes are Nixpkgs or not:

  • IN Nixpkgs
    If the attribute names match attributes in the limited subset of Nixpkgs package set¹, then these won't have to be specified in constrained_attr_set as callPackage will auto-populate them.

  • NOT in Nixpkgs
    These either must have a default value or have to be specified in constrained_attr_set.

• constrained_attr_set is an attribute set that can only be a subset of attr_set_function's input attribute set.

• overridable_att_set - See this section of the nix.dev tutorial.

Level 3. Source

https://github.com/NixOS/nixpkgs/blame/e7f2456df49b39e816e9ed71622c09a42446a581/pkgs/top-level/splice.nix#L125-L144

  splicedPackagesWithXorg = splicedPackages // builtins.removeAttrs splicedPackages.xorg [
    "callPackage"
    "newScope"
    "overrideScope"
    "packages"
  ];

  # ...

  callPackage = pkgs.newScope { };
  callPackages = ...;
  newScope = extra: lib.callPackageWith (splicedPackagesWithXorg // extra);

^{callPackages (note the extra s!) is also defined here, but omitted it, lest I muddy the waters even more.}

callPackageWith is quite long, so only posting the link, but the resources in the question explain the nitty-gritty - and the core of it is this line:

callPackage = f: args:
  f ((builtins.intersectAttrs (builtins.functionArgs f) pkgs) // args);

^{... and this is an email from 2009 that introduces it.}

Examples

NOTE
The term pkgs below will refer to the Nixpkgs package set. Such as the result of any of the following calls in nix repl:

• pkgs = import <nixpkgs> {}
• pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/06278c77b5d162e62df170fec307e83f1812d94b.tar.gz") {}
• ... and so on

• ✅

  pkgs.callPackage ({ lib, stdenv }: { ... }) {}
  

  lib and stdenv are valid attributes in Nixpkgs, so these will be auto-populated from there. Roughly equivalent to:
  pkgs.callPackage ({ lib, stdenv }: ... }) { lib = pkgs.lib; stdenv = pkgs.stdenv; }

• ✅

  pkgs.callPackage ({ lib, stdenv, lofa ? 27 }: { ... }) {}
  

  Same things apply as in the previous item, except that Nixpkgs has no lofa attribute, but there is a default value provided for it, so it will work.

• ✅

  ( pkgs.callPackage 
      ({ lib, stdenv, lofa }: { inherit lofa lib; })
      { lofa = 27; lib = "whatever"; }
  )
  

  Same as in previous item, but attributes lofa and lib are explicitly provided.

• ✅

   let
     id = x: x;
   in
     pkgs.callPackage id {}
  

  callPackage's input function (1st parameter) is called with an attribute set, so this still works.

• ❌

  pkgs.callPackage (i: i + 2) {}
  

  callPackage will choke because i will be an attribute set and the + operator will try to coerce it into a string.

• ❌

  pkgs.callPackage ({ lib, stdenv, lofa }: { inherit lofa; }) {}
  

  will raise error: Function called without required argument "lofa".

• ✅

  $ echo "{lib, stdenv, lofa ? 27}: {inherit lofa;}" > f.nix
  
  $ nix repl
  
  nix-repl> pkgs = import <nixpkgs> {}
  nix-repl> pkgs.callPackage ./f.nix {}
  

  Providing a valid file system path containing a Nix expression (that conforms to what callPackage is accepting) works too as the file will be automatically imported first.

The result of attr_set_function is usually a Nix derivation expression (which is an attribute set denoting a derivation - or something like it; look this up).

[1]: https://github.com/NixOS/nixpkgs/blame/e7f2456df49b39e816e9ed71622c09a42446a581/pkgs/top-level/splice.nix#L140-L144

posted 2 months ago

CC0

2mo ago

toraritte‭

640 reputation 32 16 99 53