From 83db0205060ce14098247dc2969338b7bdadced9 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Mon, 21 Jan 2019 11:25:32 +0100 Subject: doc: Move "System Installation" right after "Installation". * doc/guix.texi (System Installation): Move right after "Installation". --- doc/guix.texi | 12629 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 6315 insertions(+), 6314 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index eb0c5fc809..547ab8db8c 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -117,10 +117,10 @@ Project}. @menu * Introduction:: What is Guix about? * Installation:: Installing Guix. +* System Installation:: Installing the whole operating system. * Package Management:: Package installation, upgrade, etc. * Programming Interface:: Using Guix in Scheme. * Utilities:: Package management commands. -* System Installation:: Installing the whole operating system. * System Configuration:: Configuring the operating system. * Documentation:: Browsing software user manuals. * Installing Debugging Files:: Feeding the debugger. @@ -154,6 +154,16 @@ Setting Up the Daemon * Daemon Offload Setup:: Offloading builds to remote machines. * SELinux Support:: Using an SELinux policy for the daemon. +System Installation + +* Limitations:: What you can expect. +* Hardware Considerations:: Supported hardware. +* USB Stick and DVD Installation:: Preparing the installation medium. +* Preparing for Installation:: Networking, partitioning, etc. +* Proceeding with the Installation:: The real thing. +* Installing GuixSD in a VM:: GuixSD playground. +* Building the Installation Image:: How this comes to be. + Package Management * Features:: How Guix will make your life brighter. @@ -218,16 +228,6 @@ Invoking @command{guix build} * Additional Build Options:: Options specific to 'guix build'. * Debugging Build Failures:: Real life packaging experience. -System Installation - -* Limitations:: What you can expect. -* Hardware Considerations:: Supported hardware. -* USB Stick and DVD Installation:: Preparing the installation medium. -* Preparing for Installation:: Networking, partitioning, etc. -* Proceeding with the Installation:: The real thing. -* Installing GuixSD in a VM:: GuixSD playground. -* Building the Installation Image:: How this comes to be. - System Configuration * Using the Configuration System:: Customizing your GNU system. @@ -1745,8003 +1745,8004 @@ store you need to define the environment variable @c TODO What else? @c ********************************************************************* -@node Package Management -@chapter Package Management +@node System Installation +@chapter System Installation -@cindex packages -The purpose of GNU Guix is to allow users to easily install, upgrade, and -remove software packages, without having to know about their build -procedures or dependencies. Guix also goes beyond this obvious set of -features. +@cindex installing GuixSD +@cindex Guix System Distribution +This section explains how to install the Guix System Distribution (GuixSD) +on a machine. The Guix package manager can +also be installed on top of a running GNU/Linux system, +@pxref{Installation}. -This chapter describes the main features of Guix, as well as the -package management tools it provides. Along with the command-line -interface described below (@pxref{Invoking guix package, @code{guix -package}}), you may also use the Emacs-Guix interface (@pxref{Top,,, -emacs-guix, The Emacs-Guix Reference Manual}), after installing -@code{emacs-guix} package (run @kbd{M-x guix-help} command to start -with it): +@ifinfo +@quotation Note +@c This paragraph is for people reading this from tty2 of the +@c installation image. +You are reading this documentation with an Info reader. For details on +how to use it, hit the @key{RET} key (``return'' or ``enter'') on the +link that follows: @pxref{Top, Info reader,, info-stnd, Stand-alone GNU +Info}. Hit @kbd{l} afterwards to come back here. -@example -guix package -i emacs-guix -@end example +Alternately, run @command{info info} in another tty to keep the manual +available. +@end quotation +@end ifinfo @menu -* Features:: How Guix will make your life brighter. -* Invoking guix package:: Package installation, removal, etc. -* Substitutes:: Downloading pre-built binaries. -* Packages with Multiple Outputs:: Single source package, multiple outputs. -* Invoking guix gc:: Running the garbage collector. -* Invoking guix pull:: Fetching the latest Guix and distribution. -* Channels:: Customizing the package collection. -* Inferiors:: Interacting with another revision of Guix. -* Invoking guix describe:: Display information about your Guix revision. -* Invoking guix pack:: Creating software bundles. -* Invoking guix archive:: Exporting and importing store files. +* Limitations:: What you can expect. +* Hardware Considerations:: Supported hardware. +* USB Stick and DVD Installation:: Preparing the installation medium. +* Preparing for Installation:: Networking, partitioning, etc. +* Proceeding with the Installation:: The real thing. +* Installing GuixSD in a VM:: GuixSD playground. +* Building the Installation Image:: How this comes to be. @end menu -@node Features -@section Features +@node Limitations +@section Limitations -When using Guix, each package ends up in the @dfn{package store}, in its -own directory---something that resembles -@file{/gnu/store/xxx-package-1.2}, where @code{xxx} is a base32 string. +As of version @value{VERSION}, the Guix System Distribution (GuixSD) is +not production-ready. It may contain bugs and lack important +features. Thus, if you are looking for a stable production system that +respects your freedom as a computer user, a good solution at this point +is to consider @url{http://www.gnu.org/distros/free-distros.html, one of +the more established GNU/Linux distributions}. We hope you can soon switch +to the GuixSD without fear, of course. In the meantime, you can +also keep using your distribution and try out the package manager on top +of it (@pxref{Installation}). -Instead of referring to these directories, users have their own -@dfn{profile}, which points to the packages that they actually want to -use. These profiles are stored within each user's home directory, at -@code{$HOME/.guix-profile}. +Before you proceed with the installation, be aware of the following +noteworthy limitations applicable to version @value{VERSION}: -For example, @code{alice} installs GCC 4.7.2. As a result, -@file{/home/alice/.guix-profile/bin/gcc} points to -@file{/gnu/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine, -@code{bob} had already installed GCC 4.8.0. The profile of @code{bob} -simply continues to point to -@file{/gnu/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC -coexist on the same system without any interference. +@itemize +@item +The installation process does not include a graphical user interface and +requires familiarity with GNU/Linux (see the following subsections to +get a feel of what that means.) -The @command{guix package} command is the central tool to manage -packages (@pxref{Invoking guix package}). It operates on the per-user -profiles, and can be used @emph{with normal user privileges}. +@item +Support for the Logical Volume Manager (LVM) is missing. -@cindex transactions -The command provides the obvious install, remove, and upgrade -operations. Each invocation is actually a @emph{transaction}: either -the specified operation succeeds, or nothing happens. Thus, if the -@command{guix package} process is terminated during the transaction, -or if a power outage occurs during the transaction, then the user's -profile remains in its previous state, and remains usable. +@item +More and more system services are provided (@pxref{Services}), but some +may be missing. -In addition, any package transaction may be @emph{rolled back}. So, if, -for example, an upgrade installs a new version of a package that turns -out to have a serious bug, users may roll back to the previous instance -of their profile, which was known to work well. Similarly, the global -system configuration on GuixSD is subject to -transactional upgrades and roll-back -(@pxref{Using the Configuration System}). +@item +More than 8,500 packages are available, but you might +occasionally find that a useful package is missing. -All packages in the package store may be @emph{garbage-collected}. -Guix can determine which packages are still referenced by user -profiles, and remove those that are provably no longer referenced -(@pxref{Invoking guix gc}). Users may also explicitly remove old -generations of their profile so that the packages they refer to can be -collected. +@item +GNOME, Xfce, LXDE, and Enlightenment are available (@pxref{Desktop Services}), +as well as a number of X11 window managers. However, some graphical +applications may be missing, as well as KDE. +@end itemize -@cindex reproducibility -@cindex reproducible builds -Guix takes a @dfn{purely functional} approach to package -management, as described in the introduction (@pxref{Introduction}). -Each @file{/gnu/store} package directory name contains a hash of all the -inputs that were used to build that package---compiler, libraries, build -scripts, etc. This direct correspondence allows users to make sure a -given package installation matches the current state of their -distribution. It also helps maximize @dfn{build reproducibility}: -thanks to the isolated build environments that are used, a given build -is likely to yield bit-identical files when performed on different -machines (@pxref{Invoking guix-daemon, container}). +You have been warned! But more than a disclaimer, this is an invitation +to report issues (and success stories!), and to join us in improving it. +@xref{Contributing}, for more info. -@cindex substitutes -This foundation allows Guix to support @dfn{transparent binary/source -deployment}. When a pre-built binary for a @file{/gnu/store} item is -available from an external source---a @dfn{substitute}, Guix just -downloads it and unpacks it; -otherwise, it builds the package from source, locally -(@pxref{Substitutes}). Because build results are usually bit-for-bit -reproducible, users do not have to trust servers that provide -substitutes: they can force a local build and @emph{challenge} providers -(@pxref{Invoking guix challenge}). -Control over the build environment is a feature that is also useful for -developers. The @command{guix environment} command allows developers of -a package to quickly set up the right development environment for their -package, without having to manually install the dependencies of the -package into their profile (@pxref{Invoking guix environment}). +@node Hardware Considerations +@section Hardware Considerations -@cindex replication, of software environments -@cindex provenance tracking, of software artifacts -All of Guix and its package definitions is version-controlled, and -@command{guix pull} allows you to ``travel in time'' on the history of Guix -itself (@pxref{Invoking guix pull}). This makes it possible to replicate a -Guix instance on a different machine or at a later point in time, which in -turn allows you to @emph{replicate complete software environments}, while -retaining precise @dfn{provenance tracking} of the software. +@cindex hardware support on GuixSD +GNU@tie{}GuixSD focuses on respecting the user's computing freedom. It +builds around the kernel Linux-libre, which means that only hardware for +which free software drivers and firmware exist is supported. Nowadays, +a wide range of off-the-shelf hardware is supported on +GNU/Linux-libre---from keyboards to graphics cards to scanners and +Ethernet controllers. Unfortunately, there are still areas where +hardware vendors deny users control over their own computing, and such +hardware is not supported on GuixSD. -@node Invoking guix package -@section Invoking @command{guix package} +@cindex WiFi, hardware support +One of the main areas where free drivers or firmware are lacking is WiFi +devices. WiFi devices known to work include those using Atheros chips +(AR9271 and AR7010), which corresponds to the @code{ath9k} Linux-libre +driver, and those using Broadcom/AirForce chips (BCM43xx with +Wireless-Core Revision 5), which corresponds to the @code{b43-open} +Linux-libre driver. Free firmware exists for both and is available +out-of-the-box on GuixSD, as part of @var{%base-firmware} +(@pxref{operating-system Reference, @code{firmware}}). -@cindex installing packages -@cindex removing packages -@cindex package installation -@cindex package removal -The @command{guix package} command is the tool that allows users to -install, upgrade, and remove packages, as well as rolling back to -previous configurations. It operates only on the user's own profile, -and works with normal user privileges (@pxref{Features}). Its syntax -is: +@cindex RYF, Respects Your Freedom +The @uref{https://www.fsf.org/, Free Software Foundation} runs +@uref{https://www.fsf.org/ryf, @dfn{Respects Your Freedom}} (RYF), a +certification program for hardware products that respect your freedom +and your privacy and ensure that you have control over your device. We +encourage you to check the list of RYF-certified devices. -@example -guix package @var{options} -@end example -@cindex transactions -Primarily, @var{options} specifies the operations to be performed during -the transaction. Upon completion, a new profile is created, but -previous @dfn{generations} of the profile remain available, should the user -want to roll back. +Another useful resource is the @uref{https://www.h-node.org/, H-Node} +web site. It contains a catalog of hardware devices with information +about their support in GNU/Linux. -For example, to remove @code{lua} and install @code{guile} and -@code{guile-cairo} in a single transaction: -@example -guix package -r lua -i guile guile-cairo -@end example +@node USB Stick and DVD Installation +@section USB Stick and DVD Installation -@command{guix package} also supports a @dfn{declarative approach} -whereby the user specifies the exact set of packages to be available and -passes it @i{via} the @option{--manifest} option -(@pxref{profile-manifest, @option{--manifest}}). +An ISO-9660 installation image that can be written to a USB stick or +burnt to a DVD can be downloaded from +@indicateurl{https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz}, +where @var{system} is one of: -@cindex profile -For each user, a symlink to the user's default profile is automatically -created in @file{$HOME/.guix-profile}. This symlink always points to the -current generation of the user's default profile. Thus, users can add -@file{$HOME/.guix-profile/bin} to their @code{PATH} environment -variable, and so on. -@cindex search paths -If you are not using the Guix System Distribution, consider adding the -following lines to your @file{~/.bash_profile} (@pxref{Bash Startup -Files,,, bash, The GNU Bash Reference Manual}) so that newly-spawned -shells get all the right environment variable definitions: +@table @code +@item x86_64-linux +for a GNU/Linux system on Intel/AMD-compatible 64-bit CPUs; + +@item i686-linux +for a 32-bit GNU/Linux system on Intel-compatible CPUs. +@end table + +@c start duplication of authentication part from ``Binary Installation'' +Make sure to download the associated @file{.sig} file and to verify the +authenticity of the image against it, along these lines: @example -GUIX_PROFILE="$HOME/.guix-profile" ; \ -source "$HOME/.guix-profile/etc/profile" +$ wget https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig +$ gpg --verify guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig @end example -In a multi-user setup, user profiles are stored in a place registered as -a @dfn{garbage-collector root}, which @file{$HOME/.guix-profile} points -to (@pxref{Invoking guix gc}). That directory is normally -@code{@var{localstatedir}/guix/profiles/per-user/@var{user}}, where -@var{localstatedir} is the value passed to @code{configure} as -@code{--localstatedir}, and @var{user} is the user name. The -@file{per-user} directory is created when @command{guix-daemon} is -started, and the @var{user} sub-directory is created by @command{guix -package}. +If that command fails because you do not have the required public key, +then run this command to import it: -The @var{options} can be among the following: +@example +$ gpg --keyserver @value{KEY-SERVER} \ + --recv-keys @value{OPENPGP-SIGNING-KEY-ID} +@end example -@table @code +@noindent +and rerun the @code{gpg --verify} command. +@c end duplication -@item --install=@var{package} @dots{} -@itemx -i @var{package} @dots{} -Install the specified @var{package}s. +This image contains the tools necessary for an installation. +It is meant to be copied @emph{as is} to a large-enough USB stick or DVD. -Each @var{package} may specify either a simple package name, such as -@code{guile}, or a package name followed by an at-sign and version number, -such as @code{guile@@1.8.8} or simply @code{guile@@1.8} (in the latter -case, the newest version prefixed by @code{1.8} is selected.) +@unnumberedsubsec Copying to a USB Stick -If no version number is specified, the -newest available version will be selected. In addition, @var{package} -may contain a colon, followed by the name of one of the outputs of the -package, as in @code{gcc:doc} or @code{binutils@@2.22:lib} -(@pxref{Packages with Multiple Outputs}). Packages with a corresponding -name (and optionally version) are searched for among the GNU -distribution modules (@pxref{Package Modules}). +To copy the image to a USB stick, follow these steps: -@cindex propagated inputs -Sometimes packages have @dfn{propagated inputs}: these are dependencies -that automatically get installed along with the required package -(@pxref{package-propagated-inputs, @code{propagated-inputs} in -@code{package} objects}, for information about propagated inputs in -package definitions). +@enumerate +@item +Decompress the image using the @command{xz} command: -@anchor{package-cmd-propagated-inputs} -An example is the GNU MPC library: its C header files refer to those of -the GNU MPFR library, which in turn refer to those of the GMP library. -Thus, when installing MPC, the MPFR and GMP libraries also get installed -in the profile; removing MPC also removes MPFR and GMP---unless they had -also been explicitly installed by the user. +@example +xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz +@end example -Besides, packages sometimes rely on the definition of environment -variables for their search paths (see explanation of -@code{--search-paths} below). Any missing or possibly incorrect -environment variable definitions are reported here. +@item +Insert a USB stick of 1@tie{}GiB or more into your machine, and determine +its device name. Assuming that the USB stick is known as @file{/dev/sdX}, +copy the image with: -@item --install-from-expression=@var{exp} -@itemx -e @var{exp} -Install the package @var{exp} evaluates to. +@example +dd if=guixsd-install-@value{VERSION}.@var{system}.iso of=/dev/sdX +sync +@end example -@var{exp} must be a Scheme expression that evaluates to a -@code{} object. This option is notably useful to disambiguate -between same-named variants of a package, with expressions such as -@code{(@@ (gnu packages base) guile-final)}. +Access to @file{/dev/sdX} usually requires root privileges. +@end enumerate -Note that this option installs the first output of the specified -package, which may be insufficient when needing a specific output of a -multiple-output package. +@unnumberedsubsec Burning on a DVD -@item --install-from-file=@var{file} -@itemx -f @var{file} -Install the package that the code within @var{file} evaluates to. +To copy the image to a DVD, follow these steps: -As an example, @var{file} might contain a definition like this -(@pxref{Defining Packages}): +@enumerate +@item +Decompress the image using the @command{xz} command: @example -@verbatiminclude package-hello.scm +xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz @end example -Developers may find it useful to include such a @file{guix.scm} file -in the root of their project source tree that can be used to test -development snapshots and create reproducible development environments -(@pxref{Invoking guix environment}). +@item +Insert a blank DVD into your machine, and determine +its device name. Assuming that the DVD drive is known as @file{/dev/srX}, +copy the image with: -@item --remove=@var{package} @dots{} -@itemx -r @var{package} @dots{} -Remove the specified @var{package}s. +@example +growisofs -dvd-compat -Z /dev/srX=guixsd-install-@value{VERSION}.@var{system}.iso +@end example -As for @code{--install}, each @var{package} may specify a version number -and/or output name in addition to the package name. For instance, -@code{-r glibc:debug} would remove the @code{debug} output of -@code{glibc}. +Access to @file{/dev/srX} usually requires root privileges. +@end enumerate -@item --upgrade[=@var{regexp} @dots{}] -@itemx -u [@var{regexp} @dots{}] -@cindex upgrading packages -Upgrade all the installed packages. If one or more @var{regexp}s are -specified, upgrade only installed packages whose name matches a -@var{regexp}. Also see the @code{--do-not-upgrade} option below. +@unnumberedsubsec Booting -Note that this upgrades package to the latest version of packages found -in the distribution currently installed. To update your distribution, -you should regularly run @command{guix pull} (@pxref{Invoking guix -pull}). +Once this is done, you should be able to reboot the system and boot from +the USB stick or DVD. The latter usually requires you to get in the +BIOS or UEFI boot menu, where you can choose to boot from the USB stick. -@item --do-not-upgrade[=@var{regexp} @dots{}] -When used together with the @code{--upgrade} option, do @emph{not} -upgrade any packages whose name matches a @var{regexp}. For example, to -upgrade all packages in the current profile except those containing the -substring ``emacs'': +@xref{Installing GuixSD in a VM}, if, instead, you would like to install +GuixSD in a virtual machine (VM). -@example -$ guix package --upgrade . --do-not-upgrade emacs -@end example -@item @anchor{profile-manifest}--manifest=@var{file} -@itemx -m @var{file} -@cindex profile declaration -@cindex profile manifest -Create a new generation of the profile from the manifest object -returned by the Scheme code in @var{file}. +@node Preparing for Installation +@section Preparing for Installation -This allows you to @emph{declare} the profile's contents rather than -constructing it through a sequence of @code{--install} and similar -commands. The advantage is that @var{file} can be put under version -control, copied to different machines to reproduce the same profile, and -so on. +Once you have successfully booted your computer using the installation medium, +you should end up with the welcome page of the graphical installer. The +graphical installer is a text-based user interface built upon the newt +library. It shall guide you through all the different steps needed to install +GNU GuixSD. However, as the graphical installer is still under heavy +development, you might want to fallback to the original, shell based install +process, by switching to TTYs 3 to 6 with the shortcuts CTRL-ALT-F[3-6]. The +following sections describe the installation procedure assuming you're using +one of those TTYs. They are configured and can be used to run commands as +root. -@c FIXME: Add reference to (guix profile) documentation when available. -@var{file} must return a @dfn{manifest} object, which is roughly a list -of packages: +TTY2 shows this documentation, browsable using the Info reader commands +(@pxref{Top,,, info-stnd, Stand-alone GNU Info}). The installation system +runs the GPM mouse daemon, which allows you to select text with the left mouse +button and to paste it with the middle button. -@findex packages->manifest -@example -(use-package-modules guile emacs) +@quotation Note +Installation requires access to the Internet so that any missing +dependencies of your system configuration can be downloaded. See the +``Networking'' section below. +@end quotation -(packages->manifest - (list emacs - guile-2.0 - ;; Use a specific package output. - (list guile-2.0 "debug"))) -@end example +The installation system includes many common tools needed for this task. +But it is also a full-blown GuixSD system, which means that you can +install additional packages, should you need it, using @command{guix +package} (@pxref{Invoking guix package}). -@findex specifications->manifest -In this example we have to know which modules define the @code{emacs} -and @code{guile-2.0} variables to provide the right -@code{use-package-modules} line, which can be cumbersome. We can -instead provide regular package specifications and let -@code{specifications->manifest} look up the corresponding package -objects, like this: +@subsection Keyboard Layout + +@cindex keyboard layout +The installation image uses the US qwerty keyboard layout. If you want +to change it, you can use the @command{loadkeys} command. For example, +the following command selects the Dvorak keyboard layout: @example -(specifications->manifest - '("emacs" "guile@@2.2" "guile@@2.2:debug")) +loadkeys dvorak @end example -@item --roll-back -@cindex rolling back -@cindex undoing transactions -@cindex transactions, undoing -Roll back to the previous @dfn{generation} of the profile---i.e., undo -the last transaction. +See the files under @file{/run/current-system/profile/share/keymaps} for +a list of available keyboard layouts. Run @command{man loadkeys} for +more information. -When combined with options such as @code{--install}, roll back occurs -before any other actions. +@subsection Networking -When rolling back from the first generation that actually contains -installed packages, the profile is made to point to the @dfn{zeroth -generation}, which contains no files apart from its own metadata. +Run the following command to see what your network interfaces are called: -After having rolled back, installing, removing, or upgrading packages -overwrites previous future generations. Thus, the history of the -generations in a profile is always linear. +@example +ifconfig -a +@end example -@item --switch-generation=@var{pattern} -@itemx -S @var{pattern} -@cindex generations -Switch to a particular generation defined by @var{pattern}. +@noindent +@dots{} or, using the GNU/Linux-specific @command{ip} command: -@var{pattern} may be either a generation number or a number prefixed -with ``+'' or ``-''. The latter means: move forward/backward by a -specified number of generations. For example, if you want to return to -the latest generation after @code{--roll-back}, use -@code{--switch-generation=+1}. +@example +ip a +@end example -The difference between @code{--roll-back} and -@code{--switch-generation=-1} is that @code{--switch-generation} will -not make a zeroth generation, so if a specified generation does not -exist, the current generation will not be changed. +@c http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-net_id.c#n20 +Wired interfaces have a name starting with @samp{e}; for example, the +interface corresponding to the first on-board Ethernet controller is +called @samp{eno1}. Wireless interfaces have a name starting with +@samp{w}, like @samp{w1p2s0}. -@item --search-paths[=@var{kind}] -@cindex search paths -Report environment variable definitions, in Bash syntax, that may be -needed in order to use the set of installed packages. These environment -variables are used to specify @dfn{search paths} for files used by some -of the installed packages. +@table @asis +@item Wired connection +To configure a wired network run the following command, substituting +@var{interface} with the name of the wired interface you want to use. -For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH} -environment variables to be defined so it can look for headers and -libraries in the user's profile (@pxref{Environment Variables,,, gcc, -Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C -library are installed in the profile, then @code{--search-paths} will -suggest setting these variables to @code{@var{profile}/include} and -@code{@var{profile}/lib}, respectively. +@example +ifconfig @var{interface} up +@end example -The typical use case is to define these environment variables in the -shell: +@item Wireless connection +@cindex wireless +@cindex WiFi +To configure wireless networking, you can create a configuration file +for the @command{wpa_supplicant} configuration tool (its location is not +important) using one of the available text editors such as +@command{nano}: @example -$ eval `guix package --search-paths` +nano wpa_supplicant.conf @end example -@var{kind} may be one of @code{exact}, @code{prefix}, or @code{suffix}, -meaning that the returned environment variable definitions will either -be exact settings, or prefixes or suffixes of the current value of these -variables. When omitted, @var{kind} defaults to @code{exact}. +As an example, the following stanza can go to this file and will work +for many wireless networks, provided you give the actual SSID and +passphrase for the network you are connecting to: -This option can also be used to compute the @emph{combined} search paths -of several profiles. Consider this example: +@example +network=@{ + ssid="@var{my-ssid}" + key_mgmt=WPA-PSK + psk="the network's secret passphrase" +@} +@end example + +Start the wireless service and run it in the background with the +following command (substitute @var{interface} with the name of the +network interface you want to use): @example -$ guix package -p foo -i guile -$ guix package -p bar -i guile-json -$ guix package -p foo -p bar --search-paths +wpa_supplicant -c wpa_supplicant.conf -i @var{interface} -B @end example -The last command above reports about the @code{GUILE_LOAD_PATH} -variable, even though, taken individually, neither @file{foo} nor -@file{bar} would lead to that recommendation. +Run @command{man wpa_supplicant} for more information. +@end table +@cindex DHCP +At this point, you need to acquire an IP address. On a network where IP +addresses are automatically assigned @i{via} DHCP, you can run: -@item --profile=@var{profile} -@itemx -p @var{profile} -Use @var{profile} instead of the user's default profile. +@example +dhclient -v @var{interface} +@end example -@cindex collisions, in a profile -@cindex colliding packages in profiles -@cindex profile collisions -@item --allow-collisions -Allow colliding packages in the new profile. Use at your own risk! +Try to ping a server to see if networking is up and running: -By default, @command{guix package} reports as an error @dfn{collisions} -in the profile. Collisions happen when two or more different versions -or variants of a given package end up in the profile. +@example +ping -c 3 gnu.org +@end example -@item --bootstrap -Use the bootstrap Guile to build the profile. This option is only -useful to distribution developers. +Setting up network access is almost always a requirement because the +image does not contain all the software and tools that may be needed. -@end table +@cindex installing over SSH +If you want to, you can continue the installation remotely by starting +an SSH server: -In addition to these actions, @command{guix package} supports the -following options to query the current state of a profile, or the -availability of packages: +@example +herd start ssh-daemon +@end example -@table @option +Make sure to either set a password with @command{passwd}, or configure +OpenSSH public key authentication before logging in. -@item --search=@var{regexp} -@itemx -s @var{regexp} -@cindex searching for packages -List the available packages whose name, synopsis, or description matches -@var{regexp} (in a case-insensitive fashion), sorted by relevance. -Print all the metadata of matching packages in -@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, -GNU recutils manual}). +@subsection Disk Partitioning -This allows specific fields to be extracted using the @command{recsel} -command, for instance: +Unless this has already been done, the next step is to partition, and +then format the target partition(s). + +The installation image includes several partitioning tools, including +Parted (@pxref{Overview,,, parted, GNU Parted User Manual}), +@command{fdisk}, and @command{cfdisk}. Run it and set up your disk with +the partition layout you want: @example -$ guix package -s malloc | recsel -p name,version,relevance -name: jemalloc -version: 4.5.0 -relevance: 6 +cfdisk +@end example -name: glibc -version: 2.25 -relevance: 1 +If your disk uses the GUID Partition Table (GPT) format and you plan to +install BIOS-based GRUB (which is the default), make sure a BIOS Boot +Partition is available (@pxref{BIOS installation,,, grub, GNU GRUB +manual}). -name: libgc -version: 7.6.0 -relevance: 1 +@cindex EFI, installation +@cindex UEFI, installation +@cindex ESP, EFI system partition +If you instead wish to use EFI-based GRUB, a FAT32 @dfn{EFI System Partition} +(ESP) is required. This partition should be mounted at @file{/boot/efi} and +must have the @code{esp} flag set. E.g., for @command{parted}: + +@example +parted /dev/sda set 1 esp on @end example -Similarly, to show the name of all the packages available under the -terms of the GNU@tie{}LGPL version 3: +@quotation Note +@vindex grub-bootloader +@vindex grub-efi-bootloader +Unsure whether to use EFI- or BIOS-based GRUB? If the directory +@file{/sys/firmware/efi} exists in the installation image, then you should +probably perform an EFI installation, using @code{grub-efi-bootloader}. +Otherwise you should use the BIOS-based GRUB, known as +@code{grub-bootloader}. @xref{Bootloader Configuration}, for more info on +bootloaders. +@end quotation -@example -$ guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"' -name: elfutils +Once you are done partitioning the target hard disk drive, you have to +create a file system on the relevant partition(s)@footnote{Currently +GuixSD only supports ext4 and btrfs file systems. In particular, code +that reads file system UUIDs and labels only works for these file system +types.}. For the ESP, if you have one and assuming it is +@file{/dev/sda1}, run: -name: gmp -@dots{} +@example +mkfs.fat -F32 /dev/sda1 @end example -It is also possible to refine search results using several @code{-s} -flags. For example, the following command returns a list of board -games: +Preferably, assign file systems a label so that you can easily and +reliably refer to them in @code{file-system} declarations (@pxref{File +Systems}). This is typically done using the @code{-L} option of +@command{mkfs.ext4} and related commands. So, assuming the target root +partition lives at @file{/dev/sda2}, a file system with the label +@code{my-root} can be created with: @example -$ guix package -s '\' -s game | recsel -p name -name: gnubg -@dots{} +mkfs.ext4 -L my-root /dev/sda2 @end example -If we were to omit @code{-s game}, we would also get software packages -that deal with printed circuit boards; removing the angle brackets -around @code{board} would further add packages that have to do with -keyboards. - -And now for a more elaborate example. The following command searches -for cryptographic libraries, filters out Haskell, Perl, Python, and Ruby -libraries, and prints the name and synopsis of the matching packages: +@cindex encrypted disk +If you are instead planning to encrypt the root partition, you can use +the Cryptsetup/LUKS utilities to do that (see @inlinefmtifelse{html, +@uref{https://linux.die.net/man/8/cryptsetup, @code{man cryptsetup}}, +@code{man cryptsetup}} for more information.) Assuming you want to +store the root partition on @file{/dev/sda2}, the command sequence would +be along these lines: @example -$ guix package -s crypto -s library | \ - recsel -e '! (name ~ "^(ghc|perl|python|ruby)")' -p name,synopsis +cryptsetup luksFormat /dev/sda2 +cryptsetup open --type luks /dev/sda2 my-partition +mkfs.ext4 -L my-root /dev/mapper/my-partition @end example -@noindent -@xref{Selection Expressions,,, recutils, GNU recutils manual}, for more -information on @dfn{selection expressions} for @code{recsel -e}. - -@item --show=@var{package} -Show details about @var{package}, taken from the list of available packages, in -@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, GNU -recutils manual}). +Once that is done, mount the target file system under @file{/mnt} +with a command like (again, assuming @code{my-root} is the label of the +root file system): @example -$ guix package --show=python | recsel -p name,version -name: python -version: 2.7.6 - -name: python -version: 3.3.5 +mount LABEL=my-root /mnt @end example -You may also specify the full name of a package to only get details about a -specific version of it: +Also mount any other file systems you would like to use on the target +system relative to this path. If you have @file{/boot} on a separate +partition for example, mount it at @file{/mnt/boot} now so it is found +by @code{guix system init} afterwards. + +Finally, if you plan to use one or more swap partitions (@pxref{Memory +Concepts, swap space,, libc, The GNU C Library Reference Manual}), make +sure to initialize them with @command{mkswap}. Assuming you have one +swap partition on @file{/dev/sda3}, you would run: + @example -$ guix package --show=python@@3.4 | recsel -p name,version -name: python -version: 3.4.3 +mkswap /dev/sda3 +swapon /dev/sda3 @end example +Alternatively, you may use a swap file. For example, assuming that in +the new system you want to use the file @file{/swapfile} as a swap file, +you would run@footnote{This example will work for many types of file +systems (e.g., ext4). However, for copy-on-write file systems (e.g., +btrfs), the required steps may be different. For details, see the +manual pages for @command{mkswap} and @command{swapon}.}: +@example +# This is 10 GiB of swap space. Adjust "count" to change the size. +dd if=/dev/zero of=/mnt/swapfile bs=1MiB count=10240 +# For security, make the file readable and writable only by root. +chmod 600 /mnt/swapfile +mkswap /mnt/swapfile +swapon /mnt/swapfile +@end example -@item --list-installed[=@var{regexp}] -@itemx -I [@var{regexp}] -List the currently installed packages in the specified profile, with the -most recently installed packages shown last. When @var{regexp} is -specified, list only installed packages whose name matches @var{regexp}. +Note that if you have encrypted the root partition and created a swap +file in its file system as described above, then the encryption also +protects the swap file, just like any other file in that file system. -For each installed package, print the following items, separated by -tabs: the package name, its version string, the part of the package that -is installed (for instance, @code{out} for the default output, -@code{include} for its headers, etc.), and the path of this package in -the store. +@node Proceeding with the Installation +@section Proceeding with the Installation -@item --list-available[=@var{regexp}] -@itemx -A [@var{regexp}] -List packages currently available in the distribution for this system -(@pxref{GNU Distribution}). When @var{regexp} is specified, list only -installed packages whose name matches @var{regexp}. +With the target partitions ready and the target root mounted on +@file{/mnt}, we're ready to go. First, run: -For each package, print the following items separated by tabs: its name, -its version string, the parts of the package (@pxref{Packages with -Multiple Outputs}), and the source location of its definition. +@example +herd start cow-store /mnt +@end example -@item --list-generations[=@var{pattern}] -@itemx -l [@var{pattern}] -@cindex generations -Return a list of generations along with their creation dates; for each -generation, show the installed packages, with the most recently -installed packages shown last. Note that the zeroth generation is never -shown. +This makes @file{/gnu/store} copy-on-write, such that packages added to it +during the installation phase are written to the target disk on @file{/mnt} +rather than kept in memory. This is necessary because the first phase of +the @command{guix system init} command (see below) entails downloads or +builds to @file{/gnu/store} which, initially, is an in-memory file system. -For each installed package, print the following items, separated by -tabs: the name of a package, its version string, the part of the package -that is installed (@pxref{Packages with Multiple Outputs}), and the -location of this package in the store. +Next, you have to edit a file and +provide the declaration of the operating system to be installed. To +that end, the installation system comes with three text editors. We +recommend GNU nano (@pxref{Top,,, nano, GNU nano Manual}), which +supports syntax highlighting and parentheses matching; other editors +include GNU Zile (an Emacs clone), and +nvi (a clone of the original BSD @command{vi} editor). +We strongly recommend storing that file on the target root file system, say, +as @file{/mnt/etc/config.scm}. Failing to do that, you will have lost your +configuration file once you have rebooted into the newly-installed system. -When @var{pattern} is used, the command returns only matching -generations. Valid patterns include: +@xref{Using the Configuration System}, for an overview of the +configuration file. The example configurations discussed in that +section are available under @file{/etc/configuration} in the +installation image. Thus, to get started with a system configuration +providing a graphical display server (a ``desktop'' system), you can run +something along these lines: -@itemize -@item @emph{Integers and comma-separated integers}. Both patterns denote -generation numbers. For instance, @code{--list-generations=1} returns -the first one. +@example +# mkdir /mnt/etc +# cp /etc/configuration/desktop.scm /mnt/etc/config.scm +# nano /mnt/etc/config.scm +@end example -And @code{--list-generations=1,8,2} outputs three generations in the -specified order. Neither spaces nor trailing commas are allowed. +You should pay attention to what your configuration file contains, and +in particular: -@item @emph{Ranges}. @code{--list-generations=2..9} prints the -specified generations and everything in between. Note that the start of -a range must be smaller than its end. +@itemize +@item +Make sure the @code{bootloader-configuration} form refers to the target +you want to install GRUB on. It should mention @code{grub-bootloader} if +you are installing GRUB in the legacy way, or @code{grub-efi-bootloader} +for newer UEFI systems. For legacy systems, the @code{target} field +names a device, like @code{/dev/sda}; for UEFI systems it names a path +to a mounted EFI partition, like @code{/boot/efi}, and do make sure the +path is actually mounted. -It is also possible to omit the endpoint. For example, -@code{--list-generations=2..}, returns all generations starting from the -second one. +@item +Be sure that your file system labels match the value of their respective +@code{device} fields in your @code{file-system} configuration, assuming +your @code{file-system} configuration uses the @code{file-system-label} +procedure in its @code{device} field. -@item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks, -or months by passing an integer along with the first letter of the -duration. For example, @code{--list-generations=20d} lists generations -that are up to 20 days old. +@item +If there are encrypted or RAID partitions, make sure to add a +@code{mapped-devices} field to describe them (@pxref{Mapped Devices}). @end itemize -@item --delete-generations[=@var{pattern}] -@itemx -d [@var{pattern}] -When @var{pattern} is omitted, delete all generations except the current -one. - -This command accepts the same patterns as @option{--list-generations}. -When @var{pattern} is specified, delete the matching generations. When -@var{pattern} specifies a duration, generations @emph{older} than the -specified duration match. For instance, @code{--delete-generations=1m} -deletes generations that are more than one month old. - -If the current generation matches, it is @emph{not} deleted. Also, the -zeroth generation is never deleted. - -Note that deleting generations prevents rolling back to them. -Consequently, this command must be used with care. +Once you are done preparing the configuration file, the new system must +be initialized (remember that the target root file system is mounted +under @file{/mnt}): -@end table +@example +guix system init /mnt/etc/config.scm /mnt +@end example -Finally, since @command{guix package} may actually start build -processes, it supports all the common build options (@pxref{Common Build -Options}). It also supports package transformation options, such as -@option{--with-source} (@pxref{Package Transformation Options}). -However, note that package transformations are lost when upgrading; to -preserve transformations across upgrades, you should define your own -package variant in a Guile module and add it to @code{GUIX_PACKAGE_PATH} -(@pxref{Defining Packages}). +@noindent +This copies all the necessary files and installs GRUB on +@file{/dev/sdX}, unless you pass the @option{--no-bootloader} option. For +more information, @pxref{Invoking guix system}. This command may trigger +downloads or builds of missing packages, which can take some time. -@node Substitutes -@section Substitutes +Once that command has completed---and hopefully succeeded!---you can run +@command{reboot} and boot into the new system. The @code{root} password +in the new system is initially empty; other users' passwords need to be +initialized by running the @command{passwd} command as @code{root}, +unless your configuration specifies otherwise +(@pxref{user-account-password, user account passwords}). -@cindex substitutes -@cindex pre-built binaries -Guix supports transparent source/binary deployment, which means that it -can either build things locally, or download pre-built items from a -server, or both. We call these pre-built items @dfn{substitutes}---they -are substitutes for local build results. In many cases, downloading a -substitute is much faster than building things locally. +@cindex upgrading GuixSD +From then on, you can update GuixSD whenever you want by running @command{guix +pull} as @code{root} (@pxref{Invoking guix pull}), and then running +@command{guix system reconfigure /etc/config.scm}, as @code{root} too, to +build a new system generation with the latest packages and services +(@pxref{Invoking guix system}). We recommend doing that regularly so that +your system includes the latest security updates (@pxref{Security Updates}). -Substitutes can be anything resulting from a derivation build -(@pxref{Derivations}). Of course, in the common case, they are -pre-built package binaries, but source tarballs, for instance, which -also result from derivation builds, can be available as substitutes. +Join us on @code{#guix} on the Freenode IRC network or on +@email{guix-devel@@gnu.org} to share your experience---good or not so +good. -@menu -* Official Substitute Server:: One particular source of substitutes. -* Substitute Server Authorization:: How to enable or disable substitutes. -* Substitute Authentication:: How Guix verifies substitutes. -* Proxy Settings:: How to get substitutes via proxy. -* Substitution Failure:: What happens when substitution fails. -* On Trusting Binaries:: How can you trust that binary blob? -@end menu +@node Installing GuixSD in a VM +@section Installing GuixSD in a Virtual Machine -@node Official Substitute Server -@subsection Official Substitute Server +@cindex virtual machine, GuixSD installation +@cindex virtual private server (VPS) +@cindex VPS (virtual private server) +If you'd like to install GuixSD in a virtual machine (VM) or on a +virtual private server (VPS) rather than on your beloved machine, this +section is for you. -@cindex hydra -@cindex build farm -The @code{@value{SUBSTITUTE-SERVER}} server is a front-end to an official build farm -that builds packages from Guix continuously for some -architectures, and makes them available as substitutes. This is the -default source of substitutes; it can be overridden by passing the -@option{--substitute-urls} option either to @command{guix-daemon} -(@pxref{daemon-substitute-urls,, @code{guix-daemon --substitute-urls}}) -or to client tools such as @command{guix package} -(@pxref{client-substitute-urls,, client @option{--substitute-urls} -option}). +To boot a @uref{http://qemu.org/,QEMU} VM for installing GuixSD in a +disk image, follow these steps: -Substitute URLs can be either HTTP or HTTPS. -HTTPS is recommended because communications are encrypted; conversely, -using HTTP makes all communications visible to an eavesdropper, who -could use the information gathered to determine, for instance, whether -your system has unpatched security vulnerabilities. +@enumerate +@item +First, retrieve and decompress the GuixSD installation image as +described previously (@pxref{USB Stick and DVD Installation}). -Substitutes from the official build farm are enabled by default when -using the Guix System Distribution (@pxref{GNU Distribution}). However, -they are disabled by default when using Guix on a foreign distribution, -unless you have explicitly enabled them via one of the recommended -installation steps (@pxref{Installation}). The following paragraphs -describe how to enable or disable substitutes for the official build -farm; the same procedure can also be used to enable substitutes for any -other substitute server. +@item +Create a disk image that will hold the installed system. To make a +qcow2-formatted disk image, use the @command{qemu-img} command: -@node Substitute Server Authorization -@subsection Substitute Server Authorization +@example +qemu-img create -f qcow2 guixsd.img 50G +@end example -@cindex security -@cindex substitutes, authorization thereof -@cindex access control list (ACL), for substitutes -@cindex ACL (access control list), for substitutes -To allow Guix to download substitutes from @code{@value{SUBSTITUTE-SERVER}} or a -mirror thereof, you -must add its public key to the access control list (ACL) of archive -imports, using the @command{guix archive} command (@pxref{Invoking guix -archive}). Doing so implies that you trust @code{@value{SUBSTITUTE-SERVER}} to not -be compromised and to serve genuine substitutes. +The resulting file will be much smaller than 50 GB (typically less than +1 MB), but it will grow as the virtualized storage device is filled up. -The public key for @code{@value{SUBSTITUTE-SERVER}} is installed along with Guix, in -@code{@var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub}, where @var{prefix} is -the installation prefix of Guix. If you installed Guix from source, -make sure you checked the GPG signature of -@file{guix-@value{VERSION}.tar.gz}, which contains this public key file. -Then, you can run something like this: +@item +Boot the USB installation image in an VM: @example -# guix archive --authorize < @var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub +qemu-system-x86_64 -m 1024 -smp 1 \ + -net user -net nic,model=virtio -boot menu=on \ + -drive file=guixsd-install-@value{VERSION}.@var{system}.iso \ + -drive file=guixsd.img @end example -@quotation Note -Similarly, the @file{hydra.gnu.org.pub} file contains the public key -of an independent build farm also run by the project, reachable at -@indicateurl{https://mirror.hydra.gnu.org}. -@end quotation +The ordering of the drives matters. -Once this is in place, the output of a command like @code{guix build} -should change from something like: +In the VM console, quickly press the @kbd{F12} key to enter the boot +menu. Then press the @kbd{2} key and the @kbd{RET} key to validate your +selection. -@example -$ guix build emacs --dry-run -The following derivations would be built: - /gnu/store/yr7bnx8xwcayd6j95r2clmkdl1qh688w-emacs-24.3.drv - /gnu/store/x8qsh1hlhgjx6cwsjyvybnfv2i37z23w-dbus-1.6.4.tar.gz.drv - /gnu/store/1ixwp12fl950d15h2cj11c73733jay0z-alsa-lib-1.0.27.1.tar.bz2.drv - /gnu/store/nlma1pw0p603fpfiqy7kn4zm105r5dmw-util-linux-2.21.drv -@dots{} -@end example +@item +You're now root in the VM, proceed with the installation process. +@xref{Preparing for Installation}, and follow the instructions. +@end enumerate -@noindent -to something like: +Once installation is complete, you can boot the system that's on your +@file{guixsd.img} image. @xref{Running GuixSD in a VM}, for how to do +that. + +@node Building the Installation Image +@section Building the Installation Image + +@cindex installation image +The installation image described above was built using the @command{guix +system} command, specifically: @example -$ guix build emacs --dry-run -112.3 MB would be downloaded: - /gnu/store/pk3n22lbq6ydamyymqkkz7i69wiwjiwi-emacs-24.3 - /gnu/store/2ygn4ncnhrpr61rssa6z0d9x22si0va3-libjpeg-8d - /gnu/store/71yz6lgx4dazma9dwn2mcjxaah9w77jq-cairo-1.12.16 - /gnu/store/7zdhgp0n1518lvfn8mb96sxqfmvqrl7v-libxrender-0.9.7 -@dots{} +guix system disk-image gnu/system/install.scm @end example -@noindent -This indicates that substitutes from @code{@value{SUBSTITUTE-SERVER}} are usable and -will be downloaded, when possible, for future builds. - -@cindex substitutes, how to disable -The substitute mechanism can be disabled globally by running -@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking -guix-daemon}). It can also be disabled temporarily by passing the -@code{--no-substitutes} option to @command{guix package}, @command{guix -build}, and other command-line tools. +Have a look at @file{gnu/system/install.scm} in the source tree, +and see also @ref{Invoking guix system} for more information +about the installation image. -@node Substitute Authentication -@subsection Substitute Authentication +@section Building the Installation Image for ARM Boards -@cindex digital signatures -Guix detects and raises an error when attempting to use a substitute -that has been tampered with. Likewise, it ignores substitutes that are -not signed, or that are not signed by one of the keys listed in the ACL. +Many ARM boards require a specific variant of the +@uref{http://www.denx.de/wiki/U-Boot/, U-Boot} bootloader. -There is one exception though: if an unauthorized server provides -substitutes that are @emph{bit-for-bit identical} to those provided by -an authorized server, then the unauthorized server becomes eligible for -downloads. For example, assume we have chosen two substitute servers -with this option: +If you build a disk image and the bootloader is not available otherwise +(on another boot drive etc), it's advisable to build an image that +includes the bootloader, specifically: @example ---substitute-urls="https://a.example.org https://b.example.org" +guix system disk-image --system=armhf-linux -e '((@@ (gnu system install) os-with-u-boot) (@@ (gnu system install) installation-os) "A20-OLinuXino-Lime2")' @end example -@noindent -@cindex reproducible builds -If the ACL contains only the key for @code{b.example.org}, and if -@code{a.example.org} happens to serve the @emph{exact same} substitutes, -then Guix will download substitutes from @code{a.example.org} because it -comes first in the list and can be considered a mirror of -@code{b.example.org}. In practice, independent build machines usually -produce the same binaries, thanks to bit-reproducible builds (see -below). - -When using HTTPS, the server's X.509 certificate is @emph{not} validated -(in other words, the server is not authenticated), contrary to what -HTTPS clients such as Web browsers usually do. This is because Guix -authenticates substitute information itself, as explained above, which -is what we care about (whereas X.509 certificates are about -authenticating bindings between domain names and public keys.) - -@node Proxy Settings -@subsection Proxy Settings - -@vindex http_proxy -Substitutes are downloaded over HTTP or HTTPS. -The @code{http_proxy} environment -variable can be set in the environment of @command{guix-daemon} and is -honored for downloads of substitutes. Note that the value of -@code{http_proxy} in the environment where @command{guix build}, -@command{guix package}, and other client commands are run has -@emph{absolutely no effect}. +@code{A20-OLinuXino-Lime2} is the name of the board. If you specify an invalid +board, a list of possible boards will be printed. -@node Substitution Failure -@subsection Substitution Failure +@c ********************************************************************* +@node Package Management +@chapter Package Management -Even when a substitute for a derivation is available, sometimes the -substitution attempt will fail. This can happen for a variety of -reasons: the substitute server might be offline, the substitute may -recently have been deleted, the connection might have been interrupted, -etc. +@cindex packages +The purpose of GNU Guix is to allow users to easily install, upgrade, and +remove software packages, without having to know about their build +procedures or dependencies. Guix also goes beyond this obvious set of +features. -When substitutes are enabled and a substitute for a derivation is -available, but the substitution attempt fails, Guix will attempt to -build the derivation locally depending on whether or not -@code{--fallback} was given (@pxref{fallback-option,, common build -option @code{--fallback}}). Specifically, if @code{--fallback} was -omitted, then no local build will be performed, and the derivation is -considered to have failed. However, if @code{--fallback} was given, -then Guix will attempt to build the derivation locally, and the success -or failure of the derivation depends on the success or failure of the -local build. Note that when substitutes are disabled or no substitute -is available for the derivation in question, a local build will -@emph{always} be performed, regardless of whether or not -@code{--fallback} was given. +This chapter describes the main features of Guix, as well as the +package management tools it provides. Along with the command-line +interface described below (@pxref{Invoking guix package, @code{guix +package}}), you may also use the Emacs-Guix interface (@pxref{Top,,, +emacs-guix, The Emacs-Guix Reference Manual}), after installing +@code{emacs-guix} package (run @kbd{M-x guix-help} command to start +with it): -To get an idea of how many substitutes are available right now, you can -try running the @command{guix weather} command (@pxref{Invoking guix -weather}). This command provides statistics on the substitutes provided -by a server. +@example +guix package -i emacs-guix +@end example -@node On Trusting Binaries -@subsection On Trusting Binaries +@menu +* Features:: How Guix will make your life brighter. +* Invoking guix package:: Package installation, removal, etc. +* Substitutes:: Downloading pre-built binaries. +* Packages with Multiple Outputs:: Single source package, multiple outputs. +* Invoking guix gc:: Running the garbage collector. +* Invoking guix pull:: Fetching the latest Guix and distribution. +* Channels:: Customizing the package collection. +* Inferiors:: Interacting with another revision of Guix. +* Invoking guix describe:: Display information about your Guix revision. +* Invoking guix pack:: Creating software bundles. +* Invoking guix archive:: Exporting and importing store files. +@end menu -@cindex trust, of pre-built binaries -Today, each individual's control over their own computing is at the -mercy of institutions, corporations, and groups with enough power and -determination to subvert the computing infrastructure and exploit its -weaknesses. While using @code{@value{SUBSTITUTE-SERVER}} substitutes can be -convenient, we encourage users to also build on their own, or even run -their own build farm, such that @code{@value{SUBSTITUTE-SERVER}} is less of an -interesting target. One way to help is by publishing the software you -build using @command{guix publish} so that others have one more choice -of server to download substitutes from (@pxref{Invoking guix publish}). +@node Features +@section Features -Guix has the foundations to maximize build reproducibility -(@pxref{Features}). In most cases, independent builds of a given -package or derivation should yield bit-identical results. Thus, through -a diverse set of independent package builds, we can strengthen the -integrity of our systems. The @command{guix challenge} command aims to -help users assess substitute servers, and to assist developers in -finding out about non-deterministic package builds (@pxref{Invoking guix -challenge}). Similarly, the @option{--check} option of @command{guix -build} allows users to check whether previously-installed substitutes -are genuine by rebuilding them locally (@pxref{build-check, -@command{guix build --check}}). +When using Guix, each package ends up in the @dfn{package store}, in its +own directory---something that resembles +@file{/gnu/store/xxx-package-1.2}, where @code{xxx} is a base32 string. -In the future, we want Guix to have support to publish and retrieve -binaries to/from other users, in a peer-to-peer fashion. If you would -like to discuss this project, join us on @email{guix-devel@@gnu.org}. +Instead of referring to these directories, users have their own +@dfn{profile}, which points to the packages that they actually want to +use. These profiles are stored within each user's home directory, at +@code{$HOME/.guix-profile}. -@node Packages with Multiple Outputs -@section Packages with Multiple Outputs +For example, @code{alice} installs GCC 4.7.2. As a result, +@file{/home/alice/.guix-profile/bin/gcc} points to +@file{/gnu/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine, +@code{bob} had already installed GCC 4.8.0. The profile of @code{bob} +simply continues to point to +@file{/gnu/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC +coexist on the same system without any interference. -@cindex multiple-output packages -@cindex package outputs -@cindex outputs +The @command{guix package} command is the central tool to manage +packages (@pxref{Invoking guix package}). It operates on the per-user +profiles, and can be used @emph{with normal user privileges}. -Often, packages defined in Guix have a single @dfn{output}---i.e., the -source package leads to exactly one directory in the store. When running -@command{guix package -i glibc}, one installs the default output of the -GNU libc package; the default output is called @code{out}, but its name -can be omitted as shown in this command. In this particular case, the -default output of @code{glibc} contains all the C header files, shared -libraries, static libraries, Info documentation, and other supporting -files. +@cindex transactions +The command provides the obvious install, remove, and upgrade +operations. Each invocation is actually a @emph{transaction}: either +the specified operation succeeds, or nothing happens. Thus, if the +@command{guix package} process is terminated during the transaction, +or if a power outage occurs during the transaction, then the user's +profile remains in its previous state, and remains usable. -Sometimes it is more appropriate to separate the various types of files -produced from a single source package into separate outputs. For -instance, the GLib C library (used by GTK+ and related packages) -installs more than 20 MiB of reference documentation as HTML pages. -To save space for users who do not need it, the documentation goes to a -separate output, called @code{doc}. To install the main GLib output, -which contains everything but the documentation, one would run: +In addition, any package transaction may be @emph{rolled back}. So, if, +for example, an upgrade installs a new version of a package that turns +out to have a serious bug, users may roll back to the previous instance +of their profile, which was known to work well. Similarly, the global +system configuration on GuixSD is subject to +transactional upgrades and roll-back +(@pxref{Using the Configuration System}). -@example -guix package -i glib -@end example +All packages in the package store may be @emph{garbage-collected}. +Guix can determine which packages are still referenced by user +profiles, and remove those that are provably no longer referenced +(@pxref{Invoking guix gc}). Users may also explicitly remove old +generations of their profile so that the packages they refer to can be +collected. -@cindex documentation -The command to install its documentation is: +@cindex reproducibility +@cindex reproducible builds +Guix takes a @dfn{purely functional} approach to package +management, as described in the introduction (@pxref{Introduction}). +Each @file{/gnu/store} package directory name contains a hash of all the +inputs that were used to build that package---compiler, libraries, build +scripts, etc. This direct correspondence allows users to make sure a +given package installation matches the current state of their +distribution. It also helps maximize @dfn{build reproducibility}: +thanks to the isolated build environments that are used, a given build +is likely to yield bit-identical files when performed on different +machines (@pxref{Invoking guix-daemon, container}). -@example -guix package -i glib:doc -@end example +@cindex substitutes +This foundation allows Guix to support @dfn{transparent binary/source +deployment}. When a pre-built binary for a @file{/gnu/store} item is +available from an external source---a @dfn{substitute}, Guix just +downloads it and unpacks it; +otherwise, it builds the package from source, locally +(@pxref{Substitutes}). Because build results are usually bit-for-bit +reproducible, users do not have to trust servers that provide +substitutes: they can force a local build and @emph{challenge} providers +(@pxref{Invoking guix challenge}). -Some packages install programs with different ``dependency footprints''. -For instance, the WordNet package installs both command-line tools and -graphical user interfaces (GUIs). The former depend solely on the C -library, whereas the latter depend on Tcl/Tk and the underlying X -libraries. In this case, we leave the command-line tools in the default -output, whereas the GUIs are in a separate output. This allows users -who do not need the GUIs to save space. The @command{guix size} command -can help find out about such situations (@pxref{Invoking guix size}). -@command{guix graph} can also be helpful (@pxref{Invoking guix graph}). +Control over the build environment is a feature that is also useful for +developers. The @command{guix environment} command allows developers of +a package to quickly set up the right development environment for their +package, without having to manually install the dependencies of the +package into their profile (@pxref{Invoking guix environment}). -There are several such multiple-output packages in the GNU distribution. -Other conventional output names include @code{lib} for libraries and -possibly header files, @code{bin} for stand-alone programs, and -@code{debug} for debugging information (@pxref{Installing Debugging -Files}). The outputs of a packages are listed in the third column of -the output of @command{guix package --list-available} (@pxref{Invoking -guix package}). +@cindex replication, of software environments +@cindex provenance tracking, of software artifacts +All of Guix and its package definitions is version-controlled, and +@command{guix pull} allows you to ``travel in time'' on the history of Guix +itself (@pxref{Invoking guix pull}). This makes it possible to replicate a +Guix instance on a different machine or at a later point in time, which in +turn allows you to @emph{replicate complete software environments}, while +retaining precise @dfn{provenance tracking} of the software. +@node Invoking guix package +@section Invoking @command{guix package} -@node Invoking guix gc -@section Invoking @command{guix gc} +@cindex installing packages +@cindex removing packages +@cindex package installation +@cindex package removal +The @command{guix package} command is the tool that allows users to +install, upgrade, and remove packages, as well as rolling back to +previous configurations. It operates only on the user's own profile, +and works with normal user privileges (@pxref{Features}). Its syntax +is: -@cindex garbage collector -@cindex disk space -Packages that are installed, but not used, may be @dfn{garbage-collected}. -The @command{guix gc} command allows users to explicitly run the garbage -collector to reclaim space from the @file{/gnu/store} directory. It is -the @emph{only} way to remove files from @file{/gnu/store}---removing -files or directories manually may break it beyond repair! +@example +guix package @var{options} +@end example +@cindex transactions +Primarily, @var{options} specifies the operations to be performed during +the transaction. Upon completion, a new profile is created, but +previous @dfn{generations} of the profile remain available, should the user +want to roll back. -@cindex GC roots -@cindex garbage collector roots -The garbage collector has a set of known @dfn{roots}: any file under -@file{/gnu/store} reachable from a root is considered @dfn{live} and -cannot be deleted; any other file is considered @dfn{dead} and may be -deleted. The set of garbage collector roots (``GC roots'' for short) -includes default user profiles; by default, the symlinks under -@file{/var/guix/gcroots} represent these GC roots. New GC roots can be -added with @command{guix build --root}, for example (@pxref{Invoking -guix build}). +For example, to remove @code{lua} and install @code{guile} and +@code{guile-cairo} in a single transaction: -Prior to running @code{guix gc --collect-garbage} to make space, it is -often useful to remove old generations from user profiles; that way, old -package builds referenced by those generations can be reclaimed. This -is achieved by running @code{guix package --delete-generations} -(@pxref{Invoking guix package}). +@example +guix package -r lua -i guile guile-cairo +@end example -Our recommendation is to run a garbage collection periodically, or when -you are short on disk space. For instance, to guarantee that at least -5@tie{}GB are available on your disk, simply run: +@command{guix package} also supports a @dfn{declarative approach} +whereby the user specifies the exact set of packages to be available and +passes it @i{via} the @option{--manifest} option +(@pxref{profile-manifest, @option{--manifest}}). + +@cindex profile +For each user, a symlink to the user's default profile is automatically +created in @file{$HOME/.guix-profile}. This symlink always points to the +current generation of the user's default profile. Thus, users can add +@file{$HOME/.guix-profile/bin} to their @code{PATH} environment +variable, and so on. +@cindex search paths +If you are not using the Guix System Distribution, consider adding the +following lines to your @file{~/.bash_profile} (@pxref{Bash Startup +Files,,, bash, The GNU Bash Reference Manual}) so that newly-spawned +shells get all the right environment variable definitions: @example -guix gc -F 5G +GUIX_PROFILE="$HOME/.guix-profile" ; \ +source "$HOME/.guix-profile/etc/profile" @end example -It is perfectly safe to run as a non-interactive periodic job -(@pxref{Scheduled Job Execution}, for how to set up such a job on -GuixSD). Running @command{guix gc} with no arguments will collect as -much garbage as it can, but that is often inconvenient: you may find -yourself having to rebuild or re-download software that is ``dead'' from -the GC viewpoint but that is necessary to build other pieces of -software---e.g., the compiler tool chain. +In a multi-user setup, user profiles are stored in a place registered as +a @dfn{garbage-collector root}, which @file{$HOME/.guix-profile} points +to (@pxref{Invoking guix gc}). That directory is normally +@code{@var{localstatedir}/guix/profiles/per-user/@var{user}}, where +@var{localstatedir} is the value passed to @code{configure} as +@code{--localstatedir}, and @var{user} is the user name. The +@file{per-user} directory is created when @command{guix-daemon} is +started, and the @var{user} sub-directory is created by @command{guix +package}. -The @command{guix gc} command has three modes of operation: it can be -used to garbage-collect any dead files (the default), to delete specific -files (the @code{--delete} option), to print garbage-collector -information, or for more advanced queries. The garbage collection -options are as follows: +The @var{options} can be among the following: @table @code -@item --collect-garbage[=@var{min}] -@itemx -C [@var{min}] -Collect garbage---i.e., unreachable @file{/gnu/store} files and -sub-directories. This is the default operation when no option is -specified. -When @var{min} is given, stop once @var{min} bytes have been collected. -@var{min} may be a number of bytes, or it may include a unit as a -suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes -(@pxref{Block size, size specifications,, coreutils, GNU Coreutils}). +@item --install=@var{package} @dots{} +@itemx -i @var{package} @dots{} +Install the specified @var{package}s. -When @var{min} is omitted, collect all the garbage. +Each @var{package} may specify either a simple package name, such as +@code{guile}, or a package name followed by an at-sign and version number, +such as @code{guile@@1.8.8} or simply @code{guile@@1.8} (in the latter +case, the newest version prefixed by @code{1.8} is selected.) -@item --free-space=@var{free} -@itemx -F @var{free} -Collect garbage until @var{free} space is available under -@file{/gnu/store}, if possible; @var{free} denotes storage space, such -as @code{500MiB}, as described above. +If no version number is specified, the +newest available version will be selected. In addition, @var{package} +may contain a colon, followed by the name of one of the outputs of the +package, as in @code{gcc:doc} or @code{binutils@@2.22:lib} +(@pxref{Packages with Multiple Outputs}). Packages with a corresponding +name (and optionally version) are searched for among the GNU +distribution modules (@pxref{Package Modules}). -When @var{free} or more is already available in @file{/gnu/store}, do -nothing and exit immediately. +@cindex propagated inputs +Sometimes packages have @dfn{propagated inputs}: these are dependencies +that automatically get installed along with the required package +(@pxref{package-propagated-inputs, @code{propagated-inputs} in +@code{package} objects}, for information about propagated inputs in +package definitions). -@item --delete -@itemx -d -Attempt to delete all the store files and directories specified as -arguments. This fails if some of the files are not in the store, or if -they are still live. +@anchor{package-cmd-propagated-inputs} +An example is the GNU MPC library: its C header files refer to those of +the GNU MPFR library, which in turn refer to those of the GMP library. +Thus, when installing MPC, the MPFR and GMP libraries also get installed +in the profile; removing MPC also removes MPFR and GMP---unless they had +also been explicitly installed by the user. -@item --list-failures -List store items corresponding to cached build failures. +Besides, packages sometimes rely on the definition of environment +variables for their search paths (see explanation of +@code{--search-paths} below). Any missing or possibly incorrect +environment variable definitions are reported here. -This prints nothing unless the daemon was started with -@option{--cache-failures} (@pxref{Invoking guix-daemon, -@option{--cache-failures}}). +@item --install-from-expression=@var{exp} +@itemx -e @var{exp} +Install the package @var{exp} evaluates to. -@item --clear-failures -Remove the specified store items from the failed-build cache. +@var{exp} must be a Scheme expression that evaluates to a +@code{} object. This option is notably useful to disambiguate +between same-named variants of a package, with expressions such as +@code{(@@ (gnu packages base) guile-final)}. -Again, this option only makes sense when the daemon is started with -@option{--cache-failures}. Otherwise, it does nothing. +Note that this option installs the first output of the specified +package, which may be insufficient when needing a specific output of a +multiple-output package. -@item --list-dead -Show the list of dead files and directories still present in the -store---i.e., files and directories no longer reachable from any root. +@item --install-from-file=@var{file} +@itemx -f @var{file} +Install the package that the code within @var{file} evaluates to. -@item --list-live -Show the list of live store files and directories. +As an example, @var{file} might contain a definition like this +(@pxref{Defining Packages}): -@end table +@example +@verbatiminclude package-hello.scm +@end example -In addition, the references among existing store files can be queried: +Developers may find it useful to include such a @file{guix.scm} file +in the root of their project source tree that can be used to test +development snapshots and create reproducible development environments +(@pxref{Invoking guix environment}). -@table @code +@item --remove=@var{package} @dots{} +@itemx -r @var{package} @dots{} +Remove the specified @var{package}s. -@item --references -@itemx --referrers -@cindex package dependencies -List the references (respectively, the referrers) of store files given -as arguments. +As for @code{--install}, each @var{package} may specify a version number +and/or output name in addition to the package name. For instance, +@code{-r glibc:debug} would remove the @code{debug} output of +@code{glibc}. -@item --requisites -@itemx -R -@cindex closure -List the requisites of the store files passed as arguments. Requisites -include the store files themselves, their references, and the references -of these, recursively. In other words, the returned list is the -@dfn{transitive closure} of the store files. +@item --upgrade[=@var{regexp} @dots{}] +@itemx -u [@var{regexp} @dots{}] +@cindex upgrading packages +Upgrade all the installed packages. If one or more @var{regexp}s are +specified, upgrade only installed packages whose name matches a +@var{regexp}. Also see the @code{--do-not-upgrade} option below. -@xref{Invoking guix size}, for a tool to profile the size of the closure -of an element. @xref{Invoking guix graph}, for a tool to visualize -the graph of references. +Note that this upgrades package to the latest version of packages found +in the distribution currently installed. To update your distribution, +you should regularly run @command{guix pull} (@pxref{Invoking guix +pull}). -@item --derivers -@cindex derivation -Return the derivation(s) leading to the given store items -(@pxref{Derivations}). - -For example, this command: +@item --do-not-upgrade[=@var{regexp} @dots{}] +When used together with the @code{--upgrade} option, do @emph{not} +upgrade any packages whose name matches a @var{regexp}. For example, to +upgrade all packages in the current profile except those containing the +substring ``emacs'': @example -guix gc --derivers `guix package -I ^emacs$ | cut -f4` +$ guix package --upgrade . --do-not-upgrade emacs @end example -@noindent -returns the @file{.drv} file(s) leading to the @code{emacs} package -installed in your profile. +@item @anchor{profile-manifest}--manifest=@var{file} +@itemx -m @var{file} +@cindex profile declaration +@cindex profile manifest +Create a new generation of the profile from the manifest object +returned by the Scheme code in @var{file}. -Note that there may be zero matching @file{.drv} files, for instance -because these files have been garbage-collected. There can also be more -than one matching @file{.drv} due to fixed-output derivations. -@end table +This allows you to @emph{declare} the profile's contents rather than +constructing it through a sequence of @code{--install} and similar +commands. The advantage is that @var{file} can be put under version +control, copied to different machines to reproduce the same profile, and +so on. -Lastly, the following options allow you to check the integrity of the -store and to control disk usage. +@c FIXME: Add reference to (guix profile) documentation when available. +@var{file} must return a @dfn{manifest} object, which is roughly a list +of packages: -@table @option +@findex packages->manifest +@example +(use-package-modules guile emacs) -@item --verify[=@var{options}] -@cindex integrity, of the store -@cindex integrity checking -Verify the integrity of the store. +(packages->manifest + (list emacs + guile-2.0 + ;; Use a specific package output. + (list guile-2.0 "debug"))) +@end example -By default, make sure that all the store items marked as valid in the -database of the daemon actually exist in @file{/gnu/store}. +@findex specifications->manifest +In this example we have to know which modules define the @code{emacs} +and @code{guile-2.0} variables to provide the right +@code{use-package-modules} line, which can be cumbersome. We can +instead provide regular package specifications and let +@code{specifications->manifest} look up the corresponding package +objects, like this: -When provided, @var{options} must be a comma-separated list containing one -or more of @code{contents} and @code{repair}. +@example +(specifications->manifest + '("emacs" "guile@@2.2" "guile@@2.2:debug")) +@end example -When passing @option{--verify=contents}, the daemon computes the -content hash of each store item and compares it against its hash in the -database. Hash mismatches are reported as data corruptions. Because it -traverses @emph{all the files in the store}, this command can take a -long time, especially on systems with a slow disk drive. +@item --roll-back +@cindex rolling back +@cindex undoing transactions +@cindex transactions, undoing +Roll back to the previous @dfn{generation} of the profile---i.e., undo +the last transaction. -@cindex repairing the store -@cindex corruption, recovering from -Using @option{--verify=repair} or @option{--verify=contents,repair} -causes the daemon to try to repair corrupt store items by fetching -substitutes for them (@pxref{Substitutes}). Because repairing is not -atomic, and thus potentially dangerous, it is available only to the -system administrator. A lightweight alternative, when you know exactly -which items in the store are corrupt, is @command{guix build --repair} -(@pxref{Invoking guix build}). +When combined with options such as @code{--install}, roll back occurs +before any other actions. -@item --optimize -@cindex deduplication -Optimize the store by hard-linking identical files---this is -@dfn{deduplication}. +When rolling back from the first generation that actually contains +installed packages, the profile is made to point to the @dfn{zeroth +generation}, which contains no files apart from its own metadata. -The daemon performs deduplication after each successful build or archive -import, unless it was started with @code{--disable-deduplication} -(@pxref{Invoking guix-daemon, @code{--disable-deduplication}}). Thus, -this option is primarily useful when the daemon was running with -@code{--disable-deduplication}. +After having rolled back, installing, removing, or upgrading packages +overwrites previous future generations. Thus, the history of the +generations in a profile is always linear. -@end table +@item --switch-generation=@var{pattern} +@itemx -S @var{pattern} +@cindex generations +Switch to a particular generation defined by @var{pattern}. -@node Invoking guix pull -@section Invoking @command{guix pull} +@var{pattern} may be either a generation number or a number prefixed +with ``+'' or ``-''. The latter means: move forward/backward by a +specified number of generations. For example, if you want to return to +the latest generation after @code{--roll-back}, use +@code{--switch-generation=+1}. -@cindex upgrading Guix -@cindex updating Guix -@cindex @command{guix pull} -@cindex pull -Packages are installed or upgraded to the latest version available in -the distribution currently available on your local machine. To update -that distribution, along with the Guix tools, you must run @command{guix -pull}: the command downloads the latest Guix source code and package -descriptions, and deploys it. Source code is downloaded from a -@uref{https://git-scm.com, Git} repository, by default the official -GNU@tie{}Guix repository, though this can be customized. +The difference between @code{--roll-back} and +@code{--switch-generation=-1} is that @code{--switch-generation} will +not make a zeroth generation, so if a specified generation does not +exist, the current generation will not be changed. -On completion, @command{guix package} will use packages and package -versions from this just-retrieved copy of Guix. Not only that, but all -the Guix commands and Scheme modules will also be taken from that latest -version. New @command{guix} sub-commands added by the update also -become available. +@item --search-paths[=@var{kind}] +@cindex search paths +Report environment variable definitions, in Bash syntax, that may be +needed in order to use the set of installed packages. These environment +variables are used to specify @dfn{search paths} for files used by some +of the installed packages. -Any user can update their Guix copy using @command{guix pull}, and the -effect is limited to the user who run @command{guix pull}. For -instance, when user @code{root} runs @command{guix pull}, this has no -effect on the version of Guix that user @code{alice} sees, and vice -versa. +For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH} +environment variables to be defined so it can look for headers and +libraries in the user's profile (@pxref{Environment Variables,,, gcc, +Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C +library are installed in the profile, then @code{--search-paths} will +suggest setting these variables to @code{@var{profile}/include} and +@code{@var{profile}/lib}, respectively. -The result of running @command{guix pull} is a @dfn{profile} available -under @file{~/.config/guix/current} containing the latest Guix. Thus, -make sure to add it to the beginning of your search path so that you use -the latest version, and similarly for the Info manual -(@pxref{Documentation}): +The typical use case is to define these environment variables in the +shell: @example -export PATH="$HOME/.config/guix/current/bin:$PATH" -export INFOPATH="$HOME/.config/guix/current/share/info:$INFOPATH" +$ eval `guix package --search-paths` @end example -The @code{--list-generations} or @code{-l} option lists past generations -produced by @command{guix pull}, along with details about their provenance: - -@example -$ guix pull -l -Generation 1 Jun 10 2018 00:18:18 - guix 65956ad - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: origin/master - commit: 65956ad3526ba09e1f7a40722c96c6ef7c0936fe +@var{kind} may be one of @code{exact}, @code{prefix}, or @code{suffix}, +meaning that the returned environment variable definitions will either +be exact settings, or prefixes or suffixes of the current value of these +variables. When omitted, @var{kind} defaults to @code{exact}. -Generation 2 Jun 11 2018 11:02:49 - guix e0cc7f6 - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: origin/master - commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d - 2 new packages: keepalived, libnfnetlink - 6 packages upgraded: emacs-nix-mode@@2.0.4, - guile2.0-guix@@0.14.0-12.77a1aac, guix@@0.14.0-12.77a1aac, - heimdal@@7.5.0, milkytracker@@1.02.00, nix@@2.0.4 +This option can also be used to compute the @emph{combined} search paths +of several profiles. Consider this example: -Generation 3 Jun 13 2018 23:31:07 (current) - guix 844cc1c - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: origin/master - commit: 844cc1c8f394f03b404c5bb3aee086922373490c - 28 new packages: emacs-helm-ls-git, emacs-helm-mu, @dots{} - 69 packages upgraded: borg@@1.1.6, cheese@@3.28.0, @dots{} +@example +$ guix package -p foo -i guile +$ guix package -p bar -i guile-json +$ guix package -p foo -p bar --search-paths @end example -@ref{Invoking guix describe, @command{guix describe}}, for other ways to -describe the current status of Guix. +The last command above reports about the @code{GUILE_LOAD_PATH} +variable, even though, taken individually, neither @file{foo} nor +@file{bar} would lead to that recommendation. -This @code{~/.config/guix/current} profile works like any other profile -created by @command{guix package} (@pxref{Invoking guix package}). That -is, you can list generations, roll back to the previous -generation---i.e., the previous Guix---and so on: -@example -$ guix package -p ~/.config/guix/current --roll-back -switched from generation 3 to 2 -$ guix package -p ~/.config/guix/current --delete-generations=1 -deleting /var/guix/profiles/per-user/charlie/current-guix-1-link -@end example +@item --profile=@var{profile} +@itemx -p @var{profile} +Use @var{profile} instead of the user's default profile. -The @command{guix pull} command is usually invoked with no arguments, -but it supports the following options: +@cindex collisions, in a profile +@cindex colliding packages in profiles +@cindex profile collisions +@item --allow-collisions +Allow colliding packages in the new profile. Use at your own risk! -@table @code -@item --url=@var{url} -@itemx --commit=@var{commit} -@itemx --branch=@var{branch} -Download code from the specified @var{url}, at the given @var{commit} (a valid -Git commit ID represented as a hexadecimal string), or @var{branch}. +By default, @command{guix package} reports as an error @dfn{collisions} +in the profile. Collisions happen when two or more different versions +or variants of a given package end up in the profile. -@cindex @file{channels.scm}, configuration file -@cindex configuration file for channels -These options are provided for convenience, but you can also specify your -configuration in the @file{~/.config/guix/channels.scm} file or using the -@option{--channels} option (see below). +@item --bootstrap +Use the bootstrap Guile to build the profile. This option is only +useful to distribution developers. -@item --channels=@var{file} -@itemx -C @var{file} -Read the list of channels from @var{file} instead of -@file{~/.config/guix/channels.scm}. @var{file} must contain Scheme code that -evaluates to a list of channel objects. @xref{Channels}, for more -information. +@end table -@item --list-generations[=@var{pattern}] -@itemx -l [@var{pattern}] -List all the generations of @file{~/.config/guix/current} or, if @var{pattern} -is provided, the subset of generations that match @var{pattern}. -The syntax of @var{pattern} is the same as with @code{guix package ---list-generations} (@pxref{Invoking guix package}). +In addition to these actions, @command{guix package} supports the +following options to query the current state of a profile, or the +availability of packages: -@ref{Invoking guix describe}, for a way to display information about the -current generation only. +@table @option -@item --profile=@var{profile} -@itemx -p @var{profile} -Use @var{profile} instead of @file{~/.config/guix/current}. +@item --search=@var{regexp} +@itemx -s @var{regexp} +@cindex searching for packages +List the available packages whose name, synopsis, or description matches +@var{regexp} (in a case-insensitive fashion), sorted by relevance. +Print all the metadata of matching packages in +@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, +GNU recutils manual}). -@item --dry-run -@itemx -n -Show which channel commit(s) would be used and what would be built or -substituted but do not actually do it. +This allows specific fields to be extracted using the @command{recsel} +command, for instance: -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of -the system type of the build host. +@example +$ guix package -s malloc | recsel -p name,version,relevance +name: jemalloc +version: 4.5.0 +relevance: 6 -@item --verbose -Produce verbose output, writing build logs to the standard error output. +name: glibc +version: 2.25 +relevance: 1 -@item --bootstrap -Use the bootstrap Guile to build the latest Guix. This option is only -useful to Guix developers. -@end table +name: libgc +version: 7.6.0 +relevance: 1 +@end example -The @dfn{channel} mechanism allows you to instruct @command{guix pull} which -repository and branch to pull from, as well as @emph{additional} repositories -containing package modules that should be deployed. @xref{Channels}, for more -information. +Similarly, to show the name of all the packages available under the +terms of the GNU@tie{}LGPL version 3: -In addition, @command{guix pull} supports all the common build options -(@pxref{Common Build Options}). +@example +$ guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"' +name: elfutils -@node Channels -@section Channels +name: gmp +@dots{} +@end example -@cindex channels -@cindex @file{channels.scm}, configuration file -@cindex configuration file for channels -@cindex @command{guix pull}, configuration file -@cindex configuration of @command{guix pull} -Guix and its package collection are updated by running @command{guix pull} -(@pxref{Invoking guix pull}). By default @command{guix pull} downloads and -deploys Guix itself from the official GNU@tie{}Guix repository. This can be -customized by defining @dfn{channels} in the -@file{~/.config/guix/channels.scm} file. A channel specifies a URL and branch -of a Git repository to be deployed, and @command{guix pull} can be instructed -to pull from one or more channels. In other words, channels can be used to -@emph{customize} and to @emph{extend} Guix, as we will see below. +It is also possible to refine search results using several @code{-s} +flags. For example, the following command returns a list of board +games: -@subsection Using a Custom Guix Channel +@example +$ guix package -s '\' -s game | recsel -p name +name: gnubg +@dots{} +@end example -The channel called @code{guix} specifies where Guix itself---its command-line -tools as well as its package collection---should be downloaded. For instance, -suppose you want to update from your own copy of the Guix repository at -@code{example.org}, and specifically the @code{super-hacks} branch, you can -write in @code{~/.config/guix/channels.scm} this specification: +If we were to omit @code{-s game}, we would also get software packages +that deal with printed circuit boards; removing the angle brackets +around @code{board} would further add packages that have to do with +keyboards. -@lisp -;; Tell 'guix pull' to use my own repo. -(list (channel - (name 'guix) - (url "https://example.org/my-guix.git") - (branch "super-hacks"))) -@end lisp +And now for a more elaborate example. The following command searches +for cryptographic libraries, filters out Haskell, Perl, Python, and Ruby +libraries, and prints the name and synopsis of the matching packages: + +@example +$ guix package -s crypto -s library | \ + recsel -e '! (name ~ "^(ghc|perl|python|ruby)")' -p name,synopsis +@end example @noindent -From there on, @command{guix pull} will fetch code from the @code{super-hacks} -branch of the repository at @code{example.org}. +@xref{Selection Expressions,,, recutils, GNU recutils manual}, for more +information on @dfn{selection expressions} for @code{recsel -e}. -@subsection Specifying Additional Channels +@item --show=@var{package} +Show details about @var{package}, taken from the list of available packages, in +@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, GNU +recutils manual}). -@cindex extending the package collection (channels) -@cindex personal packages (channels) -@cindex channels, for personal packages -You can also specify @emph{additional channels} to pull from. Let's say you -have a bunch of custom package variants or personal packages that you think -would make little sense to contribute to the Guix project, but would like to -have these packages transparently available to you at the command line. You -would first write modules containing those package definitions (@pxref{Package -Modules}), maintain them in a Git repository, and then you and anyone else can -use it as an additional channel to get packages from. Neat, no? +@example +$ guix package --show=python | recsel -p name,version +name: python +version: 2.7.6 -@c What follows stems from discussions at -@c as well as -@c earlier discussions on guix-devel@gnu.org. -@quotation Warning -Before you, dear user, shout---``woow this is @emph{soooo coool}!''---and -publish your personal channel to the world, we would like to share a few words -of caution: +name: python +version: 3.3.5 +@end example -@itemize -@item -Before publishing a channel, please consider contributing your package -definitions to Guix proper (@pxref{Contributing}). Guix as a project is open -to free software of all sorts, and packages in Guix proper are readily -available to all Guix users and benefit from the project's quality assurance -process. +You may also specify the full name of a package to only get details about a +specific version of it: +@example +$ guix package --show=python@@3.4 | recsel -p name,version +name: python +version: 3.4.3 +@end example -@item -When you maintain package definitions outside Guix, we, Guix developers, -consider that @emph{the compatibility burden is on you}. Remember that -package modules and package definitions are just Scheme code that uses various -programming interfaces (APIs). We want to remain free to change these APIs to -keep improving Guix, possibly in ways that break your channel. We never -change APIs gratuitously, but we will @emph{not} commit to freezing APIs -either. -@item -Corollary: if you're using an external channel and that channel breaks, please -@emph{report the issue to the channel authors}, not to the Guix project. -@end itemize -You've been warned! Having said this, we believe external channels are a -practical way to exert your freedom to augment Guix' package collection and to -share your improvements, which are basic tenets of -@uref{https://www.gnu.org/philosophy/free-sw.html, free software}. Please -email us at @email{guix-devel@@gnu.org} if you'd like to discuss this. -@end quotation +@item --list-installed[=@var{regexp}] +@itemx -I [@var{regexp}] +List the currently installed packages in the specified profile, with the +most recently installed packages shown last. When @var{regexp} is +specified, list only installed packages whose name matches @var{regexp}. -Once you have a Git repository containing your own package modules, you can -write @code{~/.config/guix/channels.scm} to instruct @command{guix pull} to -pull from your personal channel @emph{in addition} to the default Guix -channel(s): +For each installed package, print the following items, separated by +tabs: the package name, its version string, the part of the package that +is installed (for instance, @code{out} for the default output, +@code{include} for its headers, etc.), and the path of this package in +the store. -@vindex %default-channels -@lisp -;; Add my personal packages to those Guix provides. -(cons (channel - (name 'my-personal-packages) - (url "https://example.org/personal-packages.git")) - %default-channels) -@end lisp +@item --list-available[=@var{regexp}] +@itemx -A [@var{regexp}] +List packages currently available in the distribution for this system +(@pxref{GNU Distribution}). When @var{regexp} is specified, list only +installed packages whose name matches @var{regexp}. -@noindent -Note that the snippet above is (as always!)@: Scheme code; we use @code{cons} to -add a channel the list of channels that the variable @code{%default-channels} -is bound to (@pxref{Pairs, @code{cons} and lists,, guile, GNU Guile Reference -Manual}). With this file in place, @command{guix pull} builds not only Guix -but also the package modules from your own repository. The result in -@file{~/.config/guix/current} is the union of Guix with your own package -modules: +For each package, print the following items separated by tabs: its name, +its version string, the parts of the package (@pxref{Packages with +Multiple Outputs}), and the source location of its definition. -@example -$ guix pull --list-generations -@dots{} -Generation 19 Aug 27 2018 16:20:48 - guix d894ab8 - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: master - commit: d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300 - my-personal-packages dd3df5e - repository URL: https://example.org/personal-packages.git - branch: master - commit: dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb - 11 new packages: my-gimp, my-emacs-with-cool-features, @dots{} - 4 packages upgraded: emacs-racket-mode@@0.0.2-2.1b78827, @dots{} -@end example +@item --list-generations[=@var{pattern}] +@itemx -l [@var{pattern}] +@cindex generations +Return a list of generations along with their creation dates; for each +generation, show the installed packages, with the most recently +installed packages shown last. Note that the zeroth generation is never +shown. -@noindent -The output of @command{guix pull} above shows that Generation@tie{}19 includes -both Guix and packages from the @code{my-personal-packages} channel. Among -the new and upgraded packages that are listed, some like @code{my-gimp} and -@code{my-emacs-with-cool-features} might come from -@code{my-personal-packages}, while others come from the Guix default channel. +For each installed package, print the following items, separated by +tabs: the name of a package, its version string, the part of the package +that is installed (@pxref{Packages with Multiple Outputs}), and the +location of this package in the store. -@cindex dependencies, channels -@cindex meta-data, channels -@subsection Declaring Channel Dependencies +When @var{pattern} is used, the command returns only matching +generations. Valid patterns include: -Channel authors may decide to augment a package collection provided by other -channels. They can declare their channel to be dependent on other channels in -a meta-data file @file{.guix-channel}, which is to be placed in the root of -the channel repository. +@itemize +@item @emph{Integers and comma-separated integers}. Both patterns denote +generation numbers. For instance, @code{--list-generations=1} returns +the first one. -The meta-data file should contain a simple S-expression like this: +And @code{--list-generations=1,8,2} outputs three generations in the +specified order. Neither spaces nor trailing commas are allowed. -@lisp -(channel - (version 0) - (dependencies - (channel - (name some-collection) - (url "https://example.org/first-collection.git")) - (channel - (name some-other-collection) - (url "https://example.org/second-collection.git") - (branch "testing")))) -@end lisp +@item @emph{Ranges}. @code{--list-generations=2..9} prints the +specified generations and everything in between. Note that the start of +a range must be smaller than its end. -In the above example this channel is declared to depend on two other channels, -which will both be fetched automatically. The modules provided by the channel -will be compiled in an environment where the modules of all these declared -channels are available. +It is also possible to omit the endpoint. For example, +@code{--list-generations=2..}, returns all generations starting from the +second one. -For the sake of reliability and maintainability, you should avoid dependencies -on channels that you don't control, and you should aim to keep the number of -dependencies to a minimum. +@item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks, +or months by passing an integer along with the first letter of the +duration. For example, @code{--list-generations=20d} lists generations +that are up to 20 days old. +@end itemize -@subsection Replicating Guix +@item --delete-generations[=@var{pattern}] +@itemx -d [@var{pattern}] +When @var{pattern} is omitted, delete all generations except the current +one. -@cindex pinning, channels -@cindex replicating Guix -@cindex reproducibility, of Guix -The @command{guix pull --list-generations} output above shows precisely which -commits were used to build this instance of Guix. We can thus replicate it, -say, on another machine, by providing a channel specification in -@file{~/.config/guix/channels.scm} that is ``pinned'' to these commits: +This command accepts the same patterns as @option{--list-generations}. +When @var{pattern} is specified, delete the matching generations. When +@var{pattern} specifies a duration, generations @emph{older} than the +specified duration match. For instance, @code{--delete-generations=1m} +deletes generations that are more than one month old. -@lisp -;; Deploy specific commits of my channels of interest. -(list (channel - (name 'guix) - (url "https://git.savannah.gnu.org/git/guix.git") - (commit "d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300")) - (channel - (name 'my-personal-packages) - (url "https://example.org/personal-packages.git") - (branch "dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb"))) -@end lisp +If the current generation matches, it is @emph{not} deleted. Also, the +zeroth generation is never deleted. -The @command{guix describe --format=channels} command can even generate this -list of channels directly (@pxref{Invoking guix describe}). +Note that deleting generations prevents rolling back to them. +Consequently, this command must be used with care. -At this point the two machines run the @emph{exact same Guix}, with access to -the @emph{exact same packages}. The output of @command{guix build gimp} on -one machine will be exactly the same, bit for bit, as the output of the same -command on the other machine. It also means both machines have access to all -the source code of Guix and, transitively, to all the source code of every -package it defines. +@end table -This gives you super powers, allowing you to track the provenance of binary -artifacts with very fine grain, and to reproduce software environments at -will---some sort of ``meta reproducibility'' capabilities, if you will. -@xref{Inferiors}, for another way to take advantage of these super powers. +Finally, since @command{guix package} may actually start build +processes, it supports all the common build options (@pxref{Common Build +Options}). It also supports package transformation options, such as +@option{--with-source} (@pxref{Package Transformation Options}). +However, note that package transformations are lost when upgrading; to +preserve transformations across upgrades, you should define your own +package variant in a Guile module and add it to @code{GUIX_PACKAGE_PATH} +(@pxref{Defining Packages}). -@node Inferiors -@section Inferiors +@node Substitutes +@section Substitutes -@c TODO: Remove this once we're more confident about API stability. -@quotation Note -The functionality described here is a ``technology preview'' as of version -@value{VERSION}. As such, the interface is subject to change. -@end quotation +@cindex substitutes +@cindex pre-built binaries +Guix supports transparent source/binary deployment, which means that it +can either build things locally, or download pre-built items from a +server, or both. We call these pre-built items @dfn{substitutes}---they +are substitutes for local build results. In many cases, downloading a +substitute is much faster than building things locally. -@cindex inferiors -@cindex composition of Guix revisions -Sometimes you might need to mix packages from the revision of Guix you're -currently running with packages available in a different revision of Guix. -Guix @dfn{inferiors} allow you to achieve that by composing different Guix -revisions in arbitrary ways. +Substitutes can be anything resulting from a derivation build +(@pxref{Derivations}). Of course, in the common case, they are +pre-built package binaries, but source tarballs, for instance, which +also result from derivation builds, can be available as substitutes. -@cindex inferior packages -Technically, an ``inferior'' is essentially a separate Guix process connected -to your main Guix process through a REPL (@pxref{Invoking guix repl}). The -@code{(guix inferior)} module allows you to create inferiors and to -communicate with them. It also provides a high-level interface to browse and -manipulate the packages that an inferior provides---@dfn{inferior packages}. +@menu +* Official Substitute Server:: One particular source of substitutes. +* Substitute Server Authorization:: How to enable or disable substitutes. +* Substitute Authentication:: How Guix verifies substitutes. +* Proxy Settings:: How to get substitutes via proxy. +* Substitution Failure:: What happens when substitution fails. +* On Trusting Binaries:: How can you trust that binary blob? +@end menu -When combined with channels (@pxref{Channels}), inferiors provide a simple way -to interact with a separate revision of Guix. For example, let's assume you -want to install in your profile the current @code{guile} package, along with -the @code{guile-json} as it existed in an older revision of Guix---perhaps -because the newer @code{guile-json} has an incompatible API and you want to -run your code against the old API@. To do that, you could write a manifest for -use by @code{guix package --manifest} (@pxref{Invoking guix package}); in that -manifest, you would create an inferior for that old Guix revision you care -about, and you would look up the @code{guile-json} package in the inferior: +@node Official Substitute Server +@subsection Official Substitute Server -@lisp -(use-modules (guix inferior) (guix channels) - (srfi srfi-1)) ;for 'first' +@cindex hydra +@cindex build farm +The @code{@value{SUBSTITUTE-SERVER}} server is a front-end to an official build farm +that builds packages from Guix continuously for some +architectures, and makes them available as substitutes. This is the +default source of substitutes; it can be overridden by passing the +@option{--substitute-urls} option either to @command{guix-daemon} +(@pxref{daemon-substitute-urls,, @code{guix-daemon --substitute-urls}}) +or to client tools such as @command{guix package} +(@pxref{client-substitute-urls,, client @option{--substitute-urls} +option}). -(define channels - ;; This is the old revision from which we want to - ;; extract guile-json. - (list (channel - (name 'guix) - (url "https://git.savannah.gnu.org/git/guix.git") - (commit - "65956ad3526ba09e1f7a40722c96c6ef7c0936fe")))) +Substitute URLs can be either HTTP or HTTPS. +HTTPS is recommended because communications are encrypted; conversely, +using HTTP makes all communications visible to an eavesdropper, who +could use the information gathered to determine, for instance, whether +your system has unpatched security vulnerabilities. -(define inferior - ;; An inferior representing the above revision. - (inferior-for-channels channels)) +Substitutes from the official build farm are enabled by default when +using the Guix System Distribution (@pxref{GNU Distribution}). However, +they are disabled by default when using Guix on a foreign distribution, +unless you have explicitly enabled them via one of the recommended +installation steps (@pxref{Installation}). The following paragraphs +describe how to enable or disable substitutes for the official build +farm; the same procedure can also be used to enable substitutes for any +other substitute server. -;; Now create a manifest with the current "guile" package -;; and the old "guile-json" package. -(packages->manifest - (list (first (lookup-inferior-packages inferior "guile-json")) - (specification->package "guile"))) -@end lisp +@node Substitute Server Authorization +@subsection Substitute Server Authorization -On its first run, @command{guix package --manifest} might have to build the -channel you specified before it can create the inferior; subsequent runs will -be much faster because the Guix revision will be cached. +@cindex security +@cindex substitutes, authorization thereof +@cindex access control list (ACL), for substitutes +@cindex ACL (access control list), for substitutes +To allow Guix to download substitutes from @code{@value{SUBSTITUTE-SERVER}} or a +mirror thereof, you +must add its public key to the access control list (ACL) of archive +imports, using the @command{guix archive} command (@pxref{Invoking guix +archive}). Doing so implies that you trust @code{@value{SUBSTITUTE-SERVER}} to not +be compromised and to serve genuine substitutes. -The @code{(guix inferior)} module provides the following procedures to open an -inferior: +The public key for @code{@value{SUBSTITUTE-SERVER}} is installed along with Guix, in +@code{@var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub}, where @var{prefix} is +the installation prefix of Guix. If you installed Guix from source, +make sure you checked the GPG signature of +@file{guix-@value{VERSION}.tar.gz}, which contains this public key file. +Then, you can run something like this: -@deffn {Scheme Procedure} inferior-for-channels @var{channels} @ - [#:cache-directory] [#:ttl] -Return an inferior for @var{channels}, a list of channels. Use the cache at -@var{cache-directory}, where entries can be reclaimed after @var{ttl} seconds. -This procedure opens a new connection to the build daemon. +@example +# guix archive --authorize < @var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub +@end example -As a side effect, this procedure may build or substitute binaries for -@var{channels}, which can take time. -@end deffn +@quotation Note +Similarly, the @file{hydra.gnu.org.pub} file contains the public key +of an independent build farm also run by the project, reachable at +@indicateurl{https://mirror.hydra.gnu.org}. +@end quotation -@deffn {Scheme Procedure} open-inferior @var{directory} @ - [#:command "bin/guix"] -Open the inferior Guix in @var{directory}, running -@code{@var{directory}/@var{command} repl} or equivalent. Return @code{#f} if -the inferior could not be launched. -@end deffn +Once this is in place, the output of a command like @code{guix build} +should change from something like: -@cindex inferior packages -The procedures listed below allow you to obtain and manipulate inferior -packages. +@example +$ guix build emacs --dry-run +The following derivations would be built: + /gnu/store/yr7bnx8xwcayd6j95r2clmkdl1qh688w-emacs-24.3.drv + /gnu/store/x8qsh1hlhgjx6cwsjyvybnfv2i37z23w-dbus-1.6.4.tar.gz.drv + /gnu/store/1ixwp12fl950d15h2cj11c73733jay0z-alsa-lib-1.0.27.1.tar.bz2.drv + /gnu/store/nlma1pw0p603fpfiqy7kn4zm105r5dmw-util-linux-2.21.drv +@dots{} +@end example -@deffn {Scheme Procedure} inferior-packages @var{inferior} -Return the list of packages known to @var{inferior}. -@end deffn +@noindent +to something like: -@deffn {Scheme Procedure} lookup-inferior-packages @var{inferior} @var{name} @ - [@var{version}] -Return the sorted list of inferior packages matching @var{name} in -@var{inferior}, with highest version numbers first. If @var{version} is true, -return only packages with a version number prefixed by @var{version}. -@end deffn +@example +$ guix build emacs --dry-run +112.3 MB would be downloaded: + /gnu/store/pk3n22lbq6ydamyymqkkz7i69wiwjiwi-emacs-24.3 + /gnu/store/2ygn4ncnhrpr61rssa6z0d9x22si0va3-libjpeg-8d + /gnu/store/71yz6lgx4dazma9dwn2mcjxaah9w77jq-cairo-1.12.16 + /gnu/store/7zdhgp0n1518lvfn8mb96sxqfmvqrl7v-libxrender-0.9.7 +@dots{} +@end example -@deffn {Scheme Procedure} inferior-package? @var{obj} -Return true if @var{obj} is an inferior package. -@end deffn - -@deffn {Scheme Procedure} inferior-package-name @var{package} -@deffnx {Scheme Procedure} inferior-package-version @var{package} -@deffnx {Scheme Procedure} inferior-package-synopsis @var{package} -@deffnx {Scheme Procedure} inferior-package-description @var{package} -@deffnx {Scheme Procedure} inferior-package-home-page @var{package} -@deffnx {Scheme Procedure} inferior-package-location @var{package} -@deffnx {Scheme Procedure} inferior-package-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-native-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-propagated-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-transitive-propagated-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-native-search-paths @var{package} -@deffnx {Scheme Procedure} inferior-package-transitive-native-search-paths @var{package} -@deffnx {Scheme Procedure} inferior-package-search-paths @var{package} -These procedures are the counterpart of package record accessors -(@pxref{package Reference}). Most of them work by querying the inferior -@var{package} comes from, so the inferior must still be live when you call -these procedures. -@end deffn +@noindent +This indicates that substitutes from @code{@value{SUBSTITUTE-SERVER}} are usable and +will be downloaded, when possible, for future builds. -Inferior packages can be used transparently like any other package or -file-like object in G-expressions (@pxref{G-Expressions}). They are also -transparently handled by the @code{packages->manifest} procedure, which is -commonly use in manifests (@pxref{Invoking guix package, the -@option{--manifest} option of @command{guix package}}). Thus you can insert -an inferior package pretty much anywhere you would insert a regular package: -in manifests, in the @code{packages} field of your @code{operating-system} -declaration, and so on. +@cindex substitutes, how to disable +The substitute mechanism can be disabled globally by running +@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking +guix-daemon}). It can also be disabled temporarily by passing the +@code{--no-substitutes} option to @command{guix package}, @command{guix +build}, and other command-line tools. -@node Invoking guix describe -@section Invoking @command{guix describe} +@node Substitute Authentication +@subsection Substitute Authentication -@cindex reproducibility -@cindex replicating Guix -Often you may want to answer questions like: ``Which revision of Guix am I -using?'' or ``Which channels am I using?'' This is useful information in many -situations: if you want to @emph{replicate} an environment on a different -machine or user account, if you want to report a bug or to determine what -change in the channels you are using caused it, or if you want to record your -system state for reproducibility purposes. The @command{guix describe} -command answers these questions. +@cindex digital signatures +Guix detects and raises an error when attempting to use a substitute +that has been tampered with. Likewise, it ignores substitutes that are +not signed, or that are not signed by one of the keys listed in the ACL. -When run from a @command{guix pull}ed @command{guix}, @command{guix describe} -displays the channel(s) that it was built from, including their repository URL -and commit IDs (@pxref{Channels}): +There is one exception though: if an unauthorized server provides +substitutes that are @emph{bit-for-bit identical} to those provided by +an authorized server, then the unauthorized server becomes eligible for +downloads. For example, assume we have chosen two substitute servers +with this option: @example -$ guix describe -Generation 10 Sep 03 2018 17:32:44 (current) - guix e0fa68c - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: master - commit: e0fa68c7718fffd33d81af415279d6ddb518f727 +--substitute-urls="https://a.example.org https://b.example.org" @end example -If you're familiar with the Git version control system, this is similar in -spirit to @command{git describe}; the output is also similar to that of -@command{guix pull --list-generations}, but limited to the current generation -(@pxref{Invoking guix pull, the @option{--list-generations} option}). Because -the Git commit ID shown above unambiguously refers to a snapshot of Guix, this -information is all it takes to describe the revision of Guix you're using, and -also to replicate it. +@noindent +@cindex reproducible builds +If the ACL contains only the key for @code{b.example.org}, and if +@code{a.example.org} happens to serve the @emph{exact same} substitutes, +then Guix will download substitutes from @code{a.example.org} because it +comes first in the list and can be considered a mirror of +@code{b.example.org}. In practice, independent build machines usually +produce the same binaries, thanks to bit-reproducible builds (see +below). -To make it easier to replicate Guix, @command{guix describe} can also be asked -to return a list of channels instead of the human-readable description above: +When using HTTPS, the server's X.509 certificate is @emph{not} validated +(in other words, the server is not authenticated), contrary to what +HTTPS clients such as Web browsers usually do. This is because Guix +authenticates substitute information itself, as explained above, which +is what we care about (whereas X.509 certificates are about +authenticating bindings between domain names and public keys.) -@example -$ guix describe -f channels -(list (channel - (name 'guix) - (url "https://git.savannah.gnu.org/git/guix.git") - (commit - "e0fa68c7718fffd33d81af415279d6ddb518f727"))) -@end example +@node Proxy Settings +@subsection Proxy Settings -@noindent -You can save this to a file and feed it to @command{guix pull -C} on some -other machine or at a later point in time, which will instantiate @emph{this -exact Guix revision} (@pxref{Invoking guix pull, the @option{-C} option}). -From there on, since you're able to deploy the same revision of Guix, you can -just as well @emph{replicate a complete software environment}. We humbly -think that this is @emph{awesome}, and we hope you'll like it too! +@vindex http_proxy +Substitutes are downloaded over HTTP or HTTPS. +The @code{http_proxy} environment +variable can be set in the environment of @command{guix-daemon} and is +honored for downloads of substitutes. Note that the value of +@code{http_proxy} in the environment where @command{guix build}, +@command{guix package}, and other client commands are run has +@emph{absolutely no effect}. -The details of the options supported by @command{guix describe} are as -follows: +@node Substitution Failure +@subsection Substitution Failure -@table @code -@item --format=@var{format} -@itemx -f @var{format} -Produce output in the specified @var{format}, one of: +Even when a substitute for a derivation is available, sometimes the +substitution attempt will fail. This can happen for a variety of +reasons: the substitute server might be offline, the substitute may +recently have been deleted, the connection might have been interrupted, +etc. -@table @code -@item human -produce human-readable output; -@item channels -produce a list of channel specifications that can be passed to @command{guix -pull -C} or installed as @file{~/.config/guix/channels.scm} (@pxref{Invoking -guix pull}); -@item json -@cindex JSON -produce a list of channel specifications in JSON format; -@item recutils -produce a list of channel specifications in Recutils format. -@end table +When substitutes are enabled and a substitute for a derivation is +available, but the substitution attempt fails, Guix will attempt to +build the derivation locally depending on whether or not +@code{--fallback} was given (@pxref{fallback-option,, common build +option @code{--fallback}}). Specifically, if @code{--fallback} was +omitted, then no local build will be performed, and the derivation is +considered to have failed. However, if @code{--fallback} was given, +then Guix will attempt to build the derivation locally, and the success +or failure of the derivation depends on the success or failure of the +local build. Note that when substitutes are disabled or no substitute +is available for the derivation in question, a local build will +@emph{always} be performed, regardless of whether or not +@code{--fallback} was given. -@item --profile=@var{profile} -@itemx -p @var{profile} -Display information about @var{profile}. -@end table +To get an idea of how many substitutes are available right now, you can +try running the @command{guix weather} command (@pxref{Invoking guix +weather}). This command provides statistics on the substitutes provided +by a server. -@node Invoking guix pack -@section Invoking @command{guix pack} +@node On Trusting Binaries +@subsection On Trusting Binaries -Occasionally you want to pass software to people who are not (yet!) -lucky enough to be using Guix. You'd tell them to run @command{guix -package -i @var{something}}, but that's not possible in this case. This -is where @command{guix pack} comes in. +@cindex trust, of pre-built binaries +Today, each individual's control over their own computing is at the +mercy of institutions, corporations, and groups with enough power and +determination to subvert the computing infrastructure and exploit its +weaknesses. While using @code{@value{SUBSTITUTE-SERVER}} substitutes can be +convenient, we encourage users to also build on their own, or even run +their own build farm, such that @code{@value{SUBSTITUTE-SERVER}} is less of an +interesting target. One way to help is by publishing the software you +build using @command{guix publish} so that others have one more choice +of server to download substitutes from (@pxref{Invoking guix publish}). -@quotation Note -If you are looking for ways to exchange binaries among machines that -already run Guix, @pxref{Invoking guix copy}, @ref{Invoking guix -publish}, and @ref{Invoking guix archive}. -@end quotation +Guix has the foundations to maximize build reproducibility +(@pxref{Features}). In most cases, independent builds of a given +package or derivation should yield bit-identical results. Thus, through +a diverse set of independent package builds, we can strengthen the +integrity of our systems. The @command{guix challenge} command aims to +help users assess substitute servers, and to assist developers in +finding out about non-deterministic package builds (@pxref{Invoking guix +challenge}). Similarly, the @option{--check} option of @command{guix +build} allows users to check whether previously-installed substitutes +are genuine by rebuilding them locally (@pxref{build-check, +@command{guix build --check}}). -@cindex pack -@cindex bundle -@cindex application bundle -@cindex software bundle -The @command{guix pack} command creates a shrink-wrapped @dfn{pack} or -@dfn{software bundle}: it creates a tarball or some other archive -containing the binaries of the software you're interested in, and all -its dependencies. The resulting archive can be used on any machine that -does not have Guix, and people can run the exact same binaries as those -you have with Guix. The pack itself is created in a bit-reproducible -fashion, so anyone can verify that it really contains the build results -that you pretend to be shipping. +In the future, we want Guix to have support to publish and retrieve +binaries to/from other users, in a peer-to-peer fashion. If you would +like to discuss this project, join us on @email{guix-devel@@gnu.org}. -For example, to create a bundle containing Guile, Emacs, Geiser, and all -their dependencies, you can run: +@node Packages with Multiple Outputs +@section Packages with Multiple Outputs -@example -$ guix pack guile emacs geiser -@dots{} -/gnu/store/@dots{}-pack.tar.gz -@end example +@cindex multiple-output packages +@cindex package outputs +@cindex outputs -The result here is a tarball containing a @file{/gnu/store} directory -with all the relevant packages. The resulting tarball contains a -@dfn{profile} with the three packages of interest; the profile is the -same as would be created by @command{guix package -i}. It is this -mechanism that is used to create Guix's own standalone binary tarball -(@pxref{Binary Installation}). +Often, packages defined in Guix have a single @dfn{output}---i.e., the +source package leads to exactly one directory in the store. When running +@command{guix package -i glibc}, one installs the default output of the +GNU libc package; the default output is called @code{out}, but its name +can be omitted as shown in this command. In this particular case, the +default output of @code{glibc} contains all the C header files, shared +libraries, static libraries, Info documentation, and other supporting +files. -Users of this pack would have to run -@file{/gnu/store/@dots{}-profile/bin/guile} to run Guile, which you may -find inconvenient. To work around it, you can create, say, a -@file{/opt/gnu/bin} symlink to the profile: +Sometimes it is more appropriate to separate the various types of files +produced from a single source package into separate outputs. For +instance, the GLib C library (used by GTK+ and related packages) +installs more than 20 MiB of reference documentation as HTML pages. +To save space for users who do not need it, the documentation goes to a +separate output, called @code{doc}. To install the main GLib output, +which contains everything but the documentation, one would run: @example -guix pack -S /opt/gnu/bin=bin guile emacs geiser +guix package -i glib @end example -@noindent -That way, users can happily type @file{/opt/gnu/bin/guile} and enjoy. - -@cindex relocatable binaries, with @command{guix pack} -What if the recipient of your pack does not have root privileges on -their machine, and thus cannot unpack it in the root file system? In -that case, you will want to use the @code{--relocatable} option (see -below). This option produces @dfn{relocatable binaries}, meaning they -they can be placed anywhere in the file system hierarchy: in the example -above, users can unpack your tarball in their home directory and -directly run @file{./opt/gnu/bin/guile}. - -@cindex Docker, build an image with guix pack -Alternatively, you can produce a pack in the Docker image format using -the following command: - -@example -guix pack -f docker guile emacs geiser -@end example - -@noindent -The result is a tarball that can be passed to the @command{docker load} -command. See the -@uref{https://docs.docker.com/engine/reference/commandline/load/, Docker -documentation} for more information. - -@cindex Singularity, build an image with guix pack -@cindex SquashFS, build an image with guix pack -Yet another option is to produce a SquashFS image with the following -command: +@cindex documentation +The command to install its documentation is: @example -guix pack -f squashfs guile emacs geiser +guix package -i glib:doc @end example -@noindent -The result is a SquashFS file system image that can either be mounted or -directly be used as a file system container image with the -@uref{http://singularity.lbl.gov, Singularity container execution -environment}, using commands like @command{singularity shell} or -@command{singularity exec}. +Some packages install programs with different ``dependency footprints''. +For instance, the WordNet package installs both command-line tools and +graphical user interfaces (GUIs). The former depend solely on the C +library, whereas the latter depend on Tcl/Tk and the underlying X +libraries. In this case, we leave the command-line tools in the default +output, whereas the GUIs are in a separate output. This allows users +who do not need the GUIs to save space. The @command{guix size} command +can help find out about such situations (@pxref{Invoking guix size}). +@command{guix graph} can also be helpful (@pxref{Invoking guix graph}). -Several command-line options allow you to customize your pack: +There are several such multiple-output packages in the GNU distribution. +Other conventional output names include @code{lib} for libraries and +possibly header files, @code{bin} for stand-alone programs, and +@code{debug} for debugging information (@pxref{Installing Debugging +Files}). The outputs of a packages are listed in the third column of +the output of @command{guix package --list-available} (@pxref{Invoking +guix package}). -@table @code -@item --format=@var{format} -@itemx -f @var{format} -Produce a pack in the given @var{format}. -The available formats are: +@node Invoking guix gc +@section Invoking @command{guix gc} -@table @code -@item tarball -This is the default format. It produces a tarball containing all the -specified binaries and symlinks. +@cindex garbage collector +@cindex disk space +Packages that are installed, but not used, may be @dfn{garbage-collected}. +The @command{guix gc} command allows users to explicitly run the garbage +collector to reclaim space from the @file{/gnu/store} directory. It is +the @emph{only} way to remove files from @file{/gnu/store}---removing +files or directories manually may break it beyond repair! -@item docker -This produces a tarball that follows the -@uref{https://github.com/docker/docker/blob/master/image/spec/v1.2.md, -Docker Image Specification}. +@cindex GC roots +@cindex garbage collector roots +The garbage collector has a set of known @dfn{roots}: any file under +@file{/gnu/store} reachable from a root is considered @dfn{live} and +cannot be deleted; any other file is considered @dfn{dead} and may be +deleted. The set of garbage collector roots (``GC roots'' for short) +includes default user profiles; by default, the symlinks under +@file{/var/guix/gcroots} represent these GC roots. New GC roots can be +added with @command{guix build --root}, for example (@pxref{Invoking +guix build}). -@item squashfs -This produces a SquashFS image containing all the specified binaries and -symlinks, as well as empty mount points for virtual file systems like -procfs. -@end table +Prior to running @code{guix gc --collect-garbage} to make space, it is +often useful to remove old generations from user profiles; that way, old +package builds referenced by those generations can be reclaimed. This +is achieved by running @code{guix package --delete-generations} +(@pxref{Invoking guix package}). -@item --relocatable -@itemx -R -Produce @dfn{relocatable binaries}---i.e., binaries that can be placed -anywhere in the file system hierarchy and run from there. For example, -if you create a pack containing Bash with: +Our recommendation is to run a garbage collection periodically, or when +you are short on disk space. For instance, to guarantee that at least +5@tie{}GB are available on your disk, simply run: @example -guix pack -R -S /mybin=bin bash +guix gc -F 5G @end example -@noindent -...@: you can copy that pack to a machine that lacks Guix, and from your -home directory as a normal user, run: - -@example -tar xf pack.tar.gz -./mybin/sh -@end example +It is perfectly safe to run as a non-interactive periodic job +(@pxref{Scheduled Job Execution}, for how to set up such a job on +GuixSD). Running @command{guix gc} with no arguments will collect as +much garbage as it can, but that is often inconvenient: you may find +yourself having to rebuild or re-download software that is ``dead'' from +the GC viewpoint but that is necessary to build other pieces of +software---e.g., the compiler tool chain. -@noindent -In that shell, if you type @code{ls /gnu/store}, you'll notice that -@file{/gnu/store} shows up and contains all the dependencies of -@code{bash}, even though the machine actually lacks @file{/gnu/store} -altogether! That is probably the simplest way to deploy Guix-built -software on a non-Guix machine. +The @command{guix gc} command has three modes of operation: it can be +used to garbage-collect any dead files (the default), to delete specific +files (the @code{--delete} option), to print garbage-collector +information, or for more advanced queries. The garbage collection +options are as follows: -There's a gotcha though: this technique relies on the @dfn{user -namespace} feature of the kernel Linux, which allows unprivileged users -to mount or change root. Old versions of Linux did not support it, and -some GNU/Linux distributions turn it off; on these systems, programs -from the pack @emph{will fail to run}, unless they are unpacked in the -root file system. +@table @code +@item --collect-garbage[=@var{min}] +@itemx -C [@var{min}] +Collect garbage---i.e., unreachable @file{/gnu/store} files and +sub-directories. This is the default operation when no option is +specified. -@item --expression=@var{expr} -@itemx -e @var{expr} -Consider the package @var{expr} evaluates to. +When @var{min} is given, stop once @var{min} bytes have been collected. +@var{min} may be a number of bytes, or it may include a unit as a +suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes +(@pxref{Block size, size specifications,, coreutils, GNU Coreutils}). -This has the same purpose as the same-named option in @command{guix -build} (@pxref{Additional Build Options, @code{--expression} in -@command{guix build}}). +When @var{min} is omitted, collect all the garbage. -@item --manifest=@var{file} -@itemx -m @var{file} -Use the packages contained in the manifest object returned by the Scheme -code in @var{file}. +@item --free-space=@var{free} +@itemx -F @var{free} +Collect garbage until @var{free} space is available under +@file{/gnu/store}, if possible; @var{free} denotes storage space, such +as @code{500MiB}, as described above. -This has a similar purpose as the same-named option in @command{guix -package} (@pxref{profile-manifest, @option{--manifest}}) and uses the -same manifest files. It allows you to define a collection of packages -once and use it both for creating profiles and for creating archives -for use on machines that do not have Guix installed. Note that you can -specify @emph{either} a manifest file @emph{or} a list of packages, -but not both. +When @var{free} or more is already available in @file{/gnu/store}, do +nothing and exit immediately. -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of -the system type of the build host. +@item --delete +@itemx -d +Attempt to delete all the store files and directories specified as +arguments. This fails if some of the files are not in the store, or if +they are still live. -@item --target=@var{triplet} -@cindex cross-compilation -Cross-build for @var{triplet}, which must be a valid GNU triplet, such -as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU -configuration triplets,, autoconf, Autoconf}). +@item --list-failures +List store items corresponding to cached build failures. -@item --compression=@var{tool} -@itemx -C @var{tool} -Compress the resulting tarball using @var{tool}---one of @code{gzip}, -@code{bzip2}, @code{xz}, @code{lzip}, or @code{none} for no compression. +This prints nothing unless the daemon was started with +@option{--cache-failures} (@pxref{Invoking guix-daemon, +@option{--cache-failures}}). -@item --symlink=@var{spec} -@itemx -S @var{spec} -Add the symlinks specified by @var{spec} to the pack. This option can -appear several times. +@item --clear-failures +Remove the specified store items from the failed-build cache. -@var{spec} has the form @code{@var{source}=@var{target}}, where -@var{source} is the symlink that will be created and @var{target} is the -symlink target. +Again, this option only makes sense when the daemon is started with +@option{--cache-failures}. Otherwise, it does nothing. -For instance, @code{-S /opt/gnu/bin=bin} creates a @file{/opt/gnu/bin} -symlink pointing to the @file{bin} sub-directory of the profile. +@item --list-dead +Show the list of dead files and directories still present in the +store---i.e., files and directories no longer reachable from any root. -@item --localstatedir -@itemx --profile-name=@var{name} -Include the ``local state directory'', @file{/var/guix}, in the resulting -pack, and notably the @file{/var/guix/profiles/per-user/root/@var{name}} -profile---by default @var{name} is @code{guix-profile}, which corresponds to -@file{~root/.guix-profile}. +@item --list-live +Show the list of live store files and directories. -@file{/var/guix} contains the store database (@pxref{The Store}) as well -as garbage-collector roots (@pxref{Invoking guix gc}). Providing it in -the pack means that the store is ``complete'' and manageable by Guix; -not providing it pack means that the store is ``dead'': items cannot be -added to it or removed from it after extraction of the pack. +@end table -One use case for this is the Guix self-contained binary tarball -(@pxref{Binary Installation}). +In addition, the references among existing store files can be queried: -@item --bootstrap -Use the bootstrap binaries to build the pack. This option is only -useful to Guix developers. -@end table +@table @code -In addition, @command{guix pack} supports all the common build options -(@pxref{Common Build Options}) and all the package transformation -options (@pxref{Package Transformation Options}). +@item --references +@itemx --referrers +@cindex package dependencies +List the references (respectively, the referrers) of store files given +as arguments. +@item --requisites +@itemx -R +@cindex closure +List the requisites of the store files passed as arguments. Requisites +include the store files themselves, their references, and the references +of these, recursively. In other words, the returned list is the +@dfn{transitive closure} of the store files. -@node Invoking guix archive -@section Invoking @command{guix archive} +@xref{Invoking guix size}, for a tool to profile the size of the closure +of an element. @xref{Invoking guix graph}, for a tool to visualize +the graph of references. -@cindex @command{guix archive} -@cindex archive -The @command{guix archive} command allows users to @dfn{export} files -from the store into a single archive, and to later @dfn{import} them on -a machine that runs Guix. -In particular, it allows store files to be transferred from one machine -to the store on another machine. - -@quotation Note -If you're looking for a way to produce archives in a format suitable for -tools other than Guix, @pxref{Invoking guix pack}. -@end quotation - -@cindex exporting store items -To export store files as an archive to standard output, run: - -@example -guix archive --export @var{options} @var{specifications}... -@end example - -@var{specifications} may be either store file names or package -specifications, as for @command{guix package} (@pxref{Invoking guix -package}). For instance, the following command creates an archive -containing the @code{gui} output of the @code{git} package and the main -output of @code{emacs}: - -@example -guix archive --export git:gui /gnu/store/...-emacs-24.3 > great.nar -@end example - -If the specified packages are not built yet, @command{guix archive} -automatically builds them. The build process may be controlled with the -common build options (@pxref{Common Build Options}). +@item --derivers +@cindex derivation +Return the derivation(s) leading to the given store items +(@pxref{Derivations}). -To transfer the @code{emacs} package to a machine connected over SSH, -one would run: +For example, this command: @example -guix archive --export -r emacs | ssh the-machine guix archive --import +guix gc --derivers `guix package -I ^emacs$ | cut -f4` @end example @noindent -Similarly, a complete user profile may be transferred from one machine -to another like this: +returns the @file{.drv} file(s) leading to the @code{emacs} package +installed in your profile. -@example -guix archive --export -r $(readlink -f ~/.guix-profile) | \ - ssh the-machine guix-archive --import -@end example +Note that there may be zero matching @file{.drv} files, for instance +because these files have been garbage-collected. There can also be more +than one matching @file{.drv} due to fixed-output derivations. +@end table -@noindent -However, note that, in both examples, all of @code{emacs} and the -profile as well as all of their dependencies are transferred (due to -@code{-r}), regardless of what is already available in the store on the -target machine. The @code{--missing} option can help figure out which -items are missing from the target store. The @command{guix copy} -command simplifies and optimizes this whole process, so this is probably -what you should use in this case (@pxref{Invoking guix copy}). +Lastly, the following options allow you to check the integrity of the +store and to control disk usage. -@cindex nar, archive format -@cindex normalized archive (nar) -Archives are stored in the ``normalized archive'' or ``nar'' format, which is -comparable in spirit to `tar', but with differences -that make it more appropriate for our purposes. First, rather than -recording all Unix metadata for each file, the nar format only mentions -the file type (regular, directory, or symbolic link); Unix permissions -and owner/group are dismissed. Second, the order in which directory -entries are stored always follows the order of file names according to -the C locale collation order. This makes archive production fully -deterministic. +@table @option -When exporting, the daemon digitally signs the contents of the archive, -and that digital signature is appended. When importing, the daemon -verifies the signature and rejects the import in case of an invalid -signature or if the signing key is not authorized. -@c FIXME: Add xref to daemon doc about signatures. +@item --verify[=@var{options}] +@cindex integrity, of the store +@cindex integrity checking +Verify the integrity of the store. -The main options are: +By default, make sure that all the store items marked as valid in the +database of the daemon actually exist in @file{/gnu/store}. -@table @code -@item --export -Export the specified store files or packages (see below.) Write the -resulting archive to the standard output. +When provided, @var{options} must be a comma-separated list containing one +or more of @code{contents} and @code{repair}. -Dependencies are @emph{not} included in the output, unless -@code{--recursive} is passed. +When passing @option{--verify=contents}, the daemon computes the +content hash of each store item and compares it against its hash in the +database. Hash mismatches are reported as data corruptions. Because it +traverses @emph{all the files in the store}, this command can take a +long time, especially on systems with a slow disk drive. -@item -r -@itemx --recursive -When combined with @code{--export}, this instructs @command{guix -archive} to include dependencies of the given items in the archive. -Thus, the resulting archive is self-contained: it contains the closure -of the exported store items. +@cindex repairing the store +@cindex corruption, recovering from +Using @option{--verify=repair} or @option{--verify=contents,repair} +causes the daemon to try to repair corrupt store items by fetching +substitutes for them (@pxref{Substitutes}). Because repairing is not +atomic, and thus potentially dangerous, it is available only to the +system administrator. A lightweight alternative, when you know exactly +which items in the store are corrupt, is @command{guix build --repair} +(@pxref{Invoking guix build}). -@item --import -Read an archive from the standard input, and import the files listed -therein into the store. Abort if the archive has an invalid digital -signature, or if it is signed by a public key not among the authorized -keys (see @code{--authorize} below.) +@item --optimize +@cindex deduplication +Optimize the store by hard-linking identical files---this is +@dfn{deduplication}. -@item --missing -Read a list of store file names from the standard input, one per line, -and write on the standard output the subset of these files missing from -the store. +The daemon performs deduplication after each successful build or archive +import, unless it was started with @code{--disable-deduplication} +(@pxref{Invoking guix-daemon, @code{--disable-deduplication}}). Thus, +this option is primarily useful when the daemon was running with +@code{--disable-deduplication}. -@item --generate-key[=@var{parameters}] -@cindex signing, archives -Generate a new key pair for the daemon. This is a prerequisite before -archives can be exported with @code{--export}. Note that this operation -usually takes time, because it needs to gather enough entropy to -generate the key pair. +@end table -The generated key pair is typically stored under @file{/etc/guix}, in -@file{signing-key.pub} (public key) and @file{signing-key.sec} (private -key, which must be kept secret.) When @var{parameters} is omitted, -an ECDSA key using the Ed25519 curve is generated, or, for Libgcrypt -versions before 1.6.0, it is a 4096-bit RSA key. -Alternatively, @var{parameters} can specify -@code{genkey} parameters suitable for Libgcrypt (@pxref{General -public-key related Functions, @code{gcry_pk_genkey},, gcrypt, The -Libgcrypt Reference Manual}). +@node Invoking guix pull +@section Invoking @command{guix pull} -@item --authorize -@cindex authorizing, archives -Authorize imports signed by the public key passed on standard input. -The public key must be in ``s-expression advanced format''---i.e., the -same format as the @file{signing-key.pub} file. +@cindex upgrading Guix +@cindex updating Guix +@cindex @command{guix pull} +@cindex pull +Packages are installed or upgraded to the latest version available in +the distribution currently available on your local machine. To update +that distribution, along with the Guix tools, you must run @command{guix +pull}: the command downloads the latest Guix source code and package +descriptions, and deploys it. Source code is downloaded from a +@uref{https://git-scm.com, Git} repository, by default the official +GNU@tie{}Guix repository, though this can be customized. -The list of authorized keys is kept in the human-editable file -@file{/etc/guix/acl}. The file contains -@url{http://people.csail.mit.edu/rivest/Sexp.txt, ``advanced-format -s-expressions''} and is structured as an access-control list in the -@url{http://theworld.com/~cme/spki.txt, Simple Public-Key Infrastructure -(SPKI)}. +On completion, @command{guix package} will use packages and package +versions from this just-retrieved copy of Guix. Not only that, but all +the Guix commands and Scheme modules will also be taken from that latest +version. New @command{guix} sub-commands added by the update also +become available. -@item --extract=@var{directory} -@itemx -x @var{directory} -Read a single-item archive as served by substitute servers -(@pxref{Substitutes}) and extract it to @var{directory}. This is a -low-level operation needed in only very narrow use cases; see below. +Any user can update their Guix copy using @command{guix pull}, and the +effect is limited to the user who run @command{guix pull}. For +instance, when user @code{root} runs @command{guix pull}, this has no +effect on the version of Guix that user @code{alice} sees, and vice +versa. -For example, the following command extracts the substitute for Emacs -served by @code{@value{SUBSTITUTE-SERVER}} to @file{/tmp/emacs}: +The result of running @command{guix pull} is a @dfn{profile} available +under @file{~/.config/guix/current} containing the latest Guix. Thus, +make sure to add it to the beginning of your search path so that you use +the latest version, and similarly for the Info manual +(@pxref{Documentation}): @example -$ wget -O - \ - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-emacs-24.5 \ - | bunzip2 | guix archive -x /tmp/emacs +export PATH="$HOME/.config/guix/current/bin:$PATH" +export INFOPATH="$HOME/.config/guix/current/share/info:$INFOPATH" @end example -Single-item archives are different from multiple-item archives produced -by @command{guix archive --export}; they contain a single store item, -and they do @emph{not} embed a signature. Thus this operation does -@emph{no} signature verification and its output should be considered -unsafe. +The @code{--list-generations} or @code{-l} option lists past generations +produced by @command{guix pull}, along with details about their provenance: -The primary purpose of this operation is to facilitate inspection of -archive contents coming from possibly untrusted substitute servers. +@example +$ guix pull -l +Generation 1 Jun 10 2018 00:18:18 + guix 65956ad + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: origin/master + commit: 65956ad3526ba09e1f7a40722c96c6ef7c0936fe -@end table +Generation 2 Jun 11 2018 11:02:49 + guix e0cc7f6 + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: origin/master + commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d + 2 new packages: keepalived, libnfnetlink + 6 packages upgraded: emacs-nix-mode@@2.0.4, + guile2.0-guix@@0.14.0-12.77a1aac, guix@@0.14.0-12.77a1aac, + heimdal@@7.5.0, milkytracker@@1.02.00, nix@@2.0.4 -@c ********************************************************************* -@node Programming Interface -@chapter Programming Interface +Generation 3 Jun 13 2018 23:31:07 (current) + guix 844cc1c + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: origin/master + commit: 844cc1c8f394f03b404c5bb3aee086922373490c + 28 new packages: emacs-helm-ls-git, emacs-helm-mu, @dots{} + 69 packages upgraded: borg@@1.1.6, cheese@@3.28.0, @dots{} +@end example -GNU Guix provides several Scheme programming interfaces (APIs) to -define, build, and query packages. The first interface allows users to -write high-level package definitions. These definitions refer to -familiar packaging concepts, such as the name and version of a package, -its build system, and its dependencies. These definitions can then be -turned into concrete build actions. +@ref{Invoking guix describe, @command{guix describe}}, for other ways to +describe the current status of Guix. -Build actions are performed by the Guix daemon, on behalf of users. In a -standard setup, the daemon has write access to the store---the -@file{/gnu/store} directory---whereas users do not. The recommended -setup also has the daemon perform builds in chroots, under a specific -build users, to minimize interference with the rest of the system. - -@cindex derivation -Lower-level APIs are available to interact with the daemon and the -store. To instruct the daemon to perform a build action, users actually -provide it with a @dfn{derivation}. A derivation is a low-level -representation of the build actions to be taken, and the environment in -which they should occur---derivations are to package definitions what -assembly is to C programs. The term ``derivation'' comes from the fact -that build results @emph{derive} from them. - -This chapter describes all these APIs in turn, starting from high-level -package definitions. - -@menu -* Defining Packages:: Defining new packages. -* Build Systems:: Specifying how packages are built. -* The Store:: Manipulating the package store. -* Derivations:: Low-level interface to package derivations. -* The Store Monad:: Purely functional interface to the store. -* G-Expressions:: Manipulating build expressions. -* Invoking guix repl:: Fiddling with Guix interactively. -@end menu - -@node Defining Packages -@section Defining Packages - -The high-level interface to package definitions is implemented in the -@code{(guix packages)} and @code{(guix build-system)} modules. As an -example, the package definition, or @dfn{recipe}, for the GNU Hello -package looks like this: +This @code{~/.config/guix/current} profile works like any other profile +created by @command{guix package} (@pxref{Invoking guix package}). That +is, you can list generations, roll back to the previous +generation---i.e., the previous Guix---and so on: @example -(define-module (gnu packages hello) - #:use-module (guix packages) - #:use-module (guix download) - #:use-module (guix build-system gnu) - #:use-module (guix licenses) - #:use-module (gnu packages gawk)) - -(define-public hello - (package - (name "hello") - (version "2.10") - (source (origin - (method url-fetch) - (uri (string-append "mirror://gnu/hello/hello-" version - ".tar.gz")) - (sha256 - (base32 - "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) - (build-system gnu-build-system) - (arguments '(#:configure-flags '("--enable-silent-rules"))) - (inputs `(("gawk" ,gawk))) - (synopsis "Hello, GNU world: An example GNU package") - (description "Guess what GNU Hello prints!") - (home-page "http://www.gnu.org/software/hello/") - (license gpl3+))) +$ guix package -p ~/.config/guix/current --roll-back +switched from generation 3 to 2 +$ guix package -p ~/.config/guix/current --delete-generations=1 +deleting /var/guix/profiles/per-user/charlie/current-guix-1-link @end example -@noindent -Without being a Scheme expert, the reader may have guessed the meaning -of the various fields here. This expression binds the variable -@code{hello} to a @code{} object, which is essentially a record -(@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}). -This package object can be inspected using procedures found in the -@code{(guix packages)} module; for instance, @code{(package-name hello)} -returns---surprise!---@code{"hello"}. - -With luck, you may be able to import part or all of the definition of -the package you are interested in from another repository, using the -@code{guix import} command (@pxref{Invoking guix import}). - -In the example above, @var{hello} is defined in a module of its own, -@code{(gnu packages hello)}. Technically, this is not strictly -necessary, but it is convenient to do so: all the packages defined in -modules under @code{(gnu packages @dots{})} are automatically known to -the command-line tools (@pxref{Package Modules}). - -There are a few points worth noting in the above package definition: - -@itemize -@item -The @code{source} field of the package is an @code{} object -(@pxref{origin Reference}, for the complete reference). -Here, the @code{url-fetch} method from @code{(guix download)} is used, -meaning that the source is a file to be downloaded over FTP or HTTP. - -The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of -the GNU mirrors defined in @code{(guix download)}. - -The @code{sha256} field specifies the expected SHA256 hash of the file -being downloaded. It is mandatory, and allows Guix to check the -integrity of the file. The @code{(base32 @dots{})} form introduces the -base32 representation of the hash. You can obtain this information with -@code{guix download} (@pxref{Invoking guix download}) and @code{guix -hash} (@pxref{Invoking guix hash}). - -@cindex patches -When needed, the @code{origin} form can also have a @code{patches} field -listing patches to be applied, and a @code{snippet} field giving a -Scheme expression to modify the source code. - -@item -@cindex GNU Build System -The @code{build-system} field specifies the procedure to build the -package (@pxref{Build Systems}). Here, @var{gnu-build-system} -represents the familiar GNU Build System, where packages may be -configured, built, and installed with the usual @code{./configure && -make && make check && make install} command sequence. - -@item -The @code{arguments} field specifies options for the build system -(@pxref{Build Systems}). Here it is interpreted by -@var{gnu-build-system} as a request run @file{configure} with the -@code{--enable-silent-rules} flag. - -@cindex quote -@cindex quoting -@findex ' -@findex quote -What about these quote (@code{'}) characters? They are Scheme syntax to -introduce a literal list; @code{'} is synonymous with @code{quote}. -@xref{Expression Syntax, quoting,, guile, GNU Guile Reference Manual}, -for details. Here the value of the @code{arguments} field is a list of -arguments passed to the build system down the road, as with @code{apply} -(@pxref{Fly Evaluation, @code{apply},, guile, GNU Guile Reference -Manual}). - -The hash-colon (@code{#:}) sequence defines a Scheme @dfn{keyword} -(@pxref{Keywords,,, guile, GNU Guile Reference Manual}), and -@code{#:configure-flags} is a keyword used to pass a keyword argument -to the build system (@pxref{Coding With Keywords,,, guile, GNU Guile -Reference Manual}). - -@item -The @code{inputs} field specifies inputs to the build process---i.e., -build-time or run-time dependencies of the package. Here, we define an -input called @code{"gawk"} whose value is that of the @var{gawk} -variable; @var{gawk} is itself bound to a @code{} object. - -@cindex backquote (quasiquote) -@findex ` -@findex quasiquote -@cindex comma (unquote) -@findex , -@findex unquote -@findex ,@@ -@findex unquote-splicing -Again, @code{`} (a backquote, synonymous with @code{quasiquote}) allows -us to introduce a literal list in the @code{inputs} field, while -@code{,} (a comma, synonymous with @code{unquote}) allows us to insert a -value in that list (@pxref{Expression Syntax, unquote,, guile, GNU Guile -Reference Manual}). +The @command{guix pull} command is usually invoked with no arguments, +but it supports the following options: -Note that GCC, Coreutils, Bash, and other essential tools do not need to -be specified as inputs here. Instead, @var{gnu-build-system} takes care -of ensuring that they are present (@pxref{Build Systems}). +@table @code +@item --url=@var{url} +@itemx --commit=@var{commit} +@itemx --branch=@var{branch} +Download code from the specified @var{url}, at the given @var{commit} (a valid +Git commit ID represented as a hexadecimal string), or @var{branch}. -However, any other dependencies need to be specified in the -@code{inputs} field. Any dependency not specified here will simply be -unavailable to the build process, possibly leading to a build failure. -@end itemize +@cindex @file{channels.scm}, configuration file +@cindex configuration file for channels +These options are provided for convenience, but you can also specify your +configuration in the @file{~/.config/guix/channels.scm} file or using the +@option{--channels} option (see below). -@xref{package Reference}, for a full description of possible fields. +@item --channels=@var{file} +@itemx -C @var{file} +Read the list of channels from @var{file} instead of +@file{~/.config/guix/channels.scm}. @var{file} must contain Scheme code that +evaluates to a list of channel objects. @xref{Channels}, for more +information. -Once a package definition is in place, the -package may actually be built using the @code{guix build} command-line -tool (@pxref{Invoking guix build}), troubleshooting any build failures -you encounter (@pxref{Debugging Build Failures}). You can easily jump back to the -package definition using the @command{guix edit} command -(@pxref{Invoking guix edit}). -@xref{Packaging Guidelines}, for -more information on how to test package definitions, and -@ref{Invoking guix lint}, for information on how to check a definition -for style conformance. -@vindex GUIX_PACKAGE_PATH -Lastly, @pxref{Channels}, for information -on how to extend the distribution by adding your own package definitions -in a ``channel''. +@item --list-generations[=@var{pattern}] +@itemx -l [@var{pattern}] +List all the generations of @file{~/.config/guix/current} or, if @var{pattern} +is provided, the subset of generations that match @var{pattern}. +The syntax of @var{pattern} is the same as with @code{guix package +--list-generations} (@pxref{Invoking guix package}). -Finally, updating the package definition to a new upstream version -can be partly automated by the @command{guix refresh} command -(@pxref{Invoking guix refresh}). +@ref{Invoking guix describe}, for a way to display information about the +current generation only. -Behind the scenes, a derivation corresponding to the @code{} -object is first computed by the @code{package-derivation} procedure. -That derivation is stored in a @code{.drv} file under @file{/gnu/store}. -The build actions it prescribes may then be realized by using the -@code{build-derivations} procedure (@pxref{The Store}). +@item --profile=@var{profile} +@itemx -p @var{profile} +Use @var{profile} instead of @file{~/.config/guix/current}. -@deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}] -Return the @code{} object of @var{package} for @var{system} -(@pxref{Derivations}). +@item --dry-run +@itemx -n +Show which channel commit(s) would be used and what would be built or +substituted but do not actually do it. -@var{package} must be a valid @code{} object, and @var{system} -must be a string denoting the target system type---e.g., -@code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store} -must be a connection to the daemon, which operates on the store -(@pxref{The Store}). -@end deffn +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of +the system type of the build host. -@noindent -@cindex cross-compilation -Similarly, it is possible to compute a derivation that cross-builds a -package for some other system: +@item --verbose +Produce verbose output, writing build logs to the standard error output. -@deffn {Scheme Procedure} package-cross-derivation @var{store} @ - @var{package} @var{target} [@var{system}] -Return the @code{} object of @var{package} cross-built from -@var{system} to @var{target}. +@item --bootstrap +Use the bootstrap Guile to build the latest Guix. This option is only +useful to Guix developers. +@end table -@var{target} must be a valid GNU triplet denoting the target hardware -and operating system, such as @code{"mips64el-linux-gnu"} -(@pxref{Configuration Names, GNU configuration triplets,, configure, GNU -Configure and Build System}). -@end deffn +The @dfn{channel} mechanism allows you to instruct @command{guix pull} which +repository and branch to pull from, as well as @emph{additional} repositories +containing package modules that should be deployed. @xref{Channels}, for more +information. -@cindex package transformations -@cindex input rewriting -@cindex dependency tree rewriting -Packages can be manipulated in arbitrary ways. An example of a useful -transformation is @dfn{input rewriting}, whereby the dependency tree of -a package is rewritten by replacing specific inputs by others: +In addition, @command{guix pull} supports all the common build options +(@pxref{Common Build Options}). -@deffn {Scheme Procedure} package-input-rewriting @var{replacements} @ - [@var{rewrite-name}] -Return a procedure that, when passed a package, replaces its direct and -indirect dependencies (but not its implicit inputs) according to -@var{replacements}. @var{replacements} is a list of package pairs; the -first element of each pair is the package to replace, and the second one -is the replacement. +@node Channels +@section Channels -Optionally, @var{rewrite-name} is a one-argument procedure that takes -the name of a package and returns its new name after rewrite. -@end deffn +@cindex channels +@cindex @file{channels.scm}, configuration file +@cindex configuration file for channels +@cindex @command{guix pull}, configuration file +@cindex configuration of @command{guix pull} +Guix and its package collection are updated by running @command{guix pull} +(@pxref{Invoking guix pull}). By default @command{guix pull} downloads and +deploys Guix itself from the official GNU@tie{}Guix repository. This can be +customized by defining @dfn{channels} in the +@file{~/.config/guix/channels.scm} file. A channel specifies a URL and branch +of a Git repository to be deployed, and @command{guix pull} can be instructed +to pull from one or more channels. In other words, channels can be used to +@emph{customize} and to @emph{extend} Guix, as we will see below. -@noindent -Consider this example: +@subsection Using a Custom Guix Channel -@example -(define libressl-instead-of-openssl - ;; This is a procedure to replace OPENSSL by LIBRESSL, - ;; recursively. - (package-input-rewriting `((,openssl . ,libressl)))) +The channel called @code{guix} specifies where Guix itself---its command-line +tools as well as its package collection---should be downloaded. For instance, +suppose you want to update from your own copy of the Guix repository at +@code{example.org}, and specifically the @code{super-hacks} branch, you can +write in @code{~/.config/guix/channels.scm} this specification: -(define git-with-libressl - (libressl-instead-of-openssl git)) -@end example +@lisp +;; Tell 'guix pull' to use my own repo. +(list (channel + (name 'guix) + (url "https://example.org/my-guix.git") + (branch "super-hacks"))) +@end lisp @noindent -Here we first define a rewriting procedure that replaces @var{openssl} -with @var{libressl}. Then we use it to define a @dfn{variant} of the -@var{git} package that uses @var{libressl} instead of @var{openssl}. -This is exactly what the @option{--with-input} command-line option does -(@pxref{Package Transformation Options, @option{--with-input}}). - -A more generic procedure to rewrite a package dependency graph is -@code{package-mapping}: it supports arbitrary changes to nodes in the -graph. - -@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] -Return a procedure that, given a package, applies @var{proc} to all the packages -depended on and returns the resulting package. The procedure stops recursion -when @var{cut?} returns true for a given package. -@end deffn - -@menu -* package Reference:: The package data type. -* origin Reference:: The origin data type. -@end menu +From there on, @command{guix pull} will fetch code from the @code{super-hacks} +branch of the repository at @code{example.org}. +@subsection Specifying Additional Channels -@node package Reference -@subsection @code{package} Reference +@cindex extending the package collection (channels) +@cindex personal packages (channels) +@cindex channels, for personal packages +You can also specify @emph{additional channels} to pull from. Let's say you +have a bunch of custom package variants or personal packages that you think +would make little sense to contribute to the Guix project, but would like to +have these packages transparently available to you at the command line. You +would first write modules containing those package definitions (@pxref{Package +Modules}), maintain them in a Git repository, and then you and anyone else can +use it as an additional channel to get packages from. Neat, no? -This section summarizes all the options available in @code{package} -declarations (@pxref{Defining Packages}). +@c What follows stems from discussions at +@c as well as +@c earlier discussions on guix-devel@gnu.org. +@quotation Warning +Before you, dear user, shout---``woow this is @emph{soooo coool}!''---and +publish your personal channel to the world, we would like to share a few words +of caution: -@deftp {Data Type} package -This is the data type representing a package recipe. +@itemize +@item +Before publishing a channel, please consider contributing your package +definitions to Guix proper (@pxref{Contributing}). Guix as a project is open +to free software of all sorts, and packages in Guix proper are readily +available to all Guix users and benefit from the project's quality assurance +process. -@table @asis -@item @code{name} -The name of the package, as a string. +@item +When you maintain package definitions outside Guix, we, Guix developers, +consider that @emph{the compatibility burden is on you}. Remember that +package modules and package definitions are just Scheme code that uses various +programming interfaces (APIs). We want to remain free to change these APIs to +keep improving Guix, possibly in ways that break your channel. We never +change APIs gratuitously, but we will @emph{not} commit to freezing APIs +either. -@item @code{version} -The version of the package, as a string. +@item +Corollary: if you're using an external channel and that channel breaks, please +@emph{report the issue to the channel authors}, not to the Guix project. +@end itemize -@item @code{source} -An object telling how the source code for the package should be -acquired. Most of the time, this is an @code{origin} object, which -denotes a file fetched from the Internet (@pxref{origin Reference}). It -can also be any other ``file-like'' object such as a @code{local-file}, -which denotes a file from the local file system (@pxref{G-Expressions, -@code{local-file}}). +You've been warned! Having said this, we believe external channels are a +practical way to exert your freedom to augment Guix' package collection and to +share your improvements, which are basic tenets of +@uref{https://www.gnu.org/philosophy/free-sw.html, free software}. Please +email us at @email{guix-devel@@gnu.org} if you'd like to discuss this. +@end quotation -@item @code{build-system} -The build system that should be used to build the package (@pxref{Build -Systems}). +Once you have a Git repository containing your own package modules, you can +write @code{~/.config/guix/channels.scm} to instruct @command{guix pull} to +pull from your personal channel @emph{in addition} to the default Guix +channel(s): -@item @code{arguments} (default: @code{'()}) -The arguments that should be passed to the build system. This is a -list, typically containing sequential keyword-value pairs. +@vindex %default-channels +@lisp +;; Add my personal packages to those Guix provides. +(cons (channel + (name 'my-personal-packages) + (url "https://example.org/personal-packages.git")) + %default-channels) +@end lisp -@item @code{inputs} (default: @code{'()}) -@itemx @code{native-inputs} (default: @code{'()}) -@itemx @code{propagated-inputs} (default: @code{'()}) -@cindex inputs, of packages -These fields list dependencies of the package. Each one is a list of -tuples, where each tuple has a label for the input (a string) as its -first element, a package, origin, or derivation as its second element, -and optionally the name of the output thereof that should be used, which -defaults to @code{"out"} (@pxref{Packages with Multiple Outputs}, for -more on package outputs). For example, the list below specifies three -inputs: +@noindent +Note that the snippet above is (as always!)@: Scheme code; we use @code{cons} to +add a channel the list of channels that the variable @code{%default-channels} +is bound to (@pxref{Pairs, @code{cons} and lists,, guile, GNU Guile Reference +Manual}). With this file in place, @command{guix pull} builds not only Guix +but also the package modules from your own repository. The result in +@file{~/.config/guix/current} is the union of Guix with your own package +modules: @example -`(("libffi" ,libffi) - ("libunistring" ,libunistring) - ("glib:bin" ,glib "bin")) ;the "bin" output of Glib +$ guix pull --list-generations +@dots{} +Generation 19 Aug 27 2018 16:20:48 + guix d894ab8 + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: master + commit: d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300 + my-personal-packages dd3df5e + repository URL: https://example.org/personal-packages.git + branch: master + commit: dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb + 11 new packages: my-gimp, my-emacs-with-cool-features, @dots{} + 4 packages upgraded: emacs-racket-mode@@0.0.2-2.1b78827, @dots{} @end example -@cindex cross compilation, package dependencies -The distinction between @code{native-inputs} and @code{inputs} is -necessary when considering cross-compilation. When cross-compiling, -dependencies listed in @code{inputs} are built for the @emph{target} -architecture; conversely, dependencies listed in @code{native-inputs} -are built for the architecture of the @emph{build} machine. +@noindent +The output of @command{guix pull} above shows that Generation@tie{}19 includes +both Guix and packages from the @code{my-personal-packages} channel. Among +the new and upgraded packages that are listed, some like @code{my-gimp} and +@code{my-emacs-with-cool-features} might come from +@code{my-personal-packages}, while others come from the Guix default channel. -@code{native-inputs} is typically used to list tools needed at -build time, but not at run time, such as Autoconf, Automake, pkg-config, -Gettext, or Bison. @command{guix lint} can report likely mistakes in -this area (@pxref{Invoking guix lint}). +@cindex dependencies, channels +@cindex meta-data, channels +@subsection Declaring Channel Dependencies -@anchor{package-propagated-inputs} -Lastly, @code{propagated-inputs} is similar to @code{inputs}, but the -specified packages will be automatically installed alongside the package -they belong to (@pxref{package-cmd-propagated-inputs, @command{guix -package}}, for information on how @command{guix package} deals with -propagated inputs.) +Channel authors may decide to augment a package collection provided by other +channels. They can declare their channel to be dependent on other channels in +a meta-data file @file{.guix-channel}, which is to be placed in the root of +the channel repository. -For example this is necessary when a C/C++ library needs headers of -another library to compile, or when a pkg-config file refers to another -one @i{via} its @code{Requires} field. +The meta-data file should contain a simple S-expression like this: -Another example where @code{propagated-inputs} is useful is for languages -that lack a facility to record the run-time search path akin to the -@code{RUNPATH} of ELF files; this includes Guile, Python, Perl, and -more. To ensure that libraries written in those languages can find -library code they depend on at run time, run-time dependencies must be -listed in @code{propagated-inputs} rather than @code{inputs}. +@lisp +(channel + (version 0) + (dependencies + (channel + (name some-collection) + (url "https://example.org/first-collection.git")) + (channel + (name some-other-collection) + (url "https://example.org/second-collection.git") + (branch "testing")))) +@end lisp -@item @code{self-native-input?} (default: @code{#f}) -This is a Boolean field telling whether the package should use itself as -a native input when cross-compiling. +In the above example this channel is declared to depend on two other channels, +which will both be fetched automatically. The modules provided by the channel +will be compiled in an environment where the modules of all these declared +channels are available. -@item @code{outputs} (default: @code{'("out")}) -The list of output names of the package. @xref{Packages with Multiple -Outputs}, for typical uses of additional outputs. +For the sake of reliability and maintainability, you should avoid dependencies +on channels that you don't control, and you should aim to keep the number of +dependencies to a minimum. -@item @code{native-search-paths} (default: @code{'()}) -@itemx @code{search-paths} (default: @code{'()}) -A list of @code{search-path-specification} objects describing -search-path environment variables honored by the package. +@subsection Replicating Guix -@item @code{replacement} (default: @code{#f}) -This must be either @code{#f} or a package object that will be used as a -@dfn{replacement} for this package. @xref{Security Updates, grafts}, -for details. +@cindex pinning, channels +@cindex replicating Guix +@cindex reproducibility, of Guix +The @command{guix pull --list-generations} output above shows precisely which +commits were used to build this instance of Guix. We can thus replicate it, +say, on another machine, by providing a channel specification in +@file{~/.config/guix/channels.scm} that is ``pinned'' to these commits: -@item @code{synopsis} -A one-line description of the package. +@lisp +;; Deploy specific commits of my channels of interest. +(list (channel + (name 'guix) + (url "https://git.savannah.gnu.org/git/guix.git") + (commit "d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300")) + (channel + (name 'my-personal-packages) + (url "https://example.org/personal-packages.git") + (branch "dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb"))) +@end lisp -@item @code{description} -A more elaborate description of the package. +The @command{guix describe --format=channels} command can even generate this +list of channels directly (@pxref{Invoking guix describe}). -@item @code{license} -@cindex license, of packages -The license of the package; a value from @code{(guix licenses)}, -or a list of such values. +At this point the two machines run the @emph{exact same Guix}, with access to +the @emph{exact same packages}. The output of @command{guix build gimp} on +one machine will be exactly the same, bit for bit, as the output of the same +command on the other machine. It also means both machines have access to all +the source code of Guix and, transitively, to all the source code of every +package it defines. -@item @code{home-page} -The URL to the home-page of the package, as a string. +This gives you super powers, allowing you to track the provenance of binary +artifacts with very fine grain, and to reproduce software environments at +will---some sort of ``meta reproducibility'' capabilities, if you will. +@xref{Inferiors}, for another way to take advantage of these super powers. -@item @code{supported-systems} (default: @var{%supported-systems}) -The list of systems supported by the package, as strings of the form -@code{architecture-kernel}, for example @code{"x86_64-linux"}. +@node Inferiors +@section Inferiors -@item @code{maintainers} (default: @code{'()}) -The list of maintainers of the package, as @code{maintainer} objects. +@c TODO: Remove this once we're more confident about API stability. +@quotation Note +The functionality described here is a ``technology preview'' as of version +@value{VERSION}. As such, the interface is subject to change. +@end quotation -@item @code{location} (default: source location of the @code{package} form) -The source location of the package. It is useful to override this when -inheriting from another package, in which case this field is not -automatically corrected. -@end table -@end deftp +@cindex inferiors +@cindex composition of Guix revisions +Sometimes you might need to mix packages from the revision of Guix you're +currently running with packages available in a different revision of Guix. +Guix @dfn{inferiors} allow you to achieve that by composing different Guix +revisions in arbitrary ways. +@cindex inferior packages +Technically, an ``inferior'' is essentially a separate Guix process connected +to your main Guix process through a REPL (@pxref{Invoking guix repl}). The +@code{(guix inferior)} module allows you to create inferiors and to +communicate with them. It also provides a high-level interface to browse and +manipulate the packages that an inferior provides---@dfn{inferior packages}. + +When combined with channels (@pxref{Channels}), inferiors provide a simple way +to interact with a separate revision of Guix. For example, let's assume you +want to install in your profile the current @code{guile} package, along with +the @code{guile-json} as it existed in an older revision of Guix---perhaps +because the newer @code{guile-json} has an incompatible API and you want to +run your code against the old API@. To do that, you could write a manifest for +use by @code{guix package --manifest} (@pxref{Invoking guix package}); in that +manifest, you would create an inferior for that old Guix revision you care +about, and you would look up the @code{guile-json} package in the inferior: -@node origin Reference -@subsection @code{origin} Reference +@lisp +(use-modules (guix inferior) (guix channels) + (srfi srfi-1)) ;for 'first' -This section summarizes all the options available in @code{origin} -declarations (@pxref{Defining Packages}). +(define channels + ;; This is the old revision from which we want to + ;; extract guile-json. + (list (channel + (name 'guix) + (url "https://git.savannah.gnu.org/git/guix.git") + (commit + "65956ad3526ba09e1f7a40722c96c6ef7c0936fe")))) -@deftp {Data Type} origin -This is the data type representing a source code origin. +(define inferior + ;; An inferior representing the above revision. + (inferior-for-channels channels)) -@table @asis -@item @code{uri} -An object containing the URI of the source. The object type depends on -the @code{method} (see below). For example, when using the -@var{url-fetch} method of @code{(guix download)}, the valid @code{uri} -values are: a URL represented as a string, or a list thereof. +;; Now create a manifest with the current "guile" package +;; and the old "guile-json" package. +(packages->manifest + (list (first (lookup-inferior-packages inferior "guile-json")) + (specification->package "guile"))) +@end lisp -@item @code{method} -A procedure that handles the URI. +On its first run, @command{guix package --manifest} might have to build the +channel you specified before it can create the inferior; subsequent runs will +be much faster because the Guix revision will be cached. -Examples include: +The @code{(guix inferior)} module provides the following procedures to open an +inferior: -@table @asis -@item @var{url-fetch} from @code{(guix download)} -download a file from the HTTP, HTTPS, or FTP URL specified in the -@code{uri} field; +@deffn {Scheme Procedure} inferior-for-channels @var{channels} @ + [#:cache-directory] [#:ttl] +Return an inferior for @var{channels}, a list of channels. Use the cache at +@var{cache-directory}, where entries can be reclaimed after @var{ttl} seconds. +This procedure opens a new connection to the build daemon. -@vindex git-fetch -@item @var{git-fetch} from @code{(guix git-download)} -clone the Git version control repository, and check out the revision -specified in the @code{uri} field as a @code{git-reference} object; a -@code{git-reference} looks like this: +As a side effect, this procedure may build or substitute binaries for +@var{channels}, which can take time. +@end deffn -@example -(git-reference - (url "git://git.debian.org/git/pkg-shadow/shadow") - (commit "v4.1.5.1")) -@end example -@end table +@deffn {Scheme Procedure} open-inferior @var{directory} @ + [#:command "bin/guix"] +Open the inferior Guix in @var{directory}, running +@code{@var{directory}/@var{command} repl} or equivalent. Return @code{#f} if +the inferior could not be launched. +@end deffn -@item @code{sha256} -A bytevector containing the SHA-256 hash of the source. Typically the -@code{base32} form is used here to generate the bytevector from a -base-32 string. +@cindex inferior packages +The procedures listed below allow you to obtain and manipulate inferior +packages. -You can obtain this information using @code{guix download} -(@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking -guix hash}). +@deffn {Scheme Procedure} inferior-packages @var{inferior} +Return the list of packages known to @var{inferior}. +@end deffn -@item @code{file-name} (default: @code{#f}) -The file name under which the source code should be saved. When this is -@code{#f}, a sensible default value will be used in most cases. In case -the source is fetched from a URL, the file name from the URL will be -used. For version control checkouts, it is recommended to provide the -file name explicitly because the default is not very descriptive. +@deffn {Scheme Procedure} lookup-inferior-packages @var{inferior} @var{name} @ + [@var{version}] +Return the sorted list of inferior packages matching @var{name} in +@var{inferior}, with highest version numbers first. If @var{version} is true, +return only packages with a version number prefixed by @var{version}. +@end deffn -@item @code{patches} (default: @code{'()}) -A list of file names, origins, or file-like objects (@pxref{G-Expressions, -file-like objects}) pointing to patches to be applied to the source. +@deffn {Scheme Procedure} inferior-package? @var{obj} +Return true if @var{obj} is an inferior package. +@end deffn -This list of patches must be unconditional. In particular, it cannot -depend on the value of @code{%current-system} or -@code{%current-target-system}. +@deffn {Scheme Procedure} inferior-package-name @var{package} +@deffnx {Scheme Procedure} inferior-package-version @var{package} +@deffnx {Scheme Procedure} inferior-package-synopsis @var{package} +@deffnx {Scheme Procedure} inferior-package-description @var{package} +@deffnx {Scheme Procedure} inferior-package-home-page @var{package} +@deffnx {Scheme Procedure} inferior-package-location @var{package} +@deffnx {Scheme Procedure} inferior-package-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-native-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-propagated-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-transitive-propagated-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-native-search-paths @var{package} +@deffnx {Scheme Procedure} inferior-package-transitive-native-search-paths @var{package} +@deffnx {Scheme Procedure} inferior-package-search-paths @var{package} +These procedures are the counterpart of package record accessors +(@pxref{package Reference}). Most of them work by querying the inferior +@var{package} comes from, so the inferior must still be live when you call +these procedures. +@end deffn -@item @code{snippet} (default: @code{#f}) -A G-expression (@pxref{G-Expressions}) or S-expression that will be run -in the source directory. This is a convenient way to modify the source, -sometimes more convenient than a patch. +Inferior packages can be used transparently like any other package or +file-like object in G-expressions (@pxref{G-Expressions}). They are also +transparently handled by the @code{packages->manifest} procedure, which is +commonly use in manifests (@pxref{Invoking guix package, the +@option{--manifest} option of @command{guix package}}). Thus you can insert +an inferior package pretty much anywhere you would insert a regular package: +in manifests, in the @code{packages} field of your @code{operating-system} +declaration, and so on. -@item @code{patch-flags} (default: @code{'("-p1")}) -A list of command-line flags that should be passed to the @code{patch} -command. +@node Invoking guix describe +@section Invoking @command{guix describe} -@item @code{patch-inputs} (default: @code{#f}) -Input packages or derivations to the patching process. When this is -@code{#f}, the usual set of inputs necessary for patching are provided, -such as GNU@tie{}Patch. +@cindex reproducibility +@cindex replicating Guix +Often you may want to answer questions like: ``Which revision of Guix am I +using?'' or ``Which channels am I using?'' This is useful information in many +situations: if you want to @emph{replicate} an environment on a different +machine or user account, if you want to report a bug or to determine what +change in the channels you are using caused it, or if you want to record your +system state for reproducibility purposes. The @command{guix describe} +command answers these questions. -@item @code{modules} (default: @code{'()}) -A list of Guile modules that should be loaded during the patching -process and while running the code in the @code{snippet} field. +When run from a @command{guix pull}ed @command{guix}, @command{guix describe} +displays the channel(s) that it was built from, including their repository URL +and commit IDs (@pxref{Channels}): -@item @code{patch-guile} (default: @code{#f}) -The Guile package that should be used in the patching process. When -this is @code{#f}, a sensible default is used. -@end table -@end deftp +@example +$ guix describe +Generation 10 Sep 03 2018 17:32:44 (current) + guix e0fa68c + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: master + commit: e0fa68c7718fffd33d81af415279d6ddb518f727 +@end example +If you're familiar with the Git version control system, this is similar in +spirit to @command{git describe}; the output is also similar to that of +@command{guix pull --list-generations}, but limited to the current generation +(@pxref{Invoking guix pull, the @option{--list-generations} option}). Because +the Git commit ID shown above unambiguously refers to a snapshot of Guix, this +information is all it takes to describe the revision of Guix you're using, and +also to replicate it. -@node Build Systems -@section Build Systems +To make it easier to replicate Guix, @command{guix describe} can also be asked +to return a list of channels instead of the human-readable description above: -@cindex build system -Each package definition specifies a @dfn{build system} and arguments for -that build system (@pxref{Defining Packages}). This @code{build-system} -field represents the build procedure of the package, as well as implicit -dependencies of that build procedure. +@example +$ guix describe -f channels +(list (channel + (name 'guix) + (url "https://git.savannah.gnu.org/git/guix.git") + (commit + "e0fa68c7718fffd33d81af415279d6ddb518f727"))) +@end example -Build systems are @code{} objects. The interface to -create and manipulate them is provided by the @code{(guix build-system)} -module, and actual build systems are exported by specific modules. +@noindent +You can save this to a file and feed it to @command{guix pull -C} on some +other machine or at a later point in time, which will instantiate @emph{this +exact Guix revision} (@pxref{Invoking guix pull, the @option{-C} option}). +From there on, since you're able to deploy the same revision of Guix, you can +just as well @emph{replicate a complete software environment}. We humbly +think that this is @emph{awesome}, and we hope you'll like it too! -@cindex bag (low-level package representation) -Under the hood, build systems first compile package objects to -@dfn{bags}. A @dfn{bag} is like a package, but with less -ornamentation---in other words, a bag is a lower-level representation of -a package, which includes all the inputs of that package, including some -that were implicitly added by the build system. This intermediate -representation is then compiled to a derivation (@pxref{Derivations}). +The details of the options supported by @command{guix describe} are as +follows: -Build systems accept an optional list of @dfn{arguments}. In package -definitions, these are passed @i{via} the @code{arguments} field -(@pxref{Defining Packages}). They are typically keyword arguments -(@pxref{Optional Arguments, keyword arguments in Guile,, guile, GNU -Guile Reference Manual}). The value of these arguments is usually -evaluated in the @dfn{build stratum}---i.e., by a Guile process launched -by the daemon (@pxref{Derivations}). +@table @code +@item --format=@var{format} +@itemx -f @var{format} +Produce output in the specified @var{format}, one of: -The main build system is @var{gnu-build-system}, which implements the -standard build procedure for GNU and many other packages. It -is provided by the @code{(guix build-system gnu)} module. +@table @code +@item human +produce human-readable output; +@item channels +produce a list of channel specifications that can be passed to @command{guix +pull -C} or installed as @file{~/.config/guix/channels.scm} (@pxref{Invoking +guix pull}); +@item json +@cindex JSON +produce a list of channel specifications in JSON format; +@item recutils +produce a list of channel specifications in Recutils format. +@end table -@defvr {Scheme Variable} gnu-build-system -@var{gnu-build-system} represents the GNU Build System, and variants -thereof (@pxref{Configuration, configuration and makefile conventions,, -standards, GNU Coding Standards}). +@item --profile=@var{profile} +@itemx -p @var{profile} +Display information about @var{profile}. +@end table -@cindex build phases -In a nutshell, packages using it are configured, built, and installed with -the usual @code{./configure && make && make check && make install} -command sequence. In practice, a few additional steps are often needed. -All these steps are split up in separate @dfn{phases}, -notably@footnote{Please see the @code{(guix build gnu-build-system)} -modules for more details about the build phases.}: +@node Invoking guix pack +@section Invoking @command{guix pack} -@table @code -@item unpack -Unpack the source tarball, and change the current directory to the -extracted source tree. If the source is actually a directory, copy it -to the build tree, and enter that directory. +Occasionally you want to pass software to people who are not (yet!) +lucky enough to be using Guix. You'd tell them to run @command{guix +package -i @var{something}}, but that's not possible in this case. This +is where @command{guix pack} comes in. -@item patch-source-shebangs -Patch shebangs encountered in source files so they refer to the right -store file names. For instance, this changes @code{#!/bin/sh} to -@code{#!/gnu/store/@dots{}-bash-4.3/bin/sh}. +@quotation Note +If you are looking for ways to exchange binaries among machines that +already run Guix, @pxref{Invoking guix copy}, @ref{Invoking guix +publish}, and @ref{Invoking guix archive}. +@end quotation -@item configure -Run the @file{configure} script with a number of default options, such -as @code{--prefix=/gnu/store/@dots{}}, as well as the options specified -by the @code{#:configure-flags} argument. +@cindex pack +@cindex bundle +@cindex application bundle +@cindex software bundle +The @command{guix pack} command creates a shrink-wrapped @dfn{pack} or +@dfn{software bundle}: it creates a tarball or some other archive +containing the binaries of the software you're interested in, and all +its dependencies. The resulting archive can be used on any machine that +does not have Guix, and people can run the exact same binaries as those +you have with Guix. The pack itself is created in a bit-reproducible +fashion, so anyone can verify that it really contains the build results +that you pretend to be shipping. -@item build -Run @code{make} with the list of flags specified with -@code{#:make-flags}. If the @code{#:parallel-build?} argument is true -(the default), build with @code{make -j}. +For example, to create a bundle containing Guile, Emacs, Geiser, and all +their dependencies, you can run: -@item check -Run @code{make check}, or some other target specified with -@code{#:test-target}, unless @code{#:tests? #f} is passed. If the -@code{#:parallel-tests?} argument is true (the default), run @code{make -check -j}. +@example +$ guix pack guile emacs geiser +@dots{} +/gnu/store/@dots{}-pack.tar.gz +@end example -@item install -Run @code{make install} with the flags listed in @code{#:make-flags}. +The result here is a tarball containing a @file{/gnu/store} directory +with all the relevant packages. The resulting tarball contains a +@dfn{profile} with the three packages of interest; the profile is the +same as would be created by @command{guix package -i}. It is this +mechanism that is used to create Guix's own standalone binary tarball +(@pxref{Binary Installation}). -@item patch-shebangs -Patch shebangs on the installed executable files. +Users of this pack would have to run +@file{/gnu/store/@dots{}-profile/bin/guile} to run Guile, which you may +find inconvenient. To work around it, you can create, say, a +@file{/opt/gnu/bin} symlink to the profile: -@item strip -Strip debugging symbols from ELF files (unless @code{#:strip-binaries?} -is false), copying them to the @code{debug} output when available -(@pxref{Installing Debugging Files}). -@end table +@example +guix pack -S /opt/gnu/bin=bin guile emacs geiser +@end example -@vindex %standard-phases -The build-side module @code{(guix build gnu-build-system)} defines -@var{%standard-phases} as the default list of build phases. -@var{%standard-phases} is a list of symbol/procedure pairs, where the -procedure implements the actual phase. +@noindent +That way, users can happily type @file{/opt/gnu/bin/guile} and enjoy. -The list of phases used for a particular package can be changed with the -@code{#:phases} parameter. For instance, passing: +@cindex relocatable binaries, with @command{guix pack} +What if the recipient of your pack does not have root privileges on +their machine, and thus cannot unpack it in the root file system? In +that case, you will want to use the @code{--relocatable} option (see +below). This option produces @dfn{relocatable binaries}, meaning they +they can be placed anywhere in the file system hierarchy: in the example +above, users can unpack your tarball in their home directory and +directly run @file{./opt/gnu/bin/guile}. + +@cindex Docker, build an image with guix pack +Alternatively, you can produce a pack in the Docker image format using +the following command: @example -#:phases (modify-phases %standard-phases (delete 'configure)) +guix pack -f docker guile emacs geiser @end example -means that all the phases described above will be used, except the -@code{configure} phase. - -In addition, this build system ensures that the ``standard'' environment -for GNU packages is available. This includes tools such as GCC, libc, -Coreutils, Bash, Make, Diffutils, grep, and sed (see the @code{(guix -build-system gnu)} module for a complete list). We call these the -@dfn{implicit inputs} of a package, because package definitions do not -have to mention them. -@end defvr +@noindent +The result is a tarball that can be passed to the @command{docker load} +command. See the +@uref{https://docs.docker.com/engine/reference/commandline/load/, Docker +documentation} for more information. -Other @code{} objects are defined to support other -conventions and tools used by free software packages. They inherit most -of @var{gnu-build-system}, and differ mainly in the set of inputs -implicitly added to the build process, and in the list of phases -executed. Some of these build systems are listed below. +@cindex Singularity, build an image with guix pack +@cindex SquashFS, build an image with guix pack +Yet another option is to produce a SquashFS image with the following +command: -@defvr {Scheme Variable} ant-build-system -This variable is exported by @code{(guix build-system ant)}. It -implements the build procedure for Java packages that can be built with -@url{http://ant.apache.org/, Ant build tool}. +@example +guix pack -f squashfs guile emacs geiser +@end example -It adds both @code{ant} and the @dfn{Java Development Kit} (JDK) as -provided by the @code{icedtea} package to the set of inputs. Different -packages can be specified with the @code{#:ant} and @code{#:jdk} -parameters, respectively. +@noindent +The result is a SquashFS file system image that can either be mounted or +directly be used as a file system container image with the +@uref{http://singularity.lbl.gov, Singularity container execution +environment}, using commands like @command{singularity shell} or +@command{singularity exec}. -When the original package does not provide a suitable Ant build file, -the parameter @code{#:jar-name} can be used to generate a minimal Ant -build file @file{build.xml} with tasks to build the specified jar -archive. In this case the parameter @code{#:source-dir} can be used to -specify the source sub-directory, defaulting to ``src''. +Several command-line options allow you to customize your pack: -The @code{#:main-class} parameter can be used with the minimal ant -buildfile to specify the main class of the resulting jar. This makes the -jar file executable. The @code{#:test-include} parameter can be used to -specify the list of junit tests to run. It defaults to -@code{(list "**/*Test.java")}. The @code{#:test-exclude} can be used to -disable some tests. It defaults to @code{(list "**/Abstract*.java")}, -because abstract classes cannot be run as tests. +@table @code +@item --format=@var{format} +@itemx -f @var{format} +Produce a pack in the given @var{format}. -The parameter @code{#:build-target} can be used to specify the Ant task -that should be run during the @code{build} phase. By default the -``jar'' task will be run. +The available formats are: -@end defvr +@table @code +@item tarball +This is the default format. It produces a tarball containing all the +specified binaries and symlinks. -@defvr {Scheme Variable} android-ndk-build-system -@cindex Android distribution -@cindex Android NDK build system -This variable is exported by @code{(guix build-system android-ndk)}. It -implements a build procedure for Android NDK (native development kit) -packages using a Guix-specific build process. +@item docker +This produces a tarball that follows the +@uref{https://github.com/docker/docker/blob/master/image/spec/v1.2.md, +Docker Image Specification}. -The build system assumes that packages install their public interface -(header) files to the subdirectory "include" of the "out" output and -their libraries to the subdirectory "lib" of the "out" output. +@item squashfs +This produces a SquashFS image containing all the specified binaries and +symlinks, as well as empty mount points for virtual file systems like +procfs. +@end table -It's also assumed that the union of all the dependencies of a package -has no conflicting files. +@item --relocatable +@itemx -R +Produce @dfn{relocatable binaries}---i.e., binaries that can be placed +anywhere in the file system hierarchy and run from there. For example, +if you create a pack containing Bash with: -For the time being, cross-compilation is not supported - so right now -the libraries and header files are assumed to be host tools. +@example +guix pack -R -S /mybin=bin bash +@end example -@end defvr +@noindent +...@: you can copy that pack to a machine that lacks Guix, and from your +home directory as a normal user, run: -@defvr {Scheme Variable} asdf-build-system/source -@defvrx {Scheme Variable} asdf-build-system/sbcl -@defvrx {Scheme Variable} asdf-build-system/ecl +@example +tar xf pack.tar.gz +./mybin/sh +@end example -These variables, exported by @code{(guix build-system asdf)}, implement -build procedures for Common Lisp packages using -@url{https://common-lisp.net/project/asdf/, ``ASDF''}. ASDF is a system -definition facility for Common Lisp programs and libraries. +@noindent +In that shell, if you type @code{ls /gnu/store}, you'll notice that +@file{/gnu/store} shows up and contains all the dependencies of +@code{bash}, even though the machine actually lacks @file{/gnu/store} +altogether! That is probably the simplest way to deploy Guix-built +software on a non-Guix machine. -The @code{asdf-build-system/source} system installs the packages in -source form, and can be loaded using any common lisp implementation, via -ASDF. The others, such as @code{asdf-build-system/sbcl}, install binary -systems in the format which a particular implementation understands. -These build systems can also be used to produce executable programs, or -lisp images which contain a set of packages pre-loaded. +There's a gotcha though: this technique relies on the @dfn{user +namespace} feature of the kernel Linux, which allows unprivileged users +to mount or change root. Old versions of Linux did not support it, and +some GNU/Linux distributions turn it off; on these systems, programs +from the pack @emph{will fail to run}, unless they are unpacked in the +root file system. -The build system uses naming conventions. For binary packages, the -package name should be prefixed with the lisp implementation, such as -@code{sbcl-} for @code{asdf-build-system/sbcl}. +@item --expression=@var{expr} +@itemx -e @var{expr} +Consider the package @var{expr} evaluates to. -Additionally, the corresponding source package should be labeled using -the same convention as python packages (see @ref{Python Modules}), using -the @code{cl-} prefix. +This has the same purpose as the same-named option in @command{guix +build} (@pxref{Additional Build Options, @code{--expression} in +@command{guix build}}). -For binary packages, each system should be defined as a Guix package. -If one package @code{origin} contains several systems, package variants -can be created in order to build all the systems. Source packages, -which use @code{asdf-build-system/source}, may contain several systems. +@item --manifest=@var{file} +@itemx -m @var{file} +Use the packages contained in the manifest object returned by the Scheme +code in @var{file}. -In order to create executable programs and images, the build-side -procedures @code{build-program} and @code{build-image} can be used. -They should be called in a build phase after the @code{create-symlinks} -phase, so that the system which was just built can be used within the -resulting image. @code{build-program} requires a list of Common Lisp -expressions to be passed as the @code{#:entry-program} argument. +This has a similar purpose as the same-named option in @command{guix +package} (@pxref{profile-manifest, @option{--manifest}}) and uses the +same manifest files. It allows you to define a collection of packages +once and use it both for creating profiles and for creating archives +for use on machines that do not have Guix installed. Note that you can +specify @emph{either} a manifest file @emph{or} a list of packages, +but not both. -If the system is not defined within its own @code{.asd} file of the same -name, then the @code{#:asd-file} parameter should be used to specify -which file the system is defined in. Furthermore, if the package -defines a system for its tests in a separate file, it will be loaded -before the tests are run if it is specified by the -@code{#:test-asd-file} parameter. If it is not set, the files -@code{-tests.asd}, @code{-test.asd}, @code{tests.asd}, -and @code{test.asd} will be tried if they exist. +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of +the system type of the build host. -If for some reason the package must be named in a different way than the -naming conventions suggest, the @code{#:asd-system-name} parameter can -be used to specify the name of the system. +@item --target=@var{triplet} +@cindex cross-compilation +Cross-build for @var{triplet}, which must be a valid GNU triplet, such +as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU +configuration triplets,, autoconf, Autoconf}). -@end defvr +@item --compression=@var{tool} +@itemx -C @var{tool} +Compress the resulting tarball using @var{tool}---one of @code{gzip}, +@code{bzip2}, @code{xz}, @code{lzip}, or @code{none} for no compression. -@defvr {Scheme Variable} cargo-build-system -@cindex Rust programming language -@cindex Cargo (Rust build system) -This variable is exported by @code{(guix build-system cargo)}. It -supports builds of packages using Cargo, the build tool of the -@uref{https://www.rust-lang.org, Rust programming language}. +@item --symlink=@var{spec} +@itemx -S @var{spec} +Add the symlinks specified by @var{spec} to the pack. This option can +appear several times. -In its @code{configure} phase, this build system replaces dependencies -specified in the @file{Carto.toml} file with inputs to the Guix package. -The @code{install} phase installs the binaries, and it also installs the -source code and @file{Cargo.toml} file. -@end defvr +@var{spec} has the form @code{@var{source}=@var{target}}, where +@var{source} is the symlink that will be created and @var{target} is the +symlink target. -@cindex Clojure (programming language) -@cindex simple Clojure build system -@defvr {Scheme Variable} clojure-build-system -This variable is exported by @code{(guix build-system clojure)}. It implements -a simple build procedure for @uref{https://clojure.org/, Clojure} packages -using plain old @code{compile} in Clojure. Cross-compilation is not supported -yet. +For instance, @code{-S /opt/gnu/bin=bin} creates a @file{/opt/gnu/bin} +symlink pointing to the @file{bin} sub-directory of the profile. -It adds @code{clojure}, @code{icedtea} and @code{zip} to the set of inputs. -Different packages can be specified with the @code{#:clojure}, @code{#:jdk} and -@code{#:zip} parameters, respectively. +@item --localstatedir +@itemx --profile-name=@var{name} +Include the ``local state directory'', @file{/var/guix}, in the resulting +pack, and notably the @file{/var/guix/profiles/per-user/root/@var{name}} +profile---by default @var{name} is @code{guix-profile}, which corresponds to +@file{~root/.guix-profile}. -A list of source directories, test directories and jar names can be specified -with the @code{#:source-dirs}, @code{#:test-dirs} and @code{#:jar-names} -parameters, respectively. Compile directory and main class can be specified -with the @code{#:compile-dir} and @code{#:main-class} parameters, respectively. -Other parameters are documented below. +@file{/var/guix} contains the store database (@pxref{The Store}) as well +as garbage-collector roots (@pxref{Invoking guix gc}). Providing it in +the pack means that the store is ``complete'' and manageable by Guix; +not providing it pack means that the store is ``dead'': items cannot be +added to it or removed from it after extraction of the pack. -This build system is an extension of @var{ant-build-system}, but with the -following phases changed: +One use case for this is the Guix self-contained binary tarball +(@pxref{Binary Installation}). -@table @code +@item --bootstrap +Use the bootstrap binaries to build the pack. This option is only +useful to Guix developers. +@end table -@item build -This phase calls @code{compile} in Clojure to compile source files and runs -@command{jar} to create jars from both source files and compiled files -according to the include list and exclude list specified in -@code{#:aot-include} and @code{#:aot-exclude}, respectively. The exclude list -has priority over the include list. These lists consist of symbols -representing Clojure libraries or the special keyword @code{#:all} representing -all Clojure libraries found under the source directories. The parameter -@code{#:omit-source?} decides if source should be included into the jars. +In addition, @command{guix pack} supports all the common build options +(@pxref{Common Build Options}) and all the package transformation +options (@pxref{Package Transformation Options}). -@item check -This phase runs tests according to the include list and exclude list specified -in @code{#:test-include} and @code{#:test-exclude}, respectively. Their -meanings are analogous to that of @code{#:aot-include} and -@code{#:aot-exclude}, except that the special keyword @code{#:all} now -stands for all Clojure libraries found under the test directories. The -parameter @code{#:tests?} decides if tests should be run. -@item install -This phase installs all jars built previously. -@end table +@node Invoking guix archive +@section Invoking @command{guix archive} -Apart from the above, this build system also contains an additional phase: +@cindex @command{guix archive} +@cindex archive +The @command{guix archive} command allows users to @dfn{export} files +from the store into a single archive, and to later @dfn{import} them on +a machine that runs Guix. +In particular, it allows store files to be transferred from one machine +to the store on another machine. -@table @code +@quotation Note +If you're looking for a way to produce archives in a format suitable for +tools other than Guix, @pxref{Invoking guix pack}. +@end quotation -@item install-doc -This phase installs all top-level files with base name matching -@var{%doc-regex}. A different regex can be specified with the -@code{#:doc-regex} parameter. All files (recursively) inside the documentation -directories specified in @code{#:doc-dirs} are installed as well. -@end table -@end defvr +@cindex exporting store items +To export store files as an archive to standard output, run: -@defvr {Scheme Variable} cmake-build-system -This variable is exported by @code{(guix build-system cmake)}. It -implements the build procedure for packages using the -@url{http://www.cmake.org, CMake build tool}. +@example +guix archive --export @var{options} @var{specifications}... +@end example -It automatically adds the @code{cmake} package to the set of inputs. -Which package is used can be specified with the @code{#:cmake} -parameter. +@var{specifications} may be either store file names or package +specifications, as for @command{guix package} (@pxref{Invoking guix +package}). For instance, the following command creates an archive +containing the @code{gui} output of the @code{git} package and the main +output of @code{emacs}: -The @code{#:configure-flags} parameter is taken as a list of flags -passed to the @command{cmake} command. The @code{#:build-type} -parameter specifies in abstract terms the flags passed to the compiler; -it defaults to @code{"RelWithDebInfo"} (short for ``release mode with -debugging information''), which roughly means that code is compiled with -@code{-O2 -g}, as is the case for Autoconf-based packages by default. -@end defvr +@example +guix archive --export git:gui /gnu/store/...-emacs-24.3 > great.nar +@end example -@defvr {Scheme Variable} dune-build-system -This variable is exported by @code{(guix build-system dune)}. It -supports builds of packages using @uref{https://dune.build/, Dune}, a build -tool for the OCaml programming language. It is implemented as an extension -of the @code{ocaml-build-system} which is described below. As such, the -@code{#:ocaml} and @code{#:findlib} parameters can be passed to this build -system. +If the specified packages are not built yet, @command{guix archive} +automatically builds them. The build process may be controlled with the +common build options (@pxref{Common Build Options}). -It automatically adds the @code{dune} package to the set of inputs. -Which package is used can be specified with the @code{#:dune} -parameter. +To transfer the @code{emacs} package to a machine connected over SSH, +one would run: -There is no @code{configure} phase because dune packages typically don't -need to be configured. The @code{#:build-flags} parameter is taken as a -list of flags passed to the @code{dune} command during the build. +@example +guix archive --export -r emacs | ssh the-machine guix archive --import +@end example -The @code{#:jbuild?} parameter can be passed to use the @code{jbuild} -command instead of the more recent @code{dune} command while building -a package. Its default value is @code{#f}. -@end defvr +@noindent +Similarly, a complete user profile may be transferred from one machine +to another like this: -@defvr {Scheme Variable} go-build-system -This variable is exported by @code{(guix build-system go)}. It -implements a build procedure for Go packages using the standard -@url{https://golang.org/cmd/go/#hdr-Compile_packages_and_dependencies, -Go build mechanisms}. +@example +guix archive --export -r $(readlink -f ~/.guix-profile) | \ + ssh the-machine guix-archive --import +@end example -The user is expected to provide a value for the key @code{#:import-path} -and, in some cases, @code{#:unpack-path}. The -@url{https://golang.org/doc/code.html#ImportPaths, import path} -corresponds to the file system path expected by the package's build -scripts and any referring packages, and provides a unique way to -refer to a Go package. It is typically based on a combination of the -package source code's remote URI and file system hierarchy structure. In -some cases, you will need to unpack the package's source code to a -different directory structure than the one indicated by the import path, -and @code{#:unpack-path} should be used in such cases. +@noindent +However, note that, in both examples, all of @code{emacs} and the +profile as well as all of their dependencies are transferred (due to +@code{-r}), regardless of what is already available in the store on the +target machine. The @code{--missing} option can help figure out which +items are missing from the target store. The @command{guix copy} +command simplifies and optimizes this whole process, so this is probably +what you should use in this case (@pxref{Invoking guix copy}). -Packages that provide Go libraries should be installed along with their -source code. The key @code{#:install-source?}, which defaults to -@code{#t}, controls whether or not the source code is installed. It can -be set to @code{#f} for packages that only provide executable files. -@end defvr +@cindex nar, archive format +@cindex normalized archive (nar) +Archives are stored in the ``normalized archive'' or ``nar'' format, which is +comparable in spirit to `tar', but with differences +that make it more appropriate for our purposes. First, rather than +recording all Unix metadata for each file, the nar format only mentions +the file type (regular, directory, or symbolic link); Unix permissions +and owner/group are dismissed. Second, the order in which directory +entries are stored always follows the order of file names according to +the C locale collation order. This makes archive production fully +deterministic. -@defvr {Scheme Variable} glib-or-gtk-build-system -This variable is exported by @code{(guix build-system glib-or-gtk)}. It -is intended for use with packages making use of GLib or GTK+. +When exporting, the daemon digitally signs the contents of the archive, +and that digital signature is appended. When importing, the daemon +verifies the signature and rejects the import in case of an invalid +signature or if the signing key is not authorized. +@c FIXME: Add xref to daemon doc about signatures. -This build system adds the following two phases to the ones defined by -@var{gnu-build-system}: +The main options are: @table @code -@item glib-or-gtk-wrap -The phase @code{glib-or-gtk-wrap} ensures that programs in -@file{bin/} are able to find GLib ``schemas'' and -@uref{https://developer.gnome.org/gtk3/stable/gtk-running.html, GTK+ -modules}. This is achieved by wrapping the programs in launch scripts -that appropriately set the @code{XDG_DATA_DIRS} and @code{GTK_PATH} -environment variables. +@item --export +Export the specified store files or packages (see below.) Write the +resulting archive to the standard output. + +Dependencies are @emph{not} included in the output, unless +@code{--recursive} is passed. + +@item -r +@itemx --recursive +When combined with @code{--export}, this instructs @command{guix +archive} to include dependencies of the given items in the archive. +Thus, the resulting archive is self-contained: it contains the closure +of the exported store items. -It is possible to exclude specific package outputs from that wrapping -process by listing their names in the -@code{#:glib-or-gtk-wrap-excluded-outputs} parameter. This is useful -when an output is known not to contain any GLib or GTK+ binaries, and -where wrapping would gratuitously add a dependency of that output on -GLib and GTK+. +@item --import +Read an archive from the standard input, and import the files listed +therein into the store. Abort if the archive has an invalid digital +signature, or if it is signed by a public key not among the authorized +keys (see @code{--authorize} below.) -@item glib-or-gtk-compile-schemas -The phase @code{glib-or-gtk-compile-schemas} makes sure that all -@uref{https://developer.gnome.org/gio/stable/glib-compile-schemas.html, -GSettings schemas} of GLib are compiled. Compilation is performed by the -@command{glib-compile-schemas} program. It is provided by the package -@code{glib:bin} which is automatically imported by the build system. -The @code{glib} package providing @command{glib-compile-schemas} can be -specified with the @code{#:glib} parameter. -@end table +@item --missing +Read a list of store file names from the standard input, one per line, +and write on the standard output the subset of these files missing from +the store. -Both phases are executed after the @code{install} phase. -@end defvr +@item --generate-key[=@var{parameters}] +@cindex signing, archives +Generate a new key pair for the daemon. This is a prerequisite before +archives can be exported with @code{--export}. Note that this operation +usually takes time, because it needs to gather enough entropy to +generate the key pair. -@defvr {Scheme Variable} guile-build-system -This build system is for Guile packages that consist exclusively of Scheme -code and that are so lean that they don't even have a makefile, let alone a -@file{configure} script. It compiles Scheme code using @command{guild -compile} (@pxref{Compilation,,, guile, GNU Guile Reference Manual}) and -installs the @file{.scm} and @file{.go} files in the right place. It also -installs documentation. +The generated key pair is typically stored under @file{/etc/guix}, in +@file{signing-key.pub} (public key) and @file{signing-key.sec} (private +key, which must be kept secret.) When @var{parameters} is omitted, +an ECDSA key using the Ed25519 curve is generated, or, for Libgcrypt +versions before 1.6.0, it is a 4096-bit RSA key. +Alternatively, @var{parameters} can specify +@code{genkey} parameters suitable for Libgcrypt (@pxref{General +public-key related Functions, @code{gcry_pk_genkey},, gcrypt, The +Libgcrypt Reference Manual}). -This build system supports cross-compilation by using the @code{--target} -option of @command{guild compile}. +@item --authorize +@cindex authorizing, archives +Authorize imports signed by the public key passed on standard input. +The public key must be in ``s-expression advanced format''---i.e., the +same format as the @file{signing-key.pub} file. -Packages built with @code{guile-build-system} must provide a Guile package in -their @code{native-inputs} field. -@end defvr +The list of authorized keys is kept in the human-editable file +@file{/etc/guix/acl}. The file contains +@url{http://people.csail.mit.edu/rivest/Sexp.txt, ``advanced-format +s-expressions''} and is structured as an access-control list in the +@url{http://theworld.com/~cme/spki.txt, Simple Public-Key Infrastructure +(SPKI)}. -@defvr {Scheme Variable} minify-build-system -This variable is exported by @code{(guix build-system minify)}. It -implements a minification procedure for simple JavaScript packages. +@item --extract=@var{directory} +@itemx -x @var{directory} +Read a single-item archive as served by substitute servers +(@pxref{Substitutes}) and extract it to @var{directory}. This is a +low-level operation needed in only very narrow use cases; see below. -It adds @code{uglify-js} to the set of inputs and uses it to compress -all JavaScript files in the @file{src} directory. A different minifier -package can be specified with the @code{#:uglify-js} parameter, but it -is expected that the package writes the minified code to the standard -output. +For example, the following command extracts the substitute for Emacs +served by @code{@value{SUBSTITUTE-SERVER}} to @file{/tmp/emacs}: -When the input JavaScript files are not all located in the @file{src} -directory, the parameter @code{#:javascript-files} can be used to -specify a list of file names to feed to the minifier. -@end defvr +@example +$ wget -O - \ + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-emacs-24.5 \ + | bunzip2 | guix archive -x /tmp/emacs +@end example -@defvr {Scheme Variable} ocaml-build-system -This variable is exported by @code{(guix build-system ocaml)}. It implements -a build procedure for @uref{https://ocaml.org, OCaml} packages, which consists -of choosing the correct set of commands to run for each package. OCaml -packages can expect many different commands to be run. This build system will -try some of them. +Single-item archives are different from multiple-item archives produced +by @command{guix archive --export}; they contain a single store item, +and they do @emph{not} embed a signature. Thus this operation does +@emph{no} signature verification and its output should be considered +unsafe. -When the package has a @file{setup.ml} file present at the top-level, it will -run @code{ocaml setup.ml -configure}, @code{ocaml setup.ml -build} and -@code{ocaml setup.ml -install}. The build system will assume that this file -was generated by @uref{http://oasis.forge.ocamlcore.org/, OASIS} and will take -care of setting the prefix and enabling tests if they are not disabled. You -can pass configure and build flags with the @code{#:configure-flags} and -@code{#:build-flags}. The @code{#:test-flags} key can be passed to change the -set of flags used to enable tests. The @code{#:use-make?} key can be used to -bypass this system in the build and install phases. +The primary purpose of this operation is to facilitate inspection of +archive contents coming from possibly untrusted substitute servers. -When the package has a @file{configure} file, it is assumed that it is a -hand-made configure script that requires a different argument format than -in the @code{gnu-build-system}. You can add more flags with the -@code{#:configure-flags} key. +@end table -When the package has a @file{Makefile} file (or @code{#:use-make?} is -@code{#t}), it will be used and more flags can be passed to the build and -install phases with the @code{#:make-flags} key. +@c ********************************************************************* +@node Programming Interface +@chapter Programming Interface -Finally, some packages do not have these files and use a somewhat standard -location for its build system. In that case, the build system will run -@code{ocaml pkg/pkg.ml} or @code{ocaml pkg/build.ml} and take care of -providing the path to the required findlib module. Additional flags can -be passed via the @code{#:build-flags} key. Install is taken care of by -@command{opam-installer}. In this case, the @code{opam} package must -be added to the @code{native-inputs} field of the package definition. +GNU Guix provides several Scheme programming interfaces (APIs) to +define, build, and query packages. The first interface allows users to +write high-level package definitions. These definitions refer to +familiar packaging concepts, such as the name and version of a package, +its build system, and its dependencies. These definitions can then be +turned into concrete build actions. -Note that most OCaml packages assume they will be installed in the same -directory as OCaml, which is not what we want in guix. In particular, they -will install @file{.so} files in their module's directory, which is usually -fine because it is in the OCaml compiler directory. In guix though, these -libraries cannot be found and we use @code{CAML_LD_LIBRARY_PATH}. This -variable points to @file{lib/ocaml/site-lib/stubslibs} and this is where -@file{.so} libraries should be installed. -@end defvr +Build actions are performed by the Guix daemon, on behalf of users. In a +standard setup, the daemon has write access to the store---the +@file{/gnu/store} directory---whereas users do not. The recommended +setup also has the daemon perform builds in chroots, under a specific +build users, to minimize interference with the rest of the system. -@defvr {Scheme Variable} python-build-system -This variable is exported by @code{(guix build-system python)}. It -implements the more or less standard build procedure used by Python -packages, which consists in running @code{python setup.py build} and -then @code{python setup.py install --prefix=/gnu/store/@dots{}}. +@cindex derivation +Lower-level APIs are available to interact with the daemon and the +store. To instruct the daemon to perform a build action, users actually +provide it with a @dfn{derivation}. A derivation is a low-level +representation of the build actions to be taken, and the environment in +which they should occur---derivations are to package definitions what +assembly is to C programs. The term ``derivation'' comes from the fact +that build results @emph{derive} from them. -For packages that install stand-alone Python programs under @code{bin/}, -it takes care of wrapping these programs so that their @code{PYTHONPATH} -environment variable points to all the Python libraries they depend on. +This chapter describes all these APIs in turn, starting from high-level +package definitions. -Which Python package is used to perform the build can be specified with -the @code{#:python} parameter. This is a useful way to force a package -to be built for a specific version of the Python interpreter, which -might be necessary if the package is only compatible with a single -interpreter version. +@menu +* Defining Packages:: Defining new packages. +* Build Systems:: Specifying how packages are built. +* The Store:: Manipulating the package store. +* Derivations:: Low-level interface to package derivations. +* The Store Monad:: Purely functional interface to the store. +* G-Expressions:: Manipulating build expressions. +* Invoking guix repl:: Fiddling with Guix interactively. +@end menu -By default guix calls @code{setup.py} under control of -@code{setuptools}, much like @command{pip} does. Some packages are not -compatible with setuptools (and pip), thus you can disable this by -setting the @code{#:use-setuptools} parameter to @code{#f}. -@end defvr +@node Defining Packages +@section Defining Packages -@defvr {Scheme Variable} perl-build-system -This variable is exported by @code{(guix build-system perl)}. It -implements the standard build procedure for Perl packages, which either -consists in running @code{perl Build.PL --prefix=/gnu/store/@dots{}}, -followed by @code{Build} and @code{Build install}; or in running -@code{perl Makefile.PL PREFIX=/gnu/store/@dots{}}, followed by -@code{make} and @code{make install}, depending on which of -@code{Build.PL} or @code{Makefile.PL} is present in the package -distribution. Preference is given to the former if both @code{Build.PL} -and @code{Makefile.PL} exist in the package distribution. This -preference can be reversed by specifying @code{#t} for the -@code{#:make-maker?} parameter. +The high-level interface to package definitions is implemented in the +@code{(guix packages)} and @code{(guix build-system)} modules. As an +example, the package definition, or @dfn{recipe}, for the GNU Hello +package looks like this: -The initial @code{perl Makefile.PL} or @code{perl Build.PL} invocation -passes flags specified by the @code{#:make-maker-flags} or -@code{#:module-build-flags} parameter, respectively. +@example +(define-module (gnu packages hello) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix build-system gnu) + #:use-module (guix licenses) + #:use-module (gnu packages gawk)) -Which Perl package is used can be specified with @code{#:perl}. -@end defvr +(define-public hello + (package + (name "hello") + (version "2.10") + (source (origin + (method url-fetch) + (uri (string-append "mirror://gnu/hello/hello-" version + ".tar.gz")) + (sha256 + (base32 + "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) + (build-system gnu-build-system) + (arguments '(#:configure-flags '("--enable-silent-rules"))) + (inputs `(("gawk" ,gawk))) + (synopsis "Hello, GNU world: An example GNU package") + (description "Guess what GNU Hello prints!") + (home-page "http://www.gnu.org/software/hello/") + (license gpl3+))) +@end example -@defvr {Scheme Variable} r-build-system -This variable is exported by @code{(guix build-system r)}. It -implements the build procedure used by @uref{http://r-project.org, R} -packages, which essentially is little more than running @code{R CMD -INSTALL --library=/gnu/store/@dots{}} in an environment where -@code{R_LIBS_SITE} contains the paths to all R package inputs. Tests -are run after installation using the R function -@code{tools::testInstalledPackage}. -@end defvr +@noindent +Without being a Scheme expert, the reader may have guessed the meaning +of the various fields here. This expression binds the variable +@code{hello} to a @code{} object, which is essentially a record +(@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}). +This package object can be inspected using procedures found in the +@code{(guix packages)} module; for instance, @code{(package-name hello)} +returns---surprise!---@code{"hello"}. -@defvr {Scheme Variable} texlive-build-system -This variable is exported by @code{(guix build-system texlive)}. It is -used to build TeX packages in batch mode with a specified engine. The -build system sets the @code{TEXINPUTS} variable to find all TeX source -files in the inputs. +With luck, you may be able to import part or all of the definition of +the package you are interested in from another repository, using the +@code{guix import} command (@pxref{Invoking guix import}). -By default it runs @code{luatex} on all files ending on @code{ins}. A -different engine and format can be specified with the -@code{#:tex-format} argument. Different build targets can be specified -with the @code{#:build-targets} argument, which expects a list of file -names. The build system adds only @code{texlive-bin} and -@code{texlive-latex-base} (both from @code{(gnu packages tex}) to the -inputs. Both can be overridden with the arguments @code{#:texlive-bin} -and @code{#:texlive-latex-base}, respectively. +In the example above, @var{hello} is defined in a module of its own, +@code{(gnu packages hello)}. Technically, this is not strictly +necessary, but it is convenient to do so: all the packages defined in +modules under @code{(gnu packages @dots{})} are automatically known to +the command-line tools (@pxref{Package Modules}). -The @code{#:tex-directory} parameter tells the build system where to -install the built files under the texmf tree. -@end defvr +There are a few points worth noting in the above package definition: -@defvr {Scheme Variable} ruby-build-system -This variable is exported by @code{(guix build-system ruby)}. It -implements the RubyGems build procedure used by Ruby packages, which -involves running @code{gem build} followed by @code{gem install}. +@itemize +@item +The @code{source} field of the package is an @code{} object +(@pxref{origin Reference}, for the complete reference). +Here, the @code{url-fetch} method from @code{(guix download)} is used, +meaning that the source is a file to be downloaded over FTP or HTTP. -The @code{source} field of a package that uses this build system -typically references a gem archive, since this is the format that Ruby -developers use when releasing their software. The build system unpacks -the gem archive, potentially patches the source, runs the test suite, -repackages the gem, and installs it. Additionally, directories and -tarballs may be referenced to allow building unreleased gems from Git or -a traditional source release tarball. +The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of +the GNU mirrors defined in @code{(guix download)}. -Which Ruby package is used can be specified with the @code{#:ruby} -parameter. A list of additional flags to be passed to the @command{gem} -command can be specified with the @code{#:gem-flags} parameter. -@end defvr +The @code{sha256} field specifies the expected SHA256 hash of the file +being downloaded. It is mandatory, and allows Guix to check the +integrity of the file. The @code{(base32 @dots{})} form introduces the +base32 representation of the hash. You can obtain this information with +@code{guix download} (@pxref{Invoking guix download}) and @code{guix +hash} (@pxref{Invoking guix hash}). -@defvr {Scheme Variable} waf-build-system -This variable is exported by @code{(guix build-system waf)}. It -implements a build procedure around the @code{waf} script. The common -phases---@code{configure}, @code{build}, and @code{install}---are -implemented by passing their names as arguments to the @code{waf} -script. +@cindex patches +When needed, the @code{origin} form can also have a @code{patches} field +listing patches to be applied, and a @code{snippet} field giving a +Scheme expression to modify the source code. -The @code{waf} script is executed by the Python interpreter. Which -Python package is used to run the script can be specified with the -@code{#:python} parameter. -@end defvr +@item +@cindex GNU Build System +The @code{build-system} field specifies the procedure to build the +package (@pxref{Build Systems}). Here, @var{gnu-build-system} +represents the familiar GNU Build System, where packages may be +configured, built, and installed with the usual @code{./configure && +make && make check && make install} command sequence. -@defvr {Scheme Variable} scons-build-system -This variable is exported by @code{(guix build-system scons)}. It -implements the build procedure used by the SCons software construction -tool. This build system runs @code{scons} to build the package, -@code{scons test} to run tests, and then @code{scons install} to install -the package. +@item +The @code{arguments} field specifies options for the build system +(@pxref{Build Systems}). Here it is interpreted by +@var{gnu-build-system} as a request run @file{configure} with the +@code{--enable-silent-rules} flag. -Additional flags to be passed to @code{scons} can be specified with the -@code{#:scons-flags} parameter. The version of Python used to run SCons -can be specified by selecting the appropriate SCons package with the -@code{#:scons} parameter. -@end defvr +@cindex quote +@cindex quoting +@findex ' +@findex quote +What about these quote (@code{'}) characters? They are Scheme syntax to +introduce a literal list; @code{'} is synonymous with @code{quote}. +@xref{Expression Syntax, quoting,, guile, GNU Guile Reference Manual}, +for details. Here the value of the @code{arguments} field is a list of +arguments passed to the build system down the road, as with @code{apply} +(@pxref{Fly Evaluation, @code{apply},, guile, GNU Guile Reference +Manual}). -@defvr {Scheme Variable} haskell-build-system -This variable is exported by @code{(guix build-system haskell)}. It -implements the Cabal build procedure used by Haskell packages, which -involves running @code{runhaskell Setup.hs configure ---prefix=/gnu/store/@dots{}} and @code{runhaskell Setup.hs build}. -Instead of installing the package by running @code{runhaskell Setup.hs -install}, to avoid trying to register libraries in the read-only -compiler store directory, the build system uses @code{runhaskell -Setup.hs copy}, followed by @code{runhaskell Setup.hs register}. In -addition, the build system generates the package documentation by -running @code{runhaskell Setup.hs haddock}, unless @code{#:haddock? #f} -is passed. Optional Haddock parameters can be passed with the help of -the @code{#:haddock-flags} parameter. If the file @code{Setup.hs} is -not found, the build system looks for @code{Setup.lhs} instead. +The hash-colon (@code{#:}) sequence defines a Scheme @dfn{keyword} +(@pxref{Keywords,,, guile, GNU Guile Reference Manual}), and +@code{#:configure-flags} is a keyword used to pass a keyword argument +to the build system (@pxref{Coding With Keywords,,, guile, GNU Guile +Reference Manual}). -Which Haskell compiler is used can be specified with the @code{#:haskell} -parameter which defaults to @code{ghc}. -@end defvr +@item +The @code{inputs} field specifies inputs to the build process---i.e., +build-time or run-time dependencies of the package. Here, we define an +input called @code{"gawk"} whose value is that of the @var{gawk} +variable; @var{gawk} is itself bound to a @code{} object. -@defvr {Scheme Variable} dub-build-system -This variable is exported by @code{(guix build-system dub)}. It -implements the Dub build procedure used by D packages, which -involves running @code{dub build} and @code{dub run}. -Installation is done by copying the files manually. +@cindex backquote (quasiquote) +@findex ` +@findex quasiquote +@cindex comma (unquote) +@findex , +@findex unquote +@findex ,@@ +@findex unquote-splicing +Again, @code{`} (a backquote, synonymous with @code{quasiquote}) allows +us to introduce a literal list in the @code{inputs} field, while +@code{,} (a comma, synonymous with @code{unquote}) allows us to insert a +value in that list (@pxref{Expression Syntax, unquote,, guile, GNU Guile +Reference Manual}). -Which D compiler is used can be specified with the @code{#:ldc} -parameter which defaults to @code{ldc}. -@end defvr +Note that GCC, Coreutils, Bash, and other essential tools do not need to +be specified as inputs here. Instead, @var{gnu-build-system} takes care +of ensuring that they are present (@pxref{Build Systems}). -@defvr {Scheme Variable} emacs-build-system -This variable is exported by @code{(guix build-system emacs)}. It -implements an installation procedure similar to the packaging system -of Emacs itself (@pxref{Packages,,, emacs, The GNU Emacs Manual}). +However, any other dependencies need to be specified in the +@code{inputs} field. Any dependency not specified here will simply be +unavailable to the build process, possibly leading to a build failure. +@end itemize -It first creates the @code{@var{package}-autoloads.el} file, then it -byte compiles all Emacs Lisp files. Differently from the Emacs -packaging system, the Info documentation files are moved to the standard -documentation directory and the @file{dir} file is deleted. Each -package is installed in its own directory under -@file{share/emacs/site-lisp/guix.d}. -@end defvr +@xref{package Reference}, for a full description of possible fields. -@defvr {Scheme Variable} font-build-system -This variable is exported by @code{(guix build-system font)}. It -implements an installation procedure for font packages where upstream -provides pre-compiled TrueType, OpenType, etc.@: font files that merely -need to be copied into place. It copies font files to standard -locations in the output directory. -@end defvr +Once a package definition is in place, the +package may actually be built using the @code{guix build} command-line +tool (@pxref{Invoking guix build}), troubleshooting any build failures +you encounter (@pxref{Debugging Build Failures}). You can easily jump back to the +package definition using the @command{guix edit} command +(@pxref{Invoking guix edit}). +@xref{Packaging Guidelines}, for +more information on how to test package definitions, and +@ref{Invoking guix lint}, for information on how to check a definition +for style conformance. +@vindex GUIX_PACKAGE_PATH +Lastly, @pxref{Channels}, for information +on how to extend the distribution by adding your own package definitions +in a ``channel''. -@defvr {Scheme Variable} meson-build-system -This variable is exported by @code{(guix build-system meson)}. It -implements the build procedure for packages that use -@url{http://mesonbuild.com, Meson} as their build system. +Finally, updating the package definition to a new upstream version +can be partly automated by the @command{guix refresh} command +(@pxref{Invoking guix refresh}). -It adds both Meson and @uref{https://ninja-build.org/, Ninja} to the set -of inputs, and they can be changed with the parameters @code{#:meson} -and @code{#:ninja} if needed. The default Meson is -@code{meson-for-build}, which is special because it doesn't clear the -@code{RUNPATH} of binaries and libraries when they are installed. +Behind the scenes, a derivation corresponding to the @code{} +object is first computed by the @code{package-derivation} procedure. +That derivation is stored in a @code{.drv} file under @file{/gnu/store}. +The build actions it prescribes may then be realized by using the +@code{build-derivations} procedure (@pxref{The Store}). -This build system is an extension of @var{gnu-build-system}, but with the -following phases changed to some specific for Meson: +@deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}] +Return the @code{} object of @var{package} for @var{system} +(@pxref{Derivations}). -@table @code +@var{package} must be a valid @code{} object, and @var{system} +must be a string denoting the target system type---e.g., +@code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store} +must be a connection to the daemon, which operates on the store +(@pxref{The Store}). +@end deffn -@item configure -The phase runs @code{meson} with the flags specified in -@code{#:configure-flags}. The flag @code{--build-type} is always set to -@code{plain} unless something else is specified in @code{#:build-type}. +@noindent +@cindex cross-compilation +Similarly, it is possible to compute a derivation that cross-builds a +package for some other system: -@item build -The phase runs @code{ninja} to build the package in parallel by default, but -this can be changed with @code{#:parallel-build?}. +@deffn {Scheme Procedure} package-cross-derivation @var{store} @ + @var{package} @var{target} [@var{system}] +Return the @code{} object of @var{package} cross-built from +@var{system} to @var{target}. -@item check -The phase runs @code{ninja} with the target specified in @code{#:test-target}, -which is @code{"test"} by default. +@var{target} must be a valid GNU triplet denoting the target hardware +and operating system, such as @code{"mips64el-linux-gnu"} +(@pxref{Configuration Names, GNU configuration triplets,, configure, GNU +Configure and Build System}). +@end deffn -@item install -The phase runs @code{ninja install} and can not be changed. -@end table +@cindex package transformations +@cindex input rewriting +@cindex dependency tree rewriting +Packages can be manipulated in arbitrary ways. An example of a useful +transformation is @dfn{input rewriting}, whereby the dependency tree of +a package is rewritten by replacing specific inputs by others: -Apart from that, the build system also adds the following phases: +@deffn {Scheme Procedure} package-input-rewriting @var{replacements} @ + [@var{rewrite-name}] +Return a procedure that, when passed a package, replaces its direct and +indirect dependencies (but not its implicit inputs) according to +@var{replacements}. @var{replacements} is a list of package pairs; the +first element of each pair is the package to replace, and the second one +is the replacement. + +Optionally, @var{rewrite-name} is a one-argument procedure that takes +the name of a package and returns its new name after rewrite. +@end deffn -@table @code +@noindent +Consider this example: -@item fix-runpath -This phase ensures that all binaries can find the libraries they need. -It searches for required libraries in subdirectories of the package being -built, and adds those to @code{RUNPATH} where needed. It also removes -references to libraries left over from the build phase by -@code{meson-for-build}, such as test dependencies, that aren't actually -required for the program to run. +@example +(define libressl-instead-of-openssl + ;; This is a procedure to replace OPENSSL by LIBRESSL, + ;; recursively. + (package-input-rewriting `((,openssl . ,libressl)))) -@item glib-or-gtk-wrap -This phase is the phase provided by @code{glib-or-gtk-build-system}, and it -is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. +(define git-with-libressl + (libressl-instead-of-openssl git)) +@end example -@item glib-or-gtk-compile-schemas -This phase is the phase provided by @code{glib-or-gtk-build-system}, and it -is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. -@end table -@end defvr +@noindent +Here we first define a rewriting procedure that replaces @var{openssl} +with @var{libressl}. Then we use it to define a @dfn{variant} of the +@var{git} package that uses @var{libressl} instead of @var{openssl}. +This is exactly what the @option{--with-input} command-line option does +(@pxref{Package Transformation Options, @option{--with-input}}). -Lastly, for packages that do not need anything as sophisticated, a -``trivial'' build system is provided. It is trivial in the sense that -it provides basically no support: it does not pull any implicit inputs, -and does not have a notion of build phases. +A more generic procedure to rewrite a package dependency graph is +@code{package-mapping}: it supports arbitrary changes to nodes in the +graph. -@defvr {Scheme Variable} trivial-build-system -This variable is exported by @code{(guix build-system trivial)}. +@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] +Return a procedure that, given a package, applies @var{proc} to all the packages +depended on and returns the resulting package. The procedure stops recursion +when @var{cut?} returns true for a given package. +@end deffn -This build system requires a @code{#:builder} argument. This argument -must be a Scheme expression that builds the package output(s)---as -with @code{build-expression->derivation} (@pxref{Derivations, -@code{build-expression->derivation}}). -@end defvr +@menu +* package Reference:: The package data type. +* origin Reference:: The origin data type. +@end menu -@node The Store -@section The Store -@cindex store -@cindex store items -@cindex store paths +@node package Reference +@subsection @code{package} Reference -Conceptually, the @dfn{store} is the place where derivations that have -been built successfully are stored---by default, @file{/gnu/store}. -Sub-directories in the store are referred to as @dfn{store items} or -sometimes @dfn{store paths}. The store has an associated database that -contains information such as the store paths referred to by each store -path, and the list of @emph{valid} store items---results of successful -builds. This database resides in @file{@var{localstatedir}/guix/db}, -where @var{localstatedir} is the state directory specified @i{via} -@option{--localstatedir} at configure time, usually @file{/var}. +This section summarizes all the options available in @code{package} +declarations (@pxref{Defining Packages}). -The store is @emph{always} accessed by the daemon on behalf of its clients -(@pxref{Invoking guix-daemon}). To manipulate the store, clients -connect to the daemon over a Unix-domain socket, send requests to it, -and read the result---these are remote procedure calls, or RPCs. +@deftp {Data Type} package +This is the data type representing a package recipe. -@quotation Note -Users must @emph{never} modify files under @file{/gnu/store} directly. -This would lead to inconsistencies and break the immutability -assumptions of Guix's functional model (@pxref{Introduction}). +@table @asis +@item @code{name} +The name of the package, as a string. -@xref{Invoking guix gc, @command{guix gc --verify}}, for information on -how to check the integrity of the store and attempt recovery from -accidental modifications. -@end quotation +@item @code{version} +The version of the package, as a string. -The @code{(guix store)} module provides procedures to connect to the -daemon, and to perform RPCs. These are described below. By default, -@code{open-connection}, and thus all the @command{guix} commands, -connect to the local daemon or to the URI specified by the -@code{GUIX_DAEMON_SOCKET} environment variable. +@item @code{source} +An object telling how the source code for the package should be +acquired. Most of the time, this is an @code{origin} object, which +denotes a file fetched from the Internet (@pxref{origin Reference}). It +can also be any other ``file-like'' object such as a @code{local-file}, +which denotes a file from the local file system (@pxref{G-Expressions, +@code{local-file}}). -@defvr {Environment Variable} GUIX_DAEMON_SOCKET -When set, the value of this variable should be a file name or a URI -designating the daemon endpoint. When it is a file name, it denotes a -Unix-domain socket to connect to. In addition to file names, the -supported URI schemes are: +@item @code{build-system} +The build system that should be used to build the package (@pxref{Build +Systems}). -@table @code -@item file -@itemx unix -These are for Unix-domain sockets. -@code{file:///var/guix/daemon-socket/socket} is equivalent to -@file{/var/guix/daemon-socket/socket}. +@item @code{arguments} (default: @code{'()}) +The arguments that should be passed to the build system. This is a +list, typically containing sequential keyword-value pairs. -@item guix -@cindex daemon, remote access -@cindex remote access to the daemon -@cindex daemon, cluster setup -@cindex clusters, daemon setup -These URIs denote connections over TCP/IP, without encryption nor -authentication of the remote host. The URI must specify the host name -and optionally a port number (by default port 44146 is used): +@item @code{inputs} (default: @code{'()}) +@itemx @code{native-inputs} (default: @code{'()}) +@itemx @code{propagated-inputs} (default: @code{'()}) +@cindex inputs, of packages +These fields list dependencies of the package. Each one is a list of +tuples, where each tuple has a label for the input (a string) as its +first element, a package, origin, or derivation as its second element, +and optionally the name of the output thereof that should be used, which +defaults to @code{"out"} (@pxref{Packages with Multiple Outputs}, for +more on package outputs). For example, the list below specifies three +inputs: @example -guix://master.guix.example.org:1234 +`(("libffi" ,libffi) + ("libunistring" ,libunistring) + ("glib:bin" ,glib "bin")) ;the "bin" output of Glib @end example -This setup is suitable on local networks, such as clusters, where only -trusted nodes may connect to the build daemon at -@code{master.guix.example.org}. +@cindex cross compilation, package dependencies +The distinction between @code{native-inputs} and @code{inputs} is +necessary when considering cross-compilation. When cross-compiling, +dependencies listed in @code{inputs} are built for the @emph{target} +architecture; conversely, dependencies listed in @code{native-inputs} +are built for the architecture of the @emph{build} machine. -The @code{--listen} option of @command{guix-daemon} can be used to -instruct it to listen for TCP connections (@pxref{Invoking guix-daemon, -@code{--listen}}). +@code{native-inputs} is typically used to list tools needed at +build time, but not at run time, such as Autoconf, Automake, pkg-config, +Gettext, or Bison. @command{guix lint} can report likely mistakes in +this area (@pxref{Invoking guix lint}). -@item ssh -@cindex SSH access to build daemons -These URIs allow you to connect to a remote daemon over -SSH@footnote{This feature requires Guile-SSH (@pxref{Requirements}).}. -A typical URL might look like this: +@anchor{package-propagated-inputs} +Lastly, @code{propagated-inputs} is similar to @code{inputs}, but the +specified packages will be automatically installed alongside the package +they belong to (@pxref{package-cmd-propagated-inputs, @command{guix +package}}, for information on how @command{guix package} deals with +propagated inputs.) -@example -ssh://charlie@@guix.example.org:22 -@end example +For example this is necessary when a C/C++ library needs headers of +another library to compile, or when a pkg-config file refers to another +one @i{via} its @code{Requires} field. -As for @command{guix copy}, the usual OpenSSH client configuration files -are honored (@pxref{Invoking guix copy}). -@end table +Another example where @code{propagated-inputs} is useful is for languages +that lack a facility to record the run-time search path akin to the +@code{RUNPATH} of ELF files; this includes Guile, Python, Perl, and +more. To ensure that libraries written in those languages can find +library code they depend on at run time, run-time dependencies must be +listed in @code{propagated-inputs} rather than @code{inputs}. -Additional URI schemes may be supported in the future. +@item @code{self-native-input?} (default: @code{#f}) +This is a Boolean field telling whether the package should use itself as +a native input when cross-compiling. -@c XXX: Remove this note when the protocol incurs fewer round trips -@c and when (guix derivations) no longer relies on file system access. -@quotation Note -The ability to connect to remote build daemons is considered -experimental as of @value{VERSION}. Please get in touch with us to -share any problems or suggestions you may have (@pxref{Contributing}). -@end quotation -@end defvr +@item @code{outputs} (default: @code{'("out")}) +The list of output names of the package. @xref{Packages with Multiple +Outputs}, for typical uses of additional outputs. -@deffn {Scheme Procedure} open-connection [@var{uri}] [#:reserve-space? #t] -Connect to the daemon over the Unix-domain socket at @var{uri} (a string). When -@var{reserve-space?} is true, instruct it to reserve a little bit of -extra space on the file system so that the garbage collector can still -operate should the disk become full. Return a server object. +@item @code{native-search-paths} (default: @code{'()}) +@itemx @code{search-paths} (default: @code{'()}) +A list of @code{search-path-specification} objects describing +search-path environment variables honored by the package. -@var{file} defaults to @var{%default-socket-path}, which is the normal -location given the options that were passed to @command{configure}. -@end deffn +@item @code{replacement} (default: @code{#f}) +This must be either @code{#f} or a package object that will be used as a +@dfn{replacement} for this package. @xref{Security Updates, grafts}, +for details. -@deffn {Scheme Procedure} close-connection @var{server} -Close the connection to @var{server}. -@end deffn +@item @code{synopsis} +A one-line description of the package. -@defvr {Scheme Variable} current-build-output-port -This variable is bound to a SRFI-39 parameter, which refers to the port -where build and error logs sent by the daemon should be written. -@end defvr +@item @code{description} +A more elaborate description of the package. -Procedures that make RPCs all take a server object as their first -argument. +@item @code{license} +@cindex license, of packages +The license of the package; a value from @code{(guix licenses)}, +or a list of such values. -@deffn {Scheme Procedure} valid-path? @var{server} @var{path} -@cindex invalid store items -Return @code{#t} when @var{path} designates a valid store item and -@code{#f} otherwise (an invalid item may exist on disk but still be -invalid, for instance because it is the result of an aborted or failed -build.) +@item @code{home-page} +The URL to the home-page of the package, as a string. + +@item @code{supported-systems} (default: @var{%supported-systems}) +The list of systems supported by the package, as strings of the form +@code{architecture-kernel}, for example @code{"x86_64-linux"}. + +@item @code{maintainers} (default: @code{'()}) +The list of maintainers of the package, as @code{maintainer} objects. + +@item @code{location} (default: source location of the @code{package} form) +The source location of the package. It is useful to override this when +inheriting from another package, in which case this field is not +automatically corrected. +@end table +@end deftp -A @code{&store-protocol-error} condition is raised if @var{path} is not -prefixed by the store directory (@file{/gnu/store}). -@end deffn -@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}] -Add @var{text} under file @var{name} in the store, and return its store -path. @var{references} is the list of store paths referred to by the -resulting store path. -@end deffn +@node origin Reference +@subsection @code{origin} Reference -@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations} -Build @var{derivations} (a list of @code{} objects or -derivation paths), and return when the worker is done building them. -Return @code{#t} on success. -@end deffn +This section summarizes all the options available in @code{origin} +declarations (@pxref{Defining Packages}). -Note that the @code{(guix monads)} module provides a monad as well as -monadic versions of the above procedures, with the goal of making it -more convenient to work with code that accesses the store (@pxref{The -Store Monad}). +@deftp {Data Type} origin +This is the data type representing a source code origin. -@c FIXME -@i{This section is currently incomplete.} +@table @asis +@item @code{uri} +An object containing the URI of the source. The object type depends on +the @code{method} (see below). For example, when using the +@var{url-fetch} method of @code{(guix download)}, the valid @code{uri} +values are: a URL represented as a string, or a list thereof. -@node Derivations -@section Derivations +@item @code{method} +A procedure that handles the URI. -@cindex derivations -Low-level build actions and the environment in which they are performed -are represented by @dfn{derivations}. A derivation contains the -following pieces of information: +Examples include: -@itemize -@item -The outputs of the derivation---derivations produce at least one file or -directory in the store, but may produce more. +@table @asis +@item @var{url-fetch} from @code{(guix download)} +download a file from the HTTP, HTTPS, or FTP URL specified in the +@code{uri} field; -@item -The inputs of the derivations, which may be other derivations or plain -files in the store (patches, build scripts, etc.) +@vindex git-fetch +@item @var{git-fetch} from @code{(guix git-download)} +clone the Git version control repository, and check out the revision +specified in the @code{uri} field as a @code{git-reference} object; a +@code{git-reference} looks like this: -@item -The system type targeted by the derivation---e.g., @code{x86_64-linux}. +@example +(git-reference + (url "git://git.debian.org/git/pkg-shadow/shadow") + (commit "v4.1.5.1")) +@end example +@end table -@item -The file name of a build script in the store, along with the arguments -to be passed. +@item @code{sha256} +A bytevector containing the SHA-256 hash of the source. Typically the +@code{base32} form is used here to generate the bytevector from a +base-32 string. -@item -A list of environment variables to be defined. +You can obtain this information using @code{guix download} +(@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking +guix hash}). -@end itemize +@item @code{file-name} (default: @code{#f}) +The file name under which the source code should be saved. When this is +@code{#f}, a sensible default value will be used in most cases. In case +the source is fetched from a URL, the file name from the URL will be +used. For version control checkouts, it is recommended to provide the +file name explicitly because the default is not very descriptive. -@cindex derivation path -Derivations allow clients of the daemon to communicate build actions to -the store. They exist in two forms: as an in-memory representation, -both on the client- and daemon-side, and as files in the store whose -name end in @code{.drv}---these files are referred to as @dfn{derivation -paths}. Derivations paths can be passed to the @code{build-derivations} -procedure to perform the build actions they prescribe (@pxref{The -Store}). +@item @code{patches} (default: @code{'()}) +A list of file names, origins, or file-like objects (@pxref{G-Expressions, +file-like objects}) pointing to patches to be applied to the source. -@cindex fixed-output derivations -Operations such as file downloads and version-control checkouts for -which the expected content hash is known in advance are modeled as -@dfn{fixed-output derivations}. Unlike regular derivations, the outputs -of a fixed-output derivation are independent of its inputs---e.g., a -source code download produces the same result regardless of the download -method and tools being used. +This list of patches must be unconditional. In particular, it cannot +depend on the value of @code{%current-system} or +@code{%current-target-system}. -The @code{(guix derivations)} module provides a representation of -derivations as Scheme objects, along with procedures to create and -otherwise manipulate derivations. The lowest-level primitive to create -a derivation is the @code{derivation} procedure: +@item @code{snippet} (default: @code{#f}) +A G-expression (@pxref{G-Expressions}) or S-expression that will be run +in the source directory. This is a convenient way to modify the source, +sometimes more convenient than a patch. -@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @ - @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ - [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @ - [#:system (%current-system)] [#:references-graphs #f] @ - [#:allowed-references #f] [#:disallowed-references #f] @ - [#:leaked-env-vars #f] [#:local-build? #f] @ - [#:substitutable? #t] [#:properties '()] -Build a derivation with the given arguments, and return the resulting -@code{} object. +@item @code{patch-flags} (default: @code{'("-p1")}) +A list of command-line flags that should be passed to the @code{patch} +command. -When @var{hash} and @var{hash-algo} are given, a -@dfn{fixed-output derivation} is created---i.e., one whose result is -known in advance, such as a file download. If, in addition, -@var{recursive?} is true, then that fixed output may be an executable -file or a directory and @var{hash} must be the hash of an archive -containing this output. +@item @code{patch-inputs} (default: @code{#f}) +Input packages or derivations to the patching process. When this is +@code{#f}, the usual set of inputs necessary for patching are provided, +such as GNU@tie{}Patch. -When @var{references-graphs} is true, it must be a list of file -name/store path pairs. In that case, the reference graph of each store -path is exported in the build environment in the corresponding file, in -a simple text format. +@item @code{modules} (default: @code{'()}) +A list of Guile modules that should be loaded during the patching +process and while running the code in the @code{snippet} field. -When @var{allowed-references} is true, it must be a list of store items -or outputs that the derivation's output may refer to. Likewise, -@var{disallowed-references}, if true, must be a list of things the -outputs may @emph{not} refer to. +@item @code{patch-guile} (default: @code{#f}) +The Guile package that should be used in the patching process. When +this is @code{#f}, a sensible default is used. +@end table +@end deftp -When @var{leaked-env-vars} is true, it must be a list of strings -denoting environment variables that are allowed to ``leak'' from the -daemon's environment to the build environment. This is only applicable -to fixed-output derivations---i.e., when @var{hash} is true. The main -use is to allow variables such as @code{http_proxy} to be passed to -derivations that download files. -When @var{local-build?} is true, declare that the derivation is not a -good candidate for offloading and should rather be built locally -(@pxref{Daemon Offload Setup}). This is the case for small derivations -where the costs of data transfers would outweigh the benefits. +@node Build Systems +@section Build Systems -When @var{substitutable?} is false, declare that substitutes of the -derivation's output should not be used (@pxref{Substitutes}). This is -useful, for instance, when building packages that capture details of the -host CPU instruction set. +@cindex build system +Each package definition specifies a @dfn{build system} and arguments for +that build system (@pxref{Defining Packages}). This @code{build-system} +field represents the build procedure of the package, as well as implicit +dependencies of that build procedure. -@var{properties} must be an association list describing ``properties'' of the -derivation. It is kept as-is, uninterpreted, in the derivation. -@end deffn +Build systems are @code{} objects. The interface to +create and manipulate them is provided by the @code{(guix build-system)} +module, and actual build systems are exported by specific modules. -@noindent -Here's an example with a shell script as its builder, assuming -@var{store} is an open connection to the daemon, and @var{bash} points -to a Bash executable in the store: +@cindex bag (low-level package representation) +Under the hood, build systems first compile package objects to +@dfn{bags}. A @dfn{bag} is like a package, but with less +ornamentation---in other words, a bag is a lower-level representation of +a package, which includes all the inputs of that package, including some +that were implicitly added by the build system. This intermediate +representation is then compiled to a derivation (@pxref{Derivations}). -@lisp -(use-modules (guix utils) - (guix store) - (guix derivations)) +Build systems accept an optional list of @dfn{arguments}. In package +definitions, these are passed @i{via} the @code{arguments} field +(@pxref{Defining Packages}). They are typically keyword arguments +(@pxref{Optional Arguments, keyword arguments in Guile,, guile, GNU +Guile Reference Manual}). The value of these arguments is usually +evaluated in the @dfn{build stratum}---i.e., by a Guile process launched +by the daemon (@pxref{Derivations}). -(let ((builder ; add the Bash script to the store - (add-text-to-store store "my-builder.sh" - "echo hello world > $out\n" '()))) - (derivation store "foo" - bash `("-e" ,builder) - #:inputs `((,bash) (,builder)) - #:env-vars '(("HOME" . "/homeless")))) -@result{} # /gnu/store/@dots{}-foo> -@end lisp +The main build system is @var{gnu-build-system}, which implements the +standard build procedure for GNU and many other packages. It +is provided by the @code{(guix build-system gnu)} module. -As can be guessed, this primitive is cumbersome to use directly. A -better approach is to write build scripts in Scheme, of course! The -best course of action for that is to write the build code as a -``G-expression'', and to pass it to @code{gexp->derivation}. For more -information, @pxref{G-Expressions}. +@defvr {Scheme Variable} gnu-build-system +@var{gnu-build-system} represents the GNU Build System, and variants +thereof (@pxref{Configuration, configuration and makefile conventions,, +standards, GNU Coding Standards}). -Once upon a time, @code{gexp->derivation} did not exist and constructing -derivations with build code written in Scheme was achieved with -@code{build-expression->derivation}, documented below. This procedure -is now deprecated in favor of the much nicer @code{gexp->derivation}. +@cindex build phases +In a nutshell, packages using it are configured, built, and installed with +the usual @code{./configure && make && make check && make install} +command sequence. In practice, a few additional steps are often needed. +All these steps are split up in separate @dfn{phases}, +notably@footnote{Please see the @code{(guix build gnu-build-system)} +modules for more details about the build phases.}: -@deffn {Scheme Procedure} build-expression->derivation @var{store} @ - @var{name} @var{exp} @ - [#:system (%current-system)] [#:inputs '()] @ - [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ - [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ - [#:references-graphs #f] [#:allowed-references #f] @ - [#:disallowed-references #f] @ - [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f] -Return a derivation that executes Scheme expression @var{exp} as a -builder for derivation @var{name}. @var{inputs} must be a list of -@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted, -@code{"out"} is assumed. @var{modules} is a list of names of Guile -modules from the current search path to be copied in the store, -compiled, and made available in the load path during the execution of -@var{exp}---e.g., @code{((guix build utils) (guix build -gnu-build-system))}. +@table @code +@item unpack +Unpack the source tarball, and change the current directory to the +extracted source tree. If the source is actually a directory, copy it +to the build tree, and enter that directory. + +@item patch-source-shebangs +Patch shebangs encountered in source files so they refer to the right +store file names. For instance, this changes @code{#!/bin/sh} to +@code{#!/gnu/store/@dots{}-bash-4.3/bin/sh}. -@var{exp} is evaluated in an environment where @code{%outputs} is bound -to a list of output/path pairs, and where @code{%build-inputs} is bound -to a list of string/output-path pairs made from @var{inputs}. -Optionally, @var{env-vars} is a list of string pairs specifying the name -and value of environment variables visible to the builder. The builder -terminates by passing the result of @var{exp} to @code{exit}; thus, when -@var{exp} returns @code{#f}, the build is considered to have failed. +@item configure +Run the @file{configure} script with a number of default options, such +as @code{--prefix=/gnu/store/@dots{}}, as well as the options specified +by the @code{#:configure-flags} argument. -@var{exp} is built using @var{guile-for-build} (a derivation). When -@var{guile-for-build} is omitted or is @code{#f}, the value of the -@code{%guile-for-build} fluid is used instead. +@item build +Run @code{make} with the list of flags specified with +@code{#:make-flags}. If the @code{#:parallel-build?} argument is true +(the default), build with @code{make -j}. -See the @code{derivation} procedure for the meaning of -@var{references-graphs}, @var{allowed-references}, -@var{disallowed-references}, @var{local-build?}, and -@var{substitutable?}. -@end deffn +@item check +Run @code{make check}, or some other target specified with +@code{#:test-target}, unless @code{#:tests? #f} is passed. If the +@code{#:parallel-tests?} argument is true (the default), run @code{make +check -j}. -@noindent -Here's an example of a single-output derivation that creates a directory -containing one file: +@item install +Run @code{make install} with the flags listed in @code{#:make-flags}. -@lisp -(let ((builder '(let ((out (assoc-ref %outputs "out"))) - (mkdir out) ; create /gnu/store/@dots{}-goo - (call-with-output-file (string-append out "/test") - (lambda (p) - (display '(hello guix) p)))))) - (build-expression->derivation store "goo" builder)) +@item patch-shebangs +Patch shebangs on the installed executable files. -@result{} # @dots{}> -@end lisp +@item strip +Strip debugging symbols from ELF files (unless @code{#:strip-binaries?} +is false), copying them to the @code{debug} output when available +(@pxref{Installing Debugging Files}). +@end table +@vindex %standard-phases +The build-side module @code{(guix build gnu-build-system)} defines +@var{%standard-phases} as the default list of build phases. +@var{%standard-phases} is a list of symbol/procedure pairs, where the +procedure implements the actual phase. -@node The Store Monad -@section The Store Monad +The list of phases used for a particular package can be changed with the +@code{#:phases} parameter. For instance, passing: -@cindex monad +@example +#:phases (modify-phases %standard-phases (delete 'configure)) +@end example -The procedures that operate on the store described in the previous -sections all take an open connection to the build daemon as their first -argument. Although the underlying model is functional, they either have -side effects or depend on the current state of the store. +means that all the phases described above will be used, except the +@code{configure} phase. -The former is inconvenient: the connection to the build daemon has to be -carried around in all those functions, making it impossible to compose -functions that do not take that parameter with functions that do. The -latter can be problematic: since store operations have side effects -and/or depend on external state, they have to be properly sequenced. +In addition, this build system ensures that the ``standard'' environment +for GNU packages is available. This includes tools such as GCC, libc, +Coreutils, Bash, Make, Diffutils, grep, and sed (see the @code{(guix +build-system gnu)} module for a complete list). We call these the +@dfn{implicit inputs} of a package, because package definitions do not +have to mention them. +@end defvr -@cindex monadic values -@cindex monadic functions -This is where the @code{(guix monads)} module comes in. This module -provides a framework for working with @dfn{monads}, and a particularly -useful monad for our uses, the @dfn{store monad}. Monads are a -construct that allows two things: associating ``context'' with values -(in our case, the context is the store), and building sequences of -computations (here computations include accesses to the store). Values -in a monad---values that carry this additional context---are called -@dfn{monadic values}; procedures that return such values are called -@dfn{monadic procedures}. +Other @code{} objects are defined to support other +conventions and tools used by free software packages. They inherit most +of @var{gnu-build-system}, and differ mainly in the set of inputs +implicitly added to the build process, and in the list of phases +executed. Some of these build systems are listed below. -Consider this ``normal'' procedure: +@defvr {Scheme Variable} ant-build-system +This variable is exported by @code{(guix build-system ant)}. It +implements the build procedure for Java packages that can be built with +@url{http://ant.apache.org/, Ant build tool}. -@example -(define (sh-symlink store) - ;; Return a derivation that symlinks the 'bash' executable. - (let* ((drv (package-derivation store bash)) - (out (derivation->output-path drv)) - (sh (string-append out "/bin/bash"))) - (build-expression->derivation store "sh" - `(symlink ,sh %output)))) -@end example +It adds both @code{ant} and the @dfn{Java Development Kit} (JDK) as +provided by the @code{icedtea} package to the set of inputs. Different +packages can be specified with the @code{#:ant} and @code{#:jdk} +parameters, respectively. -Using @code{(guix monads)} and @code{(guix gexp)}, it may be rewritten -as a monadic function: +When the original package does not provide a suitable Ant build file, +the parameter @code{#:jar-name} can be used to generate a minimal Ant +build file @file{build.xml} with tasks to build the specified jar +archive. In this case the parameter @code{#:source-dir} can be used to +specify the source sub-directory, defaulting to ``src''. -@example -(define (sh-symlink) - ;; Same, but return a monadic value. - (mlet %store-monad ((drv (package->derivation bash))) - (gexp->derivation "sh" - #~(symlink (string-append #$drv "/bin/bash") - #$output)))) -@end example +The @code{#:main-class} parameter can be used with the minimal ant +buildfile to specify the main class of the resulting jar. This makes the +jar file executable. The @code{#:test-include} parameter can be used to +specify the list of junit tests to run. It defaults to +@code{(list "**/*Test.java")}. The @code{#:test-exclude} can be used to +disable some tests. It defaults to @code{(list "**/Abstract*.java")}, +because abstract classes cannot be run as tests. -There are several things to note in the second version: the @code{store} -parameter is now implicit and is ``threaded'' in the calls to the -@code{package->derivation} and @code{gexp->derivation} monadic -procedures, and the monadic value returned by @code{package->derivation} -is @dfn{bound} using @code{mlet} instead of plain @code{let}. +The parameter @code{#:build-target} can be used to specify the Ant task +that should be run during the @code{build} phase. By default the +``jar'' task will be run. -As it turns out, the call to @code{package->derivation} can even be -omitted since it will take place implicitly, as we will see later -(@pxref{G-Expressions}): +@end defvr -@example -(define (sh-symlink) - (gexp->derivation "sh" - #~(symlink (string-append #$bash "/bin/bash") - #$output))) -@end example +@defvr {Scheme Variable} android-ndk-build-system +@cindex Android distribution +@cindex Android NDK build system +This variable is exported by @code{(guix build-system android-ndk)}. It +implements a build procedure for Android NDK (native development kit) +packages using a Guix-specific build process. -@c See -@c -@c for the funny quote. -Calling the monadic @code{sh-symlink} has no effect. As someone once -said, ``you exit a monad like you exit a building on fire: by running''. -So, to exit the monad and get the desired effect, one must use -@code{run-with-store}: +The build system assumes that packages install their public interface +(header) files to the subdirectory "include" of the "out" output and +their libraries to the subdirectory "lib" of the "out" output. -@example -(run-with-store (open-connection) (sh-symlink)) -@result{} /gnu/store/...-sh-symlink -@end example +It's also assumed that the union of all the dependencies of a package +has no conflicting files. -Note that the @code{(guix monad-repl)} module extends the Guile REPL with -new ``meta-commands'' to make it easier to deal with monadic procedures: -@code{run-in-store}, and @code{enter-store-monad}. The former is used -to ``run'' a single monadic value through the store: +For the time being, cross-compilation is not supported - so right now +the libraries and header files are assumed to be host tools. -@example -scheme@@(guile-user)> ,run-in-store (package->derivation hello) -$1 = # @dots{}> -@end example +@end defvr -The latter enters a recursive REPL, where all the return values are -automatically run through the store: +@defvr {Scheme Variable} asdf-build-system/source +@defvrx {Scheme Variable} asdf-build-system/sbcl +@defvrx {Scheme Variable} asdf-build-system/ecl -@example -scheme@@(guile-user)> ,enter-store-monad -store-monad@@(guile-user) [1]> (package->derivation hello) -$2 = # @dots{}> -store-monad@@(guile-user) [1]> (text-file "foo" "Hello!") -$3 = "/gnu/store/@dots{}-foo" -store-monad@@(guile-user) [1]> ,q -scheme@@(guile-user)> -@end example +These variables, exported by @code{(guix build-system asdf)}, implement +build procedures for Common Lisp packages using +@url{https://common-lisp.net/project/asdf/, ``ASDF''}. ASDF is a system +definition facility for Common Lisp programs and libraries. -@noindent -Note that non-monadic values cannot be returned in the -@code{store-monad} REPL. +The @code{asdf-build-system/source} system installs the packages in +source form, and can be loaded using any common lisp implementation, via +ASDF. The others, such as @code{asdf-build-system/sbcl}, install binary +systems in the format which a particular implementation understands. +These build systems can also be used to produce executable programs, or +lisp images which contain a set of packages pre-loaded. -The main syntactic forms to deal with monads in general are provided by -the @code{(guix monads)} module and are described below. +The build system uses naming conventions. For binary packages, the +package name should be prefixed with the lisp implementation, such as +@code{sbcl-} for @code{asdf-build-system/sbcl}. -@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ... -Evaluate any @code{>>=} or @code{return} forms in @var{body} as being -in @var{monad}. -@end deffn +Additionally, the corresponding source package should be labeled using +the same convention as python packages (see @ref{Python Modules}), using +the @code{cl-} prefix. -@deffn {Scheme Syntax} return @var{val} -Return a monadic value that encapsulates @var{val}. -@end deffn +For binary packages, each system should be defined as a Guix package. +If one package @code{origin} contains several systems, package variants +can be created in order to build all the systems. Source packages, +which use @code{asdf-build-system/source}, may contain several systems. -@deffn {Scheme Syntax} >>= @var{mval} @var{mproc} ... -@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic -procedures @var{mproc}@dots{}@footnote{This operation is commonly -referred to as ``bind'', but that name denotes an unrelated procedure in -Guile. Thus we use this somewhat cryptic symbol inherited from the -Haskell language.}. There can be one @var{mproc} or several of them, as -in this example: +In order to create executable programs and images, the build-side +procedures @code{build-program} and @code{build-image} can be used. +They should be called in a build phase after the @code{create-symlinks} +phase, so that the system which was just built can be used within the +resulting image. @code{build-program} requires a list of Common Lisp +expressions to be passed as the @code{#:entry-program} argument. + +If the system is not defined within its own @code{.asd} file of the same +name, then the @code{#:asd-file} parameter should be used to specify +which file the system is defined in. Furthermore, if the package +defines a system for its tests in a separate file, it will be loaded +before the tests are run if it is specified by the +@code{#:test-asd-file} parameter. If it is not set, the files +@code{-tests.asd}, @code{-test.asd}, @code{tests.asd}, +and @code{test.asd} will be tried if they exist. -@example -(run-with-state - (with-monad %state-monad - (>>= (return 1) - (lambda (x) (return (+ 1 x))) - (lambda (x) (return (* 2 x))))) - 'some-state) +If for some reason the package must be named in a different way than the +naming conventions suggest, the @code{#:asd-system-name} parameter can +be used to specify the name of the system. -@result{} 4 -@result{} some-state -@end example -@end deffn +@end defvr -@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @ - @var{body} ... -@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @ - @var{body} ... -Bind the variables @var{var} to the monadic values @var{mval} in -@var{body}, which is a sequence of expressions. As with the bind -operator, this can be thought of as ``unpacking'' the raw, non-monadic -value ``contained'' in @var{mval} and making @var{var} refer to that -raw, non-monadic value within the scope of the @var{body}. The form -(@var{var} -> @var{val}) binds @var{var} to the ``normal'' value -@var{val}, as per @code{let}. The binding operations occur in sequence -from left to right. The last expression of @var{body} must be a monadic -expression, and its result will become the result of the @code{mlet} or -@code{mlet*} when run in the @var{monad}. +@defvr {Scheme Variable} cargo-build-system +@cindex Rust programming language +@cindex Cargo (Rust build system) +This variable is exported by @code{(guix build-system cargo)}. It +supports builds of packages using Cargo, the build tool of the +@uref{https://www.rust-lang.org, Rust programming language}. -@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let} -(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}). -@end deffn +In its @code{configure} phase, this build system replaces dependencies +specified in the @file{Carto.toml} file with inputs to the Guix package. +The @code{install} phase installs the binaries, and it also installs the +source code and @file{Cargo.toml} file. +@end defvr -@deffn {Scheme System} mbegin @var{monad} @var{mexp} ... -Bind @var{mexp} and the following monadic expressions in sequence, -returning the result of the last expression. Every expression in the -sequence must be a monadic expression. +@cindex Clojure (programming language) +@cindex simple Clojure build system +@defvr {Scheme Variable} clojure-build-system +This variable is exported by @code{(guix build-system clojure)}. It implements +a simple build procedure for @uref{https://clojure.org/, Clojure} packages +using plain old @code{compile} in Clojure. Cross-compilation is not supported +yet. -This is akin to @code{mlet}, except that the return values of the -monadic expressions are ignored. In that sense, it is analogous to -@code{begin}, but applied to monadic expressions. -@end deffn +It adds @code{clojure}, @code{icedtea} and @code{zip} to the set of inputs. +Different packages can be specified with the @code{#:clojure}, @code{#:jdk} and +@code{#:zip} parameters, respectively. -@deffn {Scheme System} mwhen @var{condition} @var{mexp0} @var{mexp*} ... -When @var{condition} is true, evaluate the sequence of monadic -expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When -@var{condition} is false, return @code{*unspecified*} in the current -monad. Every expression in the sequence must be a monadic expression. -@end deffn +A list of source directories, test directories and jar names can be specified +with the @code{#:source-dirs}, @code{#:test-dirs} and @code{#:jar-names} +parameters, respectively. Compile directory and main class can be specified +with the @code{#:compile-dir} and @code{#:main-class} parameters, respectively. +Other parameters are documented below. -@deffn {Scheme System} munless @var{condition} @var{mexp0} @var{mexp*} ... -When @var{condition} is false, evaluate the sequence of monadic -expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When -@var{condition} is true, return @code{*unspecified*} in the current -monad. Every expression in the sequence must be a monadic expression. -@end deffn +This build system is an extension of @var{ant-build-system}, but with the +following phases changed: -@cindex state monad -The @code{(guix monads)} module provides the @dfn{state monad}, which -allows an additional value---the state---to be @emph{threaded} through -monadic procedure calls. +@table @code -@defvr {Scheme Variable} %state-monad -The state monad. Procedures in the state monad can access and change -the state that is threaded. +@item build +This phase calls @code{compile} in Clojure to compile source files and runs +@command{jar} to create jars from both source files and compiled files +according to the include list and exclude list specified in +@code{#:aot-include} and @code{#:aot-exclude}, respectively. The exclude list +has priority over the include list. These lists consist of symbols +representing Clojure libraries or the special keyword @code{#:all} representing +all Clojure libraries found under the source directories. The parameter +@code{#:omit-source?} decides if source should be included into the jars. -Consider the example below. The @code{square} procedure returns a value -in the state monad. It returns the square of its argument, but also -increments the current state value: +@item check +This phase runs tests according to the include list and exclude list specified +in @code{#:test-include} and @code{#:test-exclude}, respectively. Their +meanings are analogous to that of @code{#:aot-include} and +@code{#:aot-exclude}, except that the special keyword @code{#:all} now +stands for all Clojure libraries found under the test directories. The +parameter @code{#:tests?} decides if tests should be run. -@example -(define (square x) - (mlet %state-monad ((count (current-state))) - (mbegin %state-monad - (set-current-state (+ 1 count)) - (return (* x x))))) +@item install +This phase installs all jars built previously. +@end table -(run-with-state (sequence %state-monad (map square (iota 3))) 0) -@result{} (0 1 4) -@result{} 3 -@end example +Apart from the above, this build system also contains an additional phase: -When ``run'' through @var{%state-monad}, we obtain that additional state -value, which is the number of @code{square} calls. -@end defvr +@table @code -@deffn {Monadic Procedure} current-state -Return the current state as a monadic value. -@end deffn +@item install-doc +This phase installs all top-level files with base name matching +@var{%doc-regex}. A different regex can be specified with the +@code{#:doc-regex} parameter. All files (recursively) inside the documentation +directories specified in @code{#:doc-dirs} are installed as well. +@end table +@end defvr -@deffn {Monadic Procedure} set-current-state @var{value} -Set the current state to @var{value} and return the previous state as a -monadic value. -@end deffn +@defvr {Scheme Variable} cmake-build-system +This variable is exported by @code{(guix build-system cmake)}. It +implements the build procedure for packages using the +@url{http://www.cmake.org, CMake build tool}. -@deffn {Monadic Procedure} state-push @var{value} -Push @var{value} to the current state, which is assumed to be a list, -and return the previous state as a monadic value. -@end deffn +It automatically adds the @code{cmake} package to the set of inputs. +Which package is used can be specified with the @code{#:cmake} +parameter. -@deffn {Monadic Procedure} state-pop -Pop a value from the current state and return it as a monadic value. -The state is assumed to be a list. -@end deffn +The @code{#:configure-flags} parameter is taken as a list of flags +passed to the @command{cmake} command. The @code{#:build-type} +parameter specifies in abstract terms the flags passed to the compiler; +it defaults to @code{"RelWithDebInfo"} (short for ``release mode with +debugging information''), which roughly means that code is compiled with +@code{-O2 -g}, as is the case for Autoconf-based packages by default. +@end defvr -@deffn {Scheme Procedure} run-with-state @var{mval} [@var{state}] -Run monadic value @var{mval} starting with @var{state} as the initial -state. Return two values: the resulting value, and the resulting state. -@end deffn +@defvr {Scheme Variable} dune-build-system +This variable is exported by @code{(guix build-system dune)}. It +supports builds of packages using @uref{https://dune.build/, Dune}, a build +tool for the OCaml programming language. It is implemented as an extension +of the @code{ocaml-build-system} which is described below. As such, the +@code{#:ocaml} and @code{#:findlib} parameters can be passed to this build +system. -The main interface to the store monad, provided by the @code{(guix -store)} module, is as follows. +It automatically adds the @code{dune} package to the set of inputs. +Which package is used can be specified with the @code{#:dune} +parameter. -@defvr {Scheme Variable} %store-monad -The store monad---an alias for @var{%state-monad}. +There is no @code{configure} phase because dune packages typically don't +need to be configured. The @code{#:build-flags} parameter is taken as a +list of flags passed to the @code{dune} command during the build. -Values in the store monad encapsulate accesses to the store. When its -effect is needed, a value of the store monad must be ``evaluated'' by -passing it to the @code{run-with-store} procedure (see below.) +The @code{#:jbuild?} parameter can be passed to use the @code{jbuild} +command instead of the more recent @code{dune} command while building +a package. Its default value is @code{#f}. @end defvr -@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)] -Run @var{mval}, a monadic value in the store monad, in @var{store}, an -open store connection. -@end deffn +@defvr {Scheme Variable} go-build-system +This variable is exported by @code{(guix build-system go)}. It +implements a build procedure for Go packages using the standard +@url{https://golang.org/cmd/go/#hdr-Compile_packages_and_dependencies, +Go build mechanisms}. -@deffn {Monadic Procedure} text-file @var{name} @var{text} [@var{references}] -Return as a monadic value the absolute file name in the store of the file -containing @var{text}, a string. @var{references} is a list of store items that the -resulting text file refers to; it defaults to the empty list. -@end deffn +The user is expected to provide a value for the key @code{#:import-path} +and, in some cases, @code{#:unpack-path}. The +@url{https://golang.org/doc/code.html#ImportPaths, import path} +corresponds to the file system path expected by the package's build +scripts and any referring packages, and provides a unique way to +refer to a Go package. It is typically based on a combination of the +package source code's remote URI and file system hierarchy structure. In +some cases, you will need to unpack the package's source code to a +different directory structure than the one indicated by the import path, +and @code{#:unpack-path} should be used in such cases. -@deffn {Monadic Procedure} binary-file @var{name} @var{data} [@var{references}] -Return as a monadic value the absolute file name in the store of the file -containing @var{data}, a bytevector. @var{references} is a list of store -items that the resulting binary file refers to; it defaults to the empty list. -@end deffn +Packages that provide Go libraries should be installed along with their +source code. The key @code{#:install-source?}, which defaults to +@code{#t}, controls whether or not the source code is installed. It can +be set to @code{#f} for packages that only provide executable files. +@end defvr -@deffn {Monadic Procedure} interned-file @var{file} [@var{name}] @ - [#:recursive? #t] [#:select? (const #t)] -Return the name of @var{file} once interned in the store. Use -@var{name} as its store name, or the basename of @var{file} if -@var{name} is omitted. +@defvr {Scheme Variable} glib-or-gtk-build-system +This variable is exported by @code{(guix build-system glib-or-gtk)}. It +is intended for use with packages making use of GLib or GTK+. -When @var{recursive?} is true, the contents of @var{file} are added -recursively; if @var{file} designates a flat file and @var{recursive?} -is true, its contents are added, and its permission bits are kept. +This build system adds the following two phases to the ones defined by +@var{gnu-build-system}: -When @var{recursive?} is true, call @code{(@var{select?} @var{file} -@var{stat})} for each directory entry, where @var{file} is the entry's -absolute file name and @var{stat} is the result of @code{lstat}; exclude -entries for which @var{select?} does not return true. +@table @code +@item glib-or-gtk-wrap +The phase @code{glib-or-gtk-wrap} ensures that programs in +@file{bin/} are able to find GLib ``schemas'' and +@uref{https://developer.gnome.org/gtk3/stable/gtk-running.html, GTK+ +modules}. This is achieved by wrapping the programs in launch scripts +that appropriately set the @code{XDG_DATA_DIRS} and @code{GTK_PATH} +environment variables. -The example below adds a file to the store, under two different names: +It is possible to exclude specific package outputs from that wrapping +process by listing their names in the +@code{#:glib-or-gtk-wrap-excluded-outputs} parameter. This is useful +when an output is known not to contain any GLib or GTK+ binaries, and +where wrapping would gratuitously add a dependency of that output on +GLib and GTK+. + +@item glib-or-gtk-compile-schemas +The phase @code{glib-or-gtk-compile-schemas} makes sure that all +@uref{https://developer.gnome.org/gio/stable/glib-compile-schemas.html, +GSettings schemas} of GLib are compiled. Compilation is performed by the +@command{glib-compile-schemas} program. It is provided by the package +@code{glib:bin} which is automatically imported by the build system. +The @code{glib} package providing @command{glib-compile-schemas} can be +specified with the @code{#:glib} parameter. +@end table -@example -(run-with-store (open-connection) - (mlet %store-monad ((a (interned-file "README")) - (b (interned-file "README" "LEGU-MIN"))) - (return (list a b)))) +Both phases are executed after the @code{install} phase. +@end defvr -@result{} ("/gnu/store/rwm@dots{}-README" "/gnu/store/44i@dots{}-LEGU-MIN") -@end example +@defvr {Scheme Variable} guile-build-system +This build system is for Guile packages that consist exclusively of Scheme +code and that are so lean that they don't even have a makefile, let alone a +@file{configure} script. It compiles Scheme code using @command{guild +compile} (@pxref{Compilation,,, guile, GNU Guile Reference Manual}) and +installs the @file{.scm} and @file{.go} files in the right place. It also +installs documentation. -@end deffn +This build system supports cross-compilation by using the @code{--target} +option of @command{guild compile}. -The @code{(guix packages)} module exports the following package-related -monadic procedures: +Packages built with @code{guile-build-system} must provide a Guile package in +their @code{native-inputs} field. +@end defvr -@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @ - [#:system (%current-system)] [#:target #f] @ - [#:output "out"] -Return as a monadic -value in the absolute file name of @var{file} within the @var{output} -directory of @var{package}. When @var{file} is omitted, return the name -of the @var{output} directory of @var{package}. When @var{target} is -true, use it as a cross-compilation target triplet. -@end deffn +@defvr {Scheme Variable} minify-build-system +This variable is exported by @code{(guix build-system minify)}. It +implements a minification procedure for simple JavaScript packages. -@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}] -@deffnx {Monadic Procedure} package->cross-derivation @var{package} @ - @var{target} [@var{system}] -Monadic version of @code{package-derivation} and -@code{package-cross-derivation} (@pxref{Defining Packages}). -@end deffn +It adds @code{uglify-js} to the set of inputs and uses it to compress +all JavaScript files in the @file{src} directory. A different minifier +package can be specified with the @code{#:uglify-js} parameter, but it +is expected that the package writes the minified code to the standard +output. +When the input JavaScript files are not all located in the @file{src} +directory, the parameter @code{#:javascript-files} can be used to +specify a list of file names to feed to the minifier. +@end defvr -@node G-Expressions -@section G-Expressions +@defvr {Scheme Variable} ocaml-build-system +This variable is exported by @code{(guix build-system ocaml)}. It implements +a build procedure for @uref{https://ocaml.org, OCaml} packages, which consists +of choosing the correct set of commands to run for each package. OCaml +packages can expect many different commands to be run. This build system will +try some of them. -@cindex G-expression -@cindex build code quoting -So we have ``derivations'', which represent a sequence of build actions -to be performed to produce an item in the store (@pxref{Derivations}). -These build actions are performed when asking the daemon to actually -build the derivations; they are run by the daemon in a container -(@pxref{Invoking guix-daemon}). +When the package has a @file{setup.ml} file present at the top-level, it will +run @code{ocaml setup.ml -configure}, @code{ocaml setup.ml -build} and +@code{ocaml setup.ml -install}. The build system will assume that this file +was generated by @uref{http://oasis.forge.ocamlcore.org/, OASIS} and will take +care of setting the prefix and enabling tests if they are not disabled. You +can pass configure and build flags with the @code{#:configure-flags} and +@code{#:build-flags}. The @code{#:test-flags} key can be passed to change the +set of flags used to enable tests. The @code{#:use-make?} key can be used to +bypass this system in the build and install phases. -@cindex strata of code -It should come as no surprise that we like to write these build actions -in Scheme. When we do that, we end up with two @dfn{strata} of Scheme -code@footnote{The term @dfn{stratum} in this context was coined by -Manuel Serrano et al.@: in the context of their work on Hop. Oleg -Kiselyov, who has written insightful -@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code -on this topic}, refers to this kind of code generation as -@dfn{staging}.}: the ``host code''---code that defines packages, talks -to the daemon, etc.---and the ``build code''---code that actually -performs build actions, such as making directories, invoking -@command{make}, etc. +When the package has a @file{configure} file, it is assumed that it is a +hand-made configure script that requires a different argument format than +in the @code{gnu-build-system}. You can add more flags with the +@code{#:configure-flags} key. -To describe a derivation and its build actions, one typically needs to -embed build code inside host code. It boils down to manipulating build -code as data, and the homoiconicity of Scheme---code has a direct -representation as data---comes in handy for that. But we need more than -the normal @code{quasiquote} mechanism in Scheme to construct build -expressions. +When the package has a @file{Makefile} file (or @code{#:use-make?} is +@code{#t}), it will be used and more flags can be passed to the build and +install phases with the @code{#:make-flags} key. -The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of -S-expressions adapted to build expressions. G-expressions, or -@dfn{gexps}, consist essentially of three syntactic forms: @code{gexp}, -@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~}, -@code{#$}, and @code{#$@@}), which are comparable to -@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}, -respectively (@pxref{Expression Syntax, @code{quasiquote},, guile, -GNU Guile Reference Manual}). However, there are major differences: +Finally, some packages do not have these files and use a somewhat standard +location for its build system. In that case, the build system will run +@code{ocaml pkg/pkg.ml} or @code{ocaml pkg/build.ml} and take care of +providing the path to the required findlib module. Additional flags can +be passed via the @code{#:build-flags} key. Install is taken care of by +@command{opam-installer}. In this case, the @code{opam} package must +be added to the @code{native-inputs} field of the package definition. -@itemize -@item -Gexps are meant to be written to a file and run or manipulated by other -processes. +Note that most OCaml packages assume they will be installed in the same +directory as OCaml, which is not what we want in guix. In particular, they +will install @file{.so} files in their module's directory, which is usually +fine because it is in the OCaml compiler directory. In guix though, these +libraries cannot be found and we use @code{CAML_LD_LIBRARY_PATH}. This +variable points to @file{lib/ocaml/site-lib/stubslibs} and this is where +@file{.so} libraries should be installed. +@end defvr -@item -When a high-level object such as a package or derivation is unquoted -inside a gexp, the result is as if its output file name had been -introduced. +@defvr {Scheme Variable} python-build-system +This variable is exported by @code{(guix build-system python)}. It +implements the more or less standard build procedure used by Python +packages, which consists in running @code{python setup.py build} and +then @code{python setup.py install --prefix=/gnu/store/@dots{}}. -@item -Gexps carry information about the packages or derivations they refer to, -and these dependencies are automatically added as inputs to the build -processes that use them. -@end itemize +For packages that install stand-alone Python programs under @code{bin/}, +it takes care of wrapping these programs so that their @code{PYTHONPATH} +environment variable points to all the Python libraries they depend on. -@cindex lowering, of high-level objects in gexps -This mechanism is not limited to package and derivation -objects: @dfn{compilers} able to ``lower'' other high-level objects to -derivations or files in the store can be defined, -such that these objects can also be inserted -into gexps. For example, a useful type of high-level objects that can be -inserted in a gexp is ``file-like objects'', which make it easy to -add files to the store and to refer to them in -derivations and such (see @code{local-file} and @code{plain-file} -below.) +Which Python package is used to perform the build can be specified with +the @code{#:python} parameter. This is a useful way to force a package +to be built for a specific version of the Python interpreter, which +might be necessary if the package is only compatible with a single +interpreter version. -To illustrate the idea, here is an example of a gexp: +By default guix calls @code{setup.py} under control of +@code{setuptools}, much like @command{pip} does. Some packages are not +compatible with setuptools (and pip), thus you can disable this by +setting the @code{#:use-setuptools} parameter to @code{#f}. +@end defvr -@example -(define build-exp - #~(begin - (mkdir #$output) - (chdir #$output) - (symlink (string-append #$coreutils "/bin/ls") - "list-files"))) -@end example +@defvr {Scheme Variable} perl-build-system +This variable is exported by @code{(guix build-system perl)}. It +implements the standard build procedure for Perl packages, which either +consists in running @code{perl Build.PL --prefix=/gnu/store/@dots{}}, +followed by @code{Build} and @code{Build install}; or in running +@code{perl Makefile.PL PREFIX=/gnu/store/@dots{}}, followed by +@code{make} and @code{make install}, depending on which of +@code{Build.PL} or @code{Makefile.PL} is present in the package +distribution. Preference is given to the former if both @code{Build.PL} +and @code{Makefile.PL} exist in the package distribution. This +preference can be reversed by specifying @code{#t} for the +@code{#:make-maker?} parameter. -This gexp can be passed to @code{gexp->derivation}; we obtain a -derivation that builds a directory containing exactly one symlink to -@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}: +The initial @code{perl Makefile.PL} or @code{perl Build.PL} invocation +passes flags specified by the @code{#:make-maker-flags} or +@code{#:module-build-flags} parameter, respectively. -@example -(gexp->derivation "the-thing" build-exp) -@end example +Which Perl package is used can be specified with @code{#:perl}. +@end defvr -As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is -substituted to the reference to the @var{coreutils} package in the -actual build code, and @var{coreutils} is automatically made an input to -the derivation. Likewise, @code{#$output} (equivalent to @code{(ungexp -output)}) is replaced by a string containing the directory name of the -output of the derivation. +@defvr {Scheme Variable} r-build-system +This variable is exported by @code{(guix build-system r)}. It +implements the build procedure used by @uref{http://r-project.org, R} +packages, which essentially is little more than running @code{R CMD +INSTALL --library=/gnu/store/@dots{}} in an environment where +@code{R_LIBS_SITE} contains the paths to all R package inputs. Tests +are run after installation using the R function +@code{tools::testInstalledPackage}. +@end defvr -@cindex cross compilation -In a cross-compilation context, it is useful to distinguish between -references to the @emph{native} build of a package---that can run on the -host---versus references to cross builds of a package. To that end, the -@code{#+} plays the same role as @code{#$}, but is a reference to a -native package build: +@defvr {Scheme Variable} texlive-build-system +This variable is exported by @code{(guix build-system texlive)}. It is +used to build TeX packages in batch mode with a specified engine. The +build system sets the @code{TEXINPUTS} variable to find all TeX source +files in the inputs. -@example -(gexp->derivation "vi" - #~(begin - (mkdir #$output) - (system* (string-append #+coreutils "/bin/ln") - "-s" - (string-append #$emacs "/bin/emacs") - (string-append #$output "/bin/vi"))) - #:target "mips64el-linux-gnu") -@end example +By default it runs @code{luatex} on all files ending on @code{ins}. A +different engine and format can be specified with the +@code{#:tex-format} argument. Different build targets can be specified +with the @code{#:build-targets} argument, which expects a list of file +names. The build system adds only @code{texlive-bin} and +@code{texlive-latex-base} (both from @code{(gnu packages tex}) to the +inputs. Both can be overridden with the arguments @code{#:texlive-bin} +and @code{#:texlive-latex-base}, respectively. -@noindent -In the example above, the native build of @var{coreutils} is used, so -that @command{ln} can actually run on the host; but then the -cross-compiled build of @var{emacs} is referenced. +The @code{#:tex-directory} parameter tells the build system where to +install the built files under the texmf tree. +@end defvr -@cindex imported modules, for gexps -@findex with-imported-modules -Another gexp feature is @dfn{imported modules}: sometimes you want to be -able to use certain Guile modules from the ``host environment'' in the -gexp, so those modules should be imported in the ``build environment''. -The @code{with-imported-modules} form allows you to express that: +@defvr {Scheme Variable} ruby-build-system +This variable is exported by @code{(guix build-system ruby)}. It +implements the RubyGems build procedure used by Ruby packages, which +involves running @code{gem build} followed by @code{gem install}. -@example -(let ((build (with-imported-modules '((guix build utils)) - #~(begin - (use-modules (guix build utils)) - (mkdir-p (string-append #$output "/bin")))))) - (gexp->derivation "empty-dir" - #~(begin - #$build - (display "success!\n") - #t))) -@end example +The @code{source} field of a package that uses this build system +typically references a gem archive, since this is the format that Ruby +developers use when releasing their software. The build system unpacks +the gem archive, potentially patches the source, runs the test suite, +repackages the gem, and installs it. Additionally, directories and +tarballs may be referenced to allow building unreleased gems from Git or +a traditional source release tarball. -@noindent -In this example, the @code{(guix build utils)} module is automatically -pulled into the isolated build environment of our gexp, such that -@code{(use-modules (guix build utils))} works as expected. +Which Ruby package is used can be specified with the @code{#:ruby} +parameter. A list of additional flags to be passed to the @command{gem} +command can be specified with the @code{#:gem-flags} parameter. +@end defvr -@cindex module closure -@findex source-module-closure -Usually you want the @emph{closure} of the module to be imported---i.e., -the module itself and all the modules it depends on---rather than just -the module; failing to do that, attempts to use the module will fail -because of missing dependent modules. The @code{source-module-closure} -procedure computes the closure of a module by looking at its source file -headers, which comes in handy in this case: +@defvr {Scheme Variable} waf-build-system +This variable is exported by @code{(guix build-system waf)}. It +implements a build procedure around the @code{waf} script. The common +phases---@code{configure}, @code{build}, and @code{install}---are +implemented by passing their names as arguments to the @code{waf} +script. -@example -(use-modules (guix modules)) ;for 'source-module-closure' +The @code{waf} script is executed by the Python interpreter. Which +Python package is used to run the script can be specified with the +@code{#:python} parameter. +@end defvr -(with-imported-modules (source-module-closure - '((guix build utils) - (gnu build vm))) - (gexp->derivation "something-with-vms" - #~(begin - (use-modules (guix build utils) - (gnu build vm)) - @dots{}))) -@end example +@defvr {Scheme Variable} scons-build-system +This variable is exported by @code{(guix build-system scons)}. It +implements the build procedure used by the SCons software construction +tool. This build system runs @code{scons} to build the package, +@code{scons test} to run tests, and then @code{scons install} to install +the package. -@cindex extensions, for gexps -@findex with-extensions -In the same vein, sometimes you want to import not just pure-Scheme -modules, but also ``extensions'' such as Guile bindings to C libraries -or other ``full-blown'' packages. Say you need the @code{guile-json} -package available on the build side, here's how you would do it: +Additional flags to be passed to @code{scons} can be specified with the +@code{#:scons-flags} parameter. The version of Python used to run SCons +can be specified by selecting the appropriate SCons package with the +@code{#:scons} parameter. +@end defvr -@example -(use-modules (gnu packages guile)) ;for 'guile-json' +@defvr {Scheme Variable} haskell-build-system +This variable is exported by @code{(guix build-system haskell)}. It +implements the Cabal build procedure used by Haskell packages, which +involves running @code{runhaskell Setup.hs configure +--prefix=/gnu/store/@dots{}} and @code{runhaskell Setup.hs build}. +Instead of installing the package by running @code{runhaskell Setup.hs +install}, to avoid trying to register libraries in the read-only +compiler store directory, the build system uses @code{runhaskell +Setup.hs copy}, followed by @code{runhaskell Setup.hs register}. In +addition, the build system generates the package documentation by +running @code{runhaskell Setup.hs haddock}, unless @code{#:haddock? #f} +is passed. Optional Haddock parameters can be passed with the help of +the @code{#:haddock-flags} parameter. If the file @code{Setup.hs} is +not found, the build system looks for @code{Setup.lhs} instead. -(with-extensions (list guile-json) - (gexp->derivation "something-with-json" - #~(begin - (use-modules (json)) - @dots{}))) -@end example +Which Haskell compiler is used can be specified with the @code{#:haskell} +parameter which defaults to @code{ghc}. +@end defvr -The syntactic form to construct gexps is summarized below. +@defvr {Scheme Variable} dub-build-system +This variable is exported by @code{(guix build-system dub)}. It +implements the Dub build procedure used by D packages, which +involves running @code{dub build} and @code{dub run}. +Installation is done by copying the files manually. -@deffn {Scheme Syntax} #~@var{exp} -@deffnx {Scheme Syntax} (gexp @var{exp}) -Return a G-expression containing @var{exp}. @var{exp} may contain one -or more of the following forms: +Which D compiler is used can be specified with the @code{#:ldc} +parameter which defaults to @code{ldc}. +@end defvr -@table @code -@item #$@var{obj} -@itemx (ungexp @var{obj}) -Introduce a reference to @var{obj}. @var{obj} may have one of the -supported types, for example a package or a -derivation, in which case the @code{ungexp} form is replaced by its -output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}. +@defvr {Scheme Variable} emacs-build-system +This variable is exported by @code{(guix build-system emacs)}. It +implements an installation procedure similar to the packaging system +of Emacs itself (@pxref{Packages,,, emacs, The GNU Emacs Manual}). -If @var{obj} is a list, it is traversed and references to supported -objects are substituted similarly. +It first creates the @code{@var{package}-autoloads.el} file, then it +byte compiles all Emacs Lisp files. Differently from the Emacs +packaging system, the Info documentation files are moved to the standard +documentation directory and the @file{dir} file is deleted. Each +package is installed in its own directory under +@file{share/emacs/site-lisp/guix.d}. +@end defvr -If @var{obj} is another gexp, its contents are inserted and its -dependencies are added to those of the containing gexp. +@defvr {Scheme Variable} font-build-system +This variable is exported by @code{(guix build-system font)}. It +implements an installation procedure for font packages where upstream +provides pre-compiled TrueType, OpenType, etc.@: font files that merely +need to be copied into place. It copies font files to standard +locations in the output directory. +@end defvr -If @var{obj} is another kind of object, it is inserted as is. +@defvr {Scheme Variable} meson-build-system +This variable is exported by @code{(guix build-system meson)}. It +implements the build procedure for packages that use +@url{http://mesonbuild.com, Meson} as their build system. -@item #$@var{obj}:@var{output} -@itemx (ungexp @var{obj} @var{output}) -This is like the form above, but referring explicitly to the -@var{output} of @var{obj}---this is useful when @var{obj} produces -multiple outputs (@pxref{Packages with Multiple Outputs}). +It adds both Meson and @uref{https://ninja-build.org/, Ninja} to the set +of inputs, and they can be changed with the parameters @code{#:meson} +and @code{#:ninja} if needed. The default Meson is +@code{meson-for-build}, which is special because it doesn't clear the +@code{RUNPATH} of binaries and libraries when they are installed. -@item #+@var{obj} -@itemx #+@var{obj}:output -@itemx (ungexp-native @var{obj}) -@itemx (ungexp-native @var{obj} @var{output}) -Same as @code{ungexp}, but produces a reference to the @emph{native} -build of @var{obj} when used in a cross compilation context. +This build system is an extension of @var{gnu-build-system}, but with the +following phases changed to some specific for Meson: -@item #$output[:@var{output}] -@itemx (ungexp output [@var{output}]) -Insert a reference to derivation output @var{output}, or to the main -output when @var{output} is omitted. +@table @code -This only makes sense for gexps passed to @code{gexp->derivation}. +@item configure +The phase runs @code{meson} with the flags specified in +@code{#:configure-flags}. The flag @code{--build-type} is always set to +@code{plain} unless something else is specified in @code{#:build-type}. -@item #$@@@var{lst} -@itemx (ungexp-splicing @var{lst}) -Like the above, but splices the contents of @var{lst} inside the -containing list. +@item build +The phase runs @code{ninja} to build the package in parallel by default, but +this can be changed with @code{#:parallel-build?}. -@item #+@@@var{lst} -@itemx (ungexp-native-splicing @var{lst}) -Like the above, but refers to native builds of the objects listed in -@var{lst}. +@item check +The phase runs @code{ninja} with the target specified in @code{#:test-target}, +which is @code{"test"} by default. +@item install +The phase runs @code{ninja install} and can not be changed. @end table -G-expressions created by @code{gexp} or @code{#~} are run-time objects -of the @code{gexp?} type (see below.) -@end deffn +Apart from that, the build system also adds the following phases: -@deffn {Scheme Syntax} with-imported-modules @var{modules} @var{body}@dots{} -Mark the gexps defined in @var{body}@dots{} as requiring @var{modules} -in their execution environment. +@table @code -Each item in @var{modules} can be the name of a module, such as -@code{(guix build utils)}, or it can be a module name, followed by an -arrow, followed by a file-like object: +@item fix-runpath +This phase ensures that all binaries can find the libraries they need. +It searches for required libraries in subdirectories of the package being +built, and adds those to @code{RUNPATH} where needed. It also removes +references to libraries left over from the build phase by +@code{meson-for-build}, such as test dependencies, that aren't actually +required for the program to run. -@example -`((guix build utils) - (guix gcrypt) - ((guix config) => ,(scheme-file "config.scm" - #~(define-module @dots{})))) -@end example +@item glib-or-gtk-wrap +This phase is the phase provided by @code{glib-or-gtk-build-system}, and it +is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. -@noindent -In the example above, the first two modules are taken from the search -path, and the last one is created from the given file-like object. +@item glib-or-gtk-compile-schemas +This phase is the phase provided by @code{glib-or-gtk-build-system}, and it +is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. +@end table +@end defvr -This form has @emph{lexical} scope: it has an effect on the gexps -directly defined in @var{body}@dots{}, but not on those defined, say, in -procedures called from @var{body}@dots{}. -@end deffn +Lastly, for packages that do not need anything as sophisticated, a +``trivial'' build system is provided. It is trivial in the sense that +it provides basically no support: it does not pull any implicit inputs, +and does not have a notion of build phases. -@deffn {Scheme Syntax} with-extensions @var{extensions} @var{body}@dots{} -Mark the gexps defined in @var{body}@dots{} as requiring -@var{extensions} in their build and execution environment. -@var{extensions} is typically a list of package objects such as those -defined in the @code{(gnu packages guile)} module. +@defvr {Scheme Variable} trivial-build-system +This variable is exported by @code{(guix build-system trivial)}. + +This build system requires a @code{#:builder} argument. This argument +must be a Scheme expression that builds the package output(s)---as +with @code{build-expression->derivation} (@pxref{Derivations, +@code{build-expression->derivation}}). +@end defvr + +@node The Store +@section The Store -Concretely, the packages listed in @var{extensions} are added to the -load path while compiling imported modules in @var{body}@dots{}; they -are also added to the load path of the gexp returned by -@var{body}@dots{}. -@end deffn +@cindex store +@cindex store items +@cindex store paths -@deffn {Scheme Procedure} gexp? @var{obj} -Return @code{#t} if @var{obj} is a G-expression. -@end deffn +Conceptually, the @dfn{store} is the place where derivations that have +been built successfully are stored---by default, @file{/gnu/store}. +Sub-directories in the store are referred to as @dfn{store items} or +sometimes @dfn{store paths}. The store has an associated database that +contains information such as the store paths referred to by each store +path, and the list of @emph{valid} store items---results of successful +builds. This database resides in @file{@var{localstatedir}/guix/db}, +where @var{localstatedir} is the state directory specified @i{via} +@option{--localstatedir} at configure time, usually @file{/var}. -G-expressions are meant to be written to disk, either as code building -some derivation, or as plain files in the store. The monadic procedures -below allow you to do that (@pxref{The Store Monad}, for more -information about monads.) +The store is @emph{always} accessed by the daemon on behalf of its clients +(@pxref{Invoking guix-daemon}). To manipulate the store, clients +connect to the daemon over a Unix-domain socket, send requests to it, +and read the result---these are remote procedure calls, or RPCs. -@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @ - [#:system (%current-system)] [#:target #f] [#:graft? #t] @ - [#:hash #f] [#:hash-algo #f] @ - [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ - [#:module-path @var{%load-path}] @ - [#:effective-version "2.2"] @ - [#:references-graphs #f] [#:allowed-references #f] @ - [#:disallowed-references #f] @ - [#:leaked-env-vars #f] @ - [#:script-name (string-append @var{name} "-builder")] @ - [#:deprecation-warnings #f] @ - [#:local-build? #f] [#:substitutable? #t] @ - [#:properties '()] [#:guile-for-build #f] -Return a derivation @var{name} that runs @var{exp} (a gexp) with -@var{guile-for-build} (a derivation) on @var{system}; @var{exp} is -stored in a file called @var{script-name}. When @var{target} is true, -it is used as the cross-compilation target triplet for packages referred -to by @var{exp}. +@quotation Note +Users must @emph{never} modify files under @file{/gnu/store} directly. +This would lead to inconsistencies and break the immutability +assumptions of Guix's functional model (@pxref{Introduction}). -@var{modules} is deprecated in favor of @code{with-imported-modules}. -Its meaning is to -make @var{modules} available in the evaluation context of @var{exp}; -@var{modules} is a list of names of Guile modules searched in -@var{module-path} to be copied in the store, compiled, and made available in -the load path during the execution of @var{exp}---e.g., @code{((guix -build utils) (guix build gnu-build-system))}. +@xref{Invoking guix gc, @command{guix gc --verify}}, for information on +how to check the integrity of the store and attempt recovery from +accidental modifications. +@end quotation -@var{effective-version} determines the string to use when adding extensions of -@var{exp} (see @code{with-extensions}) to the search path---e.g., @code{"2.2"}. +The @code{(guix store)} module provides procedures to connect to the +daemon, and to perform RPCs. These are described below. By default, +@code{open-connection}, and thus all the @command{guix} commands, +connect to the local daemon or to the URI specified by the +@code{GUIX_DAEMON_SOCKET} environment variable. -@var{graft?} determines whether packages referred to by @var{exp} should be grafted when -applicable. +@defvr {Environment Variable} GUIX_DAEMON_SOCKET +When set, the value of this variable should be a file name or a URI +designating the daemon endpoint. When it is a file name, it denotes a +Unix-domain socket to connect to. In addition to file names, the +supported URI schemes are: -When @var{references-graphs} is true, it must be a list of tuples of one of the -following forms: +@table @code +@item file +@itemx unix +These are for Unix-domain sockets. +@code{file:///var/guix/daemon-socket/socket} is equivalent to +@file{/var/guix/daemon-socket/socket}. + +@item guix +@cindex daemon, remote access +@cindex remote access to the daemon +@cindex daemon, cluster setup +@cindex clusters, daemon setup +These URIs denote connections over TCP/IP, without encryption nor +authentication of the remote host. The URI must specify the host name +and optionally a port number (by default port 44146 is used): @example -(@var{file-name} @var{package}) -(@var{file-name} @var{package} @var{output}) -(@var{file-name} @var{derivation}) -(@var{file-name} @var{derivation} @var{output}) -(@var{file-name} @var{store-item}) +guix://master.guix.example.org:1234 @end example -The right-hand-side of each element of @var{references-graphs} is automatically made -an input of the build process of @var{exp}. In the build environment, each -@var{file-name} contains the reference graph of the corresponding item, in a simple -text format. - -@var{allowed-references} must be either @code{#f} or a list of output names and packages. -In the latter case, the list denotes store items that the result is allowed to -refer to. Any reference to another store item will lead to a build error. -Similarly for @var{disallowed-references}, which can list items that must not be -referenced by the outputs. - -@var{deprecation-warnings} determines whether to show deprecation warnings while -compiling modules. It can be @code{#f}, @code{#t}, or @code{'detailed}. +This setup is suitable on local networks, such as clusters, where only +trusted nodes may connect to the build daemon at +@code{master.guix.example.org}. -The other arguments are as for @code{derivation} (@pxref{Derivations}). -@end deffn +The @code{--listen} option of @command{guix-daemon} can be used to +instruct it to listen for TCP connections (@pxref{Invoking guix-daemon, +@code{--listen}}). -@cindex file-like objects -The @code{local-file}, @code{plain-file}, @code{computed-file}, -@code{program-file}, and @code{scheme-file} procedures below return -@dfn{file-like objects}. That is, when unquoted in a G-expression, -these objects lead to a file in the store. Consider this G-expression: +@item ssh +@cindex SSH access to build daemons +These URIs allow you to connect to a remote daemon over +SSH@footnote{This feature requires Guile-SSH (@pxref{Requirements}).}. +A typical URL might look like this: @example -#~(system* #$(file-append glibc "/sbin/nscd") "-f" - #$(local-file "/tmp/my-nscd.conf")) +ssh://charlie@@guix.example.org:22 @end example -The effect here is to ``intern'' @file{/tmp/my-nscd.conf} by copying it -to the store. Once expanded, for instance @i{via} -@code{gexp->derivation}, the G-expression refers to that copy under -@file{/gnu/store}; thus, modifying or removing the file in @file{/tmp} -does not have any effect on what the G-expression does. -@code{plain-file} can be used similarly; it differs in that the file -content is directly passed as a string. - -@deffn {Scheme Procedure} local-file @var{file} [@var{name}] @ - [#:recursive? #f] [#:select? (const #t)] -Return an object representing local file @var{file} to add to the store; this -object can be used in a gexp. If @var{file} is a relative file name, it is looked -up relative to the source file where this form appears. @var{file} will be added to -the store under @var{name}--by default the base name of @var{file}. - -When @var{recursive?} is true, the contents of @var{file} are added recursively; if @var{file} -designates a flat file and @var{recursive?} is true, its contents are added, and its -permission bits are kept. +As for @command{guix copy}, the usual OpenSSH client configuration files +are honored (@pxref{Invoking guix copy}). +@end table -When @var{recursive?} is true, call @code{(@var{select?} @var{file} -@var{stat})} for each directory entry, where @var{file} is the entry's -absolute file name and @var{stat} is the result of @code{lstat}; exclude -entries for which @var{select?} does not return true. +Additional URI schemes may be supported in the future. -This is the declarative counterpart of the @code{interned-file} monadic -procedure (@pxref{The Store Monad, @code{interned-file}}). -@end deffn +@c XXX: Remove this note when the protocol incurs fewer round trips +@c and when (guix derivations) no longer relies on file system access. +@quotation Note +The ability to connect to remote build daemons is considered +experimental as of @value{VERSION}. Please get in touch with us to +share any problems or suggestions you may have (@pxref{Contributing}). +@end quotation +@end defvr -@deffn {Scheme Procedure} plain-file @var{name} @var{content} -Return an object representing a text file called @var{name} with the given -@var{content} (a string or a bytevector) to be added to the store. +@deffn {Scheme Procedure} open-connection [@var{uri}] [#:reserve-space? #t] +Connect to the daemon over the Unix-domain socket at @var{uri} (a string). When +@var{reserve-space?} is true, instruct it to reserve a little bit of +extra space on the file system so that the garbage collector can still +operate should the disk become full. Return a server object. -This is the declarative counterpart of @code{text-file}. +@var{file} defaults to @var{%default-socket-path}, which is the normal +location given the options that were passed to @command{configure}. @end deffn -@deffn {Scheme Procedure} computed-file @var{name} @var{gexp} @ - [#:options '(#:local-build? #t)] -Return an object representing the store item @var{name}, a file or -directory computed by @var{gexp}. @var{options} -is a list of additional arguments to pass to @code{gexp->derivation}. - -This is the declarative counterpart of @code{gexp->derivation}. +@deffn {Scheme Procedure} close-connection @var{server} +Close the connection to @var{server}. @end deffn -@deffn {Monadic Procedure} gexp->script @var{name} @var{exp} @ - [#:guile (default-guile)] [#:module-path %load-path] -Return an executable script @var{name} that runs @var{exp} using -@var{guile}, with @var{exp}'s imported modules in its search path. -Look up @var{exp}'s modules in @var{module-path}. +@defvr {Scheme Variable} current-build-output-port +This variable is bound to a SRFI-39 parameter, which refers to the port +where build and error logs sent by the daemon should be written. +@end defvr -The example below builds a script that simply invokes the @command{ls} -command: +Procedures that make RPCs all take a server object as their first +argument. -@example -(use-modules (guix gexp) (gnu packages base)) +@deffn {Scheme Procedure} valid-path? @var{server} @var{path} +@cindex invalid store items +Return @code{#t} when @var{path} designates a valid store item and +@code{#f} otherwise (an invalid item may exist on disk but still be +invalid, for instance because it is the result of an aborted or failed +build.) -(gexp->script "list-files" - #~(execl #$(file-append coreutils "/bin/ls") - "ls")) -@end example +A @code{&store-protocol-error} condition is raised if @var{path} is not +prefixed by the store directory (@file{/gnu/store}). +@end deffn -When ``running'' it through the store (@pxref{The Store Monad, -@code{run-with-store}}), we obtain a derivation that produces an -executable file @file{/gnu/store/@dots{}-list-files} along these lines: +@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}] +Add @var{text} under file @var{name} in the store, and return its store +path. @var{references} is the list of store paths referred to by the +resulting store path. +@end deffn -@example -#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds -!# -(execl "/gnu/store/@dots{}-coreutils-8.22"/bin/ls" "ls") -@end example +@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations} +Build @var{derivations} (a list of @code{} objects or +derivation paths), and return when the worker is done building them. +Return @code{#t} on success. @end deffn -@deffn {Scheme Procedure} program-file @var{name} @var{exp} @ - [#:guile #f] [#:module-path %load-path] -Return an object representing the executable store item @var{name} that -runs @var{gexp}. @var{guile} is the Guile package used to execute that -script. Imported modules of @var{gexp} are looked up in @var{module-path}. +Note that the @code{(guix monads)} module provides a monad as well as +monadic versions of the above procedures, with the goal of making it +more convenient to work with code that accesses the store (@pxref{The +Store Monad}). -This is the declarative counterpart of @code{gexp->script}. -@end deffn +@c FIXME +@i{This section is currently incomplete.} -@deffn {Monadic Procedure} gexp->file @var{name} @var{exp} @ - [#:set-load-path? #t] [#:module-path %load-path] @ - [#:splice? #f] @ - [#:guile (default-guile)] -Return a derivation that builds a file @var{name} containing @var{exp}. -When @var{splice?} is true, @var{exp} is considered to be a list of -expressions that will be spliced in the resulting file. +@node Derivations +@section Derivations -When @var{set-load-path?} is true, emit code in the resulting file to -set @code{%load-path} and @code{%load-compiled-path} to honor -@var{exp}'s imported modules. Look up @var{exp}'s modules in -@var{module-path}. +@cindex derivations +Low-level build actions and the environment in which they are performed +are represented by @dfn{derivations}. A derivation contains the +following pieces of information: -The resulting file holds references to all the dependencies of @var{exp} -or a subset thereof. -@end deffn +@itemize +@item +The outputs of the derivation---derivations produce at least one file or +directory in the store, but may produce more. -@deffn {Scheme Procedure} scheme-file @var{name} @var{exp} [#:splice? #f] -Return an object representing the Scheme file @var{name} that contains -@var{exp}. +@item +The inputs of the derivations, which may be other derivations or plain +files in the store (patches, build scripts, etc.) -This is the declarative counterpart of @code{gexp->file}. -@end deffn +@item +The system type targeted by the derivation---e.g., @code{x86_64-linux}. -@deffn {Monadic Procedure} text-file* @var{name} @var{text} @dots{} -Return as a monadic value a derivation that builds a text file -containing all of @var{text}. @var{text} may list, in addition to -strings, objects of any type that can be used in a gexp: packages, -derivations, local file objects, etc. The resulting store file holds -references to all these. +@item +The file name of a build script in the store, along with the arguments +to be passed. -This variant should be preferred over @code{text-file} anytime the file -to create will reference items from the store. This is typically the -case when building a configuration file that embeds store file names, -like this: +@item +A list of environment variables to be defined. -@example -(define (profile.sh) - ;; Return the name of a shell script in the store that - ;; initializes the 'PATH' environment variable. - (text-file* "profile.sh" - "export PATH=" coreutils "/bin:" - grep "/bin:" sed "/bin\n")) -@end example +@end itemize -In this example, the resulting @file{/gnu/store/@dots{}-profile.sh} file -will reference @var{coreutils}, @var{grep}, and @var{sed}, thereby -preventing them from being garbage-collected during its lifetime. -@end deffn +@cindex derivation path +Derivations allow clients of the daemon to communicate build actions to +the store. They exist in two forms: as an in-memory representation, +both on the client- and daemon-side, and as files in the store whose +name end in @code{.drv}---these files are referred to as @dfn{derivation +paths}. Derivations paths can be passed to the @code{build-derivations} +procedure to perform the build actions they prescribe (@pxref{The +Store}). -@deffn {Scheme Procedure} mixed-text-file @var{name} @var{text} @dots{} -Return an object representing store file @var{name} containing -@var{text}. @var{text} is a sequence of strings and file-like objects, -as in: +@cindex fixed-output derivations +Operations such as file downloads and version-control checkouts for +which the expected content hash is known in advance are modeled as +@dfn{fixed-output derivations}. Unlike regular derivations, the outputs +of a fixed-output derivation are independent of its inputs---e.g., a +source code download produces the same result regardless of the download +method and tools being used. -@example -(mixed-text-file "profile" - "export PATH=" coreutils "/bin:" grep "/bin") -@end example +The @code{(guix derivations)} module provides a representation of +derivations as Scheme objects, along with procedures to create and +otherwise manipulate derivations. The lowest-level primitive to create +a derivation is the @code{derivation} procedure: -This is the declarative counterpart of @code{text-file*}. -@end deffn +@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @ + @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ + [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @ + [#:system (%current-system)] [#:references-graphs #f] @ + [#:allowed-references #f] [#:disallowed-references #f] @ + [#:leaked-env-vars #f] [#:local-build? #f] @ + [#:substitutable? #t] [#:properties '()] +Build a derivation with the given arguments, and return the resulting +@code{} object. -@deffn {Scheme Procedure} file-union @var{name} @var{files} -Return a @code{} that builds a directory containing all of @var{files}. -Each item in @var{files} must be a two-element list where the first element is the -file name to use in the new directory, and the second element is a gexp -denoting the target file. Here's an example: +When @var{hash} and @var{hash-algo} are given, a +@dfn{fixed-output derivation} is created---i.e., one whose result is +known in advance, such as a file download. If, in addition, +@var{recursive?} is true, then that fixed output may be an executable +file or a directory and @var{hash} must be the hash of an archive +containing this output. -@example -(file-union "etc" - `(("hosts" ,(plain-file "hosts" - "127.0.0.1 localhost")) - ("bashrc" ,(plain-file "bashrc" - "alias ls='ls --color=auto'")))) -@end example +When @var{references-graphs} is true, it must be a list of file +name/store path pairs. In that case, the reference graph of each store +path is exported in the build environment in the corresponding file, in +a simple text format. -This yields an @code{etc} directory containing these two files. -@end deffn +When @var{allowed-references} is true, it must be a list of store items +or outputs that the derivation's output may refer to. Likewise, +@var{disallowed-references}, if true, must be a list of things the +outputs may @emph{not} refer to. -@deffn {Scheme Procedure} directory-union @var{name} @var{things} -Return a directory that is the union of @var{things}, where @var{things} is a list of -file-like objects denoting directories. For example: +When @var{leaked-env-vars} is true, it must be a list of strings +denoting environment variables that are allowed to ``leak'' from the +daemon's environment to the build environment. This is only applicable +to fixed-output derivations---i.e., when @var{hash} is true. The main +use is to allow variables such as @code{http_proxy} to be passed to +derivations that download files. -@example -(directory-union "guile+emacs" (list guile emacs)) -@end example +When @var{local-build?} is true, declare that the derivation is not a +good candidate for offloading and should rather be built locally +(@pxref{Daemon Offload Setup}). This is the case for small derivations +where the costs of data transfers would outweigh the benefits. -yields a directory that is the union of the @code{guile} and @code{emacs} packages. -@end deffn +When @var{substitutable?} is false, declare that substitutes of the +derivation's output should not be used (@pxref{Substitutes}). This is +useful, for instance, when building packages that capture details of the +host CPU instruction set. -@deffn {Scheme Procedure} file-append @var{obj} @var{suffix} @dots{} -Return a file-like object that expands to the concatenation of @var{obj} -and @var{suffix}, where @var{obj} is a lowerable object and each -@var{suffix} is a string. +@var{properties} must be an association list describing ``properties'' of the +derivation. It is kept as-is, uninterpreted, in the derivation. +@end deffn -As an example, consider this gexp: +@noindent +Here's an example with a shell script as its builder, assuming +@var{store} is an open connection to the daemon, and @var{bash} points +to a Bash executable in the store: -@example -(gexp->script "run-uname" - #~(system* #$(file-append coreutils - "/bin/uname"))) -@end example +@lisp +(use-modules (guix utils) + (guix store) + (guix derivations)) -The same effect could be achieved with: +(let ((builder ; add the Bash script to the store + (add-text-to-store store "my-builder.sh" + "echo hello world > $out\n" '()))) + (derivation store "foo" + bash `("-e" ,builder) + #:inputs `((,bash) (,builder)) + #:env-vars '(("HOME" . "/homeless")))) +@result{} # /gnu/store/@dots{}-foo> +@end lisp -@example -(gexp->script "run-uname" - #~(system* (string-append #$coreutils - "/bin/uname"))) -@end example +As can be guessed, this primitive is cumbersome to use directly. A +better approach is to write build scripts in Scheme, of course! The +best course of action for that is to write the build code as a +``G-expression'', and to pass it to @code{gexp->derivation}. For more +information, @pxref{G-Expressions}. -There is one difference though: in the @code{file-append} case, the -resulting script contains the absolute file name as a string, whereas in -the second case, the resulting script contains a @code{(string-append -@dots{})} expression to construct the file name @emph{at run time}. -@end deffn +Once upon a time, @code{gexp->derivation} did not exist and constructing +derivations with build code written in Scheme was achieved with +@code{build-expression->derivation}, documented below. This procedure +is now deprecated in favor of the much nicer @code{gexp->derivation}. +@deffn {Scheme Procedure} build-expression->derivation @var{store} @ + @var{name} @var{exp} @ + [#:system (%current-system)] [#:inputs '()] @ + [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ + [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ + [#:references-graphs #f] [#:allowed-references #f] @ + [#:disallowed-references #f] @ + [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f] +Return a derivation that executes Scheme expression @var{exp} as a +builder for derivation @var{name}. @var{inputs} must be a list of +@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted, +@code{"out"} is assumed. @var{modules} is a list of names of Guile +modules from the current search path to be copied in the store, +compiled, and made available in the load path during the execution of +@var{exp}---e.g., @code{((guix build utils) (guix build +gnu-build-system))}. -Of course, in addition to gexps embedded in ``host'' code, there are -also modules containing build tools. To make it clear that they are -meant to be used in the build stratum, these modules are kept in the -@code{(guix build @dots{})} name space. +@var{exp} is evaluated in an environment where @code{%outputs} is bound +to a list of output/path pairs, and where @code{%build-inputs} is bound +to a list of string/output-path pairs made from @var{inputs}. +Optionally, @var{env-vars} is a list of string pairs specifying the name +and value of environment variables visible to the builder. The builder +terminates by passing the result of @var{exp} to @code{exit}; thus, when +@var{exp} returns @code{#f}, the build is considered to have failed. -@cindex lowering, of high-level objects in gexps -Internally, high-level objects are @dfn{lowered}, using their compiler, -to either derivations or store items. For instance, lowering a package -yields a derivation, and lowering a @code{plain-file} yields a store -item. This is achieved using the @code{lower-object} monadic procedure. +@var{exp} is built using @var{guile-for-build} (a derivation). When +@var{guile-for-build} is omitted or is @code{#f}, the value of the +@code{%guile-for-build} fluid is used instead. -@deffn {Monadic Procedure} lower-object @var{obj} [@var{system}] @ - [#:target #f] -Return as a value in @var{%store-monad} the derivation or store item -corresponding to @var{obj} for @var{system}, cross-compiling for -@var{target} if @var{target} is true. @var{obj} must be an object that -has an associated gexp compiler, such as a @code{}. +See the @code{derivation} procedure for the meaning of +@var{references-graphs}, @var{allowed-references}, +@var{disallowed-references}, @var{local-build?}, and +@var{substitutable?}. @end deffn -@node Invoking guix repl -@section Invoking @command{guix repl} - -@cindex REPL, read-eval-print loop -The @command{guix repl} command spawns a Guile @dfn{read-eval-print loop} -(REPL) for interactive programming (@pxref{Using Guile Interactively,,, guile, -GNU Guile Reference Manual}). Compared to just launching the @command{guile} -command, @command{guix repl} guarantees that all the Guix modules and all its -dependencies are available in the search path. You can use it this way: +@noindent +Here's an example of a single-output derivation that creates a directory +containing one file: -@example -$ guix repl -scheme@@(guile-user)> ,use (gnu packages base) -scheme@@(guile-user)> coreutils -$1 = # -@end example +@lisp +(let ((builder '(let ((out (assoc-ref %outputs "out"))) + (mkdir out) ; create /gnu/store/@dots{}-goo + (call-with-output-file (string-append out "/test") + (lambda (p) + (display '(hello guix) p)))))) + (build-expression->derivation store "goo" builder)) -@cindex inferiors -In addition, @command{guix repl} implements a simple machine-readable REPL -protocol for use by @code{(guix inferior)}, a facility to interact with -@dfn{inferiors}, separate processes running a potentially different revision -of Guix. +@result{} # @dots{}> +@end lisp -The available options are as follows: -@table @code -@item --type=@var{type} -@itemx -t @var{type} -Start a REPL of the given @var{TYPE}, which can be one of the following: +@node The Store Monad +@section The Store Monad -@table @code -@item guile -This is default, and it spawns a standard full-featured Guile REPL. -@item machine -Spawn a REPL that uses the machine-readable protocol. This is the protocol -that the @code{(guix inferior)} module speaks. -@end table +@cindex monad -@item --listen=@var{endpoint} -By default, @command{guix repl} reads from standard input and writes to -standard output. When this option is passed, it will instead listen for -connections on @var{endpoint}. Here are examples of valid options: +The procedures that operate on the store described in the previous +sections all take an open connection to the build daemon as their first +argument. Although the underlying model is functional, they either have +side effects or depend on the current state of the store. -@table @code -@item --listen=tcp:37146 -Accept connections on localhost on port 37146. +The former is inconvenient: the connection to the build daemon has to be +carried around in all those functions, making it impossible to compose +functions that do not take that parameter with functions that do. The +latter can be problematic: since store operations have side effects +and/or depend on external state, they have to be properly sequenced. -@item --listen=unix:/tmp/socket -Accept connections on the Unix-domain socket @file{/tmp/socket}. -@end table -@end table +@cindex monadic values +@cindex monadic functions +This is where the @code{(guix monads)} module comes in. This module +provides a framework for working with @dfn{monads}, and a particularly +useful monad for our uses, the @dfn{store monad}. Monads are a +construct that allows two things: associating ``context'' with values +(in our case, the context is the store), and building sequences of +computations (here computations include accesses to the store). Values +in a monad---values that carry this additional context---are called +@dfn{monadic values}; procedures that return such values are called +@dfn{monadic procedures}. -@c ********************************************************************* -@node Utilities -@chapter Utilities +Consider this ``normal'' procedure: -This section describes Guix command-line utilities. Some of them are -primarily targeted at developers and users who write new package -definitions, while others are more generally useful. They complement -the Scheme programming interface of Guix in a convenient way. +@example +(define (sh-symlink store) + ;; Return a derivation that symlinks the 'bash' executable. + (let* ((drv (package-derivation store bash)) + (out (derivation->output-path drv)) + (sh (string-append out "/bin/bash"))) + (build-expression->derivation store "sh" + `(symlink ,sh %output)))) +@end example -@menu -* Invoking guix build:: Building packages from the command line. -* Invoking guix edit:: Editing package definitions. -* Invoking guix download:: Downloading a file and printing its hash. -* Invoking guix hash:: Computing the cryptographic hash of a file. -* Invoking guix import:: Importing package definitions. -* Invoking guix refresh:: Updating package definitions. -* Invoking guix lint:: Finding errors in package definitions. -* Invoking guix size:: Profiling disk usage. -* Invoking guix graph:: Visualizing the graph of packages. -* Invoking guix environment:: Setting up development environments. -* Invoking guix publish:: Sharing substitutes. -* Invoking guix challenge:: Challenging substitute servers. -* Invoking guix copy:: Copying to and from a remote store. -* Invoking guix container:: Process isolation. -* Invoking guix weather:: Assessing substitute availability. -* Invoking guix processes:: Listing client processes. -@end menu +Using @code{(guix monads)} and @code{(guix gexp)}, it may be rewritten +as a monadic function: -@node Invoking guix build -@section Invoking @command{guix build} +@example +(define (sh-symlink) + ;; Same, but return a monadic value. + (mlet %store-monad ((drv (package->derivation bash))) + (gexp->derivation "sh" + #~(symlink (string-append #$drv "/bin/bash") + #$output)))) +@end example -@cindex package building -@cindex @command{guix build} -The @command{guix build} command builds packages or derivations and -their dependencies, and prints the resulting store paths. Note that it -does not modify the user's profile---this is the job of the -@command{guix package} command (@pxref{Invoking guix package}). Thus, -it is mainly useful for distribution developers. +There are several things to note in the second version: the @code{store} +parameter is now implicit and is ``threaded'' in the calls to the +@code{package->derivation} and @code{gexp->derivation} monadic +procedures, and the monadic value returned by @code{package->derivation} +is @dfn{bound} using @code{mlet} instead of plain @code{let}. -The general syntax is: +As it turns out, the call to @code{package->derivation} can even be +omitted since it will take place implicitly, as we will see later +(@pxref{G-Expressions}): @example -guix build @var{options} @var{package-or-derivation}@dots{} +(define (sh-symlink) + (gexp->derivation "sh" + #~(symlink (string-append #$bash "/bin/bash") + #$output))) @end example -As an example, the following command builds the latest versions of Emacs -and of Guile, displays their build logs, and finally displays the -resulting directories: +@c See +@c +@c for the funny quote. +Calling the monadic @code{sh-symlink} has no effect. As someone once +said, ``you exit a monad like you exit a building on fire: by running''. +So, to exit the monad and get the desired effect, one must use +@code{run-with-store}: @example -guix build emacs guile +(run-with-store (open-connection) (sh-symlink)) +@result{} /gnu/store/...-sh-symlink @end example -Similarly, the following command builds all the available packages: +Note that the @code{(guix monad-repl)} module extends the Guile REPL with +new ``meta-commands'' to make it easier to deal with monadic procedures: +@code{run-in-store}, and @code{enter-store-monad}. The former is used +to ``run'' a single monadic value through the store: @example -guix build --quiet --keep-going \ - `guix package -A | cut -f1,2 --output-delimiter=@@` +scheme@@(guile-user)> ,run-in-store (package->derivation hello) +$1 = # @dots{}> @end example -@var{package-or-derivation} may be either the name of a package found in -the software distribution such as @code{coreutils} or -@code{coreutils@@8.20}, or a derivation such as -@file{/gnu/store/@dots{}-coreutils-8.19.drv}. In the former case, a -package with the corresponding name (and optionally version) is searched -for among the GNU distribution modules (@pxref{Package Modules}). +The latter enters a recursive REPL, where all the return values are +automatically run through the store: -Alternatively, the @code{--expression} option may be used to specify a -Scheme expression that evaluates to a package; this is useful when -disambiguating among several same-named packages or package variants is -needed. +@example +scheme@@(guile-user)> ,enter-store-monad +store-monad@@(guile-user) [1]> (package->derivation hello) +$2 = # @dots{}> +store-monad@@(guile-user) [1]> (text-file "foo" "Hello!") +$3 = "/gnu/store/@dots{}-foo" +store-monad@@(guile-user) [1]> ,q +scheme@@(guile-user)> +@end example -There may be zero or more @var{options}. The available options are -described in the subsections below. +@noindent +Note that non-monadic values cannot be returned in the +@code{store-monad} REPL. -@menu -* Common Build Options:: Build options for most commands. -* Package Transformation Options:: Creating variants of packages. -* Additional Build Options:: Options specific to 'guix build'. -* Debugging Build Failures:: Real life packaging experience. -@end menu +The main syntactic forms to deal with monads in general are provided by +the @code{(guix monads)} module and are described below. -@node Common Build Options -@subsection Common Build Options +@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ... +Evaluate any @code{>>=} or @code{return} forms in @var{body} as being +in @var{monad}. +@end deffn -A number of options that control the build process are common to -@command{guix build} and other commands that can spawn builds, such as -@command{guix package} or @command{guix archive}. These are the -following: +@deffn {Scheme Syntax} return @var{val} +Return a monadic value that encapsulates @var{val}. +@end deffn -@table @code +@deffn {Scheme Syntax} >>= @var{mval} @var{mproc} ... +@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic +procedures @var{mproc}@dots{}@footnote{This operation is commonly +referred to as ``bind'', but that name denotes an unrelated procedure in +Guile. Thus we use this somewhat cryptic symbol inherited from the +Haskell language.}. There can be one @var{mproc} or several of them, as +in this example: -@item --load-path=@var{directory} -@itemx -L @var{directory} -Add @var{directory} to the front of the package module search path -(@pxref{Package Modules}). +@example +(run-with-state + (with-monad %state-monad + (>>= (return 1) + (lambda (x) (return (+ 1 x))) + (lambda (x) (return (* 2 x))))) + 'some-state) -This allows users to define their own packages and make them visible to -the command-line tools. +@result{} 4 +@result{} some-state +@end example +@end deffn -@item --keep-failed -@itemx -K -Keep the build tree of failed builds. Thus, if a build fails, its build -tree is kept under @file{/tmp}, in a directory whose name is shown at -the end of the build log. This is useful when debugging build issues. -@xref{Debugging Build Failures}, for tips and tricks on how to debug -build issues. +@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @ + @var{body} ... +@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @ + @var{body} ... +Bind the variables @var{var} to the monadic values @var{mval} in +@var{body}, which is a sequence of expressions. As with the bind +operator, this can be thought of as ``unpacking'' the raw, non-monadic +value ``contained'' in @var{mval} and making @var{var} refer to that +raw, non-monadic value within the scope of the @var{body}. The form +(@var{var} -> @var{val}) binds @var{var} to the ``normal'' value +@var{val}, as per @code{let}. The binding operations occur in sequence +from left to right. The last expression of @var{body} must be a monadic +expression, and its result will become the result of the @code{mlet} or +@code{mlet*} when run in the @var{monad}. -This option has no effect when connecting to a remote daemon with a -@code{guix://} URI (@pxref{The Store, the @code{GUIX_DAEMON_SOCKET} -variable}). +@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let} +(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}). +@end deffn -@item --keep-going -@itemx -k -Keep going when some of the derivations fail to build; return only once -all the builds have either completed or failed. +@deffn {Scheme System} mbegin @var{monad} @var{mexp} ... +Bind @var{mexp} and the following monadic expressions in sequence, +returning the result of the last expression. Every expression in the +sequence must be a monadic expression. -The default behavior is to stop as soon as one of the specified -derivations has failed. +This is akin to @code{mlet}, except that the return values of the +monadic expressions are ignored. In that sense, it is analogous to +@code{begin}, but applied to monadic expressions. +@end deffn -@item --dry-run -@itemx -n -Do not build the derivations. +@deffn {Scheme System} mwhen @var{condition} @var{mexp0} @var{mexp*} ... +When @var{condition} is true, evaluate the sequence of monadic +expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When +@var{condition} is false, return @code{*unspecified*} in the current +monad. Every expression in the sequence must be a monadic expression. +@end deffn -@anchor{fallback-option} -@item --fallback -When substituting a pre-built binary fails, fall back to building -packages locally (@pxref{Substitution Failure}). +@deffn {Scheme System} munless @var{condition} @var{mexp0} @var{mexp*} ... +When @var{condition} is false, evaluate the sequence of monadic +expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When +@var{condition} is true, return @code{*unspecified*} in the current +monad. Every expression in the sequence must be a monadic expression. +@end deffn -@item --substitute-urls=@var{urls} -@anchor{client-substitute-urls} -Consider @var{urls} the whitespace-separated list of substitute source -URLs, overriding the default list of URLs of @command{guix-daemon} -(@pxref{daemon-substitute-urls,, @command{guix-daemon} URLs}). +@cindex state monad +The @code{(guix monads)} module provides the @dfn{state monad}, which +allows an additional value---the state---to be @emph{threaded} through +monadic procedure calls. -This means that substitutes may be downloaded from @var{urls}, provided -they are signed by a key authorized by the system administrator -(@pxref{Substitutes}). +@defvr {Scheme Variable} %state-monad +The state monad. Procedures in the state monad can access and change +the state that is threaded. -When @var{urls} is the empty string, substitutes are effectively -disabled. +Consider the example below. The @code{square} procedure returns a value +in the state monad. It returns the square of its argument, but also +increments the current state value: -@item --no-substitutes -Do not use substitutes for build products. That is, always build things -locally instead of allowing downloads of pre-built binaries -(@pxref{Substitutes}). +@example +(define (square x) + (mlet %state-monad ((count (current-state))) + (mbegin %state-monad + (set-current-state (+ 1 count)) + (return (* x x))))) -@item --no-grafts -Do not ``graft'' packages. In practice, this means that package updates -available as grafts are not applied. @xref{Security Updates}, for more -information on grafts. +(run-with-state (sequence %state-monad (map square (iota 3))) 0) +@result{} (0 1 4) +@result{} 3 +@end example -@item --rounds=@var{n} -Build each derivation @var{n} times in a row, and raise an error if -consecutive build results are not bit-for-bit identical. +When ``run'' through @var{%state-monad}, we obtain that additional state +value, which is the number of @code{square} calls. +@end defvr -This is a useful way to detect non-deterministic builds processes. -Non-deterministic build processes are a problem because they make it -practically impossible for users to @emph{verify} whether third-party -binaries are genuine. @xref{Invoking guix challenge}, for more. +@deffn {Monadic Procedure} current-state +Return the current state as a monadic value. +@end deffn -Note that, currently, the differing build results are not kept around, -so you will have to manually investigate in case of an error---e.g., by -stashing one of the build results with @code{guix archive --export} -(@pxref{Invoking guix archive}), then rebuilding, and finally comparing -the two results. +@deffn {Monadic Procedure} set-current-state @var{value} +Set the current state to @var{value} and return the previous state as a +monadic value. +@end deffn -@item --no-build-hook -Do not attempt to offload builds @i{via} the ``build hook'' of the daemon -(@pxref{Daemon Offload Setup}). That is, always build things locally -instead of offloading builds to remote machines. +@deffn {Monadic Procedure} state-push @var{value} +Push @var{value} to the current state, which is assumed to be a list, +and return the previous state as a monadic value. +@end deffn -@item --max-silent-time=@var{seconds} -When the build or substitution process remains silent for more than -@var{seconds}, terminate it and report a build failure. +@deffn {Monadic Procedure} state-pop +Pop a value from the current state and return it as a monadic value. +The state is assumed to be a list. +@end deffn -By default, the daemon's setting is honored (@pxref{Invoking -guix-daemon, @code{--max-silent-time}}). +@deffn {Scheme Procedure} run-with-state @var{mval} [@var{state}] +Run monadic value @var{mval} starting with @var{state} as the initial +state. Return two values: the resulting value, and the resulting state. +@end deffn -@item --timeout=@var{seconds} -Likewise, when the build or substitution process lasts for more than -@var{seconds}, terminate it and report a build failure. +The main interface to the store monad, provided by the @code{(guix +store)} module, is as follows. -By default, the daemon's setting is honored (@pxref{Invoking -guix-daemon, @code{--timeout}}). +@defvr {Scheme Variable} %store-monad +The store monad---an alias for @var{%state-monad}. -@c Note: This option is actually not part of %standard-build-options but -@c most programs honor it. -@cindex verbosity, of the command-line tools -@cindex build logs, verbosity -@item -v @var{level} -@itemx --verbosity=@var{level} -Use the given verbosity @var{level}, an integer. Choosing 0 means that no -output is produced, 1 is for quiet output, and 2 shows all the build log -output on standard error. +Values in the store monad encapsulate accesses to the store. When its +effect is needed, a value of the store monad must be ``evaluated'' by +passing it to the @code{run-with-store} procedure (see below.) +@end defvr -@item --cores=@var{n} -@itemx -c @var{n} -Allow the use of up to @var{n} CPU cores for the build. The special -value @code{0} means to use as many CPU cores as available. +@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)] +Run @var{mval}, a monadic value in the store monad, in @var{store}, an +open store connection. +@end deffn -@item --max-jobs=@var{n} -@itemx -M @var{n} -Allow at most @var{n} build jobs in parallel. @xref{Invoking -guix-daemon, @code{--max-jobs}}, for details about this option and the -equivalent @command{guix-daemon} option. +@deffn {Monadic Procedure} text-file @var{name} @var{text} [@var{references}] +Return as a monadic value the absolute file name in the store of the file +containing @var{text}, a string. @var{references} is a list of store items that the +resulting text file refers to; it defaults to the empty list. +@end deffn -@item --debug=@var{level} -Produce debugging output coming from the build daemon. @var{level} must be an -integer between 0 and 5; higher means more verbose output. Setting a level of -4 or more may be helpful when debugging setup issues with the build daemon. +@deffn {Monadic Procedure} binary-file @var{name} @var{data} [@var{references}] +Return as a monadic value the absolute file name in the store of the file +containing @var{data}, a bytevector. @var{references} is a list of store +items that the resulting binary file refers to; it defaults to the empty list. +@end deffn -@end table +@deffn {Monadic Procedure} interned-file @var{file} [@var{name}] @ + [#:recursive? #t] [#:select? (const #t)] +Return the name of @var{file} once interned in the store. Use +@var{name} as its store name, or the basename of @var{file} if +@var{name} is omitted. -Behind the scenes, @command{guix build} is essentially an interface to -the @code{package-derivation} procedure of the @code{(guix packages)} -module, and to the @code{build-derivations} procedure of the @code{(guix -derivations)} module. +When @var{recursive?} is true, the contents of @var{file} are added +recursively; if @var{file} designates a flat file and @var{recursive?} +is true, its contents are added, and its permission bits are kept. -In addition to options explicitly passed on the command line, -@command{guix build} and other @command{guix} commands that support -building honor the @code{GUIX_BUILD_OPTIONS} environment variable. +When @var{recursive?} is true, call @code{(@var{select?} @var{file} +@var{stat})} for each directory entry, where @var{file} is the entry's +absolute file name and @var{stat} is the result of @code{lstat}; exclude +entries for which @var{select?} does not return true. -@defvr {Environment Variable} GUIX_BUILD_OPTIONS -Users can define this variable to a list of command line options that -will automatically be used by @command{guix build} and other -@command{guix} commands that can perform builds, as in the example -below: +The example below adds a file to the store, under two different names: @example -$ export GUIX_BUILD_OPTIONS="--no-substitutes -c 2 -L /foo/bar" -@end example +(run-with-store (open-connection) + (mlet %store-monad ((a (interned-file "README")) + (b (interned-file "README" "LEGU-MIN"))) + (return (list a b)))) -These options are parsed independently, and the result is appended to -the parsed command-line options. -@end defvr +@result{} ("/gnu/store/rwm@dots{}-README" "/gnu/store/44i@dots{}-LEGU-MIN") +@end example +@end deffn -@node Package Transformation Options -@subsection Package Transformation Options +The @code{(guix packages)} module exports the following package-related +monadic procedures: -@cindex package variants -Another set of command-line options supported by @command{guix build} -and also @command{guix package} are @dfn{package transformation -options}. These are options that make it possible to define @dfn{package -variants}---for instance, packages built from different source code. -This is a convenient way to create customized packages on the fly -without having to type in the definitions of package variants -(@pxref{Defining Packages}). +@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @ + [#:system (%current-system)] [#:target #f] @ + [#:output "out"] +Return as a monadic +value in the absolute file name of @var{file} within the @var{output} +directory of @var{package}. When @var{file} is omitted, return the name +of the @var{output} directory of @var{package}. When @var{target} is +true, use it as a cross-compilation target triplet. +@end deffn -@table @code +@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}] +@deffnx {Monadic Procedure} package->cross-derivation @var{package} @ + @var{target} [@var{system}] +Monadic version of @code{package-derivation} and +@code{package-cross-derivation} (@pxref{Defining Packages}). +@end deffn -@item --with-source=@var{source} -@itemx --with-source=@var{package}=@var{source} -@itemx --with-source=@var{package}@@@var{version}=@var{source} -Use @var{source} as the source of @var{package}, and @var{version} as -its version number. -@var{source} must be a file name or a URL, as for @command{guix -download} (@pxref{Invoking guix download}). -When @var{package} is omitted, -it is taken to be the package name specified on the -command line that matches the base of @var{source}---e.g., -if @var{source} is @code{/src/guile-2.0.10.tar.gz}, the corresponding -package is @code{guile}. +@node G-Expressions +@section G-Expressions -Likewise, when @var{version} is omitted, the version string is inferred from -@var{source}; in the previous example, it is @code{2.0.10}. +@cindex G-expression +@cindex build code quoting +So we have ``derivations'', which represent a sequence of build actions +to be performed to produce an item in the store (@pxref{Derivations}). +These build actions are performed when asking the daemon to actually +build the derivations; they are run by the daemon in a container +(@pxref{Invoking guix-daemon}). -This option allows users to try out versions of packages other than the -one provided by the distribution. The example below downloads -@file{ed-1.7.tar.gz} from a GNU mirror and uses that as the source for -the @code{ed} package: +@cindex strata of code +It should come as no surprise that we like to write these build actions +in Scheme. When we do that, we end up with two @dfn{strata} of Scheme +code@footnote{The term @dfn{stratum} in this context was coined by +Manuel Serrano et al.@: in the context of their work on Hop. Oleg +Kiselyov, who has written insightful +@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code +on this topic}, refers to this kind of code generation as +@dfn{staging}.}: the ``host code''---code that defines packages, talks +to the daemon, etc.---and the ``build code''---code that actually +performs build actions, such as making directories, invoking +@command{make}, etc. -@example -guix build ed --with-source=mirror://gnu/ed/ed-1.7.tar.gz -@end example +To describe a derivation and its build actions, one typically needs to +embed build code inside host code. It boils down to manipulating build +code as data, and the homoiconicity of Scheme---code has a direct +representation as data---comes in handy for that. But we need more than +the normal @code{quasiquote} mechanism in Scheme to construct build +expressions. -As a developer, @code{--with-source} makes it easy to test release -candidates: +The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of +S-expressions adapted to build expressions. G-expressions, or +@dfn{gexps}, consist essentially of three syntactic forms: @code{gexp}, +@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~}, +@code{#$}, and @code{#$@@}), which are comparable to +@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}, +respectively (@pxref{Expression Syntax, @code{quasiquote},, guile, +GNU Guile Reference Manual}). However, there are major differences: -@example -guix build guile --with-source=../guile-2.0.9.219-e1bb7.tar.xz -@end example +@itemize +@item +Gexps are meant to be written to a file and run or manipulated by other +processes. -@dots{} or to build from a checkout in a pristine environment: +@item +When a high-level object such as a package or derivation is unquoted +inside a gexp, the result is as if its output file name had been +introduced. -@example -$ git clone git://git.sv.gnu.org/guix.git -$ guix build guix --with-source=guix@@1.0=./guix -@end example +@item +Gexps carry information about the packages or derivations they refer to, +and these dependencies are automatically added as inputs to the build +processes that use them. +@end itemize -@item --with-input=@var{package}=@var{replacement} -Replace dependency on @var{package} by a dependency on -@var{replacement}. @var{package} must be a package name, and -@var{replacement} must be a package specification such as @code{guile} -or @code{guile@@1.8}. +@cindex lowering, of high-level objects in gexps +This mechanism is not limited to package and derivation +objects: @dfn{compilers} able to ``lower'' other high-level objects to +derivations or files in the store can be defined, +such that these objects can also be inserted +into gexps. For example, a useful type of high-level objects that can be +inserted in a gexp is ``file-like objects'', which make it easy to +add files to the store and to refer to them in +derivations and such (see @code{local-file} and @code{plain-file} +below.) -For instance, the following command builds Guix, but replaces its -dependency on the current stable version of Guile with a dependency on -the legacy version of Guile, @code{guile@@2.0}: +To illustrate the idea, here is an example of a gexp: @example -guix build --with-input=guile=guile@@2.0 guix +(define build-exp + #~(begin + (mkdir #$output) + (chdir #$output) + (symlink (string-append #$coreutils "/bin/ls") + "list-files"))) @end example -This is a recursive, deep replacement. So in this example, both -@code{guix} and its dependency @code{guile-json} (which also depends on -@code{guile}) get rebuilt against @code{guile@@2.0}. - -This is implemented using the @code{package-input-rewriting} Scheme -procedure (@pxref{Defining Packages, @code{package-input-rewriting}}). - -@item --with-graft=@var{package}=@var{replacement} -This is similar to @code{--with-input} but with an important difference: -instead of rebuilding the whole dependency chain, @var{replacement} is -built and then @dfn{grafted} onto the binaries that were initially -referring to @var{package}. @xref{Security Updates}, for more -information on grafts. - -For example, the command below grafts version 3.5.4 of GnuTLS onto Wget -and all its dependencies, replacing references to the version of GnuTLS -they currently refer to: +This gexp can be passed to @code{gexp->derivation}; we obtain a +derivation that builds a directory containing exactly one symlink to +@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}: @example -guix build --with-graft=gnutls=gnutls@@3.5.4 wget +(gexp->derivation "the-thing" build-exp) @end example -This has the advantage of being much faster than rebuilding everything. -But there is a caveat: it works if and only if @var{package} and -@var{replacement} are strictly compatible---for example, if they provide -a library, the application binary interface (ABI) of those libraries -must be compatible. If @var{replacement} is somehow incompatible with -@var{package}, then the resulting package may be unusable. Use with -care! - -@item --with-branch=@var{package}=@var{branch} -@cindex Git, using the latest commit -@cindex latest commit, building -Build @var{package} from the latest commit of @var{branch}. The @code{source} -field of @var{package} must be an origin with the @code{git-fetch} method -(@pxref{origin Reference}) or a @code{git-checkout} object; the repository URL -is taken from that @code{source}. +As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is +substituted to the reference to the @var{coreutils} package in the +actual build code, and @var{coreutils} is automatically made an input to +the derivation. Likewise, @code{#$output} (equivalent to @code{(ungexp +output)}) is replaced by a string containing the directory name of the +output of the derivation. -For instance, the following command builds @code{guile-sqlite3} from the -latest commit of its @code{master} branch, and then builds @code{guix} (which -depends on it) and @code{cuirass} (which depends on @code{guix}) against this -specific @code{guile-sqlite3} build: +@cindex cross compilation +In a cross-compilation context, it is useful to distinguish between +references to the @emph{native} build of a package---that can run on the +host---versus references to cross builds of a package. To that end, the +@code{#+} plays the same role as @code{#$}, but is a reference to a +native package build: @example -guix build --with-branch=guile-sqlite3=master cuirass +(gexp->derivation "vi" + #~(begin + (mkdir #$output) + (system* (string-append #+coreutils "/bin/ln") + "-s" + (string-append #$emacs "/bin/emacs") + (string-append #$output "/bin/vi"))) + #:target "mips64el-linux-gnu") @end example -@cindex continuous integration -Obviously, since it uses the latest commit of the given branch, the result of -such a command varies over time. Nevertheless it is a convenient way to -rebuild entire software stacks against the latest commit of one or more -packages. This is particularly useful in the context of continuous -integration (CI). - -Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up -consecutive accesses to the same repository. You may want to clean it up once -in a while to save disk space. - -@item --with-commit=@var{package}=@var{commit} -This is similar to @code{--with-branch}, except that it builds from -@var{commit} rather than the tip of a branch. @var{commit} must be a valid -Git commit SHA1 identifier. -@end table - -@node Additional Build Options -@subsection Additional Build Options - -The command-line options presented below are specific to @command{guix -build}. - -@table @code - -@item --quiet -@itemx -q -Build quietly, without displaying the build log; this is equivalent to -@code{--verbosity=0}. Upon completion, the build log is kept in @file{/var} -(or similar) and can always be retrieved using the @option{--log-file} option. - -@item --file=@var{file} -@itemx -f @var{file} -Build the package, derivation, or other file-like object that the code within -@var{file} evaluates to (@pxref{G-Expressions, file-like objects}). +@noindent +In the example above, the native build of @var{coreutils} is used, so +that @command{ln} can actually run on the host; but then the +cross-compiled build of @var{emacs} is referenced. -As an example, @var{file} might contain a package definition like this -(@pxref{Defining Packages}): +@cindex imported modules, for gexps +@findex with-imported-modules +Another gexp feature is @dfn{imported modules}: sometimes you want to be +able to use certain Guile modules from the ``host environment'' in the +gexp, so those modules should be imported in the ``build environment''. +The @code{with-imported-modules} form allows you to express that: @example -@verbatiminclude package-hello.scm +(let ((build (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils)) + (mkdir-p (string-append #$output "/bin")))))) + (gexp->derivation "empty-dir" + #~(begin + #$build + (display "success!\n") + #t))) @end example -@item --expression=@var{expr} -@itemx -e @var{expr} -Build the package or derivation @var{expr} evaluates to. +@noindent +In this example, the @code{(guix build utils)} module is automatically +pulled into the isolated build environment of our gexp, such that +@code{(use-modules (guix build utils))} works as expected. -For example, @var{expr} may be @code{(@@ (gnu packages guile) -guile-1.8)}, which unambiguously designates this specific variant of -version 1.8 of Guile. +@cindex module closure +@findex source-module-closure +Usually you want the @emph{closure} of the module to be imported---i.e., +the module itself and all the modules it depends on---rather than just +the module; failing to do that, attempts to use the module will fail +because of missing dependent modules. The @code{source-module-closure} +procedure computes the closure of a module by looking at its source file +headers, which comes in handy in this case: -Alternatively, @var{expr} may be a G-expression, in which case it is used -as a build program passed to @code{gexp->derivation} -(@pxref{G-Expressions}). +@example +(use-modules (guix modules)) ;for 'source-module-closure' -Lastly, @var{expr} may refer to a zero-argument monadic procedure -(@pxref{The Store Monad}). The procedure must return a derivation as a -monadic value, which is then passed through @code{run-with-store}. +(with-imported-modules (source-module-closure + '((guix build utils) + (gnu build vm))) + (gexp->derivation "something-with-vms" + #~(begin + (use-modules (guix build utils) + (gnu build vm)) + @dots{}))) +@end example -@item --source -@itemx -S -Build the source derivations of the packages, rather than the packages -themselves. +@cindex extensions, for gexps +@findex with-extensions +In the same vein, sometimes you want to import not just pure-Scheme +modules, but also ``extensions'' such as Guile bindings to C libraries +or other ``full-blown'' packages. Say you need the @code{guile-json} +package available on the build side, here's how you would do it: -For instance, @code{guix build -S gcc} returns something like -@file{/gnu/store/@dots{}-gcc-4.7.2.tar.bz2}, which is the GCC -source tarball. +@example +(use-modules (gnu packages guile)) ;for 'guile-json' -The returned source tarball is the result of applying any patches and -code snippets specified in the package @code{origin} (@pxref{Defining -Packages}). +(with-extensions (list guile-json) + (gexp->derivation "something-with-json" + #~(begin + (use-modules (json)) + @dots{}))) +@end example -@item --sources -Fetch and return the source of @var{package-or-derivation} and all their -dependencies, recursively. This is a handy way to obtain a local copy -of all the source code needed to build @var{packages}, allowing you to -eventually build them even without network access. It is an extension -of the @code{--source} option and can accept one of the following -optional argument values: +The syntactic form to construct gexps is summarized below. + +@deffn {Scheme Syntax} #~@var{exp} +@deffnx {Scheme Syntax} (gexp @var{exp}) +Return a G-expression containing @var{exp}. @var{exp} may contain one +or more of the following forms: @table @code -@item package -This value causes the @code{--sources} option to behave in the same way -as the @code{--source} option. +@item #$@var{obj} +@itemx (ungexp @var{obj}) +Introduce a reference to @var{obj}. @var{obj} may have one of the +supported types, for example a package or a +derivation, in which case the @code{ungexp} form is replaced by its +output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}. -@item all -Build the source derivations of all packages, including any source that -might be listed as @code{inputs}. This is the default value. +If @var{obj} is a list, it is traversed and references to supported +objects are substituted similarly. -@example -$ guix build --sources tzdata -The following derivations will be built: - /gnu/store/@dots{}-tzdata2015b.tar.gz.drv - /gnu/store/@dots{}-tzcode2015b.tar.gz.drv -@end example +If @var{obj} is another gexp, its contents are inserted and its +dependencies are added to those of the containing gexp. -@item transitive -Build the source derivations of all packages, as well of all transitive -inputs to the packages. This can be used e.g.@: to -prefetch package source for later offline building. +If @var{obj} is another kind of object, it is inserted as is. -@example -$ guix build --sources=transitive tzdata -The following derivations will be built: - /gnu/store/@dots{}-tzcode2015b.tar.gz.drv - /gnu/store/@dots{}-findutils-4.4.2.tar.xz.drv - /gnu/store/@dots{}-grep-2.21.tar.xz.drv - /gnu/store/@dots{}-coreutils-8.23.tar.xz.drv - /gnu/store/@dots{}-make-4.1.tar.xz.drv - /gnu/store/@dots{}-bash-4.3.tar.xz.drv -@dots{} -@end example +@item #$@var{obj}:@var{output} +@itemx (ungexp @var{obj} @var{output}) +This is like the form above, but referring explicitly to the +@var{output} of @var{obj}---this is useful when @var{obj} produces +multiple outputs (@pxref{Packages with Multiple Outputs}). -@end table +@item #+@var{obj} +@itemx #+@var{obj}:output +@itemx (ungexp-native @var{obj}) +@itemx (ungexp-native @var{obj} @var{output}) +Same as @code{ungexp}, but produces a reference to the @emph{native} +build of @var{obj} when used in a cross compilation context. -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of -the system type of the build host. +@item #$output[:@var{output}] +@itemx (ungexp output [@var{output}]) +Insert a reference to derivation output @var{output}, or to the main +output when @var{output} is omitted. -@quotation Note -The @code{--system} flag is for @emph{native} compilation and must not -be confused with cross-compilation. See @code{--target} below for -information on cross-compilation. -@end quotation +This only makes sense for gexps passed to @code{gexp->derivation}. -An example use of this is on Linux-based systems, which can emulate -different personalities. For instance, passing -@code{--system=i686-linux} on an @code{x86_64-linux} system or -@code{--system=armhf-linux} on an @code{aarch64-linux} system allows you -to build packages in a complete 32-bit environment. +@item #$@@@var{lst} +@itemx (ungexp-splicing @var{lst}) +Like the above, but splices the contents of @var{lst} inside the +containing list. -@quotation Note -Building for an @code{armhf-linux} system is unconditionally enabled on -@code{aarch64-linux} machines, although certain aarch64 chipsets do not -allow for this functionality, notably the ThunderX. -@end quotation +@item #+@@@var{lst} +@itemx (ungexp-native-splicing @var{lst}) +Like the above, but refers to native builds of the objects listed in +@var{lst}. -Similarly, when transparent emulation with QEMU and @code{binfmt_misc} -is enabled (@pxref{Virtualization Services, -@code{qemu-binfmt-service-type}}), you can build for any system for -which a QEMU @code{binfmt_misc} handler is installed. +@end table -Builds for a system other than that of the machine you are using can -also be offloaded to a remote machine of the right architecture. -@xref{Daemon Offload Setup}, for more information on offloading. +G-expressions created by @code{gexp} or @code{#~} are run-time objects +of the @code{gexp?} type (see below.) +@end deffn -@item --target=@var{triplet} -@cindex cross-compilation -Cross-build for @var{triplet}, which must be a valid GNU triplet, such -as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU -configuration triplets,, autoconf, Autoconf}). +@deffn {Scheme Syntax} with-imported-modules @var{modules} @var{body}@dots{} +Mark the gexps defined in @var{body}@dots{} as requiring @var{modules} +in their execution environment. -@anchor{build-check} -@item --check -@cindex determinism, checking -@cindex reproducibility, checking -Rebuild @var{package-or-derivation}, which are already available in the -store, and raise an error if the build results are not bit-for-bit -identical. +Each item in @var{modules} can be the name of a module, such as +@code{(guix build utils)}, or it can be a module name, followed by an +arrow, followed by a file-like object: -This mechanism allows you to check whether previously installed -substitutes are genuine (@pxref{Substitutes}), or whether the build result -of a package is deterministic. @xref{Invoking guix challenge}, for more -background information and tools. +@example +`((guix build utils) + (guix gcrypt) + ((guix config) => ,(scheme-file "config.scm" + #~(define-module @dots{})))) +@end example -When used in conjunction with @option{--keep-failed}, the differing -output is kept in the store, under @file{/gnu/store/@dots{}-check}. -This makes it easy to look for differences between the two results. +@noindent +In the example above, the first two modules are taken from the search +path, and the last one is created from the given file-like object. -@item --repair -@cindex repairing store items -@cindex corruption, recovering from -Attempt to repair the specified store items, if they are corrupt, by -re-downloading or rebuilding them. +This form has @emph{lexical} scope: it has an effect on the gexps +directly defined in @var{body}@dots{}, but not on those defined, say, in +procedures called from @var{body}@dots{}. +@end deffn -This operation is not atomic and thus restricted to @code{root}. +@deffn {Scheme Syntax} with-extensions @var{extensions} @var{body}@dots{} +Mark the gexps defined in @var{body}@dots{} as requiring +@var{extensions} in their build and execution environment. +@var{extensions} is typically a list of package objects such as those +defined in the @code{(gnu packages guile)} module. -@item --derivations -@itemx -d -Return the derivation paths, not the output paths, of the given -packages. +Concretely, the packages listed in @var{extensions} are added to the +load path while compiling imported modules in @var{body}@dots{}; they +are also added to the load path of the gexp returned by +@var{body}@dots{}. +@end deffn -@item --root=@var{file} -@itemx -r @var{file} -@cindex GC roots, adding -@cindex garbage collector roots, adding -Make @var{file} a symlink to the result, and register it as a garbage -collector root. +@deffn {Scheme Procedure} gexp? @var{obj} +Return @code{#t} if @var{obj} is a G-expression. +@end deffn -Consequently, the results of this @command{guix build} invocation are -protected from garbage collection until @var{file} is removed. When -that option is omitted, build results are eligible for garbage -collection as soon as the build completes. @xref{Invoking guix gc}, for -more on GC roots. +G-expressions are meant to be written to disk, either as code building +some derivation, or as plain files in the store. The monadic procedures +below allow you to do that (@pxref{The Store Monad}, for more +information about monads.) -@item --log-file -@cindex build logs, access -Return the build log file names or URLs for the given -@var{package-or-derivation}, or raise an error if build logs are -missing. +@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @ + [#:system (%current-system)] [#:target #f] [#:graft? #t] @ + [#:hash #f] [#:hash-algo #f] @ + [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ + [#:module-path @var{%load-path}] @ + [#:effective-version "2.2"] @ + [#:references-graphs #f] [#:allowed-references #f] @ + [#:disallowed-references #f] @ + [#:leaked-env-vars #f] @ + [#:script-name (string-append @var{name} "-builder")] @ + [#:deprecation-warnings #f] @ + [#:local-build? #f] [#:substitutable? #t] @ + [#:properties '()] [#:guile-for-build #f] +Return a derivation @var{name} that runs @var{exp} (a gexp) with +@var{guile-for-build} (a derivation) on @var{system}; @var{exp} is +stored in a file called @var{script-name}. When @var{target} is true, +it is used as the cross-compilation target triplet for packages referred +to by @var{exp}. -This works regardless of how packages or derivations are specified. For -instance, the following invocations are equivalent: +@var{modules} is deprecated in favor of @code{with-imported-modules}. +Its meaning is to +make @var{modules} available in the evaluation context of @var{exp}; +@var{modules} is a list of names of Guile modules searched in +@var{module-path} to be copied in the store, compiled, and made available in +the load path during the execution of @var{exp}---e.g., @code{((guix +build utils) (guix build gnu-build-system))}. -@example -guix build --log-file `guix build -d guile` -guix build --log-file `guix build guile` -guix build --log-file guile -guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)' -@end example +@var{effective-version} determines the string to use when adding extensions of +@var{exp} (see @code{with-extensions}) to the search path---e.g., @code{"2.2"}. -If a log is unavailable locally, and unless @code{--no-substitutes} is -passed, the command looks for a corresponding log on one of the -substitute servers (as specified with @code{--substitute-urls}.) +@var{graft?} determines whether packages referred to by @var{exp} should be grafted when +applicable. -So for instance, imagine you want to see the build log of GDB on MIPS, -but you are actually on an @code{x86_64} machine: +When @var{references-graphs} is true, it must be a list of tuples of one of the +following forms: @example -$ guix build --log-file gdb -s mips64el-linux -https://@value{SUBSTITUTE-SERVER}/log/@dots{}-gdb-7.10 +(@var{file-name} @var{package}) +(@var{file-name} @var{package} @var{output}) +(@var{file-name} @var{derivation}) +(@var{file-name} @var{derivation} @var{output}) +(@var{file-name} @var{store-item}) @end example -You can freely access a huge library of build logs! -@end table +The right-hand-side of each element of @var{references-graphs} is automatically made +an input of the build process of @var{exp}. In the build environment, each +@var{file-name} contains the reference graph of the corresponding item, in a simple +text format. -@node Debugging Build Failures -@subsection Debugging Build Failures +@var{allowed-references} must be either @code{#f} or a list of output names and packages. +In the latter case, the list denotes store items that the result is allowed to +refer to. Any reference to another store item will lead to a build error. +Similarly for @var{disallowed-references}, which can list items that must not be +referenced by the outputs. -@cindex build failures, debugging -When defining a new package (@pxref{Defining Packages}), you will -probably find yourself spending some time debugging and tweaking the -build until it succeeds. To do that, you need to operate the build -commands yourself in an environment as close as possible to the one the -build daemon uses. +@var{deprecation-warnings} determines whether to show deprecation warnings while +compiling modules. It can be @code{#f}, @code{#t}, or @code{'detailed}. -To that end, the first thing to do is to use the @option{--keep-failed} -or @option{-K} option of @command{guix build}, which will keep the -failed build tree in @file{/tmp} or whatever directory you specified as -@code{TMPDIR} (@pxref{Invoking guix build, @code{--keep-failed}}). +The other arguments are as for @code{derivation} (@pxref{Derivations}). +@end deffn -From there on, you can @command{cd} to the failed build tree and source -the @file{environment-variables} file, which contains all the -environment variable definitions that were in place when the build -failed. So let's say you're debugging a build failure in package -@code{foo}; a typical session would look like this: +@cindex file-like objects +The @code{local-file}, @code{plain-file}, @code{computed-file}, +@code{program-file}, and @code{scheme-file} procedures below return +@dfn{file-like objects}. That is, when unquoted in a G-expression, +these objects lead to a file in the store. Consider this G-expression: @example -$ guix build foo -K -@dots{} @i{build fails} -$ cd /tmp/guix-build-foo.drv-0 -$ source ./environment-variables -$ cd foo-1.2 +#~(system* #$(file-append glibc "/sbin/nscd") "-f" + #$(local-file "/tmp/my-nscd.conf")) @end example -Now, you can invoke commands as if you were the daemon (almost) and -troubleshoot your build process. +The effect here is to ``intern'' @file{/tmp/my-nscd.conf} by copying it +to the store. Once expanded, for instance @i{via} +@code{gexp->derivation}, the G-expression refers to that copy under +@file{/gnu/store}; thus, modifying or removing the file in @file{/tmp} +does not have any effect on what the G-expression does. +@code{plain-file} can be used similarly; it differs in that the file +content is directly passed as a string. -Sometimes it happens that, for example, a package's tests pass when you -run them manually but they fail when the daemon runs them. This can -happen because the daemon runs builds in containers where, unlike in our -environment above, network access is missing, @file{/bin/sh} does not -exist, etc. (@pxref{Build Environment Setup}). +@deffn {Scheme Procedure} local-file @var{file} [@var{name}] @ + [#:recursive? #f] [#:select? (const #t)] +Return an object representing local file @var{file} to add to the store; this +object can be used in a gexp. If @var{file} is a relative file name, it is looked +up relative to the source file where this form appears. @var{file} will be added to +the store under @var{name}--by default the base name of @var{file}. -In such cases, you may need to run inspect the build process from within -a container similar to the one the build daemon creates: +When @var{recursive?} is true, the contents of @var{file} are added recursively; if @var{file} +designates a flat file and @var{recursive?} is true, its contents are added, and its +permission bits are kept. -@example -$ guix build -K foo -@dots{} -$ cd /tmp/guix-build-foo.drv-0 -$ guix environment --no-grafts -C foo --ad-hoc strace gdb -[env]# source ./environment-variables -[env]# cd foo-1.2 -@end example +When @var{recursive?} is true, call @code{(@var{select?} @var{file} +@var{stat})} for each directory entry, where @var{file} is the entry's +absolute file name and @var{stat} is the result of @code{lstat}; exclude +entries for which @var{select?} does not return true. -Here, @command{guix environment -C} creates a container and spawns a new -shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc -strace gdb} part adds the @command{strace} and @command{gdb} commands to -the container, which would may find handy while debugging. The -@option{--no-grafts} option makes sure we get the exact same -environment, with ungrafted packages (@pxref{Security Updates}, for more -info on grafts). +This is the declarative counterpart of the @code{interned-file} monadic +procedure (@pxref{The Store Monad, @code{interned-file}}). +@end deffn -To get closer to a container like that used by the build daemon, we can -remove @file{/bin/sh}: +@deffn {Scheme Procedure} plain-file @var{name} @var{content} +Return an object representing a text file called @var{name} with the given +@var{content} (a string or a bytevector) to be added to the store. -@example -[env]# rm /bin/sh -@end example +This is the declarative counterpart of @code{text-file}. +@end deffn -(Don't worry, this is harmless: this is all happening in the throw-away -container created by @command{guix environment}.) +@deffn {Scheme Procedure} computed-file @var{name} @var{gexp} @ + [#:options '(#:local-build? #t)] +Return an object representing the store item @var{name}, a file or +directory computed by @var{gexp}. @var{options} +is a list of additional arguments to pass to @code{gexp->derivation}. -The @command{strace} command is probably not in the search path, but we -can run: +This is the declarative counterpart of @code{gexp->derivation}. +@end deffn -@example -[env]# $GUIX_ENVIRONMENT/bin/strace -f -o log make check -@end example +@deffn {Monadic Procedure} gexp->script @var{name} @var{exp} @ + [#:guile (default-guile)] [#:module-path %load-path] +Return an executable script @var{name} that runs @var{exp} using +@var{guile}, with @var{exp}'s imported modules in its search path. +Look up @var{exp}'s modules in @var{module-path}. -In this way, not only you will have reproduced the environment variables -the daemon uses, you will also be running the build process in a container -similar to the one the daemon uses. +The example below builds a script that simply invokes the @command{ls} +command: +@example +(use-modules (guix gexp) (gnu packages base)) -@node Invoking guix edit -@section Invoking @command{guix edit} +(gexp->script "list-files" + #~(execl #$(file-append coreutils "/bin/ls") + "ls")) +@end example -@cindex @command{guix edit} -@cindex package definition, editing -So many packages, so many source files! The @command{guix edit} command -facilitates the life of users and packagers by pointing their editor at -the source file containing the definition of the specified packages. -For instance: +When ``running'' it through the store (@pxref{The Store Monad, +@code{run-with-store}}), we obtain a derivation that produces an +executable file @file{/gnu/store/@dots{}-list-files} along these lines: @example -guix edit gcc@@4.9 vim +#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds +!# +(execl "/gnu/store/@dots{}-coreutils-8.22"/bin/ls" "ls") @end example +@end deffn -@noindent -launches the program specified in the @code{VISUAL} or in the -@code{EDITOR} environment variable to view the recipe of GCC@tie{}4.9.3 -and that of Vim. - -If you are using a Guix Git checkout (@pxref{Building from Git}), or -have created your own packages on @code{GUIX_PACKAGE_PATH} -(@pxref{Package Modules}), you will be able to edit the package -recipes. In other cases, you will be able to examine the read-only recipes -for packages currently in the store. +@deffn {Scheme Procedure} program-file @var{name} @var{exp} @ + [#:guile #f] [#:module-path %load-path] +Return an object representing the executable store item @var{name} that +runs @var{gexp}. @var{guile} is the Guile package used to execute that +script. Imported modules of @var{gexp} are looked up in @var{module-path}. +This is the declarative counterpart of @code{gexp->script}. +@end deffn -@node Invoking guix download -@section Invoking @command{guix download} +@deffn {Monadic Procedure} gexp->file @var{name} @var{exp} @ + [#:set-load-path? #t] [#:module-path %load-path] @ + [#:splice? #f] @ + [#:guile (default-guile)] +Return a derivation that builds a file @var{name} containing @var{exp}. +When @var{splice?} is true, @var{exp} is considered to be a list of +expressions that will be spliced in the resulting file. -@cindex @command{guix download} -@cindex downloading package sources -When writing a package definition, developers typically need to download -a source tarball, compute its SHA256 hash, and write that -hash in the package definition (@pxref{Defining Packages}). The -@command{guix download} tool helps with this task: it downloads a file -from the given URI, adds it to the store, and prints both its file name -in the store and its SHA256 hash. +When @var{set-load-path?} is true, emit code in the resulting file to +set @code{%load-path} and @code{%load-compiled-path} to honor +@var{exp}'s imported modules. Look up @var{exp}'s modules in +@var{module-path}. -The fact that the downloaded file is added to the store saves bandwidth: -when the developer eventually tries to build the newly defined package -with @command{guix build}, the source tarball will not have to be -downloaded again because it is already in the store. It is also a -convenient way to temporarily stash files, which may be deleted -eventually (@pxref{Invoking guix gc}). +The resulting file holds references to all the dependencies of @var{exp} +or a subset thereof. +@end deffn -The @command{guix download} command supports the same URIs as used in -package definitions. In particular, it supports @code{mirror://} URIs. -@code{https} URIs (HTTP over TLS) are supported @emph{provided} the -Guile bindings for GnuTLS are available in the user's environment; when -they are not available, an error is raised. @xref{Guile Preparations, -how to install the GnuTLS bindings for Guile,, gnutls-guile, -GnuTLS-Guile}, for more information. +@deffn {Scheme Procedure} scheme-file @var{name} @var{exp} [#:splice? #f] +Return an object representing the Scheme file @var{name} that contains +@var{exp}. -@command{guix download} verifies HTTPS server certificates by loading -the certificates of X.509 authorities from the directory pointed to by -the @code{SSL_CERT_DIR} environment variable (@pxref{X.509 -Certificates}), unless @option{--no-check-certificate} is used. +This is the declarative counterpart of @code{gexp->file}. +@end deffn -The following options are available: +@deffn {Monadic Procedure} text-file* @var{name} @var{text} @dots{} +Return as a monadic value a derivation that builds a text file +containing all of @var{text}. @var{text} may list, in addition to +strings, objects of any type that can be used in a gexp: packages, +derivations, local file objects, etc. The resulting store file holds +references to all these. -@table @code -@item --format=@var{fmt} -@itemx -f @var{fmt} -Write the hash in the format specified by @var{fmt}. For more -information on the valid values for @var{fmt}, @pxref{Invoking guix hash}. +This variant should be preferred over @code{text-file} anytime the file +to create will reference items from the store. This is typically the +case when building a configuration file that embeds store file names, +like this: -@item --no-check-certificate -Do not validate the X.509 certificates of HTTPS servers. +@example +(define (profile.sh) + ;; Return the name of a shell script in the store that + ;; initializes the 'PATH' environment variable. + (text-file* "profile.sh" + "export PATH=" coreutils "/bin:" + grep "/bin:" sed "/bin\n")) +@end example -When using this option, you have @emph{absolutely no guarantee} that you -are communicating with the authentic server responsible for the given -URL, which makes you vulnerable to ``man-in-the-middle'' attacks. +In this example, the resulting @file{/gnu/store/@dots{}-profile.sh} file +will reference @var{coreutils}, @var{grep}, and @var{sed}, thereby +preventing them from being garbage-collected during its lifetime. +@end deffn -@item --output=@var{file} -@itemx -o @var{file} -Save the downloaded file to @var{file} instead of adding it to the -store. -@end table +@deffn {Scheme Procedure} mixed-text-file @var{name} @var{text} @dots{} +Return an object representing store file @var{name} containing +@var{text}. @var{text} is a sequence of strings and file-like objects, +as in: -@node Invoking guix hash -@section Invoking @command{guix hash} +@example +(mixed-text-file "profile" + "export PATH=" coreutils "/bin:" grep "/bin") +@end example -@cindex @command{guix hash} -The @command{guix hash} command computes the SHA256 hash of a file. -It is primarily a convenience tool for anyone contributing to the -distribution: it computes the cryptographic hash of a file, which can be -used in the definition of a package (@pxref{Defining Packages}). +This is the declarative counterpart of @code{text-file*}. +@end deffn -The general syntax is: +@deffn {Scheme Procedure} file-union @var{name} @var{files} +Return a @code{} that builds a directory containing all of @var{files}. +Each item in @var{files} must be a two-element list where the first element is the +file name to use in the new directory, and the second element is a gexp +denoting the target file. Here's an example: @example -guix hash @var{option} @var{file} -@end example - -When @var{file} is @code{-} (a hyphen), @command{guix hash} computes the -hash of data read from standard input. @command{guix hash} has the -following options: +(file-union "etc" + `(("hosts" ,(plain-file "hosts" + "127.0.0.1 localhost")) + ("bashrc" ,(plain-file "bashrc" + "alias ls='ls --color=auto'")))) +@end example -@table @code +This yields an @code{etc} directory containing these two files. +@end deffn -@item --format=@var{fmt} -@itemx -f @var{fmt} -Write the hash in the format specified by @var{fmt}. +@deffn {Scheme Procedure} directory-union @var{name} @var{things} +Return a directory that is the union of @var{things}, where @var{things} is a list of +file-like objects denoting directories. For example: -Supported formats: @code{nix-base32}, @code{base32}, @code{base16} -(@code{hex} and @code{hexadecimal} can be used as well). +@example +(directory-union "guile+emacs" (list guile emacs)) +@end example -If the @option{--format} option is not specified, @command{guix hash} -will output the hash in @code{nix-base32}. This representation is used -in the definitions of packages. +yields a directory that is the union of the @code{guile} and @code{emacs} packages. +@end deffn -@item --recursive -@itemx -r -Compute the hash on @var{file} recursively. +@deffn {Scheme Procedure} file-append @var{obj} @var{suffix} @dots{} +Return a file-like object that expands to the concatenation of @var{obj} +and @var{suffix}, where @var{obj} is a lowerable object and each +@var{suffix} is a string. -In this case, the hash is computed on an archive containing @var{file}, -including its children if it is a directory. Some of the metadata of -@var{file} is part of the archive; for instance, when @var{file} is a -regular file, the hash is different depending on whether @var{file} is -executable or not. Metadata such as time stamps has no impact on the -hash (@pxref{Invoking guix archive}). -@c FIXME: Replace xref above with xref to an ``Archive'' section when -@c it exists. +As an example, consider this gexp: -@item --exclude-vcs -@itemx -x -When combined with @option{--recursive}, exclude version control system -directories (@file{.bzr}, @file{.git}, @file{.hg}, etc.) +@example +(gexp->script "run-uname" + #~(system* #$(file-append coreutils + "/bin/uname"))) +@end example -@vindex git-fetch -As an example, here is how you would compute the hash of a Git checkout, -which is useful when using the @code{git-fetch} method (@pxref{origin -Reference}): +The same effect could be achieved with: @example -$ git clone http://example.org/foo.git -$ cd foo -$ guix hash -rx . +(gexp->script "run-uname" + #~(system* (string-append #$coreutils + "/bin/uname"))) @end example -@end table - -@node Invoking guix import -@section Invoking @command{guix import} -@cindex importing packages -@cindex package import -@cindex package conversion -@cindex Invoking @command{guix import} -The @command{guix import} command is useful for people who would like to -add a package to the distribution with as little work as -possible---a legitimate demand. The command knows of a few -repositories from which it can ``import'' package metadata. The result -is a package definition, or a template thereof, in the format we know -(@pxref{Defining Packages}). +There is one difference though: in the @code{file-append} case, the +resulting script contains the absolute file name as a string, whereas in +the second case, the resulting script contains a @code{(string-append +@dots{})} expression to construct the file name @emph{at run time}. +@end deffn -The general syntax is: -@example -guix import @var{importer} @var{options}@dots{} -@end example +Of course, in addition to gexps embedded in ``host'' code, there are +also modules containing build tools. To make it clear that they are +meant to be used in the build stratum, these modules are kept in the +@code{(guix build @dots{})} name space. -@var{importer} specifies the source from which to import package -metadata, and @var{options} specifies a package identifier and other -options specific to @var{importer}. Currently, the available -``importers'' are: +@cindex lowering, of high-level objects in gexps +Internally, high-level objects are @dfn{lowered}, using their compiler, +to either derivations or store items. For instance, lowering a package +yields a derivation, and lowering a @code{plain-file} yields a store +item. This is achieved using the @code{lower-object} monadic procedure. -@table @code -@item gnu -Import metadata for the given GNU package. This provides a template -for the latest version of that GNU package, including the hash of its -source tarball, and its canonical synopsis and description. +@deffn {Monadic Procedure} lower-object @var{obj} [@var{system}] @ + [#:target #f] +Return as a value in @var{%store-monad} the derivation or store item +corresponding to @var{obj} for @var{system}, cross-compiling for +@var{target} if @var{target} is true. @var{obj} must be an object that +has an associated gexp compiler, such as a @code{}. +@end deffn -Additional information such as the package dependencies and its -license needs to be figured out manually. +@node Invoking guix repl +@section Invoking @command{guix repl} -For example, the following command returns a package definition for -GNU@tie{}Hello: +@cindex REPL, read-eval-print loop +The @command{guix repl} command spawns a Guile @dfn{read-eval-print loop} +(REPL) for interactive programming (@pxref{Using Guile Interactively,,, guile, +GNU Guile Reference Manual}). Compared to just launching the @command{guile} +command, @command{guix repl} guarantees that all the Guix modules and all its +dependencies are available in the search path. You can use it this way: @example -guix import gnu hello +$ guix repl +scheme@@(guile-user)> ,use (gnu packages base) +scheme@@(guile-user)> coreutils +$1 = # @end example -Specific command-line options are: +@cindex inferiors +In addition, @command{guix repl} implements a simple machine-readable REPL +protocol for use by @code{(guix inferior)}, a facility to interact with +@dfn{inferiors}, separate processes running a potentially different revision +of Guix. -@table @code -@item --key-download=@var{policy} -As for @code{guix refresh}, specify the policy to handle missing OpenPGP -keys when verifying the package signature. @xref{Invoking guix -refresh, @code{--key-download}}. -@end table +The available options are as follows: -@item pypi -@cindex pypi -Import metadata from the @uref{https://pypi.python.org/, Python Package -Index}. Information is taken from the JSON-formatted description -available at @code{pypi.python.org} and usually includes all the relevant -information, including package dependencies. For maximum efficiency, it -is recommended to install the @command{unzip} utility, so that the -importer can unzip Python wheels and gather data from them. +@table @code +@item --type=@var{type} +@itemx -t @var{type} +Start a REPL of the given @var{TYPE}, which can be one of the following: -The command below imports metadata for the @code{itsdangerous} Python -package: +@table @code +@item guile +This is default, and it spawns a standard full-featured Guile REPL. +@item machine +Spawn a REPL that uses the machine-readable protocol. This is the protocol +that the @code{(guix inferior)} module speaks. +@end table -@example -guix import pypi itsdangerous -@end example +@item --listen=@var{endpoint} +By default, @command{guix repl} reads from standard input and writes to +standard output. When this option is passed, it will instead listen for +connections on @var{endpoint}. Here are examples of valid options: @table @code -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. +@item --listen=tcp:37146 +Accept connections on localhost on port 37146. + +@item --listen=unix:/tmp/socket +Accept connections on the Unix-domain socket @file{/tmp/socket}. +@end table @end table -@item gem -@cindex gem -Import metadata from @uref{https://rubygems.org/, RubyGems}. Information -is taken from the JSON-formatted description available at -@code{rubygems.org} and includes most relevant information, including -runtime dependencies. There are some caveats, however. The metadata -doesn't distinguish between synopses and descriptions, so the same string -is used for both fields. Additionally, the details of non-Ruby -dependencies required to build native extensions is unavailable and left -as an exercise to the packager. +@c ********************************************************************* +@node Utilities +@chapter Utilities -The command below imports metadata for the @code{rails} Ruby package: +This section describes Guix command-line utilities. Some of them are +primarily targeted at developers and users who write new package +definitions, while others are more generally useful. They complement +the Scheme programming interface of Guix in a convenient way. -@example -guix import gem rails -@end example +@menu +* Invoking guix build:: Building packages from the command line. +* Invoking guix edit:: Editing package definitions. +* Invoking guix download:: Downloading a file and printing its hash. +* Invoking guix hash:: Computing the cryptographic hash of a file. +* Invoking guix import:: Importing package definitions. +* Invoking guix refresh:: Updating package definitions. +* Invoking guix lint:: Finding errors in package definitions. +* Invoking guix size:: Profiling disk usage. +* Invoking guix graph:: Visualizing the graph of packages. +* Invoking guix environment:: Setting up development environments. +* Invoking guix publish:: Sharing substitutes. +* Invoking guix challenge:: Challenging substitute servers. +* Invoking guix copy:: Copying to and from a remote store. +* Invoking guix container:: Process isolation. +* Invoking guix weather:: Assessing substitute availability. +* Invoking guix processes:: Listing client processes. +@end menu -@table @code -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. -@end table +@node Invoking guix build +@section Invoking @command{guix build} -@item cpan -@cindex CPAN -Import metadata from @uref{https://www.metacpan.org/, MetaCPAN}. -Information is taken from the JSON-formatted metadata provided through -@uref{https://fastapi.metacpan.org/, MetaCPAN's API} and includes most -relevant information, such as module dependencies. License information -should be checked closely. If Perl is available in the store, then the -@code{corelist} utility will be used to filter core modules out of the -list of dependencies. +@cindex package building +@cindex @command{guix build} +The @command{guix build} command builds packages or derivations and +their dependencies, and prints the resulting store paths. Note that it +does not modify the user's profile---this is the job of the +@command{guix package} command (@pxref{Invoking guix package}). Thus, +it is mainly useful for distribution developers. -The command command below imports metadata for the @code{Acme::Boolean} -Perl module: +The general syntax is: @example -guix import cpan Acme::Boolean +guix build @var{options} @var{package-or-derivation}@dots{} @end example -@item cran -@cindex CRAN -@cindex Bioconductor -Import metadata from @uref{https://cran.r-project.org/, CRAN}, the -central repository for the @uref{http://r-project.org, GNU@tie{}R -statistical and graphical environment}. - -Information is extracted from the @code{DESCRIPTION} file of the package. - -The command command below imports metadata for the @code{Cairo} -R package: +As an example, the following command builds the latest versions of Emacs +and of Guile, displays their build logs, and finally displays the +resulting directories: @example -guix import cran Cairo +guix build emacs guile @end example -When @code{--recursive} is added, the importer will traverse the -dependency graph of the given upstream package recursively and generate -package expressions for all those packages that are not yet in Guix. - -When @code{--archive=bioconductor} is added, metadata is imported from -@uref{https://www.bioconductor.org/, Bioconductor}, a repository of R -packages for for the analysis and comprehension of high-throughput -genomic data in bioinformatics. - -Information is extracted from the @code{DESCRIPTION} file of a package -published on the web interface of the Bioconductor SVN repository. - -The command below imports metadata for the @code{GenomicRanges} -R package: +Similarly, the following command builds all the available packages: @example -guix import cran --archive=bioconductor GenomicRanges +guix build --quiet --keep-going \ + `guix package -A | cut -f1,2 --output-delimiter=@@` @end example -@item texlive -@cindex TeX Live -@cindex CTAN -Import metadata from @uref{http://www.ctan.org/, CTAN}, the -comprehensive TeX archive network for TeX packages that are part of the -@uref{https://www.tug.org/texlive/, TeX Live distribution}. +@var{package-or-derivation} may be either the name of a package found in +the software distribution such as @code{coreutils} or +@code{coreutils@@8.20}, or a derivation such as +@file{/gnu/store/@dots{}-coreutils-8.19.drv}. In the former case, a +package with the corresponding name (and optionally version) is searched +for among the GNU distribution modules (@pxref{Package Modules}). -Information about the package is obtained through the XML API provided -by CTAN, while the source code is downloaded from the SVN repository of -the Tex Live project. This is done because the CTAN does not keep -versioned archives. +Alternatively, the @code{--expression} option may be used to specify a +Scheme expression that evaluates to a package; this is useful when +disambiguating among several same-named packages or package variants is +needed. -The command command below imports metadata for the @code{fontspec} -TeX package: +There may be zero or more @var{options}. The available options are +described in the subsections below. -@example -guix import texlive fontspec -@end example +@menu +* Common Build Options:: Build options for most commands. +* Package Transformation Options:: Creating variants of packages. +* Additional Build Options:: Options specific to 'guix build'. +* Debugging Build Failures:: Real life packaging experience. +@end menu -When @code{--archive=DIRECTORY} is added, the source code is downloaded -not from the @file{latex} sub-directory of the @file{texmf-dist/source} -tree in the TeX Live SVN repository, but from the specified sibling -directory under the same root. +@node Common Build Options +@subsection Common Build Options -The command below imports metadata for the @code{ifxetex} package from -CTAN while fetching the sources from the directory -@file{texmf/source/generic}: +A number of options that control the build process are common to +@command{guix build} and other commands that can spawn builds, such as +@command{guix package} or @command{guix archive}. These are the +following: -@example -guix import texlive --archive=generic ifxetex -@end example +@table @code -@item json -@cindex JSON, import -Import package metadata from a local JSON file. Consider the following -example package definition in JSON format: +@item --load-path=@var{directory} +@itemx -L @var{directory} +Add @var{directory} to the front of the package module search path +(@pxref{Package Modules}). -@example -@{ - "name": "hello", - "version": "2.10", - "source": "mirror://gnu/hello/hello-2.10.tar.gz", - "build-system": "gnu", - "home-page": "https://www.gnu.org/software/hello/", - "synopsis": "Hello, GNU world: An example GNU package", - "description": "GNU Hello prints a greeting.", - "license": "GPL-3.0+", - "native-inputs": ["gcc@@6"] -@} -@end example +This allows users to define their own packages and make them visible to +the command-line tools. -The field names are the same as for the @code{} record -(@xref{Defining Packages}). References to other packages are provided -as JSON lists of quoted package specification strings such as -@code{guile} or @code{guile@@2.0}. +@item --keep-failed +@itemx -K +Keep the build tree of failed builds. Thus, if a build fails, its build +tree is kept under @file{/tmp}, in a directory whose name is shown at +the end of the build log. This is useful when debugging build issues. +@xref{Debugging Build Failures}, for tips and tricks on how to debug +build issues. -The importer also supports a more explicit source definition using the -common fields for @code{} records: +This option has no effect when connecting to a remote daemon with a +@code{guix://} URI (@pxref{The Store, the @code{GUIX_DAEMON_SOCKET} +variable}). -@example -@{ - @dots{} - "source": @{ - "method": "url-fetch", - "uri": "mirror://gnu/hello/hello-2.10.tar.gz", - "sha256": @{ - "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" - @} - @} - @dots{} -@} -@end example +@item --keep-going +@itemx -k +Keep going when some of the derivations fail to build; return only once +all the builds have either completed or failed. -The command below reads metadata from the JSON file @code{hello.json} -and outputs a package expression: +The default behavior is to stop as soon as one of the specified +derivations has failed. -@example -guix import json hello.json -@end example +@item --dry-run +@itemx -n +Do not build the derivations. -@item nix -Import metadata from a local copy of the source of the -@uref{http://nixos.org/nixpkgs/, Nixpkgs distribution}@footnote{This -relies on the @command{nix-instantiate} command of -@uref{http://nixos.org/nix/, Nix}.}. Package definitions in Nixpkgs are -typically written in a mixture of Nix-language and Bash code. This -command only imports the high-level package structure that is written in -the Nix language. It normally includes all the basic fields of a -package definition. +@anchor{fallback-option} +@item --fallback +When substituting a pre-built binary fails, fall back to building +packages locally (@pxref{Substitution Failure}). -When importing a GNU package, the synopsis and descriptions are replaced -by their canonical upstream variant. +@item --substitute-urls=@var{urls} +@anchor{client-substitute-urls} +Consider @var{urls} the whitespace-separated list of substitute source +URLs, overriding the default list of URLs of @command{guix-daemon} +(@pxref{daemon-substitute-urls,, @command{guix-daemon} URLs}). -Usually, you will first need to do: +This means that substitutes may be downloaded from @var{urls}, provided +they are signed by a key authorized by the system administrator +(@pxref{Substitutes}). -@example -export NIX_REMOTE=daemon -@end example +When @var{urls} is the empty string, substitutes are effectively +disabled. -@noindent -so that @command{nix-instantiate} does not try to open the Nix database. +@item --no-substitutes +Do not use substitutes for build products. That is, always build things +locally instead of allowing downloads of pre-built binaries +(@pxref{Substitutes}). -As an example, the command below imports the package definition of -LibreOffice (more precisely, it imports the definition of the package -bound to the @code{libreoffice} top-level attribute): +@item --no-grafts +Do not ``graft'' packages. In practice, this means that package updates +available as grafts are not applied. @xref{Security Updates}, for more +information on grafts. -@example -guix import nix ~/path/to/nixpkgs libreoffice -@end example +@item --rounds=@var{n} +Build each derivation @var{n} times in a row, and raise an error if +consecutive build results are not bit-for-bit identical. -@item hackage -@cindex hackage -Import metadata from the Haskell community's central package archive -@uref{https://hackage.haskell.org/, Hackage}. Information is taken from -Cabal files and includes all the relevant information, including package -dependencies. +This is a useful way to detect non-deterministic builds processes. +Non-deterministic build processes are a problem because they make it +practically impossible for users to @emph{verify} whether third-party +binaries are genuine. @xref{Invoking guix challenge}, for more. -Specific command-line options are: +Note that, currently, the differing build results are not kept around, +so you will have to manually investigate in case of an error---e.g., by +stashing one of the build results with @code{guix archive --export} +(@pxref{Invoking guix archive}), then rebuilding, and finally comparing +the two results. -@table @code -@item --stdin -@itemx -s -Read a Cabal file from standard input. -@item --no-test-dependencies -@itemx -t -Do not include dependencies required only by the test suites. -@item --cabal-environment=@var{alist} -@itemx -e @var{alist} -@var{alist} is a Scheme alist defining the environment in which the -Cabal conditionals are evaluated. The accepted keys are: @code{os}, -@code{arch}, @code{impl} and a string representing the name of a flag. -The value associated with a flag has to be either the symbol -@code{true} or @code{false}. The value associated with other keys -has to conform to the Cabal file format definition. The default value -associated with the keys @code{os}, @code{arch} and @code{impl} is -@samp{linux}, @samp{x86_64} and @samp{ghc}, respectively. -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. -@end table +@item --no-build-hook +Do not attempt to offload builds @i{via} the ``build hook'' of the daemon +(@pxref{Daemon Offload Setup}). That is, always build things locally +instead of offloading builds to remote machines. -The command below imports metadata for the latest version of the -@code{HTTP} Haskell package without including test dependencies and -specifying the value of the flag @samp{network-uri} as @code{false}: +@item --max-silent-time=@var{seconds} +When the build or substitution process remains silent for more than +@var{seconds}, terminate it and report a build failure. -@example -guix import hackage -t -e "'((\"network-uri\" . false))" HTTP -@end example +By default, the daemon's setting is honored (@pxref{Invoking +guix-daemon, @code{--max-silent-time}}). -A specific package version may optionally be specified by following the -package name by an at-sign and a version number as in the following example: +@item --timeout=@var{seconds} +Likewise, when the build or substitution process lasts for more than +@var{seconds}, terminate it and report a build failure. + +By default, the daemon's setting is honored (@pxref{Invoking +guix-daemon, @code{--timeout}}). + +@c Note: This option is actually not part of %standard-build-options but +@c most programs honor it. +@cindex verbosity, of the command-line tools +@cindex build logs, verbosity +@item -v @var{level} +@itemx --verbosity=@var{level} +Use the given verbosity @var{level}, an integer. Choosing 0 means that no +output is produced, 1 is for quiet output, and 2 shows all the build log +output on standard error. -@example -guix import hackage mtl@@2.1.3.1 -@end example +@item --cores=@var{n} +@itemx -c @var{n} +Allow the use of up to @var{n} CPU cores for the build. The special +value @code{0} means to use as many CPU cores as available. -@item stackage -@cindex stackage -The @code{stackage} importer is a wrapper around the @code{hackage} one. -It takes a package name, looks up the package version included in a -long-term support (LTS) @uref{https://www.stackage.org, Stackage} -release and uses the @code{hackage} importer to retrieve its metadata. -Note that it is up to you to select an LTS release compatible with the -GHC compiler used by Guix. +@item --max-jobs=@var{n} +@itemx -M @var{n} +Allow at most @var{n} build jobs in parallel. @xref{Invoking +guix-daemon, @code{--max-jobs}}, for details about this option and the +equivalent @command{guix-daemon} option. -Specific command-line options are: +@item --debug=@var{level} +Produce debugging output coming from the build daemon. @var{level} must be an +integer between 0 and 5; higher means more verbose output. Setting a level of +4 or more may be helpful when debugging setup issues with the build daemon. -@table @code -@item --no-test-dependencies -@itemx -t -Do not include dependencies required only by the test suites. -@item --lts-version=@var{version} -@itemx -l @var{version} -@var{version} is the desired LTS release version. If omitted the latest -release is used. -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. @end table -The command below imports metadata for the @code{HTTP} Haskell package -included in the LTS Stackage release version 7.18: +Behind the scenes, @command{guix build} is essentially an interface to +the @code{package-derivation} procedure of the @code{(guix packages)} +module, and to the @code{build-derivations} procedure of the @code{(guix +derivations)} module. + +In addition to options explicitly passed on the command line, +@command{guix build} and other @command{guix} commands that support +building honor the @code{GUIX_BUILD_OPTIONS} environment variable. + +@defvr {Environment Variable} GUIX_BUILD_OPTIONS +Users can define this variable to a list of command line options that +will automatically be used by @command{guix build} and other +@command{guix} commands that can perform builds, as in the example +below: @example -guix import stackage --lts-version=7.18 HTTP +$ export GUIX_BUILD_OPTIONS="--no-substitutes -c 2 -L /foo/bar" @end example -@item elpa -@cindex elpa -Import metadata from an Emacs Lisp Package Archive (ELPA) package -repository (@pxref{Packages,,, emacs, The GNU Emacs Manual}). +These options are parsed independently, and the result is appended to +the parsed command-line options. +@end defvr -Specific command-line options are: -@table @code -@item --archive=@var{repo} -@itemx -a @var{repo} -@var{repo} identifies the archive repository from which to retrieve the -information. Currently the supported repositories and their identifiers -are: -@itemize - -@item -@uref{http://elpa.gnu.org/packages, GNU}, selected by the @code{gnu} -identifier. This is the default. +@node Package Transformation Options +@subsection Package Transformation Options -Packages from @code{elpa.gnu.org} are signed with one of the keys -contained in the GnuPG keyring at -@file{share/emacs/25.1/etc/package-keyring.gpg} (or similar) in the -@code{emacs} package (@pxref{Package Installation, ELPA package -signatures,, emacs, The GNU Emacs Manual}). +@cindex package variants +Another set of command-line options supported by @command{guix build} +and also @command{guix package} are @dfn{package transformation +options}. These are options that make it possible to define @dfn{package +variants}---for instance, packages built from different source code. +This is a convenient way to create customized packages on the fly +without having to type in the definitions of package variants +(@pxref{Defining Packages}). -@item -@uref{http://stable.melpa.org/packages, MELPA-Stable}, selected by the -@code{melpa-stable} identifier. +@table @code -@item -@uref{http://melpa.org/packages, MELPA}, selected by the @code{melpa} -identifier. -@end itemize +@item --with-source=@var{source} +@itemx --with-source=@var{package}=@var{source} +@itemx --with-source=@var{package}@@@var{version}=@var{source} +Use @var{source} as the source of @var{package}, and @var{version} as +its version number. +@var{source} must be a file name or a URL, as for @command{guix +download} (@pxref{Invoking guix download}). -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. -@end table +When @var{package} is omitted, +it is taken to be the package name specified on the +command line that matches the base of @var{source}---e.g., +if @var{source} is @code{/src/guile-2.0.10.tar.gz}, the corresponding +package is @code{guile}. -@item crate -@cindex crate -Import metadata from the crates.io Rust package repository -@uref{https://crates.io, crates.io}. +Likewise, when @var{version} is omitted, the version string is inferred from +@var{source}; in the previous example, it is @code{2.0.10}. -@item opam -@cindex OPAM -@cindex OCaml -Import metadata from the @uref{https://opam.ocaml.org/, OPAM} package -repository used by the OCaml community. -@end table +This option allows users to try out versions of packages other than the +one provided by the distribution. The example below downloads +@file{ed-1.7.tar.gz} from a GNU mirror and uses that as the source for +the @code{ed} package: -The structure of the @command{guix import} code is modular. It would be -useful to have more importers for other package formats, and your help -is welcome here (@pxref{Contributing}). +@example +guix build ed --with-source=mirror://gnu/ed/ed-1.7.tar.gz +@end example -@node Invoking guix refresh -@section Invoking @command{guix refresh} +As a developer, @code{--with-source} makes it easy to test release +candidates: -@cindex @command {guix refresh} -The primary audience of the @command{guix refresh} command is developers -of the GNU software distribution. By default, it reports any packages -provided by the distribution that are outdated compared to the latest -upstream version, like this: +@example +guix build guile --with-source=../guile-2.0.9.219-e1bb7.tar.xz +@end example + +@dots{} or to build from a checkout in a pristine environment: @example -$ guix refresh -gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1 -gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0 +$ git clone git://git.sv.gnu.org/guix.git +$ guix build guix --with-source=guix@@1.0=./guix @end example -Alternately, one can specify packages to consider, in which case a -warning is emitted for packages that lack an updater: +@item --with-input=@var{package}=@var{replacement} +Replace dependency on @var{package} by a dependency on +@var{replacement}. @var{package} must be a package name, and +@var{replacement} must be a package specification such as @code{guile} +or @code{guile@@1.8}. + +For instance, the following command builds Guix, but replaces its +dependency on the current stable version of Guile with a dependency on +the legacy version of Guile, @code{guile@@2.0}: @example -$ guix refresh coreutils guile guile-ssh -gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh -gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13 +guix build --with-input=guile=guile@@2.0 guix @end example -@command{guix refresh} browses the upstream repository of each package and determines -the highest version number of the releases therein. The command -knows how to update specific types of packages: GNU packages, ELPA -packages, etc.---see the documentation for @option{--type} below. There -are many packages, though, for which it lacks a method to determine -whether a new upstream release is available. However, the mechanism is -extensible, so feel free to get in touch with us to add a new method! +This is a recursive, deep replacement. So in this example, both +@code{guix} and its dependency @code{guile-json} (which also depends on +@code{guile}) get rebuilt against @code{guile@@2.0}. -@table @code +This is implemented using the @code{package-input-rewriting} Scheme +procedure (@pxref{Defining Packages, @code{package-input-rewriting}}). -@item --recursive -Consider the packages specified, and all the packages upon which they depend. +@item --with-graft=@var{package}=@var{replacement} +This is similar to @code{--with-input} but with an important difference: +instead of rebuilding the whole dependency chain, @var{replacement} is +built and then @dfn{grafted} onto the binaries that were initially +referring to @var{package}. @xref{Security Updates}, for more +information on grafts. + +For example, the command below grafts version 3.5.4 of GnuTLS onto Wget +and all its dependencies, replacing references to the version of GnuTLS +they currently refer to: @example -$ guix refresh --recursive coreutils -gnu/packages/acl.scm:35:2: warning: no updater for acl -gnu/packages/m4.scm:30:12: info: 1.4.18 is already the latest version of m4 -gnu/packages/xml.scm:68:2: warning: no updater for expat -gnu/packages/multiprecision.scm:40:12: info: 6.1.2 is already the latest version of gmp -@dots{} +guix build --with-graft=gnutls=gnutls@@3.5.4 wget @end example -@end table +This has the advantage of being much faster than rebuilding everything. +But there is a caveat: it works if and only if @var{package} and +@var{replacement} are strictly compatible---for example, if they provide +a library, the application binary interface (ABI) of those libraries +must be compatible. If @var{replacement} is somehow incompatible with +@var{package}, then the resulting package may be unusable. Use with +care! -Sometimes the upstream name differs from the package name used in Guix, -and @command{guix refresh} needs a little help. Most updaters honor the -@code{upstream-name} property in package definitions, which can be used -to that effect: +@item --with-branch=@var{package}=@var{branch} +@cindex Git, using the latest commit +@cindex latest commit, building +Build @var{package} from the latest commit of @var{branch}. The @code{source} +field of @var{package} must be an origin with the @code{git-fetch} method +(@pxref{origin Reference}) or a @code{git-checkout} object; the repository URL +is taken from that @code{source}. + +For instance, the following command builds @code{guile-sqlite3} from the +latest commit of its @code{master} branch, and then builds @code{guix} (which +depends on it) and @code{cuirass} (which depends on @code{guix}) against this +specific @code{guile-sqlite3} build: @example -(define-public network-manager - (package - (name "network-manager") - ;; @dots{} - (properties '((upstream-name . "NetworkManager"))))) +guix build --with-branch=guile-sqlite3=master cuirass @end example -When passed @code{--update}, it modifies distribution source files to -update the version numbers and source tarball hashes of those package -recipes (@pxref{Defining Packages}). This is achieved by downloading -each package's latest source tarball and its associated OpenPGP -signature, authenticating the downloaded tarball against its signature -using @command{gpg}, and finally computing its hash. When the public -key used to sign the tarball is missing from the user's keyring, an -attempt is made to automatically retrieve it from a public key server; -when this is successful, the key is added to the user's keyring; otherwise, -@command{guix refresh} reports an error. +@cindex continuous integration +Obviously, since it uses the latest commit of the given branch, the result of +such a command varies over time. Nevertheless it is a convenient way to +rebuild entire software stacks against the latest commit of one or more +packages. This is particularly useful in the context of continuous +integration (CI). + +Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up +consecutive accesses to the same repository. You may want to clean it up once +in a while to save disk space. + +@item --with-commit=@var{package}=@var{commit} +This is similar to @code{--with-branch}, except that it builds from +@var{commit} rather than the tip of a branch. @var{commit} must be a valid +Git commit SHA1 identifier. +@end table -The following options are supported: +@node Additional Build Options +@subsection Additional Build Options + +The command-line options presented below are specific to @command{guix +build}. @table @code -@item --expression=@var{expr} -@itemx -e @var{expr} -Consider the package @var{expr} evaluates to. +@item --quiet +@itemx -q +Build quietly, without displaying the build log; this is equivalent to +@code{--verbosity=0}. Upon completion, the build log is kept in @file{/var} +(or similar) and can always be retrieved using the @option{--log-file} option. -This is useful to precisely refer to a package, as in this example: +@item --file=@var{file} +@itemx -f @var{file} +Build the package, derivation, or other file-like object that the code within +@var{file} evaluates to (@pxref{G-Expressions, file-like objects}). + +As an example, @var{file} might contain a package definition like this +(@pxref{Defining Packages}): @example -guix refresh -l -e '(@@@@ (gnu packages commencement) glibc-final)' +@verbatiminclude package-hello.scm @end example -This command lists the dependents of the ``final'' libc (essentially all -the packages.) - -@item --update -@itemx -u -Update distribution source files (package recipes) in place. This is -usually run from a checkout of the Guix source tree (@pxref{Running -Guix Before It Is Installed}): +@item --expression=@var{expr} +@itemx -e @var{expr} +Build the package or derivation @var{expr} evaluates to. -@example -$ ./pre-inst-env guix refresh -s non-core -u -@end example +For example, @var{expr} may be @code{(@@ (gnu packages guile) +guile-1.8)}, which unambiguously designates this specific variant of +version 1.8 of Guile. -@xref{Defining Packages}, for more information on package definitions. +Alternatively, @var{expr} may be a G-expression, in which case it is used +as a build program passed to @code{gexp->derivation} +(@pxref{G-Expressions}). -@item --select=[@var{subset}] -@itemx -s @var{subset} -Select all the packages in @var{subset}, one of @code{core} or -@code{non-core}. +Lastly, @var{expr} may refer to a zero-argument monadic procedure +(@pxref{The Store Monad}). The procedure must return a derivation as a +monadic value, which is then passed through @code{run-with-store}. -The @code{core} subset refers to all the packages at the core of the -distribution---i.e., packages that are used to build ``everything -else''. This includes GCC, libc, Binutils, Bash, etc. Usually, -changing one of these packages in the distribution entails a rebuild of -all the others. Thus, such updates are an inconvenience to users in -terms of build time or bandwidth used to achieve the upgrade. +@item --source +@itemx -S +Build the source derivations of the packages, rather than the packages +themselves. -The @code{non-core} subset refers to the remaining packages. It is -typically useful in cases where an update of the core packages would be -inconvenient. +For instance, @code{guix build -S gcc} returns something like +@file{/gnu/store/@dots{}-gcc-4.7.2.tar.bz2}, which is the GCC +source tarball. -@item --manifest=@var{file} -@itemx -m @var{file} -Select all the packages from the manifest in @var{file}. This is useful to -check if any packages of the user manifest can be updated. +The returned source tarball is the result of applying any patches and +code snippets specified in the package @code{origin} (@pxref{Defining +Packages}). -@item --type=@var{updater} -@itemx -t @var{updater} -Select only packages handled by @var{updater} (may be a comma-separated -list of updaters). Currently, @var{updater} may be one of: +@item --sources +Fetch and return the source of @var{package-or-derivation} and all their +dependencies, recursively. This is a handy way to obtain a local copy +of all the source code needed to build @var{packages}, allowing you to +eventually build them even without network access. It is an extension +of the @code{--source} option and can accept one of the following +optional argument values: @table @code -@item gnu -the updater for GNU packages; -@item gnome -the updater for GNOME packages; -@item kde -the updater for KDE packages; -@item xorg -the updater for X.org packages; -@item kernel.org -the updater for packages hosted on kernel.org; -@item elpa -the updater for @uref{http://elpa.gnu.org/, ELPA} packages; -@item cran -the updater for @uref{https://cran.r-project.org/, CRAN} packages; -@item bioconductor -the updater for @uref{https://www.bioconductor.org/, Bioconductor} R packages; -@item cpan -the updater for @uref{http://www.cpan.org/, CPAN} packages; -@item pypi -the updater for @uref{https://pypi.python.org, PyPI} packages. -@item gem -the updater for @uref{https://rubygems.org, RubyGems} packages. -@item github -the updater for @uref{https://github.com, GitHub} packages. -@item hackage -the updater for @uref{https://hackage.haskell.org, Hackage} packages. -@item stackage -the updater for @uref{https://www.stackage.org, Stackage} packages. -@item crate -the updater for @uref{https://crates.io, Crates} packages. -@end table +@item package +This value causes the @code{--sources} option to behave in the same way +as the @code{--source} option. -For instance, the following command only checks for updates of Emacs -packages hosted at @code{elpa.gnu.org} and for updates of CRAN packages: +@item all +Build the source derivations of all packages, including any source that +might be listed as @code{inputs}. This is the default value. @example -$ guix refresh --type=elpa,cran -gnu/packages/statistics.scm:819:13: r-testthat would be upgraded from 0.10.0 to 0.11.0 -gnu/packages/emacs.scm:856:13: emacs-auctex would be upgraded from 11.88.6 to 11.88.9 +$ guix build --sources tzdata +The following derivations will be built: + /gnu/store/@dots{}-tzdata2015b.tar.gz.drv + /gnu/store/@dots{}-tzcode2015b.tar.gz.drv @end example -@end table - -In addition, @command{guix refresh} can be passed one or more package -names, as in this example: +@item transitive +Build the source derivations of all packages, as well of all transitive +inputs to the packages. This can be used e.g.@: to +prefetch package source for later offline building. @example -$ ./pre-inst-env guix refresh -u emacs idutils gcc@@4.8 +$ guix build --sources=transitive tzdata +The following derivations will be built: + /gnu/store/@dots{}-tzcode2015b.tar.gz.drv + /gnu/store/@dots{}-findutils-4.4.2.tar.xz.drv + /gnu/store/@dots{}-grep-2.21.tar.xz.drv + /gnu/store/@dots{}-coreutils-8.23.tar.xz.drv + /gnu/store/@dots{}-make-4.1.tar.xz.drv + /gnu/store/@dots{}-bash-4.3.tar.xz.drv +@dots{} @end example -@noindent -The command above specifically updates the @code{emacs} and -@code{idutils} packages. The @code{--select} option would have no -effect in this case. +@end table -When considering whether to upgrade a package, it is sometimes -convenient to know which packages would be affected by the upgrade and -should be checked for compatibility. For this the following option may -be used when passing @command{guix refresh} one or more package names: +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of +the system type of the build host. -@table @code +@quotation Note +The @code{--system} flag is for @emph{native} compilation and must not +be confused with cross-compilation. See @code{--target} below for +information on cross-compilation. +@end quotation -@item --list-updaters -@itemx -L -List available updaters and exit (see @option{--type} above.) +An example use of this is on Linux-based systems, which can emulate +different personalities. For instance, passing +@code{--system=i686-linux} on an @code{x86_64-linux} system or +@code{--system=armhf-linux} on an @code{aarch64-linux} system allows you +to build packages in a complete 32-bit environment. -For each updater, display the fraction of packages it covers; at the -end, display the fraction of packages covered by all these updaters. +@quotation Note +Building for an @code{armhf-linux} system is unconditionally enabled on +@code{aarch64-linux} machines, although certain aarch64 chipsets do not +allow for this functionality, notably the ThunderX. +@end quotation -@item --list-dependent -@itemx -l -List top-level dependent packages that would need to be rebuilt as a -result of upgrading one or more packages. +Similarly, when transparent emulation with QEMU and @code{binfmt_misc} +is enabled (@pxref{Virtualization Services, +@code{qemu-binfmt-service-type}}), you can build for any system for +which a QEMU @code{binfmt_misc} handler is installed. -@xref{Invoking guix graph, the @code{reverse-package} type of -@command{guix graph}}, for information on how to visualize the list of -dependents of a package. +Builds for a system other than that of the machine you are using can +also be offloaded to a remote machine of the right architecture. +@xref{Daemon Offload Setup}, for more information on offloading. -@end table +@item --target=@var{triplet} +@cindex cross-compilation +Cross-build for @var{triplet}, which must be a valid GNU triplet, such +as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU +configuration triplets,, autoconf, Autoconf}). + +@anchor{build-check} +@item --check +@cindex determinism, checking +@cindex reproducibility, checking +Rebuild @var{package-or-derivation}, which are already available in the +store, and raise an error if the build results are not bit-for-bit +identical. + +This mechanism allows you to check whether previously installed +substitutes are genuine (@pxref{Substitutes}), or whether the build result +of a package is deterministic. @xref{Invoking guix challenge}, for more +background information and tools. + +When used in conjunction with @option{--keep-failed}, the differing +output is kept in the store, under @file{/gnu/store/@dots{}-check}. +This makes it easy to look for differences between the two results. + +@item --repair +@cindex repairing store items +@cindex corruption, recovering from +Attempt to repair the specified store items, if they are corrupt, by +re-downloading or rebuilding them. + +This operation is not atomic and thus restricted to @code{root}. + +@item --derivations +@itemx -d +Return the derivation paths, not the output paths, of the given +packages. + +@item --root=@var{file} +@itemx -r @var{file} +@cindex GC roots, adding +@cindex garbage collector roots, adding +Make @var{file} a symlink to the result, and register it as a garbage +collector root. + +Consequently, the results of this @command{guix build} invocation are +protected from garbage collection until @var{file} is removed. When +that option is omitted, build results are eligible for garbage +collection as soon as the build completes. @xref{Invoking guix gc}, for +more on GC roots. + +@item --log-file +@cindex build logs, access +Return the build log file names or URLs for the given +@var{package-or-derivation}, or raise an error if build logs are +missing. -Be aware that the @code{--list-dependent} option only -@emph{approximates} the rebuilds that would be required as a result of -an upgrade. More rebuilds might be required under some circumstances. +This works regardless of how packages or derivations are specified. For +instance, the following invocations are equivalent: @example -$ guix refresh --list-dependent flex -Building the following 120 packages would ensure 213 dependent packages are rebuilt: -hop@@2.4.0 geiser@@0.4 notmuch@@0.18 mu@@0.9.9.5 cflow@@1.4 idutils@@4.6 @dots{} +guix build --log-file `guix build -d guile` +guix build --log-file `guix build guile` +guix build --log-file guile +guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)' @end example -The command above lists a set of packages that could be built to check -for compatibility with an upgraded @code{flex} package. - -@table @code +If a log is unavailable locally, and unless @code{--no-substitutes} is +passed, the command looks for a corresponding log on one of the +substitute servers (as specified with @code{--substitute-urls}.) -@item --list-transitive -List all the packages which one or more packages depend upon. +So for instance, imagine you want to see the build log of GDB on MIPS, +but you are actually on an @code{x86_64} machine: @example -$ guix refresh --list-transitive flex -flex@@2.6.4 depends on the following 25 packages: perl@@5.28.0 help2man@@1.47.6 -bison@@3.0.5 indent@@2.2.10 tar@@1.30 gzip@@1.9 bzip2@@1.0.6 xz@@5.2.4 file@@5.33 @dots{} +$ guix build --log-file gdb -s mips64el-linux +https://@value{SUBSTITUTE-SERVER}/log/@dots{}-gdb-7.10 @end example +You can freely access a huge library of build logs! @end table -The command above lists a set of packages which, when changed, would cause -@code{flex} to be rebuilt. - -The following options can be used to customize GnuPG operation: - -@table @code - -@item --gpg=@var{command} -Use @var{command} as the GnuPG 2.x command. @var{command} is searched -for in @code{$PATH}. +@node Debugging Build Failures +@subsection Debugging Build Failures -@item --keyring=@var{file} -Use @var{file} as the keyring for upstream keys. @var{file} must be in the -@dfn{keybox format}. Keybox files usually have a name ending in @file{.kbx} -and the GNU@tie{}Privacy Guard (GPG) can manipulate these files -(@pxref{kbxutil, @command{kbxutil},, gnupg, Using the GNU Privacy Guard}, for -information on a tool to manipulate keybox files). +@cindex build failures, debugging +When defining a new package (@pxref{Defining Packages}), you will +probably find yourself spending some time debugging and tweaking the +build until it succeeds. To do that, you need to operate the build +commands yourself in an environment as close as possible to the one the +build daemon uses. -When this option is omitted, @command{guix refresh} uses -@file{~/.config/guix/upstream/trustedkeys.kbx} as the keyring for upstream -signing keys. OpenPGP signatures are checked against keys from this keyring; -missing keys are downloaded to this keyring as well (see -@option{--key-download} below.) +To that end, the first thing to do is to use the @option{--keep-failed} +or @option{-K} option of @command{guix build}, which will keep the +failed build tree in @file{/tmp} or whatever directory you specified as +@code{TMPDIR} (@pxref{Invoking guix build, @code{--keep-failed}}). -You can export keys from your default GPG keyring into a keybox file using -commands like this one: +From there on, you can @command{cd} to the failed build tree and source +the @file{environment-variables} file, which contains all the +environment variable definitions that were in place when the build +failed. So let's say you're debugging a build failure in package +@code{foo}; a typical session would look like this: @example -gpg --export rms@@gnu.org | kbxutil --import-openpgp >> mykeyring.kbx +$ guix build foo -K +@dots{} @i{build fails} +$ cd /tmp/guix-build-foo.drv-0 +$ source ./environment-variables +$ cd foo-1.2 @end example -Likewise, you can fetch keys to a specific keybox file like this: +Now, you can invoke commands as if you were the daemon (almost) and +troubleshoot your build process. + +Sometimes it happens that, for example, a package's tests pass when you +run them manually but they fail when the daemon runs them. This can +happen because the daemon runs builds in containers where, unlike in our +environment above, network access is missing, @file{/bin/sh} does not +exist, etc. (@pxref{Build Environment Setup}). + +In such cases, you may need to run inspect the build process from within +a container similar to the one the build daemon creates: @example -gpg --no-default-keyring --keyring mykeyring.kbx \ - --recv-keys @value{OPENPGP-SIGNING-KEY-ID} +$ guix build -K foo +@dots{} +$ cd /tmp/guix-build-foo.drv-0 +$ guix environment --no-grafts -C foo --ad-hoc strace gdb +[env]# source ./environment-variables +[env]# cd foo-1.2 @end example -@ref{GPG Configuration Options, @option{--keyring},, gnupg, Using the GNU -Privacy Guard}, for more information on GPG's @option{--keyring} option. +Here, @command{guix environment -C} creates a container and spawns a new +shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc +strace gdb} part adds the @command{strace} and @command{gdb} commands to +the container, which would may find handy while debugging. The +@option{--no-grafts} option makes sure we get the exact same +environment, with ungrafted packages (@pxref{Security Updates}, for more +info on grafts). -@item --key-download=@var{policy} -Handle missing OpenPGP keys according to @var{policy}, which may be one -of: +To get closer to a container like that used by the build daemon, we can +remove @file{/bin/sh}: -@table @code -@item always -Always download missing OpenPGP keys from the key server, and add them -to the user's GnuPG keyring. +@example +[env]# rm /bin/sh +@end example -@item never -Never try to download missing OpenPGP keys. Instead just bail out. +(Don't worry, this is harmless: this is all happening in the throw-away +container created by @command{guix environment}.) -@item interactive -When a package signed with an unknown OpenPGP key is encountered, ask -the user whether to download it or not. This is the default behavior. -@end table +The @command{strace} command is probably not in the search path, but we +can run: -@item --key-server=@var{host} -Use @var{host} as the OpenPGP key server when importing a public key. +@example +[env]# $GUIX_ENVIRONMENT/bin/strace -f -o log make check +@end example -@end table +In this way, not only you will have reproduced the environment variables +the daemon uses, you will also be running the build process in a container +similar to the one the daemon uses. -The @code{github} updater uses the -@uref{https://developer.github.com/v3/, GitHub API} to query for new -releases. When used repeatedly e.g.@: when refreshing all packages, -GitHub will eventually refuse to answer any further API requests. By -default 60 API requests per hour are allowed, and a full refresh on all -GitHub packages in Guix requires more than this. Authentication with -GitHub through the use of an API token alleviates these limits. To use -an API token, set the environment variable @code{GUIX_GITHUB_TOKEN} to a -token procured from @uref{https://github.com/settings/tokens} or -otherwise. +@node Invoking guix edit +@section Invoking @command{guix edit} -@node Invoking guix lint -@section Invoking @command{guix lint} +@cindex @command{guix edit} +@cindex package definition, editing +So many packages, so many source files! The @command{guix edit} command +facilitates the life of users and packagers by pointing their editor at +the source file containing the definition of the specified packages. +For instance: -@cindex @command{guix lint} -@cindex package, checking for errors -The @command{guix lint} command is meant to help package developers avoid -common errors and use a consistent style. It runs a number of checks on -a given set of packages in order to find common mistakes in their -definitions. Available @dfn{checkers} include (see -@code{--list-checkers} for a complete list): +@example +guix edit gcc@@4.9 vim +@end example -@table @code -@item synopsis -@itemx description -Validate certain typographical and stylistic rules about package -descriptions and synopses. +@noindent +launches the program specified in the @code{VISUAL} or in the +@code{EDITOR} environment variable to view the recipe of GCC@tie{}4.9.3 +and that of Vim. -@item inputs-should-be-native -Identify inputs that should most likely be native inputs. +If you are using a Guix Git checkout (@pxref{Building from Git}), or +have created your own packages on @code{GUIX_PACKAGE_PATH} +(@pxref{Package Modules}), you will be able to edit the package +recipes. In other cases, you will be able to examine the read-only recipes +for packages currently in the store. -@item source -@itemx home-page -@itemx mirror-url -@itemx github-url -@itemx source-file-name -Probe @code{home-page} and @code{source} URLs and report those that are -invalid. Suggest a @code{mirror://} URL when applicable. If the -@code{source} URL redirects to a GitHub URL, recommend usage of the GitHub -URL. Check that the source file name is meaningful, e.g.@: is not just a -version number or ``git-checkout'', without a declared @code{file-name} -(@pxref{origin Reference}). -@item source-unstable-tarball -Parse the @code{source} URL to determine if a tarball from GitHub is -autogenerated or if it is a release tarball. Unfortunately GitHub's -autogenerated tarballs are sometimes regenerated. +@node Invoking guix download +@section Invoking @command{guix download} -@item cve -@cindex security vulnerabilities -@cindex CVE, Common Vulnerabilities and Exposures -Report known vulnerabilities found in the Common Vulnerabilities and -Exposures (CVE) databases of the current and past year -@uref{https://nvd.nist.gov/download.cfm#CVE_FEED, published by the US -NIST}. +@cindex @command{guix download} +@cindex downloading package sources +When writing a package definition, developers typically need to download +a source tarball, compute its SHA256 hash, and write that +hash in the package definition (@pxref{Defining Packages}). The +@command{guix download} tool helps with this task: it downloads a file +from the given URI, adds it to the store, and prints both its file name +in the store and its SHA256 hash. -To view information about a particular vulnerability, visit pages such as: +The fact that the downloaded file is added to the store saves bandwidth: +when the developer eventually tries to build the newly defined package +with @command{guix build}, the source tarball will not have to be +downloaded again because it is already in the store. It is also a +convenient way to temporarily stash files, which may be deleted +eventually (@pxref{Invoking guix gc}). -@itemize -@item -@indicateurl{https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-YYYY-ABCD} -@item -@indicateurl{https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-ABCD} -@end itemize +The @command{guix download} command supports the same URIs as used in +package definitions. In particular, it supports @code{mirror://} URIs. +@code{https} URIs (HTTP over TLS) are supported @emph{provided} the +Guile bindings for GnuTLS are available in the user's environment; when +they are not available, an error is raised. @xref{Guile Preparations, +how to install the GnuTLS bindings for Guile,, gnutls-guile, +GnuTLS-Guile}, for more information. -@noindent -where @code{CVE-YYYY-ABCD} is the CVE identifier---e.g., -@code{CVE-2015-7554}. +@command{guix download} verifies HTTPS server certificates by loading +the certificates of X.509 authorities from the directory pointed to by +the @code{SSL_CERT_DIR} environment variable (@pxref{X.509 +Certificates}), unless @option{--no-check-certificate} is used. -Package developers can specify in package recipes the -@uref{https://nvd.nist.gov/cpe.cfm,Common Platform Enumeration (CPE)} -name and version of the package when they differ from the name or version -that Guix uses, as in this example: +The following options are available: -@example -(package - (name "grub") - ;; @dots{} - ;; CPE calls this package "grub2". - (properties '((cpe-name . "grub2") - (cpe-version . "2.3"))) -@end example +@table @code +@item --format=@var{fmt} +@itemx -f @var{fmt} +Write the hash in the format specified by @var{fmt}. For more +information on the valid values for @var{fmt}, @pxref{Invoking guix hash}. -@c See . -Some entries in the CVE database do not specify which version of a -package they apply to, and would thus ``stick around'' forever. Package -developers who found CVE alerts and verified they can be ignored can -declare them as in this example: +@item --no-check-certificate +Do not validate the X.509 certificates of HTTPS servers. -@example -(package - (name "t1lib") - ;; @dots{} - ;; These CVEs no longer apply and can be safely ignored. - (properties `((lint-hidden-cve . ("CVE-2011-0433" - "CVE-2011-1553" - "CVE-2011-1554" - "CVE-2011-5244"))))) -@end example +When using this option, you have @emph{absolutely no guarantee} that you +are communicating with the authentic server responsible for the given +URL, which makes you vulnerable to ``man-in-the-middle'' attacks. -@item formatting -Warn about obvious source code formatting issues: trailing white space, -use of tabulations, etc. +@item --output=@var{file} +@itemx -o @var{file} +Save the downloaded file to @var{file} instead of adding it to the +store. @end table +@node Invoking guix hash +@section Invoking @command{guix hash} + +@cindex @command{guix hash} +The @command{guix hash} command computes the SHA256 hash of a file. +It is primarily a convenience tool for anyone contributing to the +distribution: it computes the cryptographic hash of a file, which can be +used in the definition of a package (@pxref{Defining Packages}). + The general syntax is: @example -guix lint @var{options} @var{package}@dots{} +guix hash @var{option} @var{file} @end example -If no package is given on the command line, then all packages are checked. -The @var{options} may be zero or more of the following: +When @var{file} is @code{-} (a hyphen), @command{guix hash} computes the +hash of data read from standard input. @command{guix hash} has the +following options: @table @code -@item --list-checkers -@itemx -l -List and describe all the available checkers that will be run on packages -and exit. -@item --checkers -@itemx -c -Only enable the checkers specified in a comma-separated list using the -names returned by @code{--list-checkers}. +@item --format=@var{fmt} +@itemx -f @var{fmt} +Write the hash in the format specified by @var{fmt}. -@end table +Supported formats: @code{nix-base32}, @code{base32}, @code{base16} +(@code{hex} and @code{hexadecimal} can be used as well). -@node Invoking guix size -@section Invoking @command{guix size} +If the @option{--format} option is not specified, @command{guix hash} +will output the hash in @code{nix-base32}. This representation is used +in the definitions of packages. -@cindex size -@cindex package size -@cindex closure -@cindex @command{guix size} -The @command{guix size} command helps package developers profile the -disk usage of packages. It is easy to overlook the impact of an -additional dependency added to a package, or the impact of using a -single output for a package that could easily be split (@pxref{Packages -with Multiple Outputs}). Such are the typical issues that -@command{guix size} can highlight. +@item --recursive +@itemx -r +Compute the hash on @var{file} recursively. -The command can be passed one or more package specifications -such as @code{gcc@@4.8} -or @code{guile:debug}, or a file name in the store. Consider this -example: +In this case, the hash is computed on an archive containing @var{file}, +including its children if it is a directory. Some of the metadata of +@var{file} is part of the archive; for instance, when @var{file} is a +regular file, the hash is different depending on whether @var{file} is +executable or not. Metadata such as time stamps has no impact on the +hash (@pxref{Invoking guix archive}). +@c FIXME: Replace xref above with xref to an ``Archive'' section when +@c it exists. -@example -$ guix size coreutils -store item total self -/gnu/store/@dots{}-gcc-5.5.0-lib 60.4 30.1 38.1% -/gnu/store/@dots{}-glibc-2.27 30.3 28.8 36.6% -/gnu/store/@dots{}-coreutils-8.28 78.9 15.0 19.0% -/gnu/store/@dots{}-gmp-6.1.2 63.1 2.7 3.4% -/gnu/store/@dots{}-bash-static-4.4.12 1.5 1.5 1.9% -/gnu/store/@dots{}-acl-2.2.52 61.1 0.4 0.5% -/gnu/store/@dots{}-attr-2.4.47 60.6 0.2 0.3% -/gnu/store/@dots{}-libcap-2.25 60.5 0.2 0.2% -total: 78.9 MiB -@end example +@item --exclude-vcs +@itemx -x +When combined with @option{--recursive}, exclude version control system +directories (@file{.bzr}, @file{.git}, @file{.hg}, etc.) -@cindex closure -The store items listed here constitute the @dfn{transitive closure} of -Coreutils---i.e., Coreutils and all its dependencies, recursively---as -would be returned by: +@vindex git-fetch +As an example, here is how you would compute the hash of a Git checkout, +which is useful when using the @code{git-fetch} method (@pxref{origin +Reference}): @example -$ guix gc -R /gnu/store/@dots{}-coreutils-8.23 +$ git clone http://example.org/foo.git +$ cd foo +$ guix hash -rx . @end example +@end table -Here the output shows three columns next to store items. The first column, -labeled ``total'', shows the size in mebibytes (MiB) of the closure of -the store item---that is, its own size plus the size of all its -dependencies. The next column, labeled ``self'', shows the size of the -item itself. The last column shows the ratio of the size of the item -itself to the space occupied by all the items listed here. - -In this example, we see that the closure of Coreutils weighs in at -79@tie{}MiB, most of which is taken by libc and GCC's run-time support -libraries. (That libc and GCC's libraries represent a large fraction of -the closure is not a problem @i{per se} because they are always available -on the system anyway.) - -When the package(s) passed to @command{guix size} are available in the -store@footnote{More precisely, @command{guix size} looks for the -@emph{ungrafted} variant of the given package(s), as returned by -@code{guix build @var{package} --no-grafts}. @xref{Security Updates}, -for information on grafts.}, @command{guix size} queries the daemon to determine its -dependencies, and measures its size in the store, similar to @command{du --ms --apparent-size} (@pxref{du invocation,,, coreutils, GNU -Coreutils}). +@node Invoking guix import +@section Invoking @command{guix import} -When the given packages are @emph{not} in the store, @command{guix size} -reports information based on the available substitutes -(@pxref{Substitutes}). This makes it possible it to profile disk usage of -store items that are not even on disk, only available remotely. +@cindex importing packages +@cindex package import +@cindex package conversion +@cindex Invoking @command{guix import} +The @command{guix import} command is useful for people who would like to +add a package to the distribution with as little work as +possible---a legitimate demand. The command knows of a few +repositories from which it can ``import'' package metadata. The result +is a package definition, or a template thereof, in the format we know +(@pxref{Defining Packages}). -You can also specify several package names: +The general syntax is: @example -$ guix size coreutils grep sed bash -store item total self -/gnu/store/@dots{}-coreutils-8.24 77.8 13.8 13.4% -/gnu/store/@dots{}-grep-2.22 73.1 0.8 0.8% -/gnu/store/@dots{}-bash-4.3.42 72.3 4.7 4.6% -/gnu/store/@dots{}-readline-6.3 67.6 1.2 1.2% -@dots{} -total: 102.3 MiB +guix import @var{importer} @var{options}@dots{} @end example -@noindent -In this example we see that the combination of the four packages takes -102.3@tie{}MiB in total, which is much less than the sum of each closure -since they have a lot of dependencies in common. +@var{importer} specifies the source from which to import package +metadata, and @var{options} specifies a package identifier and other +options specific to @var{importer}. Currently, the available +``importers'' are: -The available options are: +@table @code +@item gnu +Import metadata for the given GNU package. This provides a template +for the latest version of that GNU package, including the hash of its +source tarball, and its canonical synopsis and description. -@table @option +Additional information such as the package dependencies and its +license needs to be figured out manually. -@item --substitute-urls=@var{urls} -Use substitute information from @var{urls}. -@xref{client-substitute-urls, the same option for @code{guix build}}. +For example, the following command returns a package definition for +GNU@tie{}Hello: -@item --sort=@var{key} -Sort lines according to @var{key}, one of the following options: +@example +guix import gnu hello +@end example + +Specific command-line options are: @table @code -@item self -the size of each item (the default); -@item closure -the total size of the item's closure. +@item --key-download=@var{policy} +As for @code{guix refresh}, specify the policy to handle missing OpenPGP +keys when verifying the package signature. @xref{Invoking guix +refresh, @code{--key-download}}. @end table -@item --map-file=@var{file} -Write a graphical map of disk usage in PNG format to @var{file}. - -For the example above, the map looks like this: - -@image{images/coreutils-size-map,5in,, map of Coreutils disk usage -produced by @command{guix size}} +@item pypi +@cindex pypi +Import metadata from the @uref{https://pypi.python.org/, Python Package +Index}. Information is taken from the JSON-formatted description +available at @code{pypi.python.org} and usually includes all the relevant +information, including package dependencies. For maximum efficiency, it +is recommended to install the @command{unzip} utility, so that the +importer can unzip Python wheels and gather data from them. -This option requires that -@uref{http://wingolog.org/software/guile-charting/, Guile-Charting} be -installed and visible in Guile's module search path. When that is not -the case, @command{guix size} fails as it tries to load it. +The command below imports metadata for the @code{itsdangerous} Python +package: -@item --system=@var{system} -@itemx -s @var{system} -Consider packages for @var{system}---e.g., @code{x86_64-linux}. +@example +guix import pypi itsdangerous +@end example +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. @end table -@node Invoking guix graph -@section Invoking @command{guix graph} - -@cindex DAG -@cindex @command{guix graph} -@cindex package dependencies -Packages and their dependencies form a @dfn{graph}, specifically a -directed acyclic graph (DAG). It can quickly become difficult to have a -mental model of the package DAG, so the @command{guix graph} command -provides a visual representation of the DAG. By default, -@command{guix graph} emits a DAG representation in the input format of -@uref{http://www.graphviz.org/, Graphviz}, so its output can be passed -directly to the @command{dot} command of Graphviz. It can also emit an -HTML page with embedded JavaScript code to display a ``chord diagram'' -in a Web browser, using the @uref{https://d3js.org/, d3.js} library, or -emit Cypher queries to construct a graph in a graph database supporting -the @uref{http://www.opencypher.org/, openCypher} query language. -The general syntax is: - -@example -guix graph @var{options} @var{package}@dots{} -@end example +@item gem +@cindex gem +Import metadata from @uref{https://rubygems.org/, RubyGems}. Information +is taken from the JSON-formatted description available at +@code{rubygems.org} and includes most relevant information, including +runtime dependencies. There are some caveats, however. The metadata +doesn't distinguish between synopses and descriptions, so the same string +is used for both fields. Additionally, the details of non-Ruby +dependencies required to build native extensions is unavailable and left +as an exercise to the packager. -For example, the following command generates a PDF file representing the -package DAG for the GNU@tie{}Core Utilities, showing its build-time -dependencies: +The command below imports metadata for the @code{rails} Ruby package: @example -guix graph coreutils | dot -Tpdf > dag.pdf +guix import gem rails @end example -The output looks like this: +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table -@image{images/coreutils-graph,2in,,Dependency graph of the GNU Coreutils} +@item cpan +@cindex CPAN +Import metadata from @uref{https://www.metacpan.org/, MetaCPAN}. +Information is taken from the JSON-formatted metadata provided through +@uref{https://fastapi.metacpan.org/, MetaCPAN's API} and includes most +relevant information, such as module dependencies. License information +should be checked closely. If Perl is available in the store, then the +@code{corelist} utility will be used to filter core modules out of the +list of dependencies. -Nice little graph, no? +The command command below imports metadata for the @code{Acme::Boolean} +Perl module: -But there is more than one graph! The one above is concise: it is the -graph of package objects, omitting implicit inputs such as GCC, libc, -grep, etc. It is often useful to have such a concise graph, but -sometimes one may want to see more details. @command{guix graph} supports -several types of graphs, allowing you to choose the level of detail: +@example +guix import cpan Acme::Boolean +@end example -@table @code -@item package -This is the default type used in the example above. It shows the DAG of -package objects, excluding implicit dependencies. It is concise, but -filters out many details. +@item cran +@cindex CRAN +@cindex Bioconductor +Import metadata from @uref{https://cran.r-project.org/, CRAN}, the +central repository for the @uref{http://r-project.org, GNU@tie{}R +statistical and graphical environment}. -@item reverse-package -This shows the @emph{reverse} DAG of packages. For example: +Information is extracted from the @code{DESCRIPTION} file of the package. + +The command command below imports metadata for the @code{Cairo} +R package: @example -guix graph --type=reverse-package ocaml +guix import cran Cairo @end example -...@: yields the graph of packages that depend on OCaml. +When @code{--recursive} is added, the importer will traverse the +dependency graph of the given upstream package recursively and generate +package expressions for all those packages that are not yet in Guix. -Note that for core packages this can yield huge graphs. If all you want -is to know the number of packages that depend on a given package, use -@command{guix refresh --list-dependent} (@pxref{Invoking guix refresh, -@option{--list-dependent}}). +When @code{--archive=bioconductor} is added, metadata is imported from +@uref{https://www.bioconductor.org/, Bioconductor}, a repository of R +packages for for the analysis and comprehension of high-throughput +genomic data in bioinformatics. -@item bag-emerged -This is the package DAG, @emph{including} implicit inputs. +Information is extracted from the @code{DESCRIPTION} file of a package +published on the web interface of the Bioconductor SVN repository. -For instance, the following command: +The command below imports metadata for the @code{GenomicRanges} +R package: @example -guix graph --type=bag-emerged coreutils | dot -Tpdf > dag.pdf +guix import cran --archive=bioconductor GenomicRanges @end example -...@: yields this bigger graph: +@item texlive +@cindex TeX Live +@cindex CTAN +Import metadata from @uref{http://www.ctan.org/, CTAN}, the +comprehensive TeX archive network for TeX packages that are part of the +@uref{https://www.tug.org/texlive/, TeX Live distribution}. -@image{images/coreutils-bag-graph,,5in,Detailed dependency graph of the GNU Coreutils} +Information about the package is obtained through the XML API provided +by CTAN, while the source code is downloaded from the SVN repository of +the Tex Live project. This is done because the CTAN does not keep +versioned archives. -At the bottom of the graph, we see all the implicit inputs of -@var{gnu-build-system} (@pxref{Build Systems, @code{gnu-build-system}}). +The command command below imports metadata for the @code{fontspec} +TeX package: -Now, note that the dependencies of these implicit inputs---that is, the -@dfn{bootstrap dependencies} (@pxref{Bootstrapping})---are not shown -here, for conciseness. +@example +guix import texlive fontspec +@end example -@item bag -Similar to @code{bag-emerged}, but this time including all the bootstrap -dependencies. +When @code{--archive=DIRECTORY} is added, the source code is downloaded +not from the @file{latex} sub-directory of the @file{texmf-dist/source} +tree in the TeX Live SVN repository, but from the specified sibling +directory under the same root. -@item bag-with-origins -Similar to @code{bag}, but also showing origins and their dependencies. +The command below imports metadata for the @code{ifxetex} package from +CTAN while fetching the sources from the directory +@file{texmf/source/generic}: -@item derivation -This is the most detailed representation: It shows the DAG of -derivations (@pxref{Derivations}) and plain store items. Compared to -the above representation, many additional nodes are visible, including -build scripts, patches, Guile modules, etc. +@example +guix import texlive --archive=generic ifxetex +@end example -For this type of graph, it is also possible to pass a @file{.drv} file -name instead of a package name, as in: +@item json +@cindex JSON, import +Import package metadata from a local JSON file. Consider the following +example package definition in JSON format: @example -guix graph -t derivation `guix system build -d my-config.scm` +@{ + "name": "hello", + "version": "2.10", + "source": "mirror://gnu/hello/hello-2.10.tar.gz", + "build-system": "gnu", + "home-page": "https://www.gnu.org/software/hello/", + "synopsis": "Hello, GNU world: An example GNU package", + "description": "GNU Hello prints a greeting.", + "license": "GPL-3.0+", + "native-inputs": ["gcc@@6"] +@} @end example -@item module -This is the graph of @dfn{package modules} (@pxref{Package Modules}). -For example, the following command shows the graph for the package -module that defines the @code{guile} package: +The field names are the same as for the @code{} record +(@xref{Defining Packages}). References to other packages are provided +as JSON lists of quoted package specification strings such as +@code{guile} or @code{guile@@2.0}. + +The importer also supports a more explicit source definition using the +common fields for @code{} records: @example -guix graph -t module guile | dot -Tpdf > module-graph.pdf +@{ + @dots{} + "source": @{ + "method": "url-fetch", + "uri": "mirror://gnu/hello/hello-2.10.tar.gz", + "sha256": @{ + "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" + @} + @} + @dots{} +@} @end example -@end table -All the types above correspond to @emph{build-time dependencies}. The -following graph type represents the @emph{run-time dependencies}: +The command below reads metadata from the JSON file @code{hello.json} +and outputs a package expression: -@table @code -@item references -This is the graph of @dfn{references} of a package output, as returned -by @command{guix gc --references} (@pxref{Invoking guix gc}). +@example +guix import json hello.json +@end example -If the given package output is not available in the store, @command{guix -graph} attempts to obtain dependency information from substitutes. +@item nix +Import metadata from a local copy of the source of the +@uref{http://nixos.org/nixpkgs/, Nixpkgs distribution}@footnote{This +relies on the @command{nix-instantiate} command of +@uref{http://nixos.org/nix/, Nix}.}. Package definitions in Nixpkgs are +typically written in a mixture of Nix-language and Bash code. This +command only imports the high-level package structure that is written in +the Nix language. It normally includes all the basic fields of a +package definition. -Here you can also pass a store file name instead of a package name. For -example, the command below produces the reference graph of your profile -(which can be big!): +When importing a GNU package, the synopsis and descriptions are replaced +by their canonical upstream variant. + +Usually, you will first need to do: @example -guix graph -t references `readlink -f ~/.guix-profile` +export NIX_REMOTE=daemon @end example -@item referrers -This is the graph of the @dfn{referrers} of a store item, as returned by -@command{guix gc --referrers} (@pxref{Invoking guix gc}). +@noindent +so that @command{nix-instantiate} does not try to open the Nix database. -This relies exclusively on local information from your store. For -instance, let us suppose that the current Inkscape is available in 10 -profiles on your machine; @command{guix graph -t referrers inkscape} -will show a graph rooted at Inkscape and with those 10 profiles linked -to it. +As an example, the command below imports the package definition of +LibreOffice (more precisely, it imports the definition of the package +bound to the @code{libreoffice} top-level attribute): -It can help determine what is preventing a store item from being garbage -collected. +@example +guix import nix ~/path/to/nixpkgs libreoffice +@end example + +@item hackage +@cindex hackage +Import metadata from the Haskell community's central package archive +@uref{https://hackage.haskell.org/, Hackage}. Information is taken from +Cabal files and includes all the relevant information, including package +dependencies. +Specific command-line options are: + +@table @code +@item --stdin +@itemx -s +Read a Cabal file from standard input. +@item --no-test-dependencies +@itemx -t +Do not include dependencies required only by the test suites. +@item --cabal-environment=@var{alist} +@itemx -e @var{alist} +@var{alist} is a Scheme alist defining the environment in which the +Cabal conditionals are evaluated. The accepted keys are: @code{os}, +@code{arch}, @code{impl} and a string representing the name of a flag. +The value associated with a flag has to be either the symbol +@code{true} or @code{false}. The value associated with other keys +has to conform to the Cabal file format definition. The default value +associated with the keys @code{os}, @code{arch} and @code{impl} is +@samp{linux}, @samp{x86_64} and @samp{ghc}, respectively. +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. @end table -The available options are the following: +The command below imports metadata for the latest version of the +@code{HTTP} Haskell package without including test dependencies and +specifying the value of the flag @samp{network-uri} as @code{false}: -@table @option -@item --type=@var{type} -@itemx -t @var{type} -Produce a graph output of @var{type}, where @var{type} must be one of -the values listed above. +@example +guix import hackage -t -e "'((\"network-uri\" . false))" HTTP +@end example -@item --list-types -List the supported graph types. +A specific package version may optionally be specified by following the +package name by an at-sign and a version number as in the following example: -@item --backend=@var{backend} -@itemx -b @var{backend} -Produce a graph using the selected @var{backend}. +@example +guix import hackage mtl@@2.1.3.1 +@end example -@item --list-backends -List the supported graph backends. +@item stackage +@cindex stackage +The @code{stackage} importer is a wrapper around the @code{hackage} one. +It takes a package name, looks up the package version included in a +long-term support (LTS) @uref{https://www.stackage.org, Stackage} +release and uses the @code{hackage} importer to retrieve its metadata. +Note that it is up to you to select an LTS release compatible with the +GHC compiler used by Guix. -Currently, the available backends are Graphviz and d3.js. +Specific command-line options are: -@item --expression=@var{expr} -@itemx -e @var{expr} -Consider the package @var{expr} evaluates to. +@table @code +@item --no-test-dependencies +@itemx -t +Do not include dependencies required only by the test suites. +@item --lts-version=@var{version} +@itemx -l @var{version} +@var{version} is the desired LTS release version. If omitted the latest +release is used. +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table -This is useful to precisely refer to a package, as in this example: +The command below imports metadata for the @code{HTTP} Haskell package +included in the LTS Stackage release version 7.18: @example -guix graph -e '(@@@@ (gnu packages commencement) gnu-make-final)' +guix import stackage --lts-version=7.18 HTTP @end example -@item --system=@var{system} -@itemx -s @var{system} -Display the graph for @var{system}---e.g., @code{i686-linux}. - -The package dependency graph is largely architecture-independent, but there -are some architecture-dependent bits that this option allows you to visualize. -@end table +@item elpa +@cindex elpa +Import metadata from an Emacs Lisp Package Archive (ELPA) package +repository (@pxref{Packages,,, emacs, The GNU Emacs Manual}). +Specific command-line options are: -@node Invoking guix environment -@section Invoking @command{guix environment} +@table @code +@item --archive=@var{repo} +@itemx -a @var{repo} +@var{repo} identifies the archive repository from which to retrieve the +information. Currently the supported repositories and their identifiers +are: +@itemize - +@item +@uref{http://elpa.gnu.org/packages, GNU}, selected by the @code{gnu} +identifier. This is the default. -@cindex reproducible build environments -@cindex development environments -@cindex @command{guix environment} -@cindex environment, package build environment -The purpose of @command{guix environment} is to assist hackers in -creating reproducible development environments without polluting their -package profile. The @command{guix environment} tool takes one or more -packages, builds all of their inputs, and creates a shell -environment to use them. +Packages from @code{elpa.gnu.org} are signed with one of the keys +contained in the GnuPG keyring at +@file{share/emacs/25.1/etc/package-keyring.gpg} (or similar) in the +@code{emacs} package (@pxref{Package Installation, ELPA package +signatures,, emacs, The GNU Emacs Manual}). -The general syntax is: +@item +@uref{http://stable.melpa.org/packages, MELPA-Stable}, selected by the +@code{melpa-stable} identifier. -@example -guix environment @var{options} @var{package}@dots{} -@end example +@item +@uref{http://melpa.org/packages, MELPA}, selected by the @code{melpa} +identifier. +@end itemize -The following example spawns a new shell set up for the development of -GNU@tie{}Guile: +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table -@example -guix environment guile -@end example +@item crate +@cindex crate +Import metadata from the crates.io Rust package repository +@uref{https://crates.io, crates.io}. -If the needed dependencies are not built yet, @command{guix environment} -automatically builds them. The environment of the new shell is an augmented -version of the environment that @command{guix environment} was run in. -It contains the necessary search paths for building the given package -added to the existing environment variables. To create a ``pure'' -environment, in which the original environment variables have been unset, -use the @code{--pure} option@footnote{Users sometimes wrongfully augment -environment variables such as @code{PATH} in their @file{~/.bashrc} -file. As a consequence, when @code{guix environment} launches it, Bash -may read @file{~/.bashrc}, thereby introducing ``impurities'' in these -environment variables. It is an error to define such environment -variables in @file{.bashrc}; instead, they should be defined in -@file{.bash_profile}, which is sourced only by log-in shells. -@xref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}, for -details on Bash start-up files.}. +@item opam +@cindex OPAM +@cindex OCaml +Import metadata from the @uref{https://opam.ocaml.org/, OPAM} package +repository used by the OCaml community. +@end table -@vindex GUIX_ENVIRONMENT -@command{guix environment} defines the @code{GUIX_ENVIRONMENT} -variable in the shell it spawns; its value is the file name of the -profile of this environment. This allows users to, say, define a -specific prompt for development environments in their @file{.bashrc} -(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}): +The structure of the @command{guix import} code is modular. It would be +useful to have more importers for other package formats, and your help +is welcome here (@pxref{Contributing}). -@example -if [ -n "$GUIX_ENVIRONMENT" ] -then - export PS1="\u@@\h \w [dev]\$ " -fi -@end example +@node Invoking guix refresh +@section Invoking @command{guix refresh} -@noindent -...@: or to browse the profile: +@cindex @command {guix refresh} +The primary audience of the @command{guix refresh} command is developers +of the GNU software distribution. By default, it reports any packages +provided by the distribution that are outdated compared to the latest +upstream version, like this: @example -$ ls "$GUIX_ENVIRONMENT/bin" +$ guix refresh +gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1 +gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0 @end example -Additionally, more than one package may be specified, in which case the -union of the inputs for the given packages are used. For example, the -command below spawns a shell where all of the dependencies of both Guile -and Emacs are available: +Alternately, one can specify packages to consider, in which case a +warning is emitted for packages that lack an updater: @example -guix environment guile emacs +$ guix refresh coreutils guile guile-ssh +gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh +gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13 @end example -Sometimes an interactive shell session is not desired. An arbitrary -command may be invoked by placing the @code{--} token to separate the -command from the rest of the arguments: +@command{guix refresh} browses the upstream repository of each package and determines +the highest version number of the releases therein. The command +knows how to update specific types of packages: GNU packages, ELPA +packages, etc.---see the documentation for @option{--type} below. There +are many packages, though, for which it lacks a method to determine +whether a new upstream release is available. However, the mechanism is +extensible, so feel free to get in touch with us to add a new method! -@example -guix environment guile -- make -j4 -@end example +@table @code -In other situations, it is more convenient to specify the list of -packages needed in the environment. For example, the following command -runs @command{python} from an environment containing Python@tie{}2.7 and -NumPy: +@item --recursive +Consider the packages specified, and all the packages upon which they depend. @example -guix environment --ad-hoc python2-numpy python-2.7 -- python +$ guix refresh --recursive coreutils +gnu/packages/acl.scm:35:2: warning: no updater for acl +gnu/packages/m4.scm:30:12: info: 1.4.18 is already the latest version of m4 +gnu/packages/xml.scm:68:2: warning: no updater for expat +gnu/packages/multiprecision.scm:40:12: info: 6.1.2 is already the latest version of gmp +@dots{} @end example -Furthermore, one might want the dependencies of a package and also some -additional packages that are not build-time or runtime dependencies, but -are useful when developing nonetheless. Because of this, the -@code{--ad-hoc} flag is positional. Packages appearing before -@code{--ad-hoc} are interpreted as packages whose dependencies will be -added to the environment. Packages appearing after are interpreted as -packages that will be added to the environment directly. For example, -the following command creates a Guix development environment that -additionally includes Git and strace: - -@example -guix environment guix --ad-hoc git strace -@end example +@end table -Sometimes it is desirable to isolate the environment as much as -possible, for maximal purity and reproducibility. In particular, when -using Guix on a host distro that is not GuixSD, it is desirable to -prevent access to @file{/usr/bin} and other system-wide resources from -the development environment. For example, the following command spawns -a Guile REPL in a ``container'' where only the store and the current -working directory are mounted: +Sometimes the upstream name differs from the package name used in Guix, +and @command{guix refresh} needs a little help. Most updaters honor the +@code{upstream-name} property in package definitions, which can be used +to that effect: @example -guix environment --ad-hoc --container guile -- guile +(define-public network-manager + (package + (name "network-manager") + ;; @dots{} + (properties '((upstream-name . "NetworkManager"))))) @end example -@quotation Note -The @code{--container} option requires Linux-libre 3.19 or newer. -@end quotation - -The available options are summarized below. - -@table @code -@item --root=@var{file} -@itemx -r @var{file} -@cindex persistent environment -@cindex garbage collector root, for environments -Make @var{file} a symlink to the profile for this environment, and -register it as a garbage collector root. +When passed @code{--update}, it modifies distribution source files to +update the version numbers and source tarball hashes of those package +recipes (@pxref{Defining Packages}). This is achieved by downloading +each package's latest source tarball and its associated OpenPGP +signature, authenticating the downloaded tarball against its signature +using @command{gpg}, and finally computing its hash. When the public +key used to sign the tarball is missing from the user's keyring, an +attempt is made to automatically retrieve it from a public key server; +when this is successful, the key is added to the user's keyring; otherwise, +@command{guix refresh} reports an error. -This is useful if you want to protect your environment from garbage -collection, to make it ``persistent''. +The following options are supported: -When this option is omitted, the environment is protected from garbage -collection only for the duration of the @command{guix environment} -session. This means that next time you recreate the same environment, -you could have to rebuild or re-download packages. @xref{Invoking guix -gc}, for more on GC roots. +@table @code @item --expression=@var{expr} @itemx -e @var{expr} -Create an environment for the package or list of packages that -@var{expr} evaluates to. +Consider the package @var{expr} evaluates to. -For example, running: +This is useful to precisely refer to a package, as in this example: @example -guix environment -e '(@@ (gnu packages maths) petsc-openmpi)' +guix refresh -l -e '(@@@@ (gnu packages commencement) glibc-final)' @end example -starts a shell with the environment for this specific variant of the -PETSc package. +This command lists the dependents of the ``final'' libc (essentially all +the packages.) -Running: +@item --update +@itemx -u +Update distribution source files (package recipes) in place. This is +usually run from a checkout of the Guix source tree (@pxref{Running +Guix Before It Is Installed}): @example -guix environment --ad-hoc -e '(@@ (gnu) %base-packages)' +$ ./pre-inst-env guix refresh -s non-core -u @end example -starts a shell with all the GuixSD base packages available. - -The above commands only use the default output of the given packages. -To select other outputs, two element tuples can be specified: - -@example -guix environment --ad-hoc -e '(list (@@ (gnu packages bash) bash) "include")' -@end example +@xref{Defining Packages}, for more information on package definitions. -@item --load=@var{file} -@itemx -l @var{file} -Create an environment for the package or list of packages that the code -within @var{file} evaluates to. +@item --select=[@var{subset}] +@itemx -s @var{subset} +Select all the packages in @var{subset}, one of @code{core} or +@code{non-core}. -As an example, @var{file} might contain a definition like this -(@pxref{Defining Packages}): +The @code{core} subset refers to all the packages at the core of the +distribution---i.e., packages that are used to build ``everything +else''. This includes GCC, libc, Binutils, Bash, etc. Usually, +changing one of these packages in the distribution entails a rebuild of +all the others. Thus, such updates are an inconvenience to users in +terms of build time or bandwidth used to achieve the upgrade. -@example -@verbatiminclude environment-gdb.scm -@end example +The @code{non-core} subset refers to the remaining packages. It is +typically useful in cases where an update of the core packages would be +inconvenient. @item --manifest=@var{file} @itemx -m @var{file} -Create an environment for the packages contained in the manifest object -returned by the Scheme code in @var{file}. +Select all the packages from the manifest in @var{file}. This is useful to +check if any packages of the user manifest can be updated. -This is similar to the same-named option in @command{guix package} -(@pxref{profile-manifest, @option{--manifest}}) and uses the same -manifest files. +@item --type=@var{updater} +@itemx -t @var{updater} +Select only packages handled by @var{updater} (may be a comma-separated +list of updaters). Currently, @var{updater} may be one of: -@item --ad-hoc -Include all specified packages in the resulting environment, as if an -@i{ad hoc} package were defined with them as inputs. This option is -useful for quickly creating an environment without having to write a -package expression to contain the desired inputs. +@table @code +@item gnu +the updater for GNU packages; +@item gnome +the updater for GNOME packages; +@item kde +the updater for KDE packages; +@item xorg +the updater for X.org packages; +@item kernel.org +the updater for packages hosted on kernel.org; +@item elpa +the updater for @uref{http://elpa.gnu.org/, ELPA} packages; +@item cran +the updater for @uref{https://cran.r-project.org/, CRAN} packages; +@item bioconductor +the updater for @uref{https://www.bioconductor.org/, Bioconductor} R packages; +@item cpan +the updater for @uref{http://www.cpan.org/, CPAN} packages; +@item pypi +the updater for @uref{https://pypi.python.org, PyPI} packages. +@item gem +the updater for @uref{https://rubygems.org, RubyGems} packages. +@item github +the updater for @uref{https://github.com, GitHub} packages. +@item hackage +the updater for @uref{https://hackage.haskell.org, Hackage} packages. +@item stackage +the updater for @uref{https://www.stackage.org, Stackage} packages. +@item crate +the updater for @uref{https://crates.io, Crates} packages. +@end table -For instance, the command: +For instance, the following command only checks for updates of Emacs +packages hosted at @code{elpa.gnu.org} and for updates of CRAN packages: @example -guix environment --ad-hoc guile guile-sdl -- guile +$ guix refresh --type=elpa,cran +gnu/packages/statistics.scm:819:13: r-testthat would be upgraded from 0.10.0 to 0.11.0 +gnu/packages/emacs.scm:856:13: emacs-auctex would be upgraded from 11.88.6 to 11.88.9 @end example -runs @command{guile} in an environment where Guile and Guile-SDL are -available. +@end table -Note that this example implicitly asks for the default output of -@code{guile} and @code{guile-sdl}, but it is possible to ask for a -specific output---e.g., @code{glib:bin} asks for the @code{bin} output -of @code{glib} (@pxref{Packages with Multiple Outputs}). +In addition, @command{guix refresh} can be passed one or more package +names, as in this example: -This option may be composed with the default behavior of @command{guix -environment}. Packages appearing before @code{--ad-hoc} are interpreted -as packages whose dependencies will be added to the environment, the -default behavior. Packages appearing after are interpreted as packages -that will be added to the environment directly. +@example +$ ./pre-inst-env guix refresh -u emacs idutils gcc@@4.8 +@end example -@item --pure -Unset existing environment variables when building the new environment. -This has the effect of creating an environment in which search paths -only contain package inputs. +@noindent +The command above specifically updates the @code{emacs} and +@code{idutils} packages. The @code{--select} option would have no +effect in this case. -@item --search-paths -Display the environment variable definitions that make up the -environment. +When considering whether to upgrade a package, it is sometimes +convenient to know which packages would be affected by the upgrade and +should be checked for compatibility. For this the following option may +be used when passing @command{guix refresh} one or more package names: -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}. +@table @code -@item --container -@itemx -C -@cindex container -Run @var{command} within an isolated container. The current working -directory outside the container is mapped inside the container. -Additionally, unless overridden with @code{--user}, a dummy home -directory is created that matches the current user's home directory, and -@file{/etc/passwd} is configured accordingly. The spawned process runs -as the current user outside the container, but has root privileges in -the context of the container. +@item --list-updaters +@itemx -L +List available updaters and exit (see @option{--type} above.) -@item --network -@itemx -N -For containers, share the network namespace with the host system. -Containers created without this flag only have access to the loopback -device. +For each updater, display the fraction of packages it covers; at the +end, display the fraction of packages covered by all these updaters. -@item --link-profile -@itemx -P -For containers, link the environment profile to -@file{~/.guix-profile} within the container. This is equivalent to -running the command @command{ln -s $GUIX_ENVIRONMENT ~/.guix-profile} -within the container. Linking will fail and abort the environment if -the directory already exists, which will certainly be the case if -@command{guix environment} was invoked in the user's home directory. +@item --list-dependent +@itemx -l +List top-level dependent packages that would need to be rebuilt as a +result of upgrading one or more packages. -Certain packages are configured to look in -@code{~/.guix-profile} for configuration files and data;@footnote{For -example, the @code{fontconfig} package inspects -@file{~/.guix-profile/share/fonts} for additional fonts.} -@code{--link-profile} allows these programs to behave as expected within -the environment. +@xref{Invoking guix graph, the @code{reverse-package} type of +@command{guix graph}}, for information on how to visualize the list of +dependents of a package. -@item --user=@var{user} -@itemx -u @var{user} -For containers, use the username @var{user} in place of the current -user. The generated @file{/etc/passwd} entry within the container will -contain the name @var{user}; the home directory will be -@file{/home/USER}; and no user GECOS data will be copied. @var{user} -need not exist on the system. +@end table -Additionally, any shared or exposed path (see @code{--share} and -@code{--expose} respectively) whose target is within the current user's -home directory will be remapped relative to @file{/home/USER}; this -includes the automatic mapping of the current working directory. +Be aware that the @code{--list-dependent} option only +@emph{approximates} the rebuilds that would be required as a result of +an upgrade. More rebuilds might be required under some circumstances. @example -# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target -cd $HOME/wd -guix environment --container --user=foo \ - --expose=$HOME/test \ - --expose=/tmp/target=$HOME/target +$ guix refresh --list-dependent flex +Building the following 120 packages would ensure 213 dependent packages are rebuilt: +hop@@2.4.0 geiser@@0.4 notmuch@@0.18 mu@@0.9.9.5 cflow@@1.4 idutils@@4.6 @dots{} @end example -While this will limit the leaking of user identity through home paths -and each of the user fields, this is only one useful component of a -broader privacy/anonymity solution---not one in and of itself. +The command above lists a set of packages that could be built to check +for compatibility with an upgraded @code{flex} package. -@item --expose=@var{source}[=@var{target}] -For containers, expose the file system @var{source} from the host system -as the read-only file system @var{target} within the container. If -@var{target} is not specified, @var{source} is used as the target mount -point in the container. +@table @code + +@item --list-transitive +List all the packages which one or more packages depend upon. + +@example +$ guix refresh --list-transitive flex +flex@@2.6.4 depends on the following 25 packages: perl@@5.28.0 help2man@@1.47.6 +bison@@3.0.5 indent@@2.2.10 tar@@1.30 gzip@@1.9 bzip2@@1.0.6 xz@@5.2.4 file@@5.33 @dots{} +@end example + +@end table + +The command above lists a set of packages which, when changed, would cause +@code{flex} to be rebuilt. + +The following options can be used to customize GnuPG operation: + +@table @code + +@item --gpg=@var{command} +Use @var{command} as the GnuPG 2.x command. @var{command} is searched +for in @code{$PATH}. + +@item --keyring=@var{file} +Use @var{file} as the keyring for upstream keys. @var{file} must be in the +@dfn{keybox format}. Keybox files usually have a name ending in @file{.kbx} +and the GNU@tie{}Privacy Guard (GPG) can manipulate these files +(@pxref{kbxutil, @command{kbxutil},, gnupg, Using the GNU Privacy Guard}, for +information on a tool to manipulate keybox files). + +When this option is omitted, @command{guix refresh} uses +@file{~/.config/guix/upstream/trustedkeys.kbx} as the keyring for upstream +signing keys. OpenPGP signatures are checked against keys from this keyring; +missing keys are downloaded to this keyring as well (see +@option{--key-download} below.) -The example below spawns a Guile REPL in a container in which the user's -home directory is accessible read-only via the @file{/exchange} -directory: +You can export keys from your default GPG keyring into a keybox file using +commands like this one: @example -guix environment --container --expose=$HOME=/exchange --ad-hoc guile -- guile +gpg --export rms@@gnu.org | kbxutil --import-openpgp >> mykeyring.kbx @end example -@item --share=@var{source}[=@var{target}] -For containers, share the file system @var{source} from the host system -as the writable file system @var{target} within the container. If -@var{target} is not specified, @var{source} is used as the target mount -point in the container. - -The example below spawns a Guile REPL in a container in which the user's -home directory is accessible for both reading and writing via the -@file{/exchange} directory: +Likewise, you can fetch keys to a specific keybox file like this: @example -guix environment --container --share=$HOME=/exchange --ad-hoc guile -- guile +gpg --no-default-keyring --keyring mykeyring.kbx \ + --recv-keys @value{OPENPGP-SIGNING-KEY-ID} @end example -@end table - -@command{guix environment} -also supports all of the common build options that @command{guix -build} supports (@pxref{Common Build Options}) as well as package -transformation options (@pxref{Package Transformation Options}). +@ref{GPG Configuration Options, @option{--keyring},, gnupg, Using the GNU +Privacy Guard}, for more information on GPG's @option{--keyring} option. -@node Invoking guix publish -@section Invoking @command{guix publish} +@item --key-download=@var{policy} +Handle missing OpenPGP keys according to @var{policy}, which may be one +of: -@cindex @command{guix publish} -The purpose of @command{guix publish} is to enable users to easily share -their store with others, who can then use it as a substitute server -(@pxref{Substitutes}). +@table @code +@item always +Always download missing OpenPGP keys from the key server, and add them +to the user's GnuPG keyring. -When @command{guix publish} runs, it spawns an HTTP server which allows -anyone with network access to obtain substitutes from it. This means -that any machine running Guix can also act as if it were a build farm, -since the HTTP interface is compatible with Hydra, the software behind -the @code{@value{SUBSTITUTE-SERVER}} build farm. +@item never +Never try to download missing OpenPGP keys. Instead just bail out. -For security, each substitute is signed, allowing recipients to check -their authenticity and integrity (@pxref{Substitutes}). Because -@command{guix publish} uses the signing key of the system, which is only -readable by the system administrator, it must be started as root; the -@code{--user} option makes it drop root privileges early on. +@item interactive +When a package signed with an unknown OpenPGP key is encountered, ask +the user whether to download it or not. This is the default behavior. +@end table -The signing key pair must be generated before @command{guix publish} is -launched, using @command{guix archive --generate-key} (@pxref{Invoking -guix archive}). +@item --key-server=@var{host} +Use @var{host} as the OpenPGP key server when importing a public key. -The general syntax is: +@end table -@example -guix publish @var{options}@dots{} -@end example +The @code{github} updater uses the +@uref{https://developer.github.com/v3/, GitHub API} to query for new +releases. When used repeatedly e.g.@: when refreshing all packages, +GitHub will eventually refuse to answer any further API requests. By +default 60 API requests per hour are allowed, and a full refresh on all +GitHub packages in Guix requires more than this. Authentication with +GitHub through the use of an API token alleviates these limits. To use +an API token, set the environment variable @code{GUIX_GITHUB_TOKEN} to a +token procured from @uref{https://github.com/settings/tokens} or +otherwise. -Running @command{guix publish} without any additional arguments will -spawn an HTTP server on port 8080: -@example -guix publish -@end example +@node Invoking guix lint +@section Invoking @command{guix lint} -Once a publishing server has been authorized (@pxref{Invoking guix -archive}), the daemon may download substitutes from it: +@cindex @command{guix lint} +@cindex package, checking for errors +The @command{guix lint} command is meant to help package developers avoid +common errors and use a consistent style. It runs a number of checks on +a given set of packages in order to find common mistakes in their +definitions. Available @dfn{checkers} include (see +@code{--list-checkers} for a complete list): -@example -guix-daemon --substitute-urls=http://example.org:8080 -@end example +@table @code +@item synopsis +@itemx description +Validate certain typographical and stylistic rules about package +descriptions and synopses. -By default, @command{guix publish} compresses archives on the fly as it -serves them. This ``on-the-fly'' mode is convenient in that it requires -no setup and is immediately available. However, when serving lots of -clients, we recommend using the @option{--cache} option, which enables -caching of the archives before they are sent to clients---see below for -details. The @command{guix weather} command provides a handy way to -check what a server provides (@pxref{Invoking guix weather}). +@item inputs-should-be-native +Identify inputs that should most likely be native inputs. -As a bonus, @command{guix publish} also serves as a content-addressed -mirror for source files referenced in @code{origin} records -(@pxref{origin Reference}). For instance, assuming @command{guix -publish} is running on @code{example.org}, the following URL returns the -raw @file{hello-2.10.tar.gz} file with the given SHA256 hash -(represented in @code{nix-base32} format, @pxref{Invoking guix hash}): +@item source +@itemx home-page +@itemx mirror-url +@itemx github-url +@itemx source-file-name +Probe @code{home-page} and @code{source} URLs and report those that are +invalid. Suggest a @code{mirror://} URL when applicable. If the +@code{source} URL redirects to a GitHub URL, recommend usage of the GitHub +URL. Check that the source file name is meaningful, e.g.@: is not just a +version number or ``git-checkout'', without a declared @code{file-name} +(@pxref{origin Reference}). -@example -http://example.org/file/hello-2.10.tar.gz/sha256/0ssi1@dots{}ndq1i -@end example +@item source-unstable-tarball +Parse the @code{source} URL to determine if a tarball from GitHub is +autogenerated or if it is a release tarball. Unfortunately GitHub's +autogenerated tarballs are sometimes regenerated. -Obviously, these URLs only work for files that are in the store; in -other cases, they return 404 (``Not Found''). +@item cve +@cindex security vulnerabilities +@cindex CVE, Common Vulnerabilities and Exposures +Report known vulnerabilities found in the Common Vulnerabilities and +Exposures (CVE) databases of the current and past year +@uref{https://nvd.nist.gov/download.cfm#CVE_FEED, published by the US +NIST}. -@cindex build logs, publication -Build logs are available from @code{/log} URLs like: +To view information about a particular vulnerability, visit pages such as: -@example -http://example.org/log/gwspk@dots{}-guile-2.2.3 -@end example +@itemize +@item +@indicateurl{https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-YYYY-ABCD} +@item +@indicateurl{https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-ABCD} +@end itemize @noindent -When @command{guix-daemon} is configured to save compressed build logs, -as is the case by default (@pxref{Invoking guix-daemon}), @code{/log} -URLs return the compressed log as-is, with an appropriate -@code{Content-Type} and/or @code{Content-Encoding} header. We recommend -running @command{guix-daemon} with @code{--log-compression=gzip} since -Web browsers can automatically decompress it, which is not the case with -bzip2 compression. - -The following options are available: +where @code{CVE-YYYY-ABCD} is the CVE identifier---e.g., +@code{CVE-2015-7554}. -@table @code -@item --port=@var{port} -@itemx -p @var{port} -Listen for HTTP requests on @var{port}. +Package developers can specify in package recipes the +@uref{https://nvd.nist.gov/cpe.cfm,Common Platform Enumeration (CPE)} +name and version of the package when they differ from the name or version +that Guix uses, as in this example: -@item --listen=@var{host} -Listen on the network interface for @var{host}. The default is to -accept connections from any interface. +@example +(package + (name "grub") + ;; @dots{} + ;; CPE calls this package "grub2". + (properties '((cpe-name . "grub2") + (cpe-version . "2.3"))) +@end example -@item --user=@var{user} -@itemx -u @var{user} -Change privileges to @var{user} as soon as possible---i.e., once the -server socket is open and the signing key has been read. +@c See . +Some entries in the CVE database do not specify which version of a +package they apply to, and would thus ``stick around'' forever. Package +developers who found CVE alerts and verified they can be ignored can +declare them as in this example: -@item --compression[=@var{level}] -@itemx -C [@var{level}] -Compress data using the given @var{level}. When @var{level} is zero, -disable compression. The range 1 to 9 corresponds to different gzip -compression levels: 1 is the fastest, and 9 is the best (CPU-intensive). -The default is 3. +@example +(package + (name "t1lib") + ;; @dots{} + ;; These CVEs no longer apply and can be safely ignored. + (properties `((lint-hidden-cve . ("CVE-2011-0433" + "CVE-2011-1553" + "CVE-2011-1554" + "CVE-2011-5244"))))) +@end example -Unless @option{--cache} is used, compression occurs on the fly and -the compressed streams are not -cached. Thus, to reduce load on the machine that runs @command{guix -publish}, it may be a good idea to choose a low compression level, to -run @command{guix publish} behind a caching proxy, or to use -@option{--cache}. Using @option{--cache} has the advantage that it -allows @command{guix publish} to add @code{Content-Length} HTTP header -to its responses. +@item formatting +Warn about obvious source code formatting issues: trailing white space, +use of tabulations, etc. +@end table -@item --cache=@var{directory} -@itemx -c @var{directory} -Cache archives and meta-data (@code{.narinfo} URLs) to @var{directory} -and only serve archives that are in cache. +The general syntax is: -When this option is omitted, archives and meta-data are created -on-the-fly. This can reduce the available bandwidth, especially when -compression is enabled, since this may become CPU-bound. Another -drawback of the default mode is that the length of archives is not known -in advance, so @command{guix publish} does not add a -@code{Content-Length} HTTP header to its responses, which in turn -prevents clients from knowing the amount of data being downloaded. +@example +guix lint @var{options} @var{package}@dots{} +@end example -Conversely, when @option{--cache} is used, the first request for a store -item (@i{via} a @code{.narinfo} URL) returns 404 and triggers a -background process to @dfn{bake} the archive---computing its -@code{.narinfo} and compressing the archive, if needed. Once the -archive is cached in @var{directory}, subsequent requests succeed and -are served directly from the cache, which guarantees that clients get -the best possible bandwidth. +If no package is given on the command line, then all packages are checked. +The @var{options} may be zero or more of the following: -The ``baking'' process is performed by worker threads. By default, one -thread per CPU core is created, but this can be customized. See -@option{--workers} below. +@table @code +@item --list-checkers +@itemx -l +List and describe all the available checkers that will be run on packages +and exit. -When @option{--ttl} is used, cached entries are automatically deleted -when they have expired. +@item --checkers +@itemx -c +Only enable the checkers specified in a comma-separated list using the +names returned by @code{--list-checkers}. -@item --workers=@var{N} -When @option{--cache} is used, request the allocation of @var{N} worker -threads to ``bake'' archives. +@end table -@item --ttl=@var{ttl} -Produce @code{Cache-Control} HTTP headers that advertise a time-to-live -(TTL) of @var{ttl}. @var{ttl} must denote a duration: @code{5d} means 5 -days, @code{1m} means 1 month, and so on. +@node Invoking guix size +@section Invoking @command{guix size} -This allows the user's Guix to keep substitute information in cache for -@var{ttl}. However, note that @code{guix publish} does not itself -guarantee that the store items it provides will indeed remain available -for as long as @var{ttl}. +@cindex size +@cindex package size +@cindex closure +@cindex @command{guix size} +The @command{guix size} command helps package developers profile the +disk usage of packages. It is easy to overlook the impact of an +additional dependency added to a package, or the impact of using a +single output for a package that could easily be split (@pxref{Packages +with Multiple Outputs}). Such are the typical issues that +@command{guix size} can highlight. -Additionally, when @option{--cache} is used, cached entries that have -not been accessed for @var{ttl} and that no longer have a corresponding -item in the store, may be deleted. +The command can be passed one or more package specifications +such as @code{gcc@@4.8} +or @code{guile:debug}, or a file name in the store. Consider this +example: -@item --nar-path=@var{path} -Use @var{path} as the prefix for the URLs of ``nar'' files -(@pxref{Invoking guix archive, normalized archives}). +@example +$ guix size coreutils +store item total self +/gnu/store/@dots{}-gcc-5.5.0-lib 60.4 30.1 38.1% +/gnu/store/@dots{}-glibc-2.27 30.3 28.8 36.6% +/gnu/store/@dots{}-coreutils-8.28 78.9 15.0 19.0% +/gnu/store/@dots{}-gmp-6.1.2 63.1 2.7 3.4% +/gnu/store/@dots{}-bash-static-4.4.12 1.5 1.5 1.9% +/gnu/store/@dots{}-acl-2.2.52 61.1 0.4 0.5% +/gnu/store/@dots{}-attr-2.4.47 60.6 0.2 0.3% +/gnu/store/@dots{}-libcap-2.25 60.5 0.2 0.2% +total: 78.9 MiB +@end example -By default, nars are served at a URL such as -@code{/nar/gzip/@dots{}-coreutils-8.25}. This option allows you to -change the @code{/nar} part to @var{path}. +@cindex closure +The store items listed here constitute the @dfn{transitive closure} of +Coreutils---i.e., Coreutils and all its dependencies, recursively---as +would be returned by: -@item --public-key=@var{file} -@itemx --private-key=@var{file} -Use the specific @var{file}s as the public/private key pair used to sign -the store items being published. +@example +$ guix gc -R /gnu/store/@dots{}-coreutils-8.23 +@end example -The files must correspond to the same key pair (the private key is used -for signing and the public key is merely advertised in the signature -metadata). They must contain keys in the canonical s-expression format -as produced by @command{guix archive --generate-key} (@pxref{Invoking -guix archive}). By default, @file{/etc/guix/signing-key.pub} and -@file{/etc/guix/signing-key.sec} are used. +Here the output shows three columns next to store items. The first column, +labeled ``total'', shows the size in mebibytes (MiB) of the closure of +the store item---that is, its own size plus the size of all its +dependencies. The next column, labeled ``self'', shows the size of the +item itself. The last column shows the ratio of the size of the item +itself to the space occupied by all the items listed here. -@item --repl[=@var{port}] -@itemx -r [@var{port}] -Spawn a Guile REPL server (@pxref{REPL Servers,,, guile, GNU Guile -Reference Manual}) on @var{port} (37146 by default). This is used -primarily for debugging a running @command{guix publish} server. -@end table +In this example, we see that the closure of Coreutils weighs in at +79@tie{}MiB, most of which is taken by libc and GCC's run-time support +libraries. (That libc and GCC's libraries represent a large fraction of +the closure is not a problem @i{per se} because they are always available +on the system anyway.) -Enabling @command{guix publish} on a GuixSD system is a one-liner: just -instantiate a @code{guix-publish-service-type} service in the @code{services} field -of the @code{operating-system} declaration (@pxref{guix-publish-service-type, -@code{guix-publish-service-type}}). +When the package(s) passed to @command{guix size} are available in the +store@footnote{More precisely, @command{guix size} looks for the +@emph{ungrafted} variant of the given package(s), as returned by +@code{guix build @var{package} --no-grafts}. @xref{Security Updates}, +for information on grafts.}, @command{guix size} queries the daemon to determine its +dependencies, and measures its size in the store, similar to @command{du +-ms --apparent-size} (@pxref{du invocation,,, coreutils, GNU +Coreutils}). -If you are instead running Guix on a ``foreign distro'', follow these -instructions:” +When the given packages are @emph{not} in the store, @command{guix size} +reports information based on the available substitutes +(@pxref{Substitutes}). This makes it possible it to profile disk usage of +store items that are not even on disk, only available remotely. -@itemize -@item -If your host distro uses the systemd init system: +You can also specify several package names: @example -# ln -s ~root/.guix-profile/lib/systemd/system/guix-publish.service \ - /etc/systemd/system/ -# systemctl start guix-publish && systemctl enable guix-publish +$ guix size coreutils grep sed bash +store item total self +/gnu/store/@dots{}-coreutils-8.24 77.8 13.8 13.4% +/gnu/store/@dots{}-grep-2.22 73.1 0.8 0.8% +/gnu/store/@dots{}-bash-4.3.42 72.3 4.7 4.6% +/gnu/store/@dots{}-readline-6.3 67.6 1.2 1.2% +@dots{} +total: 102.3 MiB @end example -@item -If your host distro uses the Upstart init system: +@noindent +In this example we see that the combination of the four packages takes +102.3@tie{}MiB in total, which is much less than the sum of each closure +since they have a lot of dependencies in common. -@example -# ln -s ~root/.guix-profile/lib/upstart/system/guix-publish.conf /etc/init/ -# start guix-publish -@end example +The available options are: -@item -Otherwise, proceed similarly with your distro's init system. -@end itemize +@table @option -@node Invoking guix challenge -@section Invoking @command{guix challenge} +@item --substitute-urls=@var{urls} +Use substitute information from @var{urls}. +@xref{client-substitute-urls, the same option for @code{guix build}}. -@cindex reproducible builds -@cindex verifiable builds -@cindex @command{guix challenge} -@cindex challenge -Do the binaries provided by this server really correspond to the source -code it claims to build? Is a package build process deterministic? -These are the questions the @command{guix challenge} command attempts to -answer. +@item --sort=@var{key} +Sort lines according to @var{key}, one of the following options: -The former is obviously an important question: Before using a substitute -server (@pxref{Substitutes}), one had better @emph{verify} that it -provides the right binaries, and thus @emph{challenge} it. The latter -is what enables the former: If package builds are deterministic, then -independent builds of the package should yield the exact same result, -bit for bit; if a server provides a binary different from the one -obtained locally, it may be either corrupt or malicious. +@table @code +@item self +the size of each item (the default); +@item closure +the total size of the item's closure. +@end table -We know that the hash that shows up in @file{/gnu/store} file names is -the hash of all the inputs of the process that built the file or -directory---compilers, libraries, build scripts, -etc. (@pxref{Introduction}). Assuming deterministic build processes, -one store file name should map to exactly one build output. -@command{guix challenge} checks whether there is, indeed, a single -mapping by comparing the build outputs of several independent builds of -any given store item. +@item --map-file=@var{file} +Write a graphical map of disk usage in PNG format to @var{file}. -The command output looks like this: +For the example above, the map looks like this: -@smallexample -$ guix challenge --substitute-urls="https://@value{SUBSTITUTE-SERVER} https://guix.example.org" -updating list of substitutes from 'https://@value{SUBSTITUTE-SERVER}'... 100.0% -updating list of substitutes from 'https://guix.example.org'... 100.0% -/gnu/store/@dots{}-openssl-1.0.2d contents differ: - local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-openssl-1.0.2d: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q - https://guix.example.org/nar/@dots{}-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim -/gnu/store/@dots{}-git-2.5.0 contents differ: - local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f - https://guix.example.org/nar/@dots{}-git-2.5.0: 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73 -/gnu/store/@dots{}-pius-2.1.1 contents differ: - local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-pius-2.1.1: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax - https://guix.example.org/nar/@dots{}-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs +@image{images/coreutils-size-map,5in,, map of Coreutils disk usage +produced by @command{guix size}} -@dots{} +This option requires that +@uref{http://wingolog.org/software/guile-charting/, Guile-Charting} be +installed and visible in Guile's module search path. When that is not +the case, @command{guix size} fails as it tries to load it. -6,406 store items were analyzed: - - 4,749 (74.1%) were identical - - 525 (8.2%) differed - - 1,132 (17.7%) were inconclusive -@end smallexample +@item --system=@var{system} +@itemx -s @var{system} +Consider packages for @var{system}---e.g., @code{x86_64-linux}. -@noindent -In this example, @command{guix challenge} first scans the store to -determine the set of locally-built derivations---as opposed to store -items that were downloaded from a substitute server---and then queries -all the substitute servers. It then reports those store items for which -the servers obtained a result different from the local build. +@end table -@cindex non-determinism, in package builds -As an example, @code{guix.example.org} always gets a different answer. -Conversely, @code{@value{SUBSTITUTE-SERVER}} agrees with local builds, except in the -case of Git. This might indicate that the build process of Git is -non-deterministic, meaning that its output varies as a function of -various things that Guix does not fully control, in spite of building -packages in isolated environments (@pxref{Features}). Most common -sources of non-determinism include the addition of timestamps in build -results, the inclusion of random numbers, and directory listings sorted -by inode number. See @uref{https://reproducible-builds.org/docs/}, for -more information. +@node Invoking guix graph +@section Invoking @command{guix graph} -To find out what is wrong with this Git binary, we can do something along -these lines (@pxref{Invoking guix archive}): +@cindex DAG +@cindex @command{guix graph} +@cindex package dependencies +Packages and their dependencies form a @dfn{graph}, specifically a +directed acyclic graph (DAG). It can quickly become difficult to have a +mental model of the package DAG, so the @command{guix graph} command +provides a visual representation of the DAG. By default, +@command{guix graph} emits a DAG representation in the input format of +@uref{http://www.graphviz.org/, Graphviz}, so its output can be passed +directly to the @command{dot} command of Graphviz. It can also emit an +HTML page with embedded JavaScript code to display a ``chord diagram'' +in a Web browser, using the @uref{https://d3js.org/, d3.js} library, or +emit Cypher queries to construct a graph in a graph database supporting +the @uref{http://www.opencypher.org/, openCypher} query language. +The general syntax is: + +@example +guix graph @var{options} @var{package}@dots{} +@end example + +For example, the following command generates a PDF file representing the +package DAG for the GNU@tie{}Core Utilities, showing its build-time +dependencies: @example -$ wget -q -O - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0 \ - | guix archive -x /tmp/git -$ diff -ur --no-dereference /gnu/store/@dots{}-git.2.5.0 /tmp/git +guix graph coreutils | dot -Tpdf > dag.pdf @end example -This command shows the difference between the files resulting from the -local build, and the files resulting from the build on -@code{@value{SUBSTITUTE-SERVER}} (@pxref{Overview, Comparing and Merging Files,, -diffutils, Comparing and Merging Files}). The @command{diff} command -works great for text files. When binary files differ, a better option -is @uref{https://diffoscope.org/, Diffoscope}, a tool that helps -visualize differences for all kinds of files. +The output looks like this: -Once you have done that work, you can tell whether the differences are due -to a non-deterministic build process or to a malicious server. We try -hard to remove sources of non-determinism in packages to make it easier -to verify substitutes, but of course, this is a process that -involves not just Guix, but a large part of the free software community. -In the meantime, @command{guix challenge} is one tool to help address -the problem. +@image{images/coreutils-graph,2in,,Dependency graph of the GNU Coreutils} -If you are writing packages for Guix, you are encouraged to check -whether @code{@value{SUBSTITUTE-SERVER}} and other substitute servers obtain the -same build result as you did with: +Nice little graph, no? + +But there is more than one graph! The one above is concise: it is the +graph of package objects, omitting implicit inputs such as GCC, libc, +grep, etc. It is often useful to have such a concise graph, but +sometimes one may want to see more details. @command{guix graph} supports +several types of graphs, allowing you to choose the level of detail: + +@table @code +@item package +This is the default type used in the example above. It shows the DAG of +package objects, excluding implicit dependencies. It is concise, but +filters out many details. + +@item reverse-package +This shows the @emph{reverse} DAG of packages. For example: @example -$ guix challenge @var{package} +guix graph --type=reverse-package ocaml @end example -@noindent -where @var{package} is a package specification such as -@code{guile@@2.0} or @code{glibc:debug}. +...@: yields the graph of packages that depend on OCaml. -The general syntax is: +Note that for core packages this can yield huge graphs. If all you want +is to know the number of packages that depend on a given package, use +@command{guix refresh --list-dependent} (@pxref{Invoking guix refresh, +@option{--list-dependent}}). + +@item bag-emerged +This is the package DAG, @emph{including} implicit inputs. + +For instance, the following command: @example -guix challenge @var{options} [@var{packages}@dots{}] +guix graph --type=bag-emerged coreutils | dot -Tpdf > dag.pdf @end example -When a difference is found between the hash of a locally-built item and -that of a server-provided substitute, or among substitutes provided by -different servers, the command displays it as in the example above and -its exit code is 2 (other non-zero exit codes denote other kinds of -errors.) +...@: yields this bigger graph: -The one option that matters is: +@image{images/coreutils-bag-graph,,5in,Detailed dependency graph of the GNU Coreutils} -@table @code +At the bottom of the graph, we see all the implicit inputs of +@var{gnu-build-system} (@pxref{Build Systems, @code{gnu-build-system}}). -@item --substitute-urls=@var{urls} -Consider @var{urls} the whitespace-separated list of substitute source -URLs to compare to. +Now, note that the dependencies of these implicit inputs---that is, the +@dfn{bootstrap dependencies} (@pxref{Bootstrapping})---are not shown +here, for conciseness. -@item --verbose -@itemx -v -Show details about matches (identical contents) in addition to -information about mismatches. +@item bag +Similar to @code{bag-emerged}, but this time including all the bootstrap +dependencies. -@end table +@item bag-with-origins +Similar to @code{bag}, but also showing origins and their dependencies. -@node Invoking guix copy -@section Invoking @command{guix copy} +@item derivation +This is the most detailed representation: It shows the DAG of +derivations (@pxref{Derivations}) and plain store items. Compared to +the above representation, many additional nodes are visible, including +build scripts, patches, Guile modules, etc. -@cindex copy, of store items, over SSH -@cindex SSH, copy of store items -@cindex sharing store items across machines -@cindex transferring store items across machines -The @command{guix copy} command copies items from the store of one -machine to that of another machine over a secure shell (SSH) -connection@footnote{This command is available only when Guile-SSH was -found. @xref{Requirements}, for details.}. For example, the following -command copies the @code{coreutils} package, the user's profile, and all -their dependencies over to @var{host}, logged in as @var{user}: +For this type of graph, it is also possible to pass a @file{.drv} file +name instead of a package name, as in: @example -guix copy --to=@var{user}@@@var{host} \ - coreutils `readlink -f ~/.guix-profile` +guix graph -t derivation `guix system build -d my-config.scm` @end example -If some of the items to be copied are already present on @var{host}, -they are not actually sent. - -The command below retrieves @code{libreoffice} and @code{gimp} from -@var{host}, assuming they are available there: +@item module +This is the graph of @dfn{package modules} (@pxref{Package Modules}). +For example, the following command shows the graph for the package +module that defines the @code{guile} package: @example -guix copy --from=@var{host} libreoffice gimp +guix graph -t module guile | dot -Tpdf > module-graph.pdf @end example +@end table -The SSH connection is established using the Guile-SSH client, which is -compatible with OpenSSH: it honors @file{~/.ssh/known_hosts} and -@file{~/.ssh/config}, and uses the SSH agent for authentication. +All the types above correspond to @emph{build-time dependencies}. The +following graph type represents the @emph{run-time dependencies}: -The key used to sign items that are sent must be accepted by the remote -machine. Likewise, the key used by the remote machine to sign items you -are retrieving must be in @file{/etc/guix/acl} so it is accepted by your -own daemon. @xref{Invoking guix archive}, for more information about -store item authentication. +@table @code +@item references +This is the graph of @dfn{references} of a package output, as returned +by @command{guix gc --references} (@pxref{Invoking guix gc}). -The general syntax is: +If the given package output is not available in the store, @command{guix +graph} attempts to obtain dependency information from substitutes. + +Here you can also pass a store file name instead of a package name. For +example, the command below produces the reference graph of your profile +(which can be big!): @example -guix copy [--to=@var{spec}|--from=@var{spec}] @var{items}@dots{} +guix graph -t references `readlink -f ~/.guix-profile` @end example -You must always specify one of the following options: +@item referrers +This is the graph of the @dfn{referrers} of a store item, as returned by +@command{guix gc --referrers} (@pxref{Invoking guix gc}). + +This relies exclusively on local information from your store. For +instance, let us suppose that the current Inkscape is available in 10 +profiles on your machine; @command{guix graph -t referrers inkscape} +will show a graph rooted at Inkscape and with those 10 profiles linked +to it. + +It can help determine what is preventing a store item from being garbage +collected. -@table @code -@item --to=@var{spec} -@itemx --from=@var{spec} -Specify the host to send to or receive from. @var{spec} must be an SSH -spec such as @code{example.org}, @code{charlie@@example.org}, or -@code{charlie@@example.org:2222}. @end table -The @var{items} can be either package names, such as @code{gimp}, or -store items, such as @file{/gnu/store/@dots{}-idutils-4.6}. +The available options are the following: -When specifying the name of a package to send, it is first built if -needed, unless @option{--dry-run} was specified. Common build options -are supported (@pxref{Common Build Options}). +@table @option +@item --type=@var{type} +@itemx -t @var{type} +Produce a graph output of @var{type}, where @var{type} must be one of +the values listed above. +@item --list-types +List the supported graph types. -@node Invoking guix container -@section Invoking @command{guix container} -@cindex container -@cindex @command{guix container} -@quotation Note -As of version @value{VERSION}, this tool is experimental. The interface -is subject to radical change in the future. -@end quotation +@item --backend=@var{backend} +@itemx -b @var{backend} +Produce a graph using the selected @var{backend}. -The purpose of @command{guix container} is to manipulate processes -running within an isolated environment, commonly known as a -``container'', typically created by the @command{guix environment} -(@pxref{Invoking guix environment}) and @command{guix system container} -(@pxref{Invoking guix system}) commands. +@item --list-backends +List the supported graph backends. + +Currently, the available backends are Graphviz and d3.js. + +@item --expression=@var{expr} +@itemx -e @var{expr} +Consider the package @var{expr} evaluates to. + +This is useful to precisely refer to a package, as in this example: + +@example +guix graph -e '(@@@@ (gnu packages commencement) gnu-make-final)' +@end example + +@item --system=@var{system} +@itemx -s @var{system} +Display the graph for @var{system}---e.g., @code{i686-linux}. + +The package dependency graph is largely architecture-independent, but there +are some architecture-dependent bits that this option allows you to visualize. +@end table + + +@node Invoking guix environment +@section Invoking @command{guix environment} + +@cindex reproducible build environments +@cindex development environments +@cindex @command{guix environment} +@cindex environment, package build environment +The purpose of @command{guix environment} is to assist hackers in +creating reproducible development environments without polluting their +package profile. The @command{guix environment} tool takes one or more +packages, builds all of their inputs, and creates a shell +environment to use them. The general syntax is: @example -guix container @var{action} @var{options}@dots{} +guix environment @var{options} @var{package}@dots{} +@end example + +The following example spawns a new shell set up for the development of +GNU@tie{}Guile: + +@example +guix environment guile +@end example + +If the needed dependencies are not built yet, @command{guix environment} +automatically builds them. The environment of the new shell is an augmented +version of the environment that @command{guix environment} was run in. +It contains the necessary search paths for building the given package +added to the existing environment variables. To create a ``pure'' +environment, in which the original environment variables have been unset, +use the @code{--pure} option@footnote{Users sometimes wrongfully augment +environment variables such as @code{PATH} in their @file{~/.bashrc} +file. As a consequence, when @code{guix environment} launches it, Bash +may read @file{~/.bashrc}, thereby introducing ``impurities'' in these +environment variables. It is an error to define such environment +variables in @file{.bashrc}; instead, they should be defined in +@file{.bash_profile}, which is sourced only by log-in shells. +@xref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}, for +details on Bash start-up files.}. + +@vindex GUIX_ENVIRONMENT +@command{guix environment} defines the @code{GUIX_ENVIRONMENT} +variable in the shell it spawns; its value is the file name of the +profile of this environment. This allows users to, say, define a +specific prompt for development environments in their @file{.bashrc} +(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}): + +@example +if [ -n "$GUIX_ENVIRONMENT" ] +then + export PS1="\u@@\h \w [dev]\$ " +fi @end example -@var{action} specifies the operation to perform with a container, and -@var{options} specifies the context-specific arguments for the action. - -The following actions are available: +@noindent +...@: or to browse the profile: -@table @code -@item exec -Execute a command within the context of a running container. +@example +$ ls "$GUIX_ENVIRONMENT/bin" +@end example -The syntax is: +Additionally, more than one package may be specified, in which case the +union of the inputs for the given packages are used. For example, the +command below spawns a shell where all of the dependencies of both Guile +and Emacs are available: @example -guix container exec @var{pid} @var{program} @var{arguments}@dots{} +guix environment guile emacs @end example -@var{pid} specifies the process ID of the running container. -@var{program} specifies an executable file name within the root file -system of the container. @var{arguments} are the additional options that -will be passed to @var{program}. - -The following command launches an interactive login shell inside a -GuixSD container, started by @command{guix system container}, and whose -process ID is 9001: +Sometimes an interactive shell session is not desired. An arbitrary +command may be invoked by placing the @code{--} token to separate the +command from the rest of the arguments: @example -guix container exec 9001 /run/current-system/profile/bin/bash --login +guix environment guile -- make -j4 @end example -Note that the @var{pid} cannot be the parent process of a container. It -must be PID 1 of the container or one of its child processes. +In other situations, it is more convenient to specify the list of +packages needed in the environment. For example, the following command +runs @command{python} from an environment containing Python@tie{}2.7 and +NumPy: -@end table +@example +guix environment --ad-hoc python2-numpy python-2.7 -- python +@end example -@node Invoking guix weather -@section Invoking @command{guix weather} +Furthermore, one might want the dependencies of a package and also some +additional packages that are not build-time or runtime dependencies, but +are useful when developing nonetheless. Because of this, the +@code{--ad-hoc} flag is positional. Packages appearing before +@code{--ad-hoc} are interpreted as packages whose dependencies will be +added to the environment. Packages appearing after are interpreted as +packages that will be added to the environment directly. For example, +the following command creates a Guix development environment that +additionally includes Git and strace: -Occasionally you're grumpy because substitutes are lacking and you end -up building packages by yourself (@pxref{Substitutes}). The -@command{guix weather} command reports on substitute availability on the -specified servers so you can have an idea of whether you'll be grumpy -today. It can sometimes be useful info as a user, but it is primarily -useful to people running @command{guix publish} (@pxref{Invoking guix -publish}). +@example +guix environment guix --ad-hoc git strace +@end example -@cindex statistics, for substitutes -@cindex availability of substitutes -@cindex substitute availability -@cindex weather, substitute availability -Here's a sample run: +Sometimes it is desirable to isolate the environment as much as +possible, for maximal purity and reproducibility. In particular, when +using Guix on a host distro that is not GuixSD, it is desirable to +prevent access to @file{/usr/bin} and other system-wide resources from +the development environment. For example, the following command spawns +a Guile REPL in a ``container'' where only the store and the current +working directory are mounted: @example -$ guix weather --substitute-urls=https://guix.example.org -computing 5,872 package derivations for x86_64-linux... -looking for 6,128 store items on https://guix.example.org.. -updating list of substitutes from 'https://guix.example.org'... 100.0% -https://guix.example.org - 43.4% substitutes available (2,658 out of 6,128) - 7,032.5 MiB of nars (compressed) - 19,824.2 MiB on disk (uncompressed) - 0.030 seconds per request (182.9 seconds in total) - 33.5 requests per second - - 9.8% (342 out of 3,470) of the missing items are queued - 867 queued builds - x86_64-linux: 518 (59.7%) - i686-linux: 221 (25.5%) - aarch64-linux: 128 (14.8%) - build rate: 23.41 builds per hour - x86_64-linux: 11.16 builds per hour - i686-linux: 6.03 builds per hour - aarch64-linux: 6.41 builds per hour +guix environment --ad-hoc --container guile -- guile @end example -@cindex continuous integration, statistics -As you can see, it reports the fraction of all the packages for which -substitutes are available on the server---regardless of whether -substitutes are enabled, and regardless of whether this server's signing -key is authorized. It also reports the size of the compressed archives -(``nars'') provided by the server, the size the corresponding store -items occupy in the store (assuming deduplication is turned off), and -the server's throughput. The second part gives continuous integration -(CI) statistics, if the server supports it. - -To achieve that, @command{guix weather} queries over HTTP(S) meta-data -(@dfn{narinfos}) for all the relevant store items. Like @command{guix -challenge}, it ignores signatures on those substitutes, which is -innocuous since the command only gathers statistics and cannot install -those substitutes. +@quotation Note +The @code{--container} option requires Linux-libre 3.19 or newer. +@end quotation -Among other things, it is possible to query specific system types and -specific package sets. The available options are listed below. +The available options are summarized below. @table @code -@item --substitute-urls=@var{urls} -@var{urls} is the space-separated list of substitute server URLs to -query. When this option is omitted, the default set of substitute -servers is queried. +@item --root=@var{file} +@itemx -r @var{file} +@cindex persistent environment +@cindex garbage collector root, for environments +Make @var{file} a symlink to the profile for this environment, and +register it as a garbage collector root. -@item --system=@var{system} -@itemx -s @var{system} -Query substitutes for @var{system}---e.g., @code{aarch64-linux}. This -option can be repeated, in which case @command{guix weather} will query -substitutes for several system types. +This is useful if you want to protect your environment from garbage +collection, to make it ``persistent''. -@item --manifest=@var{file} -Instead of querying substitutes for all the packages, only ask for those -specified in @var{file}. @var{file} must contain a @dfn{manifest}, as -with the @code{-m} option of @command{guix package} (@pxref{Invoking -guix package}). -@end table +When this option is omitted, the environment is protected from garbage +collection only for the duration of the @command{guix environment} +session. This means that next time you recreate the same environment, +you could have to rebuild or re-download packages. @xref{Invoking guix +gc}, for more on GC roots. -@node Invoking guix processes -@section Invoking @command{guix processes} +@item --expression=@var{expr} +@itemx -e @var{expr} +Create an environment for the package or list of packages that +@var{expr} evaluates to. -The @command{guix processes} command can be useful to developers and system -administrators, especially on multi-user machines and on build farms: it lists -the current sessions (connections to the daemon), as well as information about -the processes involved@footnote{Remote sessions, when @command{guix-daemon} is -started with @option{--listen} specifying a TCP endpoint, are @emph{not} -listed.}. Here's an example of the information it returns: +For example, running: @example -$ sudo guix processes -SessionPID: 19002 -ClientPID: 19090 -ClientCommand: guix environment --ad-hoc python - -SessionPID: 19402 -ClientPID: 19367 -ClientCommand: guix publish -u guix-publish -p 3000 -C 9 @dots{} - -SessionPID: 19444 -ClientPID: 19419 -ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} -LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock -LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock -LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock -ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800 -ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800 -ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800 +guix environment -e '(@@ (gnu packages maths) petsc-openmpi)' @end example -In this example we see that @command{guix-daemon} has three clients: -@command{guix environment}, @command{guix publish}, and the Cuirass continuous -integration tool; their process identifier (PID) is given by the -@code{ClientPID} field. The @code{SessionPID} field gives the PID of the -@command{guix-daemon} sub-process of this particular session. - -The @code{LockHeld} fields show which store items are currently locked by this -session, which corresponds to store items being built or substituted (the -@code{LockHeld} field is not displayed when @command{guix processes} is not -running as root.) Last, by looking at the @code{ChildProcess} field, we -understand that these three builds are being offloaded (@pxref{Daemon Offload -Setup}). +starts a shell with the environment for this specific variant of the +PETSc package. -The output is in Recutils format so we can use the handy @command{recsel} -command to select sessions of interest (@pxref{Selection Expressions,,, -recutils, GNU recutils manual}). As an example, the command shows the command -line and PID of the client that triggered the build of a Perl package: +Running: @example -$ sudo guix processes | \ - recsel -p ClientPID,ClientCommand -e 'LockHeld ~ "perl"' -ClientPID: 19419 -ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} +guix environment --ad-hoc -e '(@@ (gnu) %base-packages)' @end example +starts a shell with all the GuixSD base packages available. -@node System Installation -@chapter System Installation +The above commands only use the default output of the given packages. +To select other outputs, two element tuples can be specified: -@cindex installing GuixSD -@cindex Guix System Distribution -This section explains how to install the Guix System Distribution (GuixSD) -on a machine. The Guix package manager can -also be installed on top of a running GNU/Linux system, -@pxref{Installation}. +@example +guix environment --ad-hoc -e '(list (@@ (gnu packages bash) bash) "include")' +@end example -@ifinfo -@quotation Note -@c This paragraph is for people reading this from tty2 of the -@c installation image. -You are reading this documentation with an Info reader. For details on -how to use it, hit the @key{RET} key (``return'' or ``enter'') on the -link that follows: @pxref{Top, Info reader,, info-stnd, Stand-alone GNU -Info}. Hit @kbd{l} afterwards to come back here. +@item --load=@var{file} +@itemx -l @var{file} +Create an environment for the package or list of packages that the code +within @var{file} evaluates to. -Alternately, run @command{info info} in another tty to keep the manual -available. -@end quotation -@end ifinfo +As an example, @var{file} might contain a definition like this +(@pxref{Defining Packages}): -@menu -* Limitations:: What you can expect. -* Hardware Considerations:: Supported hardware. -* USB Stick and DVD Installation:: Preparing the installation medium. -* Preparing for Installation:: Networking, partitioning, etc. -* Proceeding with the Installation:: The real thing. -* Installing GuixSD in a VM:: GuixSD playground. -* Building the Installation Image:: How this comes to be. -@end menu +@example +@verbatiminclude environment-gdb.scm +@end example -@node Limitations -@section Limitations +@item --manifest=@var{file} +@itemx -m @var{file} +Create an environment for the packages contained in the manifest object +returned by the Scheme code in @var{file}. -As of version @value{VERSION}, the Guix System Distribution (GuixSD) is -not production-ready. It may contain bugs and lack important -features. Thus, if you are looking for a stable production system that -respects your freedom as a computer user, a good solution at this point -is to consider @url{http://www.gnu.org/distros/free-distros.html, one of -the more established GNU/Linux distributions}. We hope you can soon switch -to the GuixSD without fear, of course. In the meantime, you can -also keep using your distribution and try out the package manager on top -of it (@pxref{Installation}). +This is similar to the same-named option in @command{guix package} +(@pxref{profile-manifest, @option{--manifest}}) and uses the same +manifest files. -Before you proceed with the installation, be aware of the following -noteworthy limitations applicable to version @value{VERSION}: +@item --ad-hoc +Include all specified packages in the resulting environment, as if an +@i{ad hoc} package were defined with them as inputs. This option is +useful for quickly creating an environment without having to write a +package expression to contain the desired inputs. -@itemize -@item -The installation process does not include a graphical user interface and -requires familiarity with GNU/Linux (see the following subsections to -get a feel of what that means.) +For instance, the command: -@item -Support for the Logical Volume Manager (LVM) is missing. +@example +guix environment --ad-hoc guile guile-sdl -- guile +@end example -@item -More and more system services are provided (@pxref{Services}), but some -may be missing. +runs @command{guile} in an environment where Guile and Guile-SDL are +available. -@item -More than 8,500 packages are available, but you might -occasionally find that a useful package is missing. +Note that this example implicitly asks for the default output of +@code{guile} and @code{guile-sdl}, but it is possible to ask for a +specific output---e.g., @code{glib:bin} asks for the @code{bin} output +of @code{glib} (@pxref{Packages with Multiple Outputs}). -@item -GNOME, Xfce, LXDE, and Enlightenment are available (@pxref{Desktop Services}), -as well as a number of X11 window managers. However, some graphical -applications may be missing, as well as KDE. -@end itemize +This option may be composed with the default behavior of @command{guix +environment}. Packages appearing before @code{--ad-hoc} are interpreted +as packages whose dependencies will be added to the environment, the +default behavior. Packages appearing after are interpreted as packages +that will be added to the environment directly. -You have been warned! But more than a disclaimer, this is an invitation -to report issues (and success stories!), and to join us in improving it. -@xref{Contributing}, for more info. +@item --pure +Unset existing environment variables when building the new environment. +This has the effect of creating an environment in which search paths +only contain package inputs. +@item --search-paths +Display the environment variable definitions that make up the +environment. -@node Hardware Considerations -@section Hardware Considerations +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}. -@cindex hardware support on GuixSD -GNU@tie{}GuixSD focuses on respecting the user's computing freedom. It -builds around the kernel Linux-libre, which means that only hardware for -which free software drivers and firmware exist is supported. Nowadays, -a wide range of off-the-shelf hardware is supported on -GNU/Linux-libre---from keyboards to graphics cards to scanners and -Ethernet controllers. Unfortunately, there are still areas where -hardware vendors deny users control over their own computing, and such -hardware is not supported on GuixSD. +@item --container +@itemx -C +@cindex container +Run @var{command} within an isolated container. The current working +directory outside the container is mapped inside the container. +Additionally, unless overridden with @code{--user}, a dummy home +directory is created that matches the current user's home directory, and +@file{/etc/passwd} is configured accordingly. The spawned process runs +as the current user outside the container, but has root privileges in +the context of the container. -@cindex WiFi, hardware support -One of the main areas where free drivers or firmware are lacking is WiFi -devices. WiFi devices known to work include those using Atheros chips -(AR9271 and AR7010), which corresponds to the @code{ath9k} Linux-libre -driver, and those using Broadcom/AirForce chips (BCM43xx with -Wireless-Core Revision 5), which corresponds to the @code{b43-open} -Linux-libre driver. Free firmware exists for both and is available -out-of-the-box on GuixSD, as part of @var{%base-firmware} -(@pxref{operating-system Reference, @code{firmware}}). +@item --network +@itemx -N +For containers, share the network namespace with the host system. +Containers created without this flag only have access to the loopback +device. -@cindex RYF, Respects Your Freedom -The @uref{https://www.fsf.org/, Free Software Foundation} runs -@uref{https://www.fsf.org/ryf, @dfn{Respects Your Freedom}} (RYF), a -certification program for hardware products that respect your freedom -and your privacy and ensure that you have control over your device. We -encourage you to check the list of RYF-certified devices. +@item --link-profile +@itemx -P +For containers, link the environment profile to +@file{~/.guix-profile} within the container. This is equivalent to +running the command @command{ln -s $GUIX_ENVIRONMENT ~/.guix-profile} +within the container. Linking will fail and abort the environment if +the directory already exists, which will certainly be the case if +@command{guix environment} was invoked in the user's home directory. -Another useful resource is the @uref{https://www.h-node.org/, H-Node} -web site. It contains a catalog of hardware devices with information -about their support in GNU/Linux. +Certain packages are configured to look in +@code{~/.guix-profile} for configuration files and data;@footnote{For +example, the @code{fontconfig} package inspects +@file{~/.guix-profile/share/fonts} for additional fonts.} +@code{--link-profile} allows these programs to behave as expected within +the environment. +@item --user=@var{user} +@itemx -u @var{user} +For containers, use the username @var{user} in place of the current +user. The generated @file{/etc/passwd} entry within the container will +contain the name @var{user}; the home directory will be +@file{/home/USER}; and no user GECOS data will be copied. @var{user} +need not exist on the system. -@node USB Stick and DVD Installation -@section USB Stick and DVD Installation +Additionally, any shared or exposed path (see @code{--share} and +@code{--expose} respectively) whose target is within the current user's +home directory will be remapped relative to @file{/home/USER}; this +includes the automatic mapping of the current working directory. -An ISO-9660 installation image that can be written to a USB stick or -burnt to a DVD can be downloaded from -@indicateurl{https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz}, -where @var{system} is one of: +@example +# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target +cd $HOME/wd +guix environment --container --user=foo \ + --expose=$HOME/test \ + --expose=/tmp/target=$HOME/target +@end example -@table @code -@item x86_64-linux -for a GNU/Linux system on Intel/AMD-compatible 64-bit CPUs; +While this will limit the leaking of user identity through home paths +and each of the user fields, this is only one useful component of a +broader privacy/anonymity solution---not one in and of itself. -@item i686-linux -for a 32-bit GNU/Linux system on Intel-compatible CPUs. -@end table +@item --expose=@var{source}[=@var{target}] +For containers, expose the file system @var{source} from the host system +as the read-only file system @var{target} within the container. If +@var{target} is not specified, @var{source} is used as the target mount +point in the container. -@c start duplication of authentication part from ``Binary Installation'' -Make sure to download the associated @file{.sig} file and to verify the -authenticity of the image against it, along these lines: +The example below spawns a Guile REPL in a container in which the user's +home directory is accessible read-only via the @file{/exchange} +directory: @example -$ wget https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig -$ gpg --verify guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig +guix environment --container --expose=$HOME=/exchange --ad-hoc guile -- guile @end example -If that command fails because you do not have the required public key, -then run this command to import it: +@item --share=@var{source}[=@var{target}] +For containers, share the file system @var{source} from the host system +as the writable file system @var{target} within the container. If +@var{target} is not specified, @var{source} is used as the target mount +point in the container. + +The example below spawns a Guile REPL in a container in which the user's +home directory is accessible for both reading and writing via the +@file{/exchange} directory: @example -$ gpg --keyserver @value{KEY-SERVER} \ - --recv-keys @value{OPENPGP-SIGNING-KEY-ID} +guix environment --container --share=$HOME=/exchange --ad-hoc guile -- guile @end example +@end table -@noindent -and rerun the @code{gpg --verify} command. -@c end duplication +@command{guix environment} +also supports all of the common build options that @command{guix +build} supports (@pxref{Common Build Options}) as well as package +transformation options (@pxref{Package Transformation Options}). -This image contains the tools necessary for an installation. -It is meant to be copied @emph{as is} to a large-enough USB stick or DVD. -@unnumberedsubsec Copying to a USB Stick +@node Invoking guix publish +@section Invoking @command{guix publish} -To copy the image to a USB stick, follow these steps: +@cindex @command{guix publish} +The purpose of @command{guix publish} is to enable users to easily share +their store with others, who can then use it as a substitute server +(@pxref{Substitutes}). -@enumerate -@item -Decompress the image using the @command{xz} command: +When @command{guix publish} runs, it spawns an HTTP server which allows +anyone with network access to obtain substitutes from it. This means +that any machine running Guix can also act as if it were a build farm, +since the HTTP interface is compatible with Hydra, the software behind +the @code{@value{SUBSTITUTE-SERVER}} build farm. + +For security, each substitute is signed, allowing recipients to check +their authenticity and integrity (@pxref{Substitutes}). Because +@command{guix publish} uses the signing key of the system, which is only +readable by the system administrator, it must be started as root; the +@code{--user} option makes it drop root privileges early on. + +The signing key pair must be generated before @command{guix publish} is +launched, using @command{guix archive --generate-key} (@pxref{Invoking +guix archive}). + +The general syntax is: @example -xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz +guix publish @var{options}@dots{} @end example -@item -Insert a USB stick of 1@tie{}GiB or more into your machine, and determine -its device name. Assuming that the USB stick is known as @file{/dev/sdX}, -copy the image with: +Running @command{guix publish} without any additional arguments will +spawn an HTTP server on port 8080: @example -dd if=guixsd-install-@value{VERSION}.@var{system}.iso of=/dev/sdX -sync +guix publish @end example -Access to @file{/dev/sdX} usually requires root privileges. -@end enumerate +Once a publishing server has been authorized (@pxref{Invoking guix +archive}), the daemon may download substitutes from it: -@unnumberedsubsec Burning on a DVD +@example +guix-daemon --substitute-urls=http://example.org:8080 +@end example -To copy the image to a DVD, follow these steps: +By default, @command{guix publish} compresses archives on the fly as it +serves them. This ``on-the-fly'' mode is convenient in that it requires +no setup and is immediately available. However, when serving lots of +clients, we recommend using the @option{--cache} option, which enables +caching of the archives before they are sent to clients---see below for +details. The @command{guix weather} command provides a handy way to +check what a server provides (@pxref{Invoking guix weather}). -@enumerate -@item -Decompress the image using the @command{xz} command: +As a bonus, @command{guix publish} also serves as a content-addressed +mirror for source files referenced in @code{origin} records +(@pxref{origin Reference}). For instance, assuming @command{guix +publish} is running on @code{example.org}, the following URL returns the +raw @file{hello-2.10.tar.gz} file with the given SHA256 hash +(represented in @code{nix-base32} format, @pxref{Invoking guix hash}): @example -xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz +http://example.org/file/hello-2.10.tar.gz/sha256/0ssi1@dots{}ndq1i @end example -@item -Insert a blank DVD into your machine, and determine -its device name. Assuming that the DVD drive is known as @file{/dev/srX}, -copy the image with: +Obviously, these URLs only work for files that are in the store; in +other cases, they return 404 (``Not Found''). + +@cindex build logs, publication +Build logs are available from @code{/log} URLs like: @example -growisofs -dvd-compat -Z /dev/srX=guixsd-install-@value{VERSION}.@var{system}.iso +http://example.org/log/gwspk@dots{}-guile-2.2.3 @end example -Access to @file{/dev/srX} usually requires root privileges. -@end enumerate +@noindent +When @command{guix-daemon} is configured to save compressed build logs, +as is the case by default (@pxref{Invoking guix-daemon}), @code{/log} +URLs return the compressed log as-is, with an appropriate +@code{Content-Type} and/or @code{Content-Encoding} header. We recommend +running @command{guix-daemon} with @code{--log-compression=gzip} since +Web browsers can automatically decompress it, which is not the case with +bzip2 compression. -@unnumberedsubsec Booting +The following options are available: -Once this is done, you should be able to reboot the system and boot from -the USB stick or DVD. The latter usually requires you to get in the -BIOS or UEFI boot menu, where you can choose to boot from the USB stick. +@table @code +@item --port=@var{port} +@itemx -p @var{port} +Listen for HTTP requests on @var{port}. -@xref{Installing GuixSD in a VM}, if, instead, you would like to install -GuixSD in a virtual machine (VM). +@item --listen=@var{host} +Listen on the network interface for @var{host}. The default is to +accept connections from any interface. +@item --user=@var{user} +@itemx -u @var{user} +Change privileges to @var{user} as soon as possible---i.e., once the +server socket is open and the signing key has been read. -@node Preparing for Installation -@section Preparing for Installation +@item --compression[=@var{level}] +@itemx -C [@var{level}] +Compress data using the given @var{level}. When @var{level} is zero, +disable compression. The range 1 to 9 corresponds to different gzip +compression levels: 1 is the fastest, and 9 is the best (CPU-intensive). +The default is 3. -Once you have successfully booted your computer using the installation medium, -you should end up with the welcome page of the graphical installer. The -graphical installer is a text-based user interface built upon the newt -library. It shall guide you through all the different steps needed to install -GNU GuixSD. However, as the graphical installer is still under heavy -development, you might want to fallback to the original, shell based install -process, by switching to TTYs 3 to 6 with the shortcuts CTRL-ALT-F[3-6]. The -following sections describe the installation procedure assuming you're using -one of those TTYs. They are configured and can be used to run commands as -root. +Unless @option{--cache} is used, compression occurs on the fly and +the compressed streams are not +cached. Thus, to reduce load on the machine that runs @command{guix +publish}, it may be a good idea to choose a low compression level, to +run @command{guix publish} behind a caching proxy, or to use +@option{--cache}. Using @option{--cache} has the advantage that it +allows @command{guix publish} to add @code{Content-Length} HTTP header +to its responses. -TTY2 shows this documentation, browsable using the Info reader commands -(@pxref{Top,,, info-stnd, Stand-alone GNU Info}). The installation system -runs the GPM mouse daemon, which allows you to select text with the left mouse -button and to paste it with the middle button. +@item --cache=@var{directory} +@itemx -c @var{directory} +Cache archives and meta-data (@code{.narinfo} URLs) to @var{directory} +and only serve archives that are in cache. -@quotation Note -Installation requires access to the Internet so that any missing -dependencies of your system configuration can be downloaded. See the -``Networking'' section below. -@end quotation +When this option is omitted, archives and meta-data are created +on-the-fly. This can reduce the available bandwidth, especially when +compression is enabled, since this may become CPU-bound. Another +drawback of the default mode is that the length of archives is not known +in advance, so @command{guix publish} does not add a +@code{Content-Length} HTTP header to its responses, which in turn +prevents clients from knowing the amount of data being downloaded. -The installation system includes many common tools needed for this task. -But it is also a full-blown GuixSD system, which means that you can -install additional packages, should you need it, using @command{guix -package} (@pxref{Invoking guix package}). +Conversely, when @option{--cache} is used, the first request for a store +item (@i{via} a @code{.narinfo} URL) returns 404 and triggers a +background process to @dfn{bake} the archive---computing its +@code{.narinfo} and compressing the archive, if needed. Once the +archive is cached in @var{directory}, subsequent requests succeed and +are served directly from the cache, which guarantees that clients get +the best possible bandwidth. -@subsection Keyboard Layout +The ``baking'' process is performed by worker threads. By default, one +thread per CPU core is created, but this can be customized. See +@option{--workers} below. -@cindex keyboard layout -The installation image uses the US qwerty keyboard layout. If you want -to change it, you can use the @command{loadkeys} command. For example, -the following command selects the Dvorak keyboard layout: +When @option{--ttl} is used, cached entries are automatically deleted +when they have expired. -@example -loadkeys dvorak -@end example +@item --workers=@var{N} +When @option{--cache} is used, request the allocation of @var{N} worker +threads to ``bake'' archives. -See the files under @file{/run/current-system/profile/share/keymaps} for -a list of available keyboard layouts. Run @command{man loadkeys} for -more information. +@item --ttl=@var{ttl} +Produce @code{Cache-Control} HTTP headers that advertise a time-to-live +(TTL) of @var{ttl}. @var{ttl} must denote a duration: @code{5d} means 5 +days, @code{1m} means 1 month, and so on. -@subsection Networking +This allows the user's Guix to keep substitute information in cache for +@var{ttl}. However, note that @code{guix publish} does not itself +guarantee that the store items it provides will indeed remain available +for as long as @var{ttl}. -Run the following command to see what your network interfaces are called: +Additionally, when @option{--cache} is used, cached entries that have +not been accessed for @var{ttl} and that no longer have a corresponding +item in the store, may be deleted. -@example -ifconfig -a -@end example +@item --nar-path=@var{path} +Use @var{path} as the prefix for the URLs of ``nar'' files +(@pxref{Invoking guix archive, normalized archives}). -@noindent -@dots{} or, using the GNU/Linux-specific @command{ip} command: +By default, nars are served at a URL such as +@code{/nar/gzip/@dots{}-coreutils-8.25}. This option allows you to +change the @code{/nar} part to @var{path}. -@example -ip a -@end example +@item --public-key=@var{file} +@itemx --private-key=@var{file} +Use the specific @var{file}s as the public/private key pair used to sign +the store items being published. -@c http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-net_id.c#n20 -Wired interfaces have a name starting with @samp{e}; for example, the -interface corresponding to the first on-board Ethernet controller is -called @samp{eno1}. Wireless interfaces have a name starting with -@samp{w}, like @samp{w1p2s0}. +The files must correspond to the same key pair (the private key is used +for signing and the public key is merely advertised in the signature +metadata). They must contain keys in the canonical s-expression format +as produced by @command{guix archive --generate-key} (@pxref{Invoking +guix archive}). By default, @file{/etc/guix/signing-key.pub} and +@file{/etc/guix/signing-key.sec} are used. -@table @asis -@item Wired connection -To configure a wired network run the following command, substituting -@var{interface} with the name of the wired interface you want to use. +@item --repl[=@var{port}] +@itemx -r [@var{port}] +Spawn a Guile REPL server (@pxref{REPL Servers,,, guile, GNU Guile +Reference Manual}) on @var{port} (37146 by default). This is used +primarily for debugging a running @command{guix publish} server. +@end table -@example -ifconfig @var{interface} up -@end example +Enabling @command{guix publish} on a GuixSD system is a one-liner: just +instantiate a @code{guix-publish-service-type} service in the @code{services} field +of the @code{operating-system} declaration (@pxref{guix-publish-service-type, +@code{guix-publish-service-type}}). -@item Wireless connection -@cindex wireless -@cindex WiFi -To configure wireless networking, you can create a configuration file -for the @command{wpa_supplicant} configuration tool (its location is not -important) using one of the available text editors such as -@command{nano}: +If you are instead running Guix on a ``foreign distro'', follow these +instructions:” + +@itemize +@item +If your host distro uses the systemd init system: @example -nano wpa_supplicant.conf +# ln -s ~root/.guix-profile/lib/systemd/system/guix-publish.service \ + /etc/systemd/system/ +# systemctl start guix-publish && systemctl enable guix-publish @end example -As an example, the following stanza can go to this file and will work -for many wireless networks, provided you give the actual SSID and -passphrase for the network you are connecting to: +@item +If your host distro uses the Upstart init system: @example -network=@{ - ssid="@var{my-ssid}" - key_mgmt=WPA-PSK - psk="the network's secret passphrase" -@} +# ln -s ~root/.guix-profile/lib/upstart/system/guix-publish.conf /etc/init/ +# start guix-publish @end example -Start the wireless service and run it in the background with the -following command (substitute @var{interface} with the name of the -network interface you want to use): +@item +Otherwise, proceed similarly with your distro's init system. +@end itemize -@example -wpa_supplicant -c wpa_supplicant.conf -i @var{interface} -B -@end example +@node Invoking guix challenge +@section Invoking @command{guix challenge} -Run @command{man wpa_supplicant} for more information. -@end table +@cindex reproducible builds +@cindex verifiable builds +@cindex @command{guix challenge} +@cindex challenge +Do the binaries provided by this server really correspond to the source +code it claims to build? Is a package build process deterministic? +These are the questions the @command{guix challenge} command attempts to +answer. -@cindex DHCP -At this point, you need to acquire an IP address. On a network where IP -addresses are automatically assigned @i{via} DHCP, you can run: +The former is obviously an important question: Before using a substitute +server (@pxref{Substitutes}), one had better @emph{verify} that it +provides the right binaries, and thus @emph{challenge} it. The latter +is what enables the former: If package builds are deterministic, then +independent builds of the package should yield the exact same result, +bit for bit; if a server provides a binary different from the one +obtained locally, it may be either corrupt or malicious. + +We know that the hash that shows up in @file{/gnu/store} file names is +the hash of all the inputs of the process that built the file or +directory---compilers, libraries, build scripts, +etc. (@pxref{Introduction}). Assuming deterministic build processes, +one store file name should map to exactly one build output. +@command{guix challenge} checks whether there is, indeed, a single +mapping by comparing the build outputs of several independent builds of +any given store item. + +The command output looks like this: + +@smallexample +$ guix challenge --substitute-urls="https://@value{SUBSTITUTE-SERVER} https://guix.example.org" +updating list of substitutes from 'https://@value{SUBSTITUTE-SERVER}'... 100.0% +updating list of substitutes from 'https://guix.example.org'... 100.0% +/gnu/store/@dots{}-openssl-1.0.2d contents differ: + local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-openssl-1.0.2d: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q + https://guix.example.org/nar/@dots{}-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim +/gnu/store/@dots{}-git-2.5.0 contents differ: + local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f + https://guix.example.org/nar/@dots{}-git-2.5.0: 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73 +/gnu/store/@dots{}-pius-2.1.1 contents differ: + local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-pius-2.1.1: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax + https://guix.example.org/nar/@dots{}-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs -@example -dhclient -v @var{interface} -@end example +@dots{} -Try to ping a server to see if networking is up and running: +6,406 store items were analyzed: + - 4,749 (74.1%) were identical + - 525 (8.2%) differed + - 1,132 (17.7%) were inconclusive +@end smallexample -@example -ping -c 3 gnu.org -@end example +@noindent +In this example, @command{guix challenge} first scans the store to +determine the set of locally-built derivations---as opposed to store +items that were downloaded from a substitute server---and then queries +all the substitute servers. It then reports those store items for which +the servers obtained a result different from the local build. -Setting up network access is almost always a requirement because the -image does not contain all the software and tools that may be needed. +@cindex non-determinism, in package builds +As an example, @code{guix.example.org} always gets a different answer. +Conversely, @code{@value{SUBSTITUTE-SERVER}} agrees with local builds, except in the +case of Git. This might indicate that the build process of Git is +non-deterministic, meaning that its output varies as a function of +various things that Guix does not fully control, in spite of building +packages in isolated environments (@pxref{Features}). Most common +sources of non-determinism include the addition of timestamps in build +results, the inclusion of random numbers, and directory listings sorted +by inode number. See @uref{https://reproducible-builds.org/docs/}, for +more information. -@cindex installing over SSH -If you want to, you can continue the installation remotely by starting -an SSH server: +To find out what is wrong with this Git binary, we can do something along +these lines (@pxref{Invoking guix archive}): @example -herd start ssh-daemon +$ wget -q -O - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0 \ + | guix archive -x /tmp/git +$ diff -ur --no-dereference /gnu/store/@dots{}-git.2.5.0 /tmp/git @end example -Make sure to either set a password with @command{passwd}, or configure -OpenSSH public key authentication before logging in. - -@subsection Disk Partitioning +This command shows the difference between the files resulting from the +local build, and the files resulting from the build on +@code{@value{SUBSTITUTE-SERVER}} (@pxref{Overview, Comparing and Merging Files,, +diffutils, Comparing and Merging Files}). The @command{diff} command +works great for text files. When binary files differ, a better option +is @uref{https://diffoscope.org/, Diffoscope}, a tool that helps +visualize differences for all kinds of files. -Unless this has already been done, the next step is to partition, and -then format the target partition(s). +Once you have done that work, you can tell whether the differences are due +to a non-deterministic build process or to a malicious server. We try +hard to remove sources of non-determinism in packages to make it easier +to verify substitutes, but of course, this is a process that +involves not just Guix, but a large part of the free software community. +In the meantime, @command{guix challenge} is one tool to help address +the problem. -The installation image includes several partitioning tools, including -Parted (@pxref{Overview,,, parted, GNU Parted User Manual}), -@command{fdisk}, and @command{cfdisk}. Run it and set up your disk with -the partition layout you want: +If you are writing packages for Guix, you are encouraged to check +whether @code{@value{SUBSTITUTE-SERVER}} and other substitute servers obtain the +same build result as you did with: @example -cfdisk +$ guix challenge @var{package} @end example -If your disk uses the GUID Partition Table (GPT) format and you plan to -install BIOS-based GRUB (which is the default), make sure a BIOS Boot -Partition is available (@pxref{BIOS installation,,, grub, GNU GRUB -manual}). +@noindent +where @var{package} is a package specification such as +@code{guile@@2.0} or @code{glibc:debug}. -@cindex EFI, installation -@cindex UEFI, installation -@cindex ESP, EFI system partition -If you instead wish to use EFI-based GRUB, a FAT32 @dfn{EFI System Partition} -(ESP) is required. This partition should be mounted at @file{/boot/efi} and -must have the @code{esp} flag set. E.g., for @command{parted}: +The general syntax is: @example -parted /dev/sda set 1 esp on +guix challenge @var{options} [@var{packages}@dots{}] @end example -@quotation Note -@vindex grub-bootloader -@vindex grub-efi-bootloader -Unsure whether to use EFI- or BIOS-based GRUB? If the directory -@file{/sys/firmware/efi} exists in the installation image, then you should -probably perform an EFI installation, using @code{grub-efi-bootloader}. -Otherwise you should use the BIOS-based GRUB, known as -@code{grub-bootloader}. @xref{Bootloader Configuration}, for more info on -bootloaders. -@end quotation +When a difference is found between the hash of a locally-built item and +that of a server-provided substitute, or among substitutes provided by +different servers, the command displays it as in the example above and +its exit code is 2 (other non-zero exit codes denote other kinds of +errors.) -Once you are done partitioning the target hard disk drive, you have to -create a file system on the relevant partition(s)@footnote{Currently -GuixSD only supports ext4 and btrfs file systems. In particular, code -that reads file system UUIDs and labels only works for these file system -types.}. For the ESP, if you have one and assuming it is -@file{/dev/sda1}, run: +The one option that matters is: -@example -mkfs.fat -F32 /dev/sda1 -@end example +@table @code -Preferably, assign file systems a label so that you can easily and -reliably refer to them in @code{file-system} declarations (@pxref{File -Systems}). This is typically done using the @code{-L} option of -@command{mkfs.ext4} and related commands. So, assuming the target root -partition lives at @file{/dev/sda2}, a file system with the label -@code{my-root} can be created with: +@item --substitute-urls=@var{urls} +Consider @var{urls} the whitespace-separated list of substitute source +URLs to compare to. -@example -mkfs.ext4 -L my-root /dev/sda2 -@end example +@item --verbose +@itemx -v +Show details about matches (identical contents) in addition to +information about mismatches. -@cindex encrypted disk -If you are instead planning to encrypt the root partition, you can use -the Cryptsetup/LUKS utilities to do that (see @inlinefmtifelse{html, -@uref{https://linux.die.net/man/8/cryptsetup, @code{man cryptsetup}}, -@code{man cryptsetup}} for more information.) Assuming you want to -store the root partition on @file{/dev/sda2}, the command sequence would -be along these lines: +@end table -@example -cryptsetup luksFormat /dev/sda2 -cryptsetup open --type luks /dev/sda2 my-partition -mkfs.ext4 -L my-root /dev/mapper/my-partition -@end example +@node Invoking guix copy +@section Invoking @command{guix copy} -Once that is done, mount the target file system under @file{/mnt} -with a command like (again, assuming @code{my-root} is the label of the -root file system): +@cindex copy, of store items, over SSH +@cindex SSH, copy of store items +@cindex sharing store items across machines +@cindex transferring store items across machines +The @command{guix copy} command copies items from the store of one +machine to that of another machine over a secure shell (SSH) +connection@footnote{This command is available only when Guile-SSH was +found. @xref{Requirements}, for details.}. For example, the following +command copies the @code{coreutils} package, the user's profile, and all +their dependencies over to @var{host}, logged in as @var{user}: @example -mount LABEL=my-root /mnt +guix copy --to=@var{user}@@@var{host} \ + coreutils `readlink -f ~/.guix-profile` @end example -Also mount any other file systems you would like to use on the target -system relative to this path. If you have @file{/boot} on a separate -partition for example, mount it at @file{/mnt/boot} now so it is found -by @code{guix system init} afterwards. +If some of the items to be copied are already present on @var{host}, +they are not actually sent. -Finally, if you plan to use one or more swap partitions (@pxref{Memory -Concepts, swap space,, libc, The GNU C Library Reference Manual}), make -sure to initialize them with @command{mkswap}. Assuming you have one -swap partition on @file{/dev/sda3}, you would run: +The command below retrieves @code{libreoffice} and @code{gimp} from +@var{host}, assuming they are available there: @example -mkswap /dev/sda3 -swapon /dev/sda3 +guix copy --from=@var{host} libreoffice gimp @end example -Alternatively, you may use a swap file. For example, assuming that in -the new system you want to use the file @file{/swapfile} as a swap file, -you would run@footnote{This example will work for many types of file -systems (e.g., ext4). However, for copy-on-write file systems (e.g., -btrfs), the required steps may be different. For details, see the -manual pages for @command{mkswap} and @command{swapon}.}: +The SSH connection is established using the Guile-SSH client, which is +compatible with OpenSSH: it honors @file{~/.ssh/known_hosts} and +@file{~/.ssh/config}, and uses the SSH agent for authentication. + +The key used to sign items that are sent must be accepted by the remote +machine. Likewise, the key used by the remote machine to sign items you +are retrieving must be in @file{/etc/guix/acl} so it is accepted by your +own daemon. @xref{Invoking guix archive}, for more information about +store item authentication. + +The general syntax is: @example -# This is 10 GiB of swap space. Adjust "count" to change the size. -dd if=/dev/zero of=/mnt/swapfile bs=1MiB count=10240 -# For security, make the file readable and writable only by root. -chmod 600 /mnt/swapfile -mkswap /mnt/swapfile -swapon /mnt/swapfile +guix copy [--to=@var{spec}|--from=@var{spec}] @var{items}@dots{} @end example -Note that if you have encrypted the root partition and created a swap -file in its file system as described above, then the encryption also -protects the swap file, just like any other file in that file system. +You must always specify one of the following options: -@node Proceeding with the Installation -@section Proceeding with the Installation +@table @code +@item --to=@var{spec} +@itemx --from=@var{spec} +Specify the host to send to or receive from. @var{spec} must be an SSH +spec such as @code{example.org}, @code{charlie@@example.org}, or +@code{charlie@@example.org:2222}. +@end table -With the target partitions ready and the target root mounted on -@file{/mnt}, we're ready to go. First, run: +The @var{items} can be either package names, such as @code{gimp}, or +store items, such as @file{/gnu/store/@dots{}-idutils-4.6}. -@example -herd start cow-store /mnt -@end example +When specifying the name of a package to send, it is first built if +needed, unless @option{--dry-run} was specified. Common build options +are supported (@pxref{Common Build Options}). -This makes @file{/gnu/store} copy-on-write, such that packages added to it -during the installation phase are written to the target disk on @file{/mnt} -rather than kept in memory. This is necessary because the first phase of -the @command{guix system init} command (see below) entails downloads or -builds to @file{/gnu/store} which, initially, is an in-memory file system. -Next, you have to edit a file and -provide the declaration of the operating system to be installed. To -that end, the installation system comes with three text editors. We -recommend GNU nano (@pxref{Top,,, nano, GNU nano Manual}), which -supports syntax highlighting and parentheses matching; other editors -include GNU Zile (an Emacs clone), and -nvi (a clone of the original BSD @command{vi} editor). -We strongly recommend storing that file on the target root file system, say, -as @file{/mnt/etc/config.scm}. Failing to do that, you will have lost your -configuration file once you have rebooted into the newly-installed system. +@node Invoking guix container +@section Invoking @command{guix container} +@cindex container +@cindex @command{guix container} +@quotation Note +As of version @value{VERSION}, this tool is experimental. The interface +is subject to radical change in the future. +@end quotation -@xref{Using the Configuration System}, for an overview of the -configuration file. The example configurations discussed in that -section are available under @file{/etc/configuration} in the -installation image. Thus, to get started with a system configuration -providing a graphical display server (a ``desktop'' system), you can run -something along these lines: +The purpose of @command{guix container} is to manipulate processes +running within an isolated environment, commonly known as a +``container'', typically created by the @command{guix environment} +(@pxref{Invoking guix environment}) and @command{guix system container} +(@pxref{Invoking guix system}) commands. + +The general syntax is: @example -# mkdir /mnt/etc -# cp /etc/configuration/desktop.scm /mnt/etc/config.scm -# nano /mnt/etc/config.scm +guix container @var{action} @var{options}@dots{} @end example -You should pay attention to what your configuration file contains, and -in particular: - -@itemize -@item -Make sure the @code{bootloader-configuration} form refers to the target -you want to install GRUB on. It should mention @code{grub-bootloader} if -you are installing GRUB in the legacy way, or @code{grub-efi-bootloader} -for newer UEFI systems. For legacy systems, the @code{target} field -names a device, like @code{/dev/sda}; for UEFI systems it names a path -to a mounted EFI partition, like @code{/boot/efi}, and do make sure the -path is actually mounted. +@var{action} specifies the operation to perform with a container, and +@var{options} specifies the context-specific arguments for the action. -@item -Be sure that your file system labels match the value of their respective -@code{device} fields in your @code{file-system} configuration, assuming -your @code{file-system} configuration uses the @code{file-system-label} -procedure in its @code{device} field. +The following actions are available: -@item -If there are encrypted or RAID partitions, make sure to add a -@code{mapped-devices} field to describe them (@pxref{Mapped Devices}). -@end itemize +@table @code +@item exec +Execute a command within the context of a running container. -Once you are done preparing the configuration file, the new system must -be initialized (remember that the target root file system is mounted -under @file{/mnt}): +The syntax is: @example -guix system init /mnt/etc/config.scm /mnt +guix container exec @var{pid} @var{program} @var{arguments}@dots{} @end example -@noindent -This copies all the necessary files and installs GRUB on -@file{/dev/sdX}, unless you pass the @option{--no-bootloader} option. For -more information, @pxref{Invoking guix system}. This command may trigger -downloads or builds of missing packages, which can take some time. - -Once that command has completed---and hopefully succeeded!---you can run -@command{reboot} and boot into the new system. The @code{root} password -in the new system is initially empty; other users' passwords need to be -initialized by running the @command{passwd} command as @code{root}, -unless your configuration specifies otherwise -(@pxref{user-account-password, user account passwords}). +@var{pid} specifies the process ID of the running container. +@var{program} specifies an executable file name within the root file +system of the container. @var{arguments} are the additional options that +will be passed to @var{program}. -@cindex upgrading GuixSD -From then on, you can update GuixSD whenever you want by running @command{guix -pull} as @code{root} (@pxref{Invoking guix pull}), and then running -@command{guix system reconfigure /etc/config.scm}, as @code{root} too, to -build a new system generation with the latest packages and services -(@pxref{Invoking guix system}). We recommend doing that regularly so that -your system includes the latest security updates (@pxref{Security Updates}). +The following command launches an interactive login shell inside a +GuixSD container, started by @command{guix system container}, and whose +process ID is 9001: -Join us on @code{#guix} on the Freenode IRC network or on -@email{guix-devel@@gnu.org} to share your experience---good or not so -good. +@example +guix container exec 9001 /run/current-system/profile/bin/bash --login +@end example -@node Installing GuixSD in a VM -@section Installing GuixSD in a Virtual Machine +Note that the @var{pid} cannot be the parent process of a container. It +must be PID 1 of the container or one of its child processes. -@cindex virtual machine, GuixSD installation -@cindex virtual private server (VPS) -@cindex VPS (virtual private server) -If you'd like to install GuixSD in a virtual machine (VM) or on a -virtual private server (VPS) rather than on your beloved machine, this -section is for you. +@end table -To boot a @uref{http://qemu.org/,QEMU} VM for installing GuixSD in a -disk image, follow these steps: +@node Invoking guix weather +@section Invoking @command{guix weather} -@enumerate -@item -First, retrieve and decompress the GuixSD installation image as -described previously (@pxref{USB Stick and DVD Installation}). +Occasionally you're grumpy because substitutes are lacking and you end +up building packages by yourself (@pxref{Substitutes}). The +@command{guix weather} command reports on substitute availability on the +specified servers so you can have an idea of whether you'll be grumpy +today. It can sometimes be useful info as a user, but it is primarily +useful to people running @command{guix publish} (@pxref{Invoking guix +publish}). -@item -Create a disk image that will hold the installed system. To make a -qcow2-formatted disk image, use the @command{qemu-img} command: +@cindex statistics, for substitutes +@cindex availability of substitutes +@cindex substitute availability +@cindex weather, substitute availability +Here's a sample run: @example -qemu-img create -f qcow2 guixsd.img 50G -@end example +$ guix weather --substitute-urls=https://guix.example.org +computing 5,872 package derivations for x86_64-linux... +looking for 6,128 store items on https://guix.example.org.. +updating list of substitutes from 'https://guix.example.org'... 100.0% +https://guix.example.org + 43.4% substitutes available (2,658 out of 6,128) + 7,032.5 MiB of nars (compressed) + 19,824.2 MiB on disk (uncompressed) + 0.030 seconds per request (182.9 seconds in total) + 33.5 requests per second -The resulting file will be much smaller than 50 GB (typically less than -1 MB), but it will grow as the virtualized storage device is filled up. + 9.8% (342 out of 3,470) of the missing items are queued + 867 queued builds + x86_64-linux: 518 (59.7%) + i686-linux: 221 (25.5%) + aarch64-linux: 128 (14.8%) + build rate: 23.41 builds per hour + x86_64-linux: 11.16 builds per hour + i686-linux: 6.03 builds per hour + aarch64-linux: 6.41 builds per hour +@end example -@item -Boot the USB installation image in an VM: +@cindex continuous integration, statistics +As you can see, it reports the fraction of all the packages for which +substitutes are available on the server---regardless of whether +substitutes are enabled, and regardless of whether this server's signing +key is authorized. It also reports the size of the compressed archives +(``nars'') provided by the server, the size the corresponding store +items occupy in the store (assuming deduplication is turned off), and +the server's throughput. The second part gives continuous integration +(CI) statistics, if the server supports it. -@example -qemu-system-x86_64 -m 1024 -smp 1 \ - -net user -net nic,model=virtio -boot menu=on \ - -drive file=guixsd-install-@value{VERSION}.@var{system}.iso \ - -drive file=guixsd.img -@end example +To achieve that, @command{guix weather} queries over HTTP(S) meta-data +(@dfn{narinfos}) for all the relevant store items. Like @command{guix +challenge}, it ignores signatures on those substitutes, which is +innocuous since the command only gathers statistics and cannot install +those substitutes. -The ordering of the drives matters. +Among other things, it is possible to query specific system types and +specific package sets. The available options are listed below. -In the VM console, quickly press the @kbd{F12} key to enter the boot -menu. Then press the @kbd{2} key and the @kbd{RET} key to validate your -selection. +@table @code +@item --substitute-urls=@var{urls} +@var{urls} is the space-separated list of substitute server URLs to +query. When this option is omitted, the default set of substitute +servers is queried. -@item -You're now root in the VM, proceed with the installation process. -@xref{Preparing for Installation}, and follow the instructions. -@end enumerate +@item --system=@var{system} +@itemx -s @var{system} +Query substitutes for @var{system}---e.g., @code{aarch64-linux}. This +option can be repeated, in which case @command{guix weather} will query +substitutes for several system types. -Once installation is complete, you can boot the system that's on your -@file{guixsd.img} image. @xref{Running GuixSD in a VM}, for how to do -that. +@item --manifest=@var{file} +Instead of querying substitutes for all the packages, only ask for those +specified in @var{file}. @var{file} must contain a @dfn{manifest}, as +with the @code{-m} option of @command{guix package} (@pxref{Invoking +guix package}). +@end table -@node Building the Installation Image -@section Building the Installation Image +@node Invoking guix processes +@section Invoking @command{guix processes} -@cindex installation image -The installation image described above was built using the @command{guix -system} command, specifically: +The @command{guix processes} command can be useful to developers and system +administrators, especially on multi-user machines and on build farms: it lists +the current sessions (connections to the daemon), as well as information about +the processes involved@footnote{Remote sessions, when @command{guix-daemon} is +started with @option{--listen} specifying a TCP endpoint, are @emph{not} +listed.}. Here's an example of the information it returns: @example -guix system disk-image gnu/system/install.scm -@end example +$ sudo guix processes +SessionPID: 19002 +ClientPID: 19090 +ClientCommand: guix environment --ad-hoc python -Have a look at @file{gnu/system/install.scm} in the source tree, -and see also @ref{Invoking guix system} for more information -about the installation image. +SessionPID: 19402 +ClientPID: 19367 +ClientCommand: guix publish -u guix-publish -p 3000 -C 9 @dots{} -@section Building the Installation Image for ARM Boards +SessionPID: 19444 +ClientPID: 19419 +ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} +LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock +LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock +LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock +ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800 +ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800 +ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800 +@end example -Many ARM boards require a specific variant of the -@uref{http://www.denx.de/wiki/U-Boot/, U-Boot} bootloader. +In this example we see that @command{guix-daemon} has three clients: +@command{guix environment}, @command{guix publish}, and the Cuirass continuous +integration tool; their process identifier (PID) is given by the +@code{ClientPID} field. The @code{SessionPID} field gives the PID of the +@command{guix-daemon} sub-process of this particular session. -If you build a disk image and the bootloader is not available otherwise -(on another boot drive etc), it's advisable to build an image that -includes the bootloader, specifically: +The @code{LockHeld} fields show which store items are currently locked by this +session, which corresponds to store items being built or substituted (the +@code{LockHeld} field is not displayed when @command{guix processes} is not +running as root.) Last, by looking at the @code{ChildProcess} field, we +understand that these three builds are being offloaded (@pxref{Daemon Offload +Setup}). + +The output is in Recutils format so we can use the handy @command{recsel} +command to select sessions of interest (@pxref{Selection Expressions,,, +recutils, GNU recutils manual}). As an example, the command shows the command +line and PID of the client that triggered the build of a Perl package: @example -guix system disk-image --system=armhf-linux -e '((@@ (gnu system install) os-with-u-boot) (@@ (gnu system install) installation-os) "A20-OLinuXino-Lime2")' +$ sudo guix processes | \ + recsel -p ClientPID,ClientCommand -e 'LockHeld ~ "perl"' +ClientPID: 19419 +ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} @end example -@code{A20-OLinuXino-Lime2} is the name of the board. If you specify an invalid -board, a list of possible boards will be printed. @node System Configuration @chapter System Configuration -- cgit v1.2.3