guix.texi

actions, such as making directories, invoking @command{make}, etc.

To describe a derivation and its build actions, one typically needs to
embed build code inside host code.  It boils down to manipulating build
code as data, and Scheme's homoiconicity---code has a direct
representation as data---comes in handy for that.  But we need more than
Scheme's normal @code{quasiquote} mechanism to construct build
expressions.

The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of
S-expressions adapted to build expressions.  G-expressions, or
@dfn{gexps}, consist essentially in three syntactic forms: @code{gexp},
@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~},
@code{#$}, and @code{#$@@}), which are comparable respectively to
@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}
(@pxref{Expression Syntax, @code{quasiquote},, guile, GNU Guile
Reference Manual}).  However, there are major differences:

@itemize
@item
Gexps are meant to be written to a file and run or manipulated by other
processes.

@item
When a package or derivation is unquoted inside a gexp, the result is as
if its output file name had been introduced.

@item
Gexps carry information about the packages or derivations they refer to,
and these dependencies are automatically added as inputs to the build
processes that use them.
@end itemize

To illustrate the idea, here is an example of a gexp:

@example
(define build-exp
  #~(begin
      (mkdir #$output)
      (chdir #$output)
      (symlink (string-append #$coreutils "/bin/ls") 
               "list-files")))
@end example

This gexp can be passed to @code{gexp->derivation}; we obtain a
derivation that builds a directory containing exactly one symlink to
@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}:

@example
(gexp->derivation "the-thing" build-exp)
@end example

As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is
substituted to the reference to the @var{coreutils} package in the
actual build code, and @var{coreutils} is automatically made an input to
the derivation.  Likewise, @code{#$output} (equivalent to @code{(ungexp
output)}) is replaced by a string containing the derivation's output
directory name.  The syntactic form to construct gexps is summarized
below.

@deffn {Scheme Syntax} #~@var{exp}
@deffnx {Scheme Syntax} (gexp @var{exp})
Return a G-expression containing @var{exp}.  @var{exp} may contain one
or more of the following forms:

@table @code
@item #$@var{obj}
@itemx (ungexp @var{obj})
Introduce a reference to @var{obj}.  @var{obj} may be a package or a
derivation, in which case the @code{ungexp} form is replaced by its
output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}.

If @var{obj} is a list, it is traversed and any package or derivation
references are substituted similarly.

If @var{obj} is another gexp, its contents are inserted and its
dependencies are added to those of the containing gexp.

If @var{obj} is another kind of object, it is inserted as is.

@item #$@var{package-or-derivation}:@var{output}
@itemx (ungexp @var{package-or-derivation} @var{output})
This is like the form above, but referring explicitly to the
@var{output} of @var{package-or-derivation}---this is useful when
@var{package-or-derivation} produces multiple outputs (@pxref{Packages
with Multiple Outputs}).

@item #$output[:@var{output}]
@itemx (ungexp output [@var{output}])
Insert a reference to derivation output @var{output}, or to the main
output when @var{output} is omitted.

This only makes sense for gexps passed to @code{gexp->derivation}.

@item #$@@@var{lst}
@itemx (ungexp-splicing @var{lst})
Like the above, but splices the contents of @var{lst} inside the
containing list.

@end table

G-expressions created by @code{gexp} or @code{#~} are run-time objects
of the @code{gexp?} type (see below.)
@end deffn

@deffn {Scheme Procedure} gexp? @var{obj}
Return @code{#t} if @var{obj} is a G-expression.
@end deffn

G-expressions are meant to be written to disk, either as code building
some derivation, or as plain files in the store.  The monadic procedures
below allow you to do that (@pxref{The Store Monad}, for more
information about monads.)

@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @
       [#:system (%current-system)] [#:inputs '()] @
       [#:hash #f] [#:hash-algo #f] @
       [#:recursive? #f] [#:env-vars '()] [#:modules '()] @
       [#:references-graphs #f] [#:local-build? #f] @
       [#:guile-for-build #f]
Return a derivation @var{name} that runs @var{exp} (a gexp) with
@var{guile-for-build} (a derivation) on @var{system}.

Make @var{modules} available in the evaluation context of @var{EXP};
@var{MODULES} is a list of names of Guile modules from the current
search path to be copied in the store, compiled, and made available in
the load path during the execution of @var{exp}---e.g., @code{((guix
build utils) (guix build gnu-build-system))}.

The other arguments are as for @code{derivation} (@pxref{Derivations}).
@end deffn

@deffn {Monadic Procedure} gexp->script @var{name} @var{exp}
Return an executable script @var{name} that runs @var{exp} using
@var{guile} with @var{modules} in its search path.

The example below builds a script that simply invokes the @command{ls}
command:

@example
(use-modules (guix gexp) (gnu packages base))

(gexp->script "list-files"
              #~(execl (string-append #$coreutils "/bin/ls")
                       "ls"))
@end example

When ``running'' it through the store (@pxref{The Store Monad,
@code{run-with-store}}), we obtain a derivation that produces an
executable file @file{/gnu/store/@dots{}-list-files} along these lines:

@example
#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds
!#
(execl (string-append "/gnu/store/@dots{}-coreutils-8.22"/bin/ls")
       "ls")
@end example
@end deffn

@deffn {Monadic Procedure} gexp->file @var{name} @var{exp}
Return a derivation that builds a file @var{name} containing @var{exp}.

The resulting file holds references to all the dependencies of @var{exp}
or a subset thereof.
@end deffn

Of course, in addition to gexps embedded in ``host'' code, there are
also modules containing build tools.  To make it clear that they are
meant to be used in the build stratum, these modules are kept in the
@code{(guix build @dots{})} name space.


@c *********************************************************************
@node Utilities
@chapter Utilities

This section describes tools primarily targeted at developers and users
who write new package definitions.  They complement the Scheme
programming interface of Guix in a convenient way.

@menu
* Invoking guix build::         Building packages from the command line.
* Invoking guix download::      Downloading a file and printing its hash.
* Invoking guix hash::          Computing the cryptographic hash of a file.
* Invoking guix refresh::       Updating package definitions.
@end menu

@node Invoking guix build
@section Invoking @command{guix build}

The @command{guix build} command builds packages or derivations and
their dependencies, and prints the resulting store paths.  Note that it
does not modify the user's profile---this is the job of the
@command{guix package} command (@pxref{Invoking guix package}).  Thus,
it is mainly useful for distribution developers.

The general syntax is:

@example
guix build @var{options} @var{package-or-derivation}@dots{}
@end example

@var{package-or-derivation} may be either the name of a package found in
the software distribution such as @code{coreutils} or
@code{coreutils-8.20}, or a derivation such as
@file{/gnu/store/@dots{}-coreutils-8.19.drv}.  In the former case, a
package with the corresponding name (and optionally version) is searched
for among the GNU distribution modules (@pxref{Package Modules}).

Alternatively, the @code{--expression} option may be used to specify a
Scheme expression that evaluates to a package; this is useful when
disambiguation among several same-named packages or package variants is
needed.

The @var{options} may be zero or more of the following:

@table @code

@item --expression=@var{expr}
@itemx -e @var{expr}
Build the package or derivation @var{expr} evaluates to.

For example, @var{expr} may be @code{(@@ (gnu packages guile)
guile-1.8)}, which unambiguously designates this specific variant of
version 1.8 of Guile.

Alternately, @var{expr} may refer to a zero-argument monadic procedure
(@pxref{The Store Monad}).  The procedure must return a derivation as a
monadic value, which is then passed through @code{run-with-store}.

@item --source
@itemx -S
Build the packages' source derivations, rather than the packages
themselves.

For instance, @code{guix build -S gcc} returns something like
@file{/gnu/store/@dots{}-gcc-4.7.2.tar.bz2}, which is GCC's source tarball.

The returned source tarball is the result of applying any patches and
code snippets specified in the package's @code{origin} (@pxref{Defining
Packages}).

@item --system=@var{system}
@itemx -s @var{system}
Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of
the host's system type.

An example use of this is on Linux-based systems, which can emulate
different personalities.  For instance, passing
@code{--system=i686-linux} on an @code{x86_64-linux} system allows users
to build packages in a complete 32-bit environment.

@item --target=@var{triplet}
@cindex cross-compilation
Cross-build for @var{triplet}, which must be a valid GNU triplet, such
as @code{"mips64el-linux-gnu"} (@pxref{Configuration Names, GNU
configuration triplets,, configure, GNU Configure and Build System}).

@item --with-source=@var{source}
Use @var{source} as the source of the corresponding package.
@var{source} must be a file name or a URL, as for @command{guix
download} (@pxref{Invoking guix download}).

The ``corresponding package'' is taken to be one specified on the
command line whose name matches the base of @var{source}---e.g., if
@var{source} is @code{/src/guile-2.0.10.tar.gz}, the corresponding
package is @code{guile}.  Likewise, the version string is inferred from
@var{source}; in the previous example, it's @code{2.0.10}.

This option allows users to try out versions of packages other than the
one provided by the distribution.  The example below downloads
@file{ed-1.7.tar.gz} from a GNU mirror and uses that as the source for
the @code{ed} package:

@example
guix build ed --with-source=mirror://gnu/ed/ed-1.7.tar.gz
@end example

As a developer, @code{--with-source} makes it easy to test release
candidates:

@example
guix build guile --with-source=../guile-2.0.9.219-e1bb7.tar.xz
@end example


@item --derivations
@itemx -d
Return the derivation paths, not the output paths, of the given
packages.

@item --root=@var{file}
@itemx -r @var{file}
Make @var{file} a symlink to the result, and register it as a garbage
collector root.

@item --log-file
Return the build log file names for the given
@var{package-or-derivation}s, or raise an error if build logs are
missing.

This works regardless of how packages or derivations are specified.  For
instance, the following invocations are equivalent:

@example
guix build --log-file `guix build -d guile`
guix build --log-file `guix build guile`
guix build --log-file guile
guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)'
@end example


@end table

@cindex common build options
In addition, a number of options that control the build process are
common to @command{guix build} and other commands that can spawn builds,
such as @command{guix package} or @command{guix archive}.  These are the
following:

@table @code

@item --keep-failed
@itemx -K
Keep the build tree of failed builds.  Thus, if a build fail, its build
tree is kept under @file{/tmp}, in a directory whose name is shown at
the end of the build log.  This is useful when debugging build issues.

@item --dry-run
@itemx -n
Do not build the derivations.

@item --fallback
When substituting a pre-built binary fails, fall back to building
packages locally.

@item --no-substitutes
Do not use substitutes for build products.  That is, always build things
locally instead of allowing downloads of pre-built binaries
(@pxref{Substitutes}).

@item --no-build-hook
Do not attempt to offload builds @i{via} the daemon's ``build hook''
(@pxref{Daemon Offload Setup}).  That is, always build things locally
instead of offloading builds to remote machines.

@item --max-silent-time=@var{seconds}
When the build or substitution process remains silent for more than
@var{seconds}, terminate it and report a build failure.

@item --timeout=@var{seconds}
Likewise, when the build or substitution process lasts for more than
@var{seconds}, terminate it and report a build failure.

By default there is no timeout.  This behavior can be restored with
@code{--timeout=0}.

@item --verbosity=@var{level}
Use the given verbosity level.  @var{level} must be an integer between 0
and 5; higher means more verbose output.  Setting a level of 4 or more
may be helpful when debugging setup issues with the build daemon.

@item --cores=@var{n}
@itemx -c @var{n}
Allow the use of up to @var{n} CPU cores for the build.  The special
value @code{0} means to use as many CPU cores as available.

@end table

Behind the scenes, @command{guix build} is essentially an interface to
the @code{package-derivation} procedure of the @code{(guix packages)}
module, and to the @code{build-derivations} procedure of the @code{(guix
store)} module.

@node Invoking guix download
@section Invoking @command{guix download}

When writing a package definition, developers typically need to download
the package's source tarball, compute its SHA256 hash, and write that
hash in the package definition (@pxref{Defining Packages}).  The
@command{guix download} tool helps with this task: it downloads a file
from the given URI, adds it to the store, and prints both its file name
in the store and its SHA256 hash.

The fact that the downloaded file is added to the store saves bandwidth:
when the developer eventually tries to build the newly defined package
with @command{guix build}, the source tarball will not have to be
downloaded again because it is already in the store.  It is also a
convenient way to temporarily stash files, which may be deleted
eventually (@pxref{Invoking guix gc}).

The @command{guix download} command supports the same URIs as used in
package definitions.  In particular, it supports @code{mirror://} URIs.
@code{https} URIs (HTTP over TLS) are supported @emph{provided} the
Guile bindings for GnuTLS are available in the user's environment; when
they are not available, an error is raised.

The following option is available:

@table @code
@item --format=@var{fmt}
@itemx -f @var{fmt}
Write the hash in the format specified by @var{fmt}.  For more
information on the valid values for @var{fmt}, @ref{Invoking guix hash}.
@end table

@node Invoking guix hash
@section Invoking @command{guix hash}

The @command{guix hash} command computes the SHA256 hash of a file.
It is primarily a convenience tool for anyone contributing to the
distribution: it computes the cryptographic hash of a file, which can be
used in the definition of a package (@pxref{Defining Packages}).

The general syntax is:

@example
guix hash @var{option} @var{file}
@end example

@command{guix hash} has the following option:

@table @code

@item --format=@var{fmt}
@itemx -f @var{fmt}
Write the hash in the format specified by @var{fmt}.

Supported formats: @code{nix-base32}, @code{base32}, @code{base16}
(@code{hex} and @code{hexadecimal} can be used as well).

If the @option{--format} option is not specified, @command{guix hash}
will output the hash in @code{nix-base32}.  This representation is used
in the definitions of packages.

@item --recursive
@itemx -r
Compute the hash on @var{file} recursively.

In this case, the hash is computed on an archive containing @var{file},
including its children if it is a directory.  Some of @var{file}'s
meta-data is part of the archive; for instance, when @var{file} is a
regular file, the hash is different depending on whether @var{file} is
executable or not.  Meta-data such as time stamps has no impact on the
hash (@pxref{Invoking guix archive}).
@c FIXME: Replace xref above with xref to an ``Archive'' section when
@c it exists.

@end table

@node Invoking guix refresh
@section Invoking @command{guix refresh}

The primary audience of the @command{guix refresh} command is developers
of the GNU software distribution.  By default, it reports any packages
provided by the distribution that are outdated compared to the latest
upstream version, like this:

@example
$ guix refresh
gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1
gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0
@end example

It does so by browsing each package's FTP directory and determining the
highest version number of the source tarballs
therein@footnote{Currently, this only works for GNU packages.}.

When passed @code{--update}, it modifies distribution source files to
update the version numbers and source tarball hashes of those packages'
recipes (@pxref{Defining Packages}).  This is achieved by downloading
each package's latest source tarball and its associated OpenPGP
signature, authenticating the downloaded tarball against its signature
using @command{gpg}, and finally computing its hash.  When the public
key used to sign the tarball is missing from the user's keyring, an
attempt is made to automatically retrieve it from a public key server;
when it's successful, the key is added to the user's keyring; otherwise,
@command{guix refresh} reports an error.

The following options are supported:

@table @code

@item --update
@itemx -u
Update distribution source files (package recipes) in place.
@ref{Defining Packages}, for more information on package definitions.

@item --select=[@var{subset}]
@itemx -s @var{subset}
Select all the packages in @var{subset}, one of @code{core} or
@code{non-core}.

The @code{core} subset refers to all the packages at the core of the
distribution---i.e., packages that are used to build ``everything
else''.  This includes GCC, libc, Binutils, Bash, etc.  Usually,
changing one of these packages in the distribution entails a rebuild of
all the others.  Thus, such updates are an inconvenience to users in
terms of build time or bandwidth used to achieve the upgrade.

The @code{non-core} subset refers to the remaining packages.  It is
typically useful in cases where an update of the core packages would be
inconvenient.

@end table

In addition, @command{guix refresh} can be passed one or more package
names, as in this example:

@example
guix refresh -u emacs idutils
@end example

@noindent
The command above specifically updates the @code{emacs} and
@code{idutils} packages.  The @code{--select} option would have no
effect in this case.

The following options can be used to customize GnuPG operation:

@table @code

@item --key-server=@var{host}
Use @var{host} as the OpenPGP key server when importing a public key.

@item --gpg=@var{command}
Use @var{command} as the GnuPG 2.x command.  @var{command} is searched
for in @code{$PATH}.

@end table


@c *********************************************************************
@node GNU Distribution
@chapter GNU Distribution

Guix comes with a distribution of free software@footnote{The term
``free'' here refers to the
@url{http://www.gnu.org/philosophy/free-sw.html,freedom provided to
users of that software}.} that forms the basis of the GNU system.  This
includes core GNU packages such as GNU libc, GCC, and Binutils, as well
as many GNU and non-GNU applications.  The complete list of available
packages can be browsed
@url{http://www.gnu.org/software/guix/package-list.html,on-line} or by
running @command{guix package} (@pxref{Invoking guix package}):

@example
guix package --list-available
@end example

Our goal is to build a practical 100% free software distribution of
Linux-based and other variants of GNU, with a focus on the promotion and
tight integration of GNU components, and an emphasis on programs and
tools that help users exert that freedom.

The GNU distribution is currently available on the following platforms:

@table @code

@item x86_64-linux
Intel/AMD @code{x86_64} architecture, Linux-Libre kernel;

@item i686-linux
Intel 32-bit architecture (IA32), Linux-Libre kernel;

@item mips64el-linux
little-endian 64-bit MIPS processors, specifically the Loongson series,
n32 application binary interface (ABI), and Linux-Libre kernel.

@end table

@noindent
For information on porting to other architectures or kernels,
@xref{Porting}.

@menu
* Installing Debugging Files::  Feeding the debugger.
* Package Modules::             Packages from the programmer's viewpoint.
* Packaging Guidelines::        Growing the distribution.
* Bootstrapping::               GNU/Linux built from scratch.
* Porting::                     Targeting another platform or kernel.
* System Configuration::        Configuring a GNU system.
@end menu

Building this distribution is a cooperative effort, and you are invited
to join!  @ref{Contributing}, for information about how you can help.


@node Installing Debugging Files
@section Installing Debugging Files

Program binaries, as produced by the GCC compilers for instance, are
typically written in the ELF format, with a section containing
@dfn{debugging information}.  Debugging information is what allows the
debugger, GDB, to map binary code to source code; it is required to
debug a compiled program in good conditions.

The problem with debugging information is that is takes up a fair amount
of disk space.  For example, debugging information for the GNU C Library
weighs in at more than 60 MiB.  Thus, as a user, keeping all the
debugging info of all the installed programs is usually not an option.
Yet, space savings should not come at the cost of an impediment to
debugging---especially in the GNU system, which should make it easier
for users to exert their computing freedom (@pxref{GNU Distribution}).

Thankfully, the GNU Binary Utilities (Binutils) and GDB provide a
mechanism that allows users to get the best of both worlds: debugging
information can be stripped from the binaries and stored in separate
files.  GDB is then able to load debugging information from those files,
when they are available (@pxref{Separate Debug Files,,, gdb, Debugging
with GDB}).

The GNU distribution takes advantage of this by storing debugging
information in the @code{lib/debug} sub-directory of a separate package
output unimaginatively called @code{debug} (@pxref{Packages with
Multiple Outputs}).  Users can choose to install the @code{debug} output
of a package when they need it.  For instance, the following command
installs the debugging information for the GNU C Library and for GNU
Guile:

@example
guix package -i glibc:debug -i guile:debug
@end example

GDB must then be told to look for debug files in the user's profile, by
setting the @code{debug-file-directory} variable (consider setting it
from the @file{~/.gdbinit} file, @pxref{Startup,,, gdb, Debugging with
GDB}):

@example
(gdb) set debug-file-directory ~/.guix-profile/lib/debug
@end example

From there on, GDB will pick up debugging information from the
@code{.debug} files under @file{~/.guix-profile/lib/debug}.

@c XXX: keep me up-to-date
The @code{debug} output mechanism in Guix is implemented by the
@code{gnu-build-system} (@pxref{Defining Packages}).  Currently, it is
opt-in---debugging information is available only for those packages
whose definition explicitly declares a @code{debug} output.  This may be
changed to opt-out in the future, if our build farm servers can handle
the load.  To check whether a package has a @code{debug} output, use
@command{guix package --list-available} (@pxref{Invoking guix package}).


@node Package Modules
@section Package Modules

From a programming viewpoint, the package definitions of the
GNU distribution are provided by Guile modules in the @code{(gnu packages
@dots{})} name space@footnote{Note that packages under the @code{(gnu
packages @dots{})} module name space are not necessarily ``GNU
packages''.  This module naming scheme follows the usual Guile module
naming convention: @code{gnu} means that these modules are distributed
as part of the GNU system, and @code{packages} identifies modules that
define packages.}  (@pxref{Modules, Guile modules,, guile, GNU Guile
Reference Manual}).  For instance, the @code{(gnu packages emacs)}
module exports a variable named @code{emacs}, which is bound to a
@code{<package>} object (@pxref{Defining Packages}).

The @code{(gnu packages @dots{})} module name space is special: it is
automatically scanned for packages by the command-line tools.  For
instance, when running @code{guix package -i emacs}, all the @code{(gnu
packages @dots{})} modules are scanned until one that exports a package
object whose name is @code{emacs} is found.  This package search
facility is implemented in the @code{(gnu packages)} module.

Users can store package definitions in modules with different
names---e.g., @code{(my-packages emacs)}.  In that case, commands such
as @command{guix package} and @command{guix build} have to be used with
the @code{-e} option so that they know where to find the package.

The distribution is fully @dfn{bootstrapped} and @dfn{self-contained}:
each package is built based solely on other packages in the
distribution.  The root of this dependency graph is a small set of
@dfn{bootstrap binaries}, provided by the @code{(gnu packages
bootstrap)} module.  For more information on bootstrapping,
@ref{Bootstrapping}.

@node Packaging Guidelines
@section Packaging Guidelines

The GNU distribution is nascent and may well lack some of your favorite
packages.  This section describes how you can help make the distribution
grow.  @xref{Contributing}, for additional information on how you can
help.

Free software packages are usually distributed in the form of
@dfn{source code tarballs}---typically @file{tar.gz} files that contain
all the source files.  Adding a package to the distribution means
essentially two things: adding a @dfn{recipe} that describes how to
build the package, including a list of other packages required to build
it, and adding @dfn{package meta-data} along with that recipe, such as a
description and licensing information.

In Guix all this information is embodied in @dfn{package definitions}.
Package definitions provide a high-level view of the package.  They are
written using the syntax of the Scheme programming language; in fact,
for each package we define a variable bound to the package definition,
and export that variable from a module (@pxref{Package Modules}).
However, in-depth Scheme knowledge is @emph{not} a prerequisite for
creating packages.  For more information on package definitions,
@ref{Defining Packages}.

Once a package definition is in place, stored in a file in the Guix
source tree, it can be tested using the @command{guix build} command
(@pxref{Invoking guix build}).  For example, assuming the new package is
called @code{gnew}, you may run this command from the Guix build tree:

@example
./pre-inst-env guix build gnew --keep-failed
@end example

Using @code{--keep-failed} makes it easier to debug build failures since
it provides access to the failed build tree.  Another useful
command-line option when debugging is @code{--log-file}, to access the
build log.

If the package is unknown to the @command{guix} command, it may be that
the source file contains a syntax error, or lacks a @code{define-public}
clause to export the package variable.  To figure it out, you may load
the module from Guile to get more information about the actual error:

@example
./pre-inst-env guile -c '(use-modules (gnu packages gnew))'
@end example

Once your package builds correctly, please send us a patch
(@pxref{Contributing}).  Well, if you need help, we will be happy to
help you too.  Once the patch is committed in the Guix repository, the
new package automatically gets built on the supported platforms by
@url{http://hydra.gnu.org/gnu/master, our continuous integration
system}.

@cindex substituter
Users can obtain the new package definition simply by running
@command{guix pull} (@pxref{Invoking guix pull}).  When
@code{hydra.gnu.org} is done building the package, installing the
package automatically downloads binaries from there
(@pxref{Substitutes}).  The only place where human intervention is
needed is to review and apply the patch.


@menu
* Software Freedom::     What may go into the distribution.
* Package Naming::       What's in a name?
* Version Numbers::      When the name is not enough.
* Python Modules::       Taming the snake.
@end menu

@node Software Freedom
@subsection Software Freedom

@c Adapted from http://www.gnu.org/philosophy/philosophy.html.

The GNU operating system has been developed so that users can have
freedom in their computing.  GNU is @dfn{free software}, meaning that
users have the @url{http://www.gnu.org/philosophy/free-sw.html,four
essential freedoms}: to run the program, to study and change the program
in source code form, to redistribute exact copies, and to distribute
modified versions.  Packages found in the GNU distribution provide only
software that conveys these four freedoms.

In addition, the GNU distribution follow the
@url{http://www.gnu.org/distros/free-system-distribution-guidelines.html,free
software distribution guidelines}.  Among other things, these guidelines
reject non-free firmware, recommendations of non-free software, and
discuss ways to deal with trademarks and patents.

Some packages contain a small and optional subset that violates the
above guidelines, for instance because this subset is itself non-free
code.  When that happens, the offending items are removed with
appropriate patches or code snippets in the package definition's
@code{origin} form (@pxref{Defining Packages}).  That way, @code{guix
build --source} returns the ``freed'' source rather than the unmodified
upstream source.


@node Package Naming
@subsection Package Naming

A package has actually two names associated with it:
First, there is the name of the @emph{Scheme variable}, the one following
@code{define-public}.  By this name, the package can be made known in the
Scheme code, for instance as input to another package.  Second, there is
the string in the @code{name} field of a package definition.  This name
is used by package management commands such as
@command{guix package} and @command{guix build}.

Both are usually the same and correspond to the lowercase conversion of the
project name chosen upstream.  For instance, the GNUnet project is packaged
as @code{gnunet}.  We do not add @code{lib} prefixes for library packages,
unless these are already part of the official project name.  But see
@ref{Python Modules} for special rules concerning modules for
the Python language.


@node Version Numbers
@subsection Version Numbers

We usually package only the latest version of a given free software
project.  But sometimes, for instance for incompatible library versions,
two (or more) versions of the same package are needed.  These require
different Scheme variable names.  We use the name as defined
in @ref{Package Naming}
for the most recent version; previous versions use the same name, suffixed
by @code{-} and the smallest prefix of the version number that may
distinguish the two versions.

The name inside the package definition is the same for all versions of a
package and does not contain any version number.

For instance, the versions 2.24.20 and 3.9.12 of GTK+ may be packaged as follows:

@example
(define-public gtk+
  (package
   (name "gtk+")
   (version "3.9.12")
   ...))
(define-public gtk+-2
  (package
   (name "gtk+")
   (version "2.24.20")
   ...))
@end example
If we also wanted GTK+ 3.8.2, this would be packaged as
@example
(define-public gtk+-3.8
  (package
   (name "gtk+")
   (version "3.8.2")
   ...))
@end example


@node Python Modules
@subsection Python Modules

We currently package Python 2 and Python 3, under the Scheme variable names
@code{python-2} and @code{python} as explained in @ref{Version Numbers}.
To avoid confusion and naming clashes with other programming languages, it
seems desirable that the name of a package for a Python module contains
the word @code{python}.

Some modules are compatible with only one version of Python, others with both.
If the package Foo compiles only with Python 3, we name it
@code{python-foo}; if it compiles only with Python 2, we name it
@code{python2-foo}. If it is compatible with both versions, we create two
packages with the corresponding names.

If a project already contains the word @code{python}, we drop this;
for instance, the module python-dateutil is packaged under the names
@code{python-dateutil} and @code{python2-dateutil}.





@node Bootstrapping
@section Bootstrapping

@c Adapted from the ELS 2013 paper.

@cindex bootstrapping

Bootstrapping in our context refers to how the distribution gets built
``from nothing''.  Remember that the build environment of a derivation
contains nothing but its declared inputs (@pxref{Introduction}).  So
there's an obvious chicken-and-egg problem: how does the first package
get built?  How does the first compiler get compiled?  Note that this is
a question of interest only to the curious hacker, not to the regular
user, so you can shamelessly skip this section if you consider yourself
a ``regular user''.

@cindex bootstrap binaries
The GNU system is primarily made of C code, with libc at its core.  The
GNU build system itself assumes the availability of a Bourne shell and
command-line tools provided by GNU Coreutils, Awk, Findutils, `sed', and
`grep'.  Furthermore, build programs---programs that run
@code{./configure}, @code{make}, etc.---are written in Guile Scheme
(@pxref{Derivations}).  Consequently, to be able to build anything at
all, from scratch, Guix relies on pre-built binaries of Guile, GCC,
Binutils, libc, and the other packages mentioned above---the
@dfn{bootstrap binaries}.

These bootstrap binaries are ``taken for granted'', though we can also
re-create them if needed (more on that later).

@unnumberedsubsec Preparing to Use the Bootstrap Binaries

@c As of Emacs 24.3, Info-mode displays the image, but since it's a
@c large image, it's hard to scroll.  Oh well.
@image{images/bootstrap-graph,6in,,Dependency graph of the early bootstrap derivations}

The figure above shows the very beginning of the dependency graph of the
distribution, corresponding to the package definitions of the @code{(gnu
packages bootstrap)} module.  At this level of detail, things are
slightly complex.  First, Guile itself consists of an ELF executable,
along with many source and compiled Scheme files that are dynamically
loaded when it runs.  This gets stored in the @file{guile-2.0.7.tar.xz}
tarball shown in this graph.  This tarball is part of Guix's ``source''
distribution, and gets inserted into the store with @code{add-to-store}
(@pxref{The Store}).

But how do we write a derivation that unpacks this tarball and adds it
to the store?  To solve this problem, the @code{guile-bootstrap-2.0.drv}
derivation---the first one that gets built---uses @code{bash} as its
builder, which runs @code{build-bootstrap-guile.sh}, which in turn calls
@code{tar} to unpack the tarball.  Thus, @file{bash}, @file{tar},
@file{xz}, and @file{mkdir} are statically-linked binaries, also part of
the Guix source distribution, whose sole purpose is to allow the Guile
tarball to be unpacked.

Once @code{guile-bootstrap-2.0.drv} is built, we have a functioning
Guile that can be used to run subsequent build programs.  Its first task
is to download tarballs containing the other pre-built binaries---this
is what the @code{.tar.xz.drv} derivations do.  Guix modules such as
@code{ftp-client.scm} are used for this purpose.  The
@code{module-import.drv} derivations import those modules in a directory
in the store, using the original layout.  The
@code{module-import-compiled.drv} derivations compile those modules, and
write them in an output directory with the right layout.  This
corresponds to the @code{#:modules} argument of
@code{build-expression->derivation} (@pxref{Derivations}).

Finally, the various tarballs are unpacked by the
derivations @code{gcc-bootstrap-0.drv}, @code{glibc-bootstrap-0.drv},
etc., at which point we have a working C tool chain.


@unnumberedsubsec Building the Build Tools

@c TODO: Add a package-level dependency graph generated from (gnu
@c packages base).

Bootstrapping is complete when we have a full tool chain that does not
depend on the pre-built bootstrap tools discussed above.  This
no-dependency requirement is verified by checking whether the files of
the final tool chain contain references to the @file{/gnu/store}
directories of the bootstrap inputs.  The process that leads to this
``final'' tool chain is described by the package definitions found in
the @code{(gnu packages base)} module.

@c See <http://lists.gnu.org/archive/html/gnu-system-discuss/2012-10/msg00000.html>.
The first tool that gets built with the bootstrap binaries is
GNU Make, which is a prerequisite for all the following packages.
From there Findutils and Diffutils get built.

Then come the first-stage Binutils and GCC, built as pseudo cross
tools---i.e., with @code{--target} equal to @code{--host}.  They are
used to build libc.  Thanks to this cross-build trick, this libc is
guaranteed not to hold any reference to the initial tool chain.

From there the final Binutils and GCC are built.  GCC uses @code{ld}
from the final Binutils, and links programs against the just-built libc.
This tool chain is used to build the other packages used by Guix and by
the GNU Build System: Guile, Bash, Coreutils, etc.

And voilà!  At this point we have the complete set of build tools that
the GNU Build System expects.  These are in the @code{%final-inputs}
variables of the @code{(gnu packages base)} module, and are implicitly
used by any package that uses @code{gnu-build-system} (@pxref{Defining
Packages}).


@unnumberedsubsec Building the Bootstrap Binaries

Because the final tool chain does not depend on the bootstrap binaries,
those rarely need to be updated.  Nevertheless, it is useful to have an
automated way to produce them, should an update occur, and this is what
the @code{(gnu packages make-bootstrap)} module provides.

The following command builds the tarballs containing the bootstrap
binaries (Guile, Binutils, GCC, libc, and a tarball containing a mixture
of Coreutils and other basic command-line tools):

@example
guix build bootstrap-tarballs
@end example

The generated tarballs are those that should be referred to in the
@code{(gnu packages bootstrap)} module mentioned at the beginning of
this section.

Still here?  Then perhaps by now you've started to wonder: when do we
reach a fixed point?  That is an interesting question!  The answer is
unknown, but if you would like to investigate further (and have
significant computational and storage resources to do so), then let us
know.

@node Porting
@section Porting to a New Platform

As discussed above, the GNU distribution is self-contained, and
self-containment is achieved by relying on pre-built ``bootstrap
binaries'' (@pxref{Bootstrapping}).  These binaries are specific to an
operating system kernel, CPU architecture, and application binary
interface (ABI).  Thus, to port the distribution to a platform that is