guix.texi

Guix has the foundations to maximize build reproducibility
(@pxref{Features}).  In most cases, independent builds of a given
package or derivation should yield bit-identical results.  Thus, through
a diverse set of independent package builds, we can strengthen the
integrity of our systems.

In the future, we want Guix to have support to publish and retrieve
binaries to/from other users, in a peer-to-peer fashion.  If you would
like to discuss this project, join us on @email{guix-devel@@gnu.org}.


@node Packages with Multiple Outputs
@section Packages with Multiple Outputs

@cindex multiple-output packages
@cindex package outputs

Often, packages defined in Guix have a single @dfn{output}---i.e., the
source package leads exactly one directory in the store.  When running
@command{guix package -i glibc}, one installs the default output of the
GNU libc package; the default output is called @code{out}, but its name
can be omitted as shown in this command.  In this particular case, the
default output of @code{glibc} contains all the C header files, shared
libraries, static libraries, Info documentation, and other supporting
files.

Sometimes it is more appropriate to separate the various types of files
produced from a single source package into separate outputs.  For
instance, the GLib C library (used by GTK+ and related packages)
installs more than 20 MiB of reference documentation as HTML pages.
To save space for users who do not need it, the documentation goes to a
separate output, called @code{doc}.  To install the main GLib output,
which contains everything but the documentation, one would run:

@example
guix package -i glib
@end example

The command to install its documentation is:

@example
guix package -i glib:doc
@end example

Some packages install programs with different ``dependency footprints''.
For instance, the WordNet package install both command-line tools and
graphical user interfaces (GUIs).  The former depend solely on the C
library, whereas the latter depend on Tcl/Tk and the underlying X
libraries.  In this case, we leave the command-line tools in the default
output, whereas the GUIs are in a separate output.  This allows users
who do not need the GUIs to save space.

There are several such multiple-output packages in the GNU distribution.
Other conventional output names include @code{lib} for libraries and
possibly header files, @code{bin} for stand-alone programs, and
@code{debug} for debugging information (@pxref{Installing Debugging
Files}).  The outputs of a packages are listed in the third column of
the output of @command{guix package --list-available} (@pxref{Invoking
guix package}).


@node Invoking guix gc
@section Invoking @command{guix gc}

@cindex garbage collector
Packages that are installed but not used may be @dfn{garbage-collected}.
The @command{guix gc} command allows users to explicitly run the garbage
collector to reclaim space from the @file{/gnu/store} directory.

The garbage collector has a set of known @dfn{roots}: any file under
@file{/gnu/store} reachable from a root is considered @dfn{live} and
cannot be deleted; any other file is considered @dfn{dead} and may be
deleted.  The set of garbage collector roots includes default user
profiles, and may be augmented with @command{guix build --root}, for
example (@pxref{Invoking guix build}).

Prior to running @code{guix gc --collect-garbage} to make space, it is
often useful to remove old generations from user profiles; that way, old
package builds referenced by those generations can be reclaimed.  This
is achieved by running @code{guix package --delete-generations}
(@pxref{Invoking guix package}).

The @command{guix gc} command has three modes of operation: it can be
used to garbage-collect any dead files (the default), to delete specific
files (the @code{--delete} option), or to print garbage-collector
information.  The available options are listed below:

@table @code
@item --collect-garbage[=@var{min}]
@itemx -C [@var{min}]
Collect garbage---i.e., unreachable @file{/gnu/store} files and
sub-directories.  This is the default operation when no option is
specified.

When @var{min} is given, stop once @var{min} bytes have been collected.
@var{min} may be a number of bytes, or it may include a unit as a
suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes.

When @var{min} is omitted, collect all the garbage.

@item --delete
@itemx -d
Attempt to delete all the store files and directories specified as
arguments.  This fails if some of the files are not in the store, or if
they are still live.

@item --list-dead
Show the list of dead files and directories still present in the
store---i.e., files and directories no longer reachable from any root.

@item --list-live
Show the list of live store files and directories.

@end table

In addition, the references among existing store files can be queried:

@table @code

@item --references
@itemx --referrers
List the references (respectively, the referrers) of store files given
as arguments.

@item --requisites
@itemx -R
List the requisites of the store files passed as arguments.  Requisites
include the store files themselves, their references, and the references
of these, recursively.  In other words, the returned list is the
@dfn{transitive closure} of the store files.

@end table


@node Invoking guix pull
@section Invoking @command{guix pull}

Packages are installed or upgraded to the latest version available in
the distribution currently available on your local machine.  To update
that distribution, along with the Guix tools, you must run @command{guix
pull}: the command downloads the latest Guix source code and package
descriptions, and deploys it.

On completion, @command{guix package} will use packages and package
versions from this just-retrieved copy of Guix.  Not only that, but all
the Guix commands and Scheme modules will also be taken from that latest
version.  New @command{guix} sub-commands added by the update also
become available.

The @command{guix pull} command is usually invoked with no arguments,
but it supports the following options:

@table @code
@item --verbose
Produce verbose output, writing build logs to the standard error output.

@item --url=@var{url}
Download the source tarball of Guix from @var{url}.

By default, the tarball is taken from its canonical address at
@code{gnu.org}, for the stable branch of Guix.

@item --bootstrap
Use the bootstrap Guile to build the latest Guix.  This option is only
useful to Guix developers.
@end table


@node Invoking guix archive
@section Invoking @command{guix archive}

The @command{guix archive} command allows users to @dfn{export} files
from the store into a single archive, and to later @dfn{import} them.
In particular, it allows store files to be transferred from one machine
to another machine's store.  For example, to transfer the @code{emacs}
package to a machine connected over SSH, one would run:

@example
guix archive --export emacs | ssh the-machine guix archive --import
@end example

@noindent
However, note that, in this example, all of @code{emacs} and its
dependencies are transferred, regardless of what is already available in
the target machine's store.  The @code{--missing} option can help figure
out which items are missing from the target's store.

Archives are stored in the ``Nix archive'' or ``Nar'' format, which is
comparable in spirit to `tar', but with a few noteworthy differences
that make it more appropriate for our purposes.  First, rather than
recording all Unix meta-data for each file, the Nar format only mentions
the file type (regular, directory, or symbolic link); Unix permissions
and owner/group are dismissed.  Second, the order in which directory
entries are stored always follows the order of file names according to
the C locale collation order.  This makes archive production fully
deterministic.

When exporting, the daemon digitally signs the contents of the archive,
and that digital signature is appended.  When importing, the daemon
verifies the signature and rejects the import in case of an invalid
signature or if the signing key is not authorized.
@c FIXME: Add xref to daemon doc about signatures.

The main options are:

@table @code
@item --export
Export the specified store files or packages (see below.)  Write the
resulting archive to the standard output.

@item --import
Read an archive from the standard input, and import the files listed
therein into the store.  Abort if the archive has an invalid digital
signature, or if it is signed by a public key not among the authorized
keys (see @code{--authorize} below.)

@item --missing
Read a list of store file names from the standard input, one per line,
and write on the standard output the subset of these files missing from
the store.

@item --generate-key[=@var{parameters}]
@cindex signing, archives
Generate a new key pair for the daemons.  This is a prerequisite before
archives can be exported with @code{--export}.  Note that this operation
usually takes time, because it needs to gather enough entropy to
generate the key pair.

The generated key pair is typically stored under @file{/etc/guix}, in
@file{signing-key.pub} (public key) and @file{signing-key.sec} (private
key, which must be kept secret.)  When @var{parameters} is omitted, it
is a 4096-bit RSA key.  Alternately, @var{parameters} can specify
@code{genkey} parameters suitable for Libgcrypt (@pxref{General
public-key related Functions, @code{gcry_pk_genkey},, gcrypt, The
Libgcrypt Reference Manual}).

@item --authorize
@cindex authorizing, archives
Authorize imports signed by the public key passed on standard input.
The public key must be in ``s-expression advanced format''---i.e., the
same format as the @file{signing-key.pub} file.

The list of authorized keys is kept in the human-editable file
@file{/etc/guix/acl}.  The file contains
@url{http://people.csail.mit.edu/rivest/Sexp.txt, ``advanced-format
s-expressions''} and is structured as an access-control list in the
@url{http://theworld.com/~cme/spki.txt, Simple Public-Key Infrastructure
(SPKI)}.
@end table

To export store files as an archive to the standard output, run:

@example
guix archive --export @var{options} @var{specifications}...
@end example

@var{specifications} may be either store file names or package
specifications, as for @command{guix package} (@pxref{Invoking guix
package}).  For instance, the following command creates an archive
containing the @code{gui} output of the @code{git} package and the main
output of @code{emacs}:

@example
guix archive --export git:gui /gnu/store/...-emacs-24.3 > great.nar
@end example

If the specified packages are not built yet, @command{guix archive}
automatically builds them.  The build process may be controlled with the
same options that can be passed to the @command{guix build} command
(@pxref{Invoking guix build, common build options}).


@c *********************************************************************
@node Programming Interface
@chapter Programming Interface

GNU Guix provides several Scheme programming interfaces (APIs) to
define, build, and query packages.  The first interface allows users to
write high-level package definitions.  These definitions refer to
familiar packaging concepts, such as the name and version of a package,
its build system, and its dependencies.  These definitions can then be
turned into concrete build actions.

Build actions are performed by the Guix daemon, on behalf of users.  In a
standard setup, the daemon has write access to the store---the
@file{/gnu/store} directory---whereas users do not.  The recommended
setup also has the daemon perform builds in chroots, under a specific
build users, to minimize interference with the rest of the system.

@cindex derivation
Lower-level APIs are available to interact with the daemon and the
store.  To instruct the daemon to perform a build action, users actually
provide it with a @dfn{derivation}.  A derivation is a low-level
representation of the build actions to be taken, and the environment in
which they should occur---derivations are to package definitions what
assembly is to C programs.

This chapter describes all these APIs in turn, starting from high-level
package definitions.

@menu
* Defining Packages::           Defining new packages.
* Build Systems::               Specifying how packages are built.
* The Store::                   Manipulating the package store.
* Derivations::                 Low-level interface to package derivations.
* The Store Monad::             Purely functional interface to the store.
* G-Expressions::               Manipulating build expressions.
@end menu

@node Defining Packages
@section Defining Packages

The high-level interface to package definitions is implemented in the
@code{(guix packages)} and @code{(guix build-system)} modules.  As an
example, the package definition, or @dfn{recipe}, for the GNU Hello
package looks like this:

@example
(define-module (gnu packages hello)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (guix build-system gnu)
  #:use-module (guix licenses))

(define hello
  (package
    (name "hello")
    (version "2.8")
    (source (origin
             (method url-fetch)
             (uri (string-append "mirror://gnu/hello/hello-" version
                                 ".tar.gz"))
             (sha256
              (base32 "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6"))))
    (build-system gnu-build-system)
    (arguments `(#:configure-flags '("--enable-silent-rules")))
    (inputs `(("gawk" ,gawk)))
    (synopsis "Hello, GNU world: An example GNU package")
    (description "Guess what GNU Hello prints!")
    (home-page "http://www.gnu.org/software/hello/")
    (license gpl3+)))
@end example

@noindent
Without being a Scheme expert, the reader may have guessed the meaning
of the various fields here.  This expression binds variable @code{hello}
to a @code{<package>} object, which is essentially a record
(@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}).
This package object can be inspected using procedures found in the
@code{(guix packages)} module; for instance, @code{(package-name hello)}
returns---surprise!---@code{"hello"}.

In the example above, @var{hello} is defined into a module of its own,
@code{(gnu packages hello)}.  Technically, this is not strictly
necessary, but it is convenient to do so: all the packages defined in
modules under @code{(gnu packages @dots{})} are automatically known to
the command-line tools (@pxref{Package Modules}).

There are a few points worth noting in the above package definition:

@itemize
@item
The @code{source} field of the package is an @code{<origin>} object.
Here, the @code{url-fetch} method from @code{(guix download)} is used,
meaning that the source is a file to be downloaded over FTP or HTTP.

The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of
the GNU mirrors defined in @code{(guix download)}.

The @code{sha256} field specifies the expected SHA256 hash of the file
being downloaded.  It is mandatory, and allows Guix to check the
integrity of the file.  The @code{(base32 @dots{})} form introduces the
base32 representation of the hash.  You can obtain this information with
@code{guix download} (@pxref{Invoking guix download}) and @code{guix
hash} (@pxref{Invoking guix hash}).

@cindex patches
When needed, the @code{origin} form can also have a @code{patches} field
listing patches to be applied, and a @code{snippet} field giving a
Scheme expression to modify the source code.

@item
@cindex GNU Build System
The @code{build-system} field specifies the procedure to build the
package (@pxref{Build Systems}).  Here, @var{gnu-build-system}
represents the familiar GNU Build System, where packages may be
configured, built, and installed with the usual @code{./configure &&
make && make check && make install} command sequence.

@item
The @code{arguments} field specifies options for the build system
(@pxref{Build Systems}).  Here it is interpreted by
@var{gnu-build-system} as a request run @file{configure} with the
@code{--enable-silent-rules} flag.

@item
The @code{inputs} field specifies inputs to the build process---i.e.,
build-time or run-time dependencies of the package.  Here, we define an
input called @code{"gawk"} whose value is that of the @var{gawk}
variable; @var{gawk} is itself bound to a @code{<package>} object.

Note that GCC, Coreutils, Bash, and other essential tools do not need to
be specified as inputs here.  Instead, @var{gnu-build-system} takes care
of ensuring that they are present (@pxref{Build Systems}).

However, any other dependencies need to be specified in the
@code{inputs} field.  Any dependency not specified here will simply be
unavailable to the build process, possibly leading to a build failure.
@end itemize

Once a package definition is in place@footnote{Simple package
definitions like the one above may be automatically converted from the
Nixpkgs distribution using the @command{guix import} command.}, the
package may actually be built using the @code{guix build} command-line
tool (@pxref{Invoking guix build}).  @xref{Packaging Guidelines}, for
more information on how to test package definitions.

Eventually, updating the package definition to a new upstream version
can be partly automated by the @command{guix refresh} command
(@pxref{Invoking guix refresh}).

Behind the scenes, a derivation corresponding to the @code{<package>}
object is first computed by the @code{package-derivation} procedure.
That derivation is stored in a @code{.drv} file under @file{/gnu/store}.
The build actions it prescribes may then be realized by using the
@code{build-derivations} procedure (@pxref{The Store}).

@deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}]
Return the @code{<derivation>} object of @var{package} for @var{system}
(@pxref{Derivations}).

@var{package} must be a valid @code{<package>} object, and @var{system}
must be a string denoting the target system type---e.g.,
@code{"x86_64-linux"} for an x86_64 Linux-based GNU system.  @var{store}
must be a connection to the daemon, which operates on the store
(@pxref{The Store}).
@end deffn

@noindent
@cindex cross-compilation
Similarly, it is possible to compute a derivation that cross-builds a
package for some other system:

@deffn {Scheme Procedure} package-cross-derivation @var{store} @
            @var{package} @var{target} [@var{system}]
Return the @code{<derivation>} object of @var{package} cross-built from
@var{system} to @var{target}.

@var{target} must be a valid GNU triplet denoting the target hardware
and operating system, such as @code{"mips64el-linux-gnu"}
(@pxref{Configuration Names, GNU configuration triplets,, configure, GNU
Configure and Build System}).
@end deffn


@node Build Systems
@section Build Systems

@cindex build system
Each package definition specifies a @dfn{build system} and arguments for
that build system (@pxref{Defining Packages}).  This @code{build-system}
field represents the build procedure of the package, as well implicit
dependencies of that build procedure.

Build systems are @code{<build-system>} objects.  The interface to
create and manipulate them is provided by the @code{(guix build-system)}
module, and actual build systems are exported by specific modules.

Build systems accept an optional list of @dfn{arguments}.  In package
definitions, these are passed @i{via} the @code{arguments} field
(@pxref{Defining Packages}).  They are typically keyword arguments
(@pxref{Optional Arguments, keyword arguments in Guile,, guile, GNU
Guile Reference Manual}).  The value of these arguments is usually
evaluated in the @dfn{build stratum}---i.e., by a Guile process launched
by the daemon (@pxref{Derivations}).

The main build system is @var{gnu-build-system}, which implements the
standard build procedure for GNU packages and many other packages.  It
is provided by the @code{(guix build-system gnu)} module.

@defvr {Scheme Variable} gnu-build-system
@var{gnu-build-system} represents the GNU Build System, and variants
thereof (@pxref{Configuration, configuration and makefile conventions,,
standards, GNU Coding Standards}).

@cindex build phases
In a nutshell, packages using it configured, built, and installed with
the usual @code{./configure && make && make check && make install}
command sequence.  In practice, a few additional steps are often needed.
All these steps are split up in separate @dfn{phases},
notably@footnote{Please see the @code{(guix build gnu-build-system)}
modules for more details about the build phases.}:

@table @code
@item unpack
Unpack the source tarball, and change the current directory to the
extracted source tree.  If the source is actually a directory, copy it
to the build tree, and enter that directory.

@item patch-source-shebangs
Patch shebangs encountered in source files so they refer to the right
store file names.  For instance, this changes @code{#!/bin/sh} to
@code{#!/gnu/store/@dots{}-bash-4.3/bin/sh}.

@item configure
Run the @file{configure} script with a number of default options, such
as @code{--prefix=/gnu/store/@dots{}}, as well as the options specified
by the @code{#:configure-flags} argument.

@item build
Run @code{make} with the list of flags specified with
@code{#:make-flags}.  If the @code{#:parallel-builds?} argument is true
(the default), build with @code{make -j}.

@item check
Run @code{make check}, or some other target specified with
@code{#:test-target}, unless @code{#:tests? #f} is passed.  If the
@code{#:parallel-tests?} argument is true (the default), run @code{make
check -j}.

@item install
Run @code{make install} with the flags listed in @code{#:make-flags}.

@item patch-shebangs
Patch shebangs on the installed executable files.

@item strip
Strip debugging symbols from ELF files (unless @code{#:strip-binaries?}
is false), copying them to the @code{debug} output when available
(@pxref{Installing Debugging Files}).
@end table

@vindex %standard-phases
The build-side module @code{(guix build gnu-build-system)} defines
@var{%standard-phases} as the default list of build phases.
@var{%standard-phases} is a list of symbol/procedure pairs, where the
procedure implements the actual phase.

The list of phases used for a particular package can be changed with the
@code{#:phases} parameter.  For instance, passing:

@example
#:phases (alist-delete 'configure %standard-phases)
@end example

means that all the phases described above will be used, except the
@code{configure} phase.

In addition, this build system ensures that the ``standard'' environment
for GNU packages is available.  This includes tools such as GCC, libc,
Coreutils, Bash, Make, Diffutils, grep, and sed (see the @code{(guix
build-system gnu)} module for a complete list.)  We call these the
@dfn{implicit inputs} of a package, because package definitions don't
have to mention them.
@end defvr

Other @code{<build-system>} objects are defined to support other
conventions and tools used by free software packages.  They inherit most
of @var{gnu-build-system}, and differ mainly in the set of inputs
implicitly added to the build process, and in the list of phases
executed.  Some of these build systems are listed below.

@defvr {Scheme Variable} cmake-build-system
This variable is exported by @code{(guix build-system cmake)}.  It
implements the build procedure for packages using the
@url{http://www.cmake.org, CMake build tool}.

It automatically adds the @code{cmake} package to the set of inputs.
Which package is used can be specified with the @code{#:cmake}
parameter.
@end defvr

@defvr {Scheme Variable} python-build-system
This variable is exported by @code{(guix build-system python)}.  It
implements the more or less standard build procedure used by Python
packages, which consists in running @code{python setup.py build} and
then @code{python setup.py install --prefix=/gnu/store/@dots{}}.

For packages that install stand-alone Python programs under @code{bin/},
it takes care of wrapping these programs so their @code{PYTHONPATH}
environment variable points to all the Python libraries they depend on.

Which Python package is used can be specified with the @code{#:python}
parameter.
@end defvr

@defvr {Scheme Variable} perl-build-system
This variable is exported by @code{(guix build-system perl)}.  It
implements the standard build procedure for Perl packages, which
consists in running @code{perl Makefile.PL PREFIX=/gnu/store/@dots{}},
followed by @code{make} and @code{make install}.

The initial @code{perl Makefile.PL} invocation passes flags specified by
the @code{#:make-maker-flags} parameter.

Which Perl package is used can be specified with @code{#:perl}.
@end defvr


Lastly, for packages that do not need anything as sophisticated, a
``trivial'' build system is provided.  It is trivial in the sense that
it provides basically no support: it does not pull any implicit inputs,
and does not have a notion of build phases.

@defvr {Scheme Variable} trivial-build-system
This variable is exported by @code{(guix build-system trivial)}.

This build system requires a @code{#:builder} argument.  This argument
must be a Scheme expression that builds the package's output(s)---as
with @code{build-expression->derivation} (@pxref{Derivations,
@code{build-expression->derivation}}).
@end defvr

@node The Store
@section The Store

@cindex store
@cindex store paths

Conceptually, the @dfn{store} is where derivations that have been
successfully built are stored---by default, under @file{/gnu/store}.
Sub-directories in the store are referred to as @dfn{store paths}.  The
store has an associated database that contains information such has the
store paths referred to by each store path, and the list of @emph{valid}
store paths---paths that result from a successful build.

The store is always accessed by the daemon on behalf of its clients
(@pxref{Invoking guix-daemon}).  To manipulate the store, clients
connect to the daemon over a Unix-domain socket, send it requests, and
read the result---these are remote procedure calls, or RPCs.

The @code{(guix store)} module provides procedures to connect to the
daemon, and to perform RPCs.  These are described below.

@deffn {Scheme Procedure} open-connection [@var{file}] [#:reserve-space? #t]
Connect to the daemon over the Unix-domain socket at @var{file}.  When
@var{reserve-space?} is true, instruct it to reserve a little bit of
extra space on the file system so that the garbage collector can still
operate, should the disk become full.  Return a server object.

@var{file} defaults to @var{%default-socket-path}, which is the normal
location given the options that were passed to @command{configure}.
@end deffn

@deffn {Scheme Procedure} close-connection @var{server}
Close the connection to @var{server}.
@end deffn

@defvr {Scheme Variable} current-build-output-port
This variable is bound to a SRFI-39 parameter, which refers to the port
where build and error logs sent by the daemon should be written.
@end defvr

Procedures that make RPCs all take a server object as their first
argument.

@deffn {Scheme Procedure} valid-path? @var{server} @var{path}
Return @code{#t} when @var{path} is a valid store path.
@end deffn

@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}]
Add @var{text} under file @var{name} in the store, and return its store
path.  @var{references} is the list of store paths referred to by the
resulting store path.
@end deffn

@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations}
Build @var{derivations} (a list of @code{<derivation>} objects or
derivation paths), and return when the worker is done building them.
Return @code{#t} on success.
@end deffn

Note that the @code{(guix monads)} module provides a monad as well as
monadic versions of the above procedures, with the goal of making it
more convenient to work with code that accesses the store (@pxref{The
Store Monad}).

@c FIXME
@i{This section is currently incomplete.}

@node Derivations
@section Derivations

@cindex derivations
Low-level build actions and the environment in which they are performed
are represented by @dfn{derivations}.  A derivation contain the
following pieces of information:

@itemize
@item
The outputs of the derivation---derivations produce at least one file or
directory in the store, but may produce more.

@item
The inputs of the derivations, which may be other derivations or plain
files in the store (patches, build scripts, etc.)

@item
The system type targeted by the derivation---e.g., @code{x86_64-linux}.

@item
The file name of a build script in the store, along with the arguments
to be passed.

@item
A list of environment variables to be defined.

@end itemize

@cindex derivation path
Derivations allow clients of the daemon to communicate build actions to
the store.  They exist in two forms: as an in-memory representation,
both on the client- and daemon-side, and as files in the store whose
name end in @code{.drv}---these files are referred to as @dfn{derivation
paths}.  Derivations paths can be passed to the @code{build-derivations}
procedure to perform the build actions they prescribe (@pxref{The
Store}).

The @code{(guix derivations)} module provides a representation of
derivations as Scheme objects, along with procedures to create and
otherwise manipulate derivations.  The lowest-level primitive to create
a derivation is the @code{derivation} procedure:

@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @
  @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
  [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @
  [#:system (%current-system)] [#:references-graphs #f] @
  [#:local-build? #f]
Build a derivation with the given arguments, and return the resulting
@code{<derivation>} object.

When @var{hash} and @var{hash-algo} are given, a
@dfn{fixed-output derivation} is created---i.e., one whose result is
known in advance, such as a file download.  If, in addition,
@var{recursive?} is true, then that fixed output may be an executable
file or a directory and @var{hash} must be the hash of an archive
containing this output.

When @var{references-graphs} is true, it must be a list of file
name/store path pairs.  In that case, the reference graph of each store
path is exported in the build environment in the corresponding file, in
a simple text format.

When @var{local-build?} is true, declare that the derivation is not a
good candidate for offloading and should rather be built locally
(@pxref{Daemon Offload Setup}).  This is the case for small derivations
where the costs of data transfers would outweigh the benefits.
@end deffn

@noindent
Here's an example with a shell script as its builder, assuming
@var{store} is an open connection to the daemon, and @var{bash} points
to a Bash executable in the store:

@lisp
(use-modules (guix utils)
             (guix store)
             (guix derivations))

(let ((builder   ; add the Bash script to the store
        (add-text-to-store store "my-builder.sh"
                           "echo hello world > $out\n" '())))
  (derivation store "foo"
              bash `("-e" ,builder)
              #:inputs `((,bash) (,builder))
              #:env-vars '(("HOME" . "/homeless"))))
@result{} #<derivation /gnu/store/@dots{}-foo.drv => /gnu/store/@dots{}-foo>
@end lisp

As can be guessed, this primitive is cumbersome to use directly.  A
better approach is to write build scripts in Scheme, of course!  The
best course of action for that is to write the build code as a
``G-expression'', and to pass it to @code{gexp->derivation}.  For more
information, @ref{G-Expressions}.

Once upon a time, @code{gexp->derivation} did not exist and constructing
derivations with build code written in Scheme was achieved with
@code{build-expression->derivation}, documented below.  This procedure
is now deprecated in favor of the much nicer @code{gexp->derivation}.

@deffn {Scheme Procedure} build-expression->derivation @var{store} @
       @var{name} @var{exp} @
       [#:system (%current-system)] [#:inputs '()] @
       [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
       [#:recursive? #f] [#:env-vars '()] [#:modules '()] @
       [#:references-graphs #f] [#:local-build? #f] [#:guile-for-build #f]
Return a derivation that executes Scheme expression @var{exp} as a
builder for derivation @var{name}.  @var{inputs} must be a list of
@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
@code{"out"} is assumed.  @var{modules} is a list of names of Guile
modules from the current search path to be copied in the store,
compiled, and made available in the load path during the execution of
@var{exp}---e.g., @code{((guix build utils) (guix build
gnu-build-system))}.

@var{exp} is evaluated in an environment where @code{%outputs} is bound
to a list of output/path pairs, and where @code{%build-inputs} is bound
to a list of string/output-path pairs made from @var{inputs}.
Optionally, @var{env-vars} is a list of string pairs specifying the name
and value of environment variables visible to the builder.  The builder
terminates by passing the result of @var{exp} to @code{exit}; thus, when
@var{exp} returns @code{#f}, the build is considered to have failed.

@var{exp} is built using @var{guile-for-build} (a derivation).  When
@var{guile-for-build} is omitted or is @code{#f}, the value of the
@code{%guile-for-build} fluid is used instead.

See the @code{derivation} procedure for the meaning of @var{references-graphs}
and @var{local-build?}.
@end deffn

@noindent
Here's an example of a single-output derivation that creates a directory
containing one file:

@lisp
(let ((builder '(let ((out (assoc-ref %outputs "out")))
                  (mkdir out)    ; create /gnu/store/@dots{}-goo
                  (call-with-output-file (string-append out "/test")
                    (lambda (p)
                      (display '(hello guix) p))))))
  (build-expression->derivation store "goo" builder))

@result{} #<derivation /gnu/store/@dots{}-goo.drv => @dots{}>
@end lisp


@node The Store Monad
@section The Store Monad

@cindex monad

The procedures that operate on the store described in the previous
sections all take an open connection to the build daemon as their first
argument.  Although the underlying model is functional, they either have
side effects or depend on the current state of the store.

The former is inconvenient: the connection to the build daemon has to be
carried around in all those functions, making it impossible to compose
functions that do not take that parameter with functions that do.  The
latter can be problematic: since store operations have side effects
and/or depend on external state, they have to be properly sequenced.

@cindex monadic values
@cindex monadic functions
This is where the @code{(guix monads)} module comes in.  This module
provides a framework for working with @dfn{monads}, and a particularly
useful monad for our uses, the @dfn{store monad}.  Monads are a
construct that allows two things: associating ``context'' with values
(in our case, the context is the store), and building sequences of
computations (here computations includes accesses to the store.)  Values
in a monad---values that carry this additional context---are called
@dfn{monadic values}; procedures that return such values are called
@dfn{monadic procedures}.

Consider this ``normal'' procedure:

@example
(define (sh-symlink store)
  ;; Return a derivation that symlinks the 'bash' executable.
  (let* ((drv (package-derivation store bash))
         (out (derivation->output-path drv))
         (sh  (string-append out "/bin/bash")))
    (build-expression->derivation store "sh"
                                  `(symlink ,sh %output))))
@end example

Using @code{(guix monads)}, it may be rewritten as a monadic function:

@example
(define (sh-symlink)
  ;; Same, but return a monadic value.
  (mlet %store-monad ((sh (package-file bash "bin")))
    (derivation-expression "sh" `(symlink ,sh %output))))
@end example

There are two things to note in the second version: the @code{store}
parameter is now implicit, and the monadic value returned by
@code{package-file}---a wrapper around @code{package-derivation} and
@code{derivation->output-path}---is @dfn{bound} using @code{mlet}
instead of plain @code{let}.

Calling the monadic @code{profile.sh} has no effect.  To get the desired
effect, one must use @code{run-with-store}:

@example
(run-with-store (open-connection) (profile.sh))
@result{} /gnu/store/...-profile.sh
@end example

The main syntactic forms to deal with monads in general are described
below.

@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ...
Evaluate any @code{>>=} or @code{return} forms in @var{body} as being
in @var{monad}.
@end deffn

@deffn {Scheme Syntax} return @var{val}
Return a monadic value that encapsulates @var{val}.
@end deffn

@deffn {Scheme Syntax} >>= @var{mval} @var{mproc}
@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic
procedure @var{mproc}@footnote{This operation is commonly referred to as
``bind'', but that name denotes an unrelated procedure in Guile.  Thus
we use this somewhat cryptic symbol inherited from the Haskell
language.}.
@end deffn

@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @
       @var{body} ...
@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @
       @var{body} ...
Bind the variables @var{var} to the monadic values @var{mval} in
@var{body}.  The form (@var{var} -> @var{val}) binds @var{var} to the
``normal'' value @var{val}, as per @code{let}.

@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let}
(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}).
@end deffn

The interface to the store monad provided by @code{(guix monads)} is as
follows.

@defvr {Scheme Variable} %store-monad
The store monad.  Values in the store monad encapsulate accesses to the
store.  When its effect is needed, a value of the store monad must be
``evaluated'' by passing it to the @code{run-with-store} procedure (see
below.)
@end defvr

@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)]
Run @var{mval}, a monadic value in the store monad, in @var{store}, an
open store connection.
@end deffn

@deffn {Monadic Procedure} text-file @var{name} @var{text}
Return as a monadic value the absolute file name in the store of the file
containing @var{text}, a string.
@end deffn

@deffn {Monadic Procedure} text-file* @var{name} @var{text} @dots{}
Return as a monadic value a derivation that builds a text file
containing all of @var{text}.  @var{text} may list, in addition to
strings, packages, derivations, and store file names; the resulting
store file holds references to all these.

This variant should be preferred over @code{text-file} anytime the file
to create will reference items from the store.  This is typically the
case when building a configuration file that embeds store file names,
like this:

@example
(define (profile.sh)
  ;; Return the name of a shell script in the store that
  ;; initializes the 'PATH' environment variable.
  (text-file* "profile.sh"
              "export PATH=" coreutils "/bin:"
              grep "/bin:" sed "/bin\n"))
@end example

In this example, the resulting @file{/gnu/store/@dots{}-profile.sh} file
will references @var{coreutils}, @var{grep}, and @var{sed}, thereby
preventing them from being garbage-collected during its lifetime.
@end deffn

@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @
       [#:system (%current-system)] [#:output "out"] Return as a monadic
value in the absolute file name of @var{file} within the @var{output}
directory of @var{package}.  When @var{file} is omitted, return the name
of the @var{output} directory of @var{package}.
@end deffn

@deffn {Monadic Procedure} derivation-expression @var{name} @var{exp} @
       [#:system (%current-system)] [#:inputs '()] @
       [#:outputs '("out")] [#:hash #f] @
       [#:hash-algo #f] [#:env-vars '()] [#:modules '()] @
       [#:references-graphs #f] [#:guile-for-build #f]
Monadic version of @code{build-expression->derivation}
(@pxref{Derivations}).
@end deffn

@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
Monadic version of @code{package-derivation} (@pxref{Defining
Packages}).
@end deffn


@node G-Expressions
@section G-Expressions

@cindex G-expression
@cindex build code quoting
So we have ``derivations'', which represent a sequence of build actions
to be performed to produce an item in the store (@pxref{Derivations}).
Those build actions are performed when asking the daemon to actually
build the derivations; they are run by the daemon in a container
(@pxref{Invoking guix-daemon}).