guix.texi

@end defvr

@defvr {Scheme Variable} glib-or-gtk-build-system
This variable is exported by @code{(guix build-system glib-or-gtk)}.  It
is intended for use with packages making use of GLib or GTK+.

This build system adds the following two phases to the ones defined by
@var{gnu-build-system}:

@table @code
@item glib-or-gtk-wrap
The phase @code{glib-or-gtk-wrap} ensures that programs in
@file{bin/} are able to find GLib ``schemas'' and
@uref{https://developer.gnome.org/gtk3/stable/gtk-running.html, GTK+
modules}.  This is achieved by wrapping the programs in launch scripts
that appropriately set the @code{XDG_DATA_DIRS} and @code{GTK_PATH}
environment variables.

It is possible to exclude specific package outputs from that wrapping
process by listing their names in the
@code{#:glib-or-gtk-wrap-excluded-outputs} parameter.  This is useful
when an output is known not to contain any GLib or GTK+ binaries, and
where wrapping would gratuitously add a dependency of that output on
GLib and GTK+.

@item glib-or-gtk-compile-schemas
The phase @code{glib-or-gtk-compile-schemas} makes sure that all
@uref{https://developer.gnome.org/gio/stable/glib-compile-schemas.html,
GSettings schemas} of GLib are compiled.  Compilation is performed by the
@command{glib-compile-schemas} program.  It is provided by the package
@code{glib:bin} which is automatically imported by the build system.
The @code{glib} package providing @command{glib-compile-schemas} can be
specified with the @code{#:glib} parameter.
@end table

Both phases are executed after the @code{install} phase.
@end defvr

@defvr {Scheme Variable} minify-build-system
This variable is exported by @code{(guix build-system minify)}.  It
implements a minification procedure for simple JavaScript packages.

It adds @code{uglify-js} to the set of inputs and uses it to compress
all JavaScript files in the @file{src} directory.  A different minifier
package can be specified with the @code{#:uglify-js} parameter, but it
is expected that the package writes the minified code to the standard
output.

When the input JavaScript files are not all located in the @file{src}
directory, the parameter @code{#:javascript-files} can be used to
specify a list of file names to feed to the minifier.
@end defvr

@defvr {Scheme Variable} ocaml-build-system
This variable is exported by @code{(guix build-system ocaml)}.  It implements
a build procedure for @uref{https://ocaml.org, OCaml} packages, which consists
of choosing the correct set of commands to run for each package.  OCaml
packages can expect many different commands to be run.  This build system will
try some of them.

When the package has a @file{setup.ml} file present at the top-level, it will
run @code{ocaml setup.ml -configure}, @code{ocaml setup.ml -build} and
@code{ocaml setup.ml -install}.  The build system will assume that this file
was generated by @uref{http://oasis.forge.ocamlcore.org/, OASIS} and will take
care of setting the prefix and enabling tests if they are not disabled.  You
can pass configure and build flags with the @code{#:configure-flags} and
@code{#:build-flags}.  The @code{#:test-flags} key can be passed to change the
set of flags used to enable tests.  The @code{#:use-make?} key can be used to
bypass this system in the build and install phases.

When the package has a @file{configure} file, it is assumed that it is a
hand-made configure script that requires a different argument format than
in the @code{gnu-build-system}.  You can add more flags with the
@code{#:configure-flags} key.

When the package has a @file{Makefile} file (or @code{#:use-make?} is
@code{#t}), it will be used and more flags can be passed to the build and
install phases with the @code{#:make-flags} key.

Finally, some packages do not have these files and use a somewhat standard
location for its build system.  In that case, the build system will run
@code{ocaml pkg/pkg.ml} or @code{ocaml pkg/build.ml} and take care of
providing the path to the required findlib module.  Additional flags can
be passed via the @code{#:build-flags} key.  Install is taken care of by
@command{opam-installer}.  In this case, the @code{opam} package must
be added to the @code{native-inputs} field of the package definition.

Note that most OCaml packages assume they will be installed in the same
directory as OCaml, which is not what we want in guix.  In particular, they
will install @file{.so} files in their module's directory, which is usually
fine because it is in the OCaml compiler directory.  In guix though, these
libraries cannot be found and we use @code{CAML_LD_LIBRARY_PATH}.  This
variable points to @file{lib/ocaml/site-lib/stubslibs} and this is where
@file{.so} libraries should be installed.
@end defvr

@defvr {Scheme Variable} python-build-system
This variable is exported by @code{(guix build-system python)}.  It
implements the more or less standard build procedure used by Python
packages, which consists in running @code{python setup.py build} and
then @code{python setup.py install --prefix=/gnu/store/@dots{}}.

For packages that install stand-alone Python programs under @code{bin/},
it takes care of wrapping these programs so that their @code{PYTHONPATH}
environment variable points to all the Python libraries they depend on.

Which Python package is used to perform the build can be specified with
the @code{#:python} parameter.  This is a useful way to force a package
to be built for a specific version of the Python interpreter, which
might be necessary if the package is only compatible with a single
interpreter version.

By default guix calls @code{setup.py} under control of
@code{setuptools}, much like @command{pip} does.  Some packages are not
compatible with setuptools (and pip), thus you can disable this by
setting the @code{#:use-setuptools} parameter to @code{#f}.
@end defvr

@defvr {Scheme Variable} perl-build-system
This variable is exported by @code{(guix build-system perl)}.  It
implements the standard build procedure for Perl packages, which either
consists in running @code{perl Build.PL --prefix=/gnu/store/@dots{}},
followed by @code{Build} and @code{Build install}; or in running
@code{perl Makefile.PL PREFIX=/gnu/store/@dots{}}, followed by
@code{make} and @code{make install}, depending on which of
@code{Build.PL} or @code{Makefile.PL} is present in the package
distribution.  Preference is given to the former if both @code{Build.PL}
and @code{Makefile.PL} exist in the package distribution.  This
preference can be reversed by specifying @code{#t} for the
@code{#:make-maker?} parameter.

The initial @code{perl Makefile.PL} or @code{perl Build.PL} invocation
passes flags specified by the @code{#:make-maker-flags} or
@code{#:module-build-flags} parameter, respectively.

Which Perl package is used can be specified with @code{#:perl}.
@end defvr

@defvr {Scheme Variable} r-build-system
This variable is exported by @code{(guix build-system r)}.  It
implements the build procedure used by @uref{http://r-project.org, R}
packages, which essentially is little more than running @code{R CMD
INSTALL --library=/gnu/store/@dots{}} in an environment where
@code{R_LIBS_SITE} contains the paths to all R package inputs.  Tests
are run after installation using the R function
@code{tools::testInstalledPackage}.
@end defvr

@defvr {Scheme Variable} texlive-build-system
This variable is exported by @code{(guix build-system texlive)}.  It is
used to build TeX packages in batch mode with a specified engine.  The
build system sets the @code{TEXINPUTS} variable to find all TeX source
files in the inputs.

By default it runs @code{luatex} on all files ending on @code{ins}.  A
different engine and format can be specified with the
@code{#:tex-format} argument.  Different build targets can be specified
with the @code{#:build-targets} argument, which expects a list of file
names.  The build system adds only @code{texlive-bin} and
@code{texlive-latex-base} (both from @code{(gnu packages tex}) to the
inputs.  Both can be overridden with the arguments @code{#:texlive-bin}
and @code{#:texlive-latex-base}, respectively.

The @code{#:tex-directory} parameter tells the build system where to
install the built files under the texmf tree.
@end defvr

@defvr {Scheme Variable} ruby-build-system
This variable is exported by @code{(guix build-system ruby)}.  It
implements the RubyGems build procedure used by Ruby packages, which
involves running @code{gem build} followed by @code{gem install}.

The @code{source} field of a package that uses this build system
typically references a gem archive, since this is the format that Ruby
developers use when releasing their software.  The build system unpacks
the gem archive, potentially patches the source, runs the test suite,
repackages the gem, and installs it.  Additionally, directories and
tarballs may be referenced to allow building unreleased gems from Git or
a traditional source release tarball.

Which Ruby package is used can be specified with the @code{#:ruby}
parameter.  A list of additional flags to be passed to the @command{gem}
command can be specified with the @code{#:gem-flags} parameter.
@end defvr

@defvr {Scheme Variable} waf-build-system
This variable is exported by @code{(guix build-system waf)}.  It
implements a build procedure around the @code{waf} script.  The common
phases---@code{configure}, @code{build}, and @code{install}---are
implemented by passing their names as arguments to the @code{waf}
script.

The @code{waf} script is executed by the Python interpreter.  Which
Python package is used to run the script can be specified with the
@code{#:python} parameter.
@end defvr

@defvr {Scheme Variable} scons-build-system
This variable is exported by @code{(guix build-system scons)}.  It
implements the build procedure used by the SCons software construction
tool.  This build system runs @code{scons} to build the package,
@code{scons test} to run tests, and then @code{scons install} to install
the package.

Additional flags to be passed to @code{scons} can be specified with the
@code{#:scons-flags} parameter.  The version of Python used to run SCons
can be specified by selecting the appropriate SCons package with the
@code{#:scons} parameter.
@end defvr

@defvr {Scheme Variable} haskell-build-system
This variable is exported by @code{(guix build-system haskell)}.  It
implements the Cabal build procedure used by Haskell packages, which
involves running @code{runhaskell Setup.hs configure
--prefix=/gnu/store/@dots{}} and @code{runhaskell Setup.hs build}.
Instead of installing the package by running @code{runhaskell Setup.hs
install}, to avoid trying to register libraries in the read-only
compiler store directory, the build system uses @code{runhaskell
Setup.hs copy}, followed by @code{runhaskell Setup.hs register}.  In
addition, the build system generates the package documentation by
running @code{runhaskell Setup.hs haddock}, unless @code{#:haddock? #f}
is passed.  Optional Haddock parameters can be passed with the help of
the @code{#:haddock-flags} parameter.  If the file @code{Setup.hs} is
not found, the build system looks for @code{Setup.lhs} instead.

Which Haskell compiler is used can be specified with the @code{#:haskell}
parameter which defaults to @code{ghc}.
@end defvr

@defvr {Scheme Variable} dub-build-system
This variable is exported by @code{(guix build-system dub)}.  It
implements the Dub build procedure used by D packages, which
involves running @code{dub build} and @code{dub run}.
Installation is done by copying the files manually.

Which D compiler is used can be specified with the @code{#:ldc}
parameter which defaults to @code{ldc}.
@end defvr

@defvr {Scheme Variable} emacs-build-system
This variable is exported by @code{(guix build-system emacs)}.  It
implements an installation procedure similar to the packaging system
of Emacs itself (@pxref{Packages,,, emacs, The GNU Emacs Manual}).

It first creates the @code{@var{package}-autoloads.el} file, then it
byte compiles all Emacs Lisp files.  Differently from the Emacs
packaging system, the Info documentation files are moved to the standard
documentation directory and the @file{dir} file is deleted.  Each
package is installed in its own directory under
@file{share/emacs/site-lisp/guix.d}.
@end defvr

@defvr {Scheme Variable} font-build-system
This variable is exported by @code{(guix build-system font)}.  It
implements an installation procedure for font packages where upstream
provides pre-compiled TrueType, OpenType, etc. font files that merely
need to be copied into place.  It copies font files to standard
locations in the output directory.
@end defvr

@defvr {Scheme Variable} meson-build-system
This variable is exported by @code{(guix build-system meson)}.  It
implements the build procedure for packages that use
@url{http://mesonbuild.com, Meson} as their build system.

It adds both Meson and @uref{https://ninja-build.org/, Ninja} to the set
of inputs, and they can be changed with the parameters @code{#:meson}
and @code{#:ninja} if needed.  The default Meson is
@code{meson-for-build}, which is special because it doesn't clear the
@code{RUNPATH} of binaries and libraries when they are installed.

This build system is an extension of @var{gnu-build-system}, but with the
following phases changed to some specific for Meson:

@table @code

@item configure
The phase runs @code{meson} with the flags specified in
@code{#:configure-flags}.  The flag @code{--build-type} is always set to
@code{plain} unless something else is specified in @code{#:build-type}.

@item build
The phase runs @code{ninja} to build the package in parallel by default, but
this can be changed with @code{#:parallel-build?}.

@item check
The phase runs @code{ninja} with the target specified in @code{#:test-target},
which is @code{"test"} by default.

@item install
The phase runs @code{ninja install} and can not be changed.
@end table

Apart from that, the build system also adds the following phases:

@table @code

@item fix-runpath
This phase ensures that all binaries can find the libraries they need.
It searches for required libraries in subdirectories of the package being
built, and adds those to @code{RUNPATH} where needed.  It also removes
references to libraries left over from the build phase by
@code{meson-for-build}, such as test dependencies, that aren't actually
required for the program to run.

@item glib-or-gtk-wrap
This phase is the phase provided by @code{glib-or-gtk-build-system}, and it
is not enabled by default.  It can be enabled with @code{#:glib-or-gtk?}.

@item glib-or-gtk-compile-schemas
This phase is the phase provided by @code{glib-or-gtk-build-system}, and it
is not enabled by default.  It can be enabled with @code{#:glib-or-gtk?}.
@end table
@end defvr

Lastly, for packages that do not need anything as sophisticated, a
``trivial'' build system is provided.  It is trivial in the sense that
it provides basically no support: it does not pull any implicit inputs,
and does not have a notion of build phases.

@defvr {Scheme Variable} trivial-build-system
This variable is exported by @code{(guix build-system trivial)}.

This build system requires a @code{#:builder} argument.  This argument
must be a Scheme expression that builds the package output(s)---as
with @code{build-expression->derivation} (@pxref{Derivations,
@code{build-expression->derivation}}).
@end defvr

@node The Store
@section The Store

@cindex store
@cindex store items
@cindex store paths

Conceptually, the @dfn{store} is the place where derivations that have
been built successfully are stored---by default, @file{/gnu/store}.
Sub-directories in the store are referred to as @dfn{store items} or
sometimes @dfn{store paths}.  The store has an associated database that
contains information such as the store paths referred to by each store
path, and the list of @emph{valid} store items---results of successful
builds.  This database resides in @file{@var{localstatedir}/guix/db},
where @var{localstatedir} is the state directory specified @i{via}
@option{--localstatedir} at configure time, usually @file{/var}.

The store is @emph{always} accessed by the daemon on behalf of its clients
(@pxref{Invoking guix-daemon}).  To manipulate the store, clients
connect to the daemon over a Unix-domain socket, send requests to it,
and read the result---these are remote procedure calls, or RPCs.

@quotation Note
Users must @emph{never} modify files under @file{/gnu/store} directly.
This would lead to inconsistencies and break the immutability
assumptions of Guix's functional model (@pxref{Introduction}).

@xref{Invoking guix gc, @command{guix gc --verify}}, for information on
how to check the integrity of the store and attempt recovery from
accidental modifications.
@end quotation

The @code{(guix store)} module provides procedures to connect to the
daemon, and to perform RPCs.  These are described below.  By default,
@code{open-connection}, and thus all the @command{guix} commands,
connect to the local daemon or to the URI specified by the
@code{GUIX_DAEMON_SOCKET} environment variable.

@defvr {Environment Variable} GUIX_DAEMON_SOCKET
When set, the value of this variable should be a file name or a URI
designating the daemon endpoint.  When it is a file name, it denotes a
Unix-domain socket to connect to.  In addition to file names, the
supported URI schemes are:

@table @code
@item file
@itemx unix
These are for Unix-domain sockets.
@code{file:///var/guix/daemon-socket/socket} is equivalent to
@file{/var/guix/daemon-socket/socket}.

@item guix
@cindex daemon, remote access
@cindex remote access to the daemon
@cindex daemon, cluster setup
@cindex clusters, daemon setup
These URIs denote connections over TCP/IP, without encryption nor
authentication of the remote host.  The URI must specify the host name
and optionally a port number (by default port 44146 is used):

@example
guix://master.guix.example.org:1234
@end example

This setup is suitable on local networks, such as clusters, where only
trusted nodes may connect to the build daemon at
@code{master.guix.example.org}.

The @code{--listen} option of @command{guix-daemon} can be used to
instruct it to listen for TCP connections (@pxref{Invoking guix-daemon,
@code{--listen}}).

@item ssh
@cindex SSH access to build daemons
These URIs allow you to connect to a remote daemon over
SSH@footnote{This feature requires Guile-SSH (@pxref{Requirements}).}.
A typical URL might look like this:

@example
ssh://charlie@@guix.example.org:22
@end example

As for @command{guix copy}, the usual OpenSSH client configuration files
are honored (@pxref{Invoking guix copy}).
@end table

Additional URI schemes may be supported in the future.

@c XXX: Remove this note when the protocol incurs fewer round trips
@c and when (guix derivations) no longer relies on file system access.
@quotation Note
The ability to connect to remote build daemons is considered
experimental as of @value{VERSION}.  Please get in touch with us to
share any problems or suggestions you may have (@pxref{Contributing}).
@end quotation
@end defvr

@deffn {Scheme Procedure} open-connection [@var{uri}] [#:reserve-space? #t]
Connect to the daemon over the Unix-domain socket at @var{uri} (a string).  When
@var{reserve-space?} is true, instruct it to reserve a little bit of
extra space on the file system so that the garbage collector can still
operate should the disk become full.  Return a server object.

@var{file} defaults to @var{%default-socket-path}, which is the normal
location given the options that were passed to @command{configure}.
@end deffn

@deffn {Scheme Procedure} close-connection @var{server}
Close the connection to @var{server}.
@end deffn

@defvr {Scheme Variable} current-build-output-port
This variable is bound to a SRFI-39 parameter, which refers to the port
where build and error logs sent by the daemon should be written.
@end defvr

Procedures that make RPCs all take a server object as their first
argument.

@deffn {Scheme Procedure} valid-path? @var{server} @var{path}
@cindex invalid store items
Return @code{#t} when @var{path} designates a valid store item and
@code{#f} otherwise (an invalid item may exist on disk but still be
invalid, for instance because it is the result of an aborted or failed
build.)

A @code{&nix-protocol-error} condition is raised if @var{path} is not
prefixed by the store directory (@file{/gnu/store}).
@end deffn

@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}]
Add @var{text} under file @var{name} in the store, and return its store
path.  @var{references} is the list of store paths referred to by the
resulting store path.
@end deffn

@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations}
Build @var{derivations} (a list of @code{<derivation>} objects or
derivation paths), and return when the worker is done building them.
Return @code{#t} on success.
@end deffn

Note that the @code{(guix monads)} module provides a monad as well as
monadic versions of the above procedures, with the goal of making it
more convenient to work with code that accesses the store (@pxref{The
Store Monad}).

@c FIXME
@i{This section is currently incomplete.}

@node Derivations
@section Derivations

@cindex derivations
Low-level build actions and the environment in which they are performed
are represented by @dfn{derivations}.  A derivation contains the
following pieces of information:

@itemize
@item
The outputs of the derivation---derivations produce at least one file or
directory in the store, but may produce more.

@item
The inputs of the derivations, which may be other derivations or plain
files in the store (patches, build scripts, etc.)

@item
The system type targeted by the derivation---e.g., @code{x86_64-linux}.

@item
The file name of a build script in the store, along with the arguments
to be passed.

@item
A list of environment variables to be defined.

@end itemize

@cindex derivation path
Derivations allow clients of the daemon to communicate build actions to
the store.  They exist in two forms: as an in-memory representation,
both on the client- and daemon-side, and as files in the store whose
name end in @code{.drv}---these files are referred to as @dfn{derivation
paths}.  Derivations paths can be passed to the @code{build-derivations}
procedure to perform the build actions they prescribe (@pxref{The
Store}).

@cindex fixed-output derivations
Operations such as file downloads and version-control checkouts for
which the expected content hash is known in advance are modeled as
@dfn{fixed-output derivations}.  Unlike regular derivations, the outputs
of a fixed-output derivation are independent of its inputs---e.g., a
source code download produces the same result regardless of the download
method and tools being used.

The @code{(guix derivations)} module provides a representation of
derivations as Scheme objects, along with procedures to create and
otherwise manipulate derivations.  The lowest-level primitive to create
a derivation is the @code{derivation} procedure:

@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @
  @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
  [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @
  [#:system (%current-system)] [#:references-graphs #f] @
  [#:allowed-references #f] [#:disallowed-references #f] @
  [#:leaked-env-vars #f] [#:local-build? #f] @
  [#:substitutable? #t]
Build a derivation with the given arguments, and return the resulting
@code{<derivation>} object.

When @var{hash} and @var{hash-algo} are given, a
@dfn{fixed-output derivation} is created---i.e., one whose result is
known in advance, such as a file download.  If, in addition,
@var{recursive?} is true, then that fixed output may be an executable
file or a directory and @var{hash} must be the hash of an archive
containing this output.

When @var{references-graphs} is true, it must be a list of file
name/store path pairs.  In that case, the reference graph of each store
path is exported in the build environment in the corresponding file, in
a simple text format.

When @var{allowed-references} is true, it must be a list of store items
or outputs that the derivation's output may refer to.  Likewise,
@var{disallowed-references}, if true, must be a list of things the
outputs may @emph{not} refer to.

When @var{leaked-env-vars} is true, it must be a list of strings
denoting environment variables that are allowed to ``leak'' from the
daemon's environment to the build environment.  This is only applicable
to fixed-output derivations---i.e., when @var{hash} is true.  The main
use is to allow variables such as @code{http_proxy} to be passed to
derivations that download files.

When @var{local-build?} is true, declare that the derivation is not a
good candidate for offloading and should rather be built locally
(@pxref{Daemon Offload Setup}).  This is the case for small derivations
where the costs of data transfers would outweigh the benefits.

When @var{substitutable?} is false, declare that substitutes of the
derivation's output should not be used (@pxref{Substitutes}).  This is
useful, for instance, when building packages that capture details of the
host CPU instruction set.
@end deffn

@noindent
Here's an example with a shell script as its builder, assuming
@var{store} is an open connection to the daemon, and @var{bash} points
to a Bash executable in the store:

@lisp
(use-modules (guix utils)
             (guix store)
             (guix derivations))

(let ((builder   ; add the Bash script to the store
        (add-text-to-store store "my-builder.sh"
                           "echo hello world > $out\n" '())))
  (derivation store "foo"
              bash `("-e" ,builder)
              #:inputs `((,bash) (,builder))
              #:env-vars '(("HOME" . "/homeless"))))
@result{} #<derivation /gnu/store/@dots{}-foo.drv => /gnu/store/@dots{}-foo>
@end lisp

As can be guessed, this primitive is cumbersome to use directly.  A
better approach is to write build scripts in Scheme, of course!  The
best course of action for that is to write the build code as a
``G-expression'', and to pass it to @code{gexp->derivation}.  For more
information, @pxref{G-Expressions}.

Once upon a time, @code{gexp->derivation} did not exist and constructing
derivations with build code written in Scheme was achieved with
@code{build-expression->derivation}, documented below.  This procedure
is now deprecated in favor of the much nicer @code{gexp->derivation}.

@deffn {Scheme Procedure} build-expression->derivation @var{store} @
       @var{name} @var{exp} @
       [#:system (%current-system)] [#:inputs '()] @
       [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
       [#:recursive? #f] [#:env-vars '()] [#:modules '()] @
       [#:references-graphs #f] [#:allowed-references #f] @
       [#:disallowed-references #f] @
       [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f]
Return a derivation that executes Scheme expression @var{exp} as a
builder for derivation @var{name}.  @var{inputs} must be a list of
@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
@code{"out"} is assumed.  @var{modules} is a list of names of Guile
modules from the current search path to be copied in the store,
compiled, and made available in the load path during the execution of
@var{exp}---e.g., @code{((guix build utils) (guix build
gnu-build-system))}.

@var{exp} is evaluated in an environment where @code{%outputs} is bound
to a list of output/path pairs, and where @code{%build-inputs} is bound
to a list of string/output-path pairs made from @var{inputs}.
Optionally, @var{env-vars} is a list of string pairs specifying the name
and value of environment variables visible to the builder.  The builder
terminates by passing the result of @var{exp} to @code{exit}; thus, when
@var{exp} returns @code{#f}, the build is considered to have failed.

@var{exp} is built using @var{guile-for-build} (a derivation).  When
@var{guile-for-build} is omitted or is @code{#f}, the value of the
@code{%guile-for-build} fluid is used instead.

See the @code{derivation} procedure for the meaning of
@var{references-graphs}, @var{allowed-references},
@var{disallowed-references}, @var{local-build?}, and
@var{substitutable?}.
@end deffn

@noindent
Here's an example of a single-output derivation that creates a directory
containing one file:

@lisp
(let ((builder '(let ((out (assoc-ref %outputs "out")))
                  (mkdir out)    ; create /gnu/store/@dots{}-goo
                  (call-with-output-file (string-append out "/test")
                    (lambda (p)
                      (display '(hello guix) p))))))
  (build-expression->derivation store "goo" builder))

@result{} #<derivation /gnu/store/@dots{}-goo.drv => @dots{}>
@end lisp


@node The Store Monad
@section The Store Monad

@cindex monad

The procedures that operate on the store described in the previous
sections all take an open connection to the build daemon as their first
argument.  Although the underlying model is functional, they either have
side effects or depend on the current state of the store.

The former is inconvenient: the connection to the build daemon has to be
carried around in all those functions, making it impossible to compose
functions that do not take that parameter with functions that do.  The
latter can be problematic: since store operations have side effects
and/or depend on external state, they have to be properly sequenced.

@cindex monadic values
@cindex monadic functions
This is where the @code{(guix monads)} module comes in.  This module
provides a framework for working with @dfn{monads}, and a particularly
useful monad for our uses, the @dfn{store monad}.  Monads are a
construct that allows two things: associating ``context'' with values
(in our case, the context is the store), and building sequences of
computations (here computations include accesses to the store).  Values
in a monad---values that carry this additional context---are called
@dfn{monadic values}; procedures that return such values are called
@dfn{monadic procedures}.

Consider this ``normal'' procedure:

@example
(define (sh-symlink store)
  ;; Return a derivation that symlinks the 'bash' executable.
  (let* ((drv (package-derivation store bash))
         (out (derivation->output-path drv))
         (sh  (string-append out "/bin/bash")))
    (build-expression->derivation store "sh"
                                  `(symlink ,sh %output))))
@end example

Using @code{(guix monads)} and @code{(guix gexp)}, it may be rewritten
as a monadic function:

@example
(define (sh-symlink)
  ;; Same, but return a monadic value.
  (mlet %store-monad ((drv (package->derivation bash)))
    (gexp->derivation "sh"
                      #~(symlink (string-append #$drv "/bin/bash")
                                 #$output))))
@end example

There are several things to note in the second version: the @code{store}
parameter is now implicit and is ``threaded'' in the calls to the
@code{package->derivation} and @code{gexp->derivation} monadic
procedures, and the monadic value returned by @code{package->derivation}
is @dfn{bound} using @code{mlet} instead of plain @code{let}.

As it turns out, the call to @code{package->derivation} can even be
omitted since it will take place implicitly, as we will see later
(@pxref{G-Expressions}):

@example
(define (sh-symlink)
  (gexp->derivation "sh"
                    #~(symlink (string-append #$bash "/bin/bash")
                               #$output)))
@end example

@c See
@c <https://syntaxexclamation.wordpress.com/2014/06/26/escaping-continuations/>
@c for the funny quote.
Calling the monadic @code{sh-symlink} has no effect.  As someone once
said, ``you exit a monad like you exit a building on fire: by running''.
So, to exit the monad and get the desired effect, one must use
@code{run-with-store}:

@example
(run-with-store (open-connection) (sh-symlink))
@result{} /gnu/store/...-sh-symlink
@end example

Note that the @code{(guix monad-repl)} module extends the Guile REPL with
new ``meta-commands'' to make it easier to deal with monadic procedures:
@code{run-in-store}, and @code{enter-store-monad}.  The former is used
to ``run'' a single monadic value through the store:

@example
scheme@@(guile-user)> ,run-in-store (package->derivation hello)
$1 = #<derivation /gnu/store/@dots{}-hello-2.9.drv => @dots{}>
@end example

The latter enters a recursive REPL, where all the return values are
automatically run through the store:

@example
scheme@@(guile-user)> ,enter-store-monad
store-monad@@(guile-user) [1]> (package->derivation hello)
$2 = #<derivation /gnu/store/@dots{}-hello-2.9.drv => @dots{}>
store-monad@@(guile-user) [1]> (text-file "foo" "Hello!")
$3 = "/gnu/store/@dots{}-foo"
store-monad@@(guile-user) [1]> ,q
scheme@@(guile-user)>
@end example

@noindent
Note that non-monadic values cannot be returned in the
@code{store-monad} REPL.

The main syntactic forms to deal with monads in general are provided by
the @code{(guix monads)} module and are described below.

@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ...
Evaluate any @code{>>=} or @code{return} forms in @var{body} as being
in @var{monad}.
@end deffn

@deffn {Scheme Syntax} return @var{val}
Return a monadic value that encapsulates @var{val}.
@end deffn

@deffn {Scheme Syntax} >>= @var{mval} @var{mproc} ...
@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic
procedures @var{mproc}@dots{}@footnote{This operation is commonly
referred to as ``bind'', but that name denotes an unrelated procedure in
Guile.  Thus we use this somewhat cryptic symbol inherited from the
Haskell language.}.  There can be one @var{mproc} or several of them, as
in this example:

@example
(run-with-state
    (with-monad %state-monad
      (>>= (return 1)
           (lambda (x) (return (+ 1 x)))
           (lambda (x) (return (* 2 x)))))
  'some-state)

@result{} 4
@result{} some-state
@end example
@end deffn

@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @
       @var{body} ...
@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @
       @var{body} ...
Bind the variables @var{var} to the monadic values @var{mval} in
@var{body}, which is a sequence of expressions.  As with the bind
operator, this can be thought of as ``unpacking'' the raw, non-monadic
value ``contained'' in @var{mval} and making @var{var} refer to that
raw, non-monadic value within the scope of the @var{body}.  The form
(@var{var} -> @var{val}) binds @var{var} to the ``normal'' value
@var{val}, as per @code{let}.  The binding operations occur in sequence
from left to right.  The last expression of @var{body} must be a monadic
expression, and its result will become the result of the @code{mlet} or
@code{mlet*} when run in the @var{monad}.

@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let}
(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}).
@end deffn

@deffn {Scheme System} mbegin @var{monad} @var{mexp} ...
Bind @var{mexp} and the following monadic expressions in sequence,
returning the result of the last expression.  Every expression in the
sequence must be a monadic expression.

This is akin to @code{mlet}, except that the return values of the
monadic expressions are ignored.  In that sense, it is analogous to
@code{begin}, but applied to monadic expressions.
@end deffn

@deffn {Scheme System} mwhen @var{condition} @var{mexp0} @var{mexp*} ...
When @var{condition} is true, evaluate the sequence of monadic
expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}.  When
@var{condition} is false, return @code{*unspecified*} in the current
monad.  Every expression in the sequence must be a monadic expression.
@end deffn

@deffn {Scheme System} munless @var{condition} @var{mexp0} @var{mexp*} ...
When @var{condition} is false, evaluate the sequence of monadic
expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}.  When
@var{condition} is true, return @code{*unspecified*} in the current
monad.  Every expression in the sequence must be a monadic expression.
@end deffn

@cindex state monad
The @code{(guix monads)} module provides the @dfn{state monad}, which
allows an additional value---the state---to be @emph{threaded} through
monadic procedure calls.

@defvr {Scheme Variable} %state-monad
The state monad.  Procedures in the state monad can access and change
the state that is threaded.

Consider the example below.  The @code{square} procedure returns a value
in the state monad.  It returns the square of its argument, but also
increments the current state value:

@example
(define (square x)
  (mlet %state-monad ((count (current-state)))
    (mbegin %state-monad
      (set-current-state (+ 1 count))
      (return (* x x)))))

(run-with-state (sequence %state-monad (map square (iota 3))) 0)
@result{} (0 1 4)
@result{} 3
@end example

When ``run'' through @var{%state-monad}, we obtain that additional state
value, which is the number of @code{square} calls.
@end defvr

@deffn {Monadic Procedure} current-state
Return the current state as a monadic value.
@end deffn

@deffn {Monadic Procedure} set-current-state @var{value}
Set the current state to @var{value} and return the previous state as a
monadic value.
@end deffn

@deffn {Monadic Procedure} state-push @var{value}
Push @var{value} to the current state, which is assumed to be a list,
and return the previous state as a monadic value.
@end deffn

@deffn {Monadic Procedure} state-pop
Pop a value from the current state and return it as a monadic value.
The state is assumed to be a list.
@end deffn

@deffn {Scheme Procedure} run-with-state @var{mval} [@var{state}]
Run monadic value @var{mval} starting with @var{state} as the initial
state.  Return two values: the resulting value, and the resulting state.
@end deffn

The main interface to the store monad, provided by the @code{(guix
store)} module, is as follows.

@defvr {Scheme Variable} %store-monad
The store monad---an alias for @var{%state-monad}.

Values in the store monad encapsulate accesses to the store.  When its
effect is needed, a value of the store monad must be ``evaluated'' by
passing it to the @code{run-with-store} procedure (see below.)
@end defvr

@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)]
Run @var{mval}, a monadic value in the store monad, in @var{store}, an
open store connection.
@end deffn

@deffn {Monadic Procedure} text-file @var{name} @var{text} [@var{references}]
Return as a monadic value the absolute file name in the store of the file
containing @var{text}, a string.  @var{references} is a list of store items that the
resulting text file refers to; it defaults to the empty list.
@end deffn

@deffn {Monadic Procedure} interned-file @var{file} [@var{name}] @
         [#:recursive? #t] [#:select? (const #t)]
Return the name of @var{file} once interned in the store.  Use
@var{name} as its store name, or the basename of @var{file} if
@var{name} is omitted.

When @var{recursive?} is true, the contents of @var{file} are added
recursively; if @var{file} designates a flat file and @var{recursive?}
is true, its contents are added, and its permission bits are kept.

When @var{recursive?} is true, call @code{(@var{select?} @var{file}
@var{stat})} for each directory entry, where @var{file} is the entry's
absolute file name and @var{stat} is the result of @code{lstat}; exclude
entries for which @var{select?} does not return true.

The example below adds a file to the store, under two different names:

@example
(run-with-store (open-connection)
  (mlet %store-monad ((a (interned-file "README"))
                      (b (interned-file "README" "LEGU-MIN")))
    (return (list a b))))

@result{} ("/gnu/store/rwm@dots{}-README" "/gnu/store/44i@dots{}-LEGU-MIN")
@end example

@end deffn

The @code{(guix packages)} module exports the following package-related
monadic procedures:

@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @
       [#:system (%current-system)] [#:target #f] @
       [#:output "out"]
Return as a monadic
value in the absolute file name of @var{file} within the @var{output}
directory of @var{package}.  When @var{file} is omitted, return the name
of the @var{output} directory of @var{package}.  When @var{target} is
true, use it as a cross-compilation target triplet.
@end deffn

@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
@deffnx {Monadic Procedure} package->cross-derivation @var{package} @
          @var{target} [@var{system}]
Monadic version of @code{package-derivation} and
@code{package-cross-derivation} (@pxref{Defining Packages}).
@end deffn


@node G-Expressions
@section G-Expressions

@cindex G-expression
@cindex build code quoting
So we have ``derivations'', which represent a sequence of build actions
to be performed to produce an item in the store (@pxref{Derivations}).
These build actions are performed when asking the daemon to actually
build the derivations; they are run by the daemon in a container
(@pxref{Invoking guix-daemon}).

@cindex strata of code
It should come as no surprise that we like to write these build actions
in Scheme.  When we do that, we end up with two @dfn{strata} of Scheme
code@footnote{The term @dfn{stratum} in this context was coined by
Manuel Serrano et al.@: in the context of their work on Hop.  Oleg
Kiselyov, who has written insightful
@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code
on this topic}, refers to this kind of code generation as
@dfn{staging}.}: the ``host code''---code that defines packages, talks
to the daemon, etc.---and the ``build code''---code that actually
performs build actions, such as making directories, invoking
@command{make}, etc.

To describe a derivation and its build actions, one typically needs to
embed build code inside host code.  It boils down to manipulating build
code as data, and the homoiconicity of Scheme---code has a direct
representation as data---comes in handy for that.  But we need more than
the normal @code{quasiquote} mechanism in Scheme to construct build
expressions.

The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of
S-expressions adapted to build expressions.  G-expressions, or
@dfn{gexps}, consist essentially of three syntactic forms: @code{gexp},