guix.texi

   (version "3.9.12")
   ...))
(define-public gtk+-2
  (package
   (name "gtk+")
   (version "2.24.20")
   ...))
@end example
If we also wanted GTK+ 3.8.2, this would be packaged as
@example
(define-public gtk+-3.8
  (package
   (name "gtk+")
   (version "3.8.2")
   ...))
@end example


@node Python Modules
@subsection Python Modules

We currently package Python 2 and Python 3, under the Scheme variable names
@code{python-2} and @code{python} as explained in @ref{Version Numbers}.
To avoid confusion and naming clashes with other programming languages, it
seems desirable that the name of a package for a Python module contains
the word @code{python}.

Some modules are compatible with only one version of Python, others with both.
If the package Foo compiles only with Python 3, we name it
@code{python-foo}; if it compiles only with Python 2, we name it
@code{python2-foo}. If it is compatible with both versions, we create two
packages with the corresponding names.

If a project already contains the word @code{python}, we drop this;
for instance, the module python-dateutil is packaged under the names
@code{python-dateutil} and @code{python2-dateutil}.





@node Bootstrapping
@section Bootstrapping

@c Adapted from the ELS 2013 paper.

@cindex bootstrapping

Bootstrapping in our context refers to how the distribution gets built
``from nothing''.  Remember that the build environment of a derivation
contains nothing but its declared inputs (@pxref{Introduction}).  So
there's an obvious chicken-and-egg problem: how does the first package
get built?  How does the first compiler get compiled?  Note that this is
a question of interest only to the curious hacker, not to the regular
user, so you can shamelessly skip this section if you consider yourself
a ``regular user''.

@cindex bootstrap binaries
The GNU system is primarily made of C code, with libc at its core.  The
GNU build system itself assumes the availability of a Bourne shell and
command-line tools provided by GNU Coreutils, Awk, Findutils, `sed', and
`grep'.  Furthermore, build programs---programs that run
@code{./configure}, @code{make}, etc.---are written in Guile Scheme
(@pxref{Derivations}).  Consequently, to be able to build anything at
all, from scratch, Guix relies on pre-built binaries of Guile, GCC,
Binutils, libc, and the other packages mentioned above---the
@dfn{bootstrap binaries}.

These bootstrap binaries are ``taken for granted'', though we can also
re-create them if needed (more on that later).

@unnumberedsubsec Preparing to Use the Bootstrap Binaries

@c As of Emacs 24.3, Info-mode displays the image, but since it's a
@c large image, it's hard to scroll.  Oh well.
@image{images/bootstrap-graph,6in,,Dependency graph of the early bootstrap derivations}

The figure above shows the very beginning of the dependency graph of the
distribution, corresponding to the package definitions of the @code{(gnu
packages bootstrap)} module.  At this level of detail, things are
slightly complex.  First, Guile itself consists of an ELF executable,
along with many source and compiled Scheme files that are dynamically
loaded when it runs.  This gets stored in the @file{guile-2.0.7.tar.xz}
tarball shown in this graph.  This tarball is part of Guix's ``source''
distribution, and gets inserted into the store with @code{add-to-store}
(@pxref{The Store}).

But how do we write a derivation that unpacks this tarball and adds it
to the store?  To solve this problem, the @code{guile-bootstrap-2.0.drv}
derivation---the first one that gets built---uses @code{bash} as its
builder, which runs @code{build-bootstrap-guile.sh}, which in turn calls
@code{tar} to unpack the tarball.  Thus, @file{bash}, @file{tar},
@file{xz}, and @file{mkdir} are statically-linked binaries, also part of
the Guix source distribution, whose sole purpose is to allow the Guile
tarball to be unpacked.

Once @code{guile-bootstrap-2.0.drv} is built, we have a functioning
Guile that can be used to run subsequent build programs.  Its first task
is to download tarballs containing the other pre-built binaries---this
is what the @code{.tar.xz.drv} derivations do.  Guix modules such as
@code{ftp-client.scm} are used for this purpose.  The
@code{module-import.drv} derivations import those modules in a directory
in the store, using the original layout.  The
@code{module-import-compiled.drv} derivations compile those modules, and
write them in an output directory with the right layout.  This
corresponds to the @code{#:modules} argument of
@code{build-expression->derivation} (@pxref{Derivations}).

Finally, the various tarballs are unpacked by the
derivations @code{gcc-bootstrap-0.drv}, @code{glibc-bootstrap-0.drv},
etc., at which point we have a working C tool chain.


@unnumberedsubsec Building the Build Tools

@c TODO: Add a package-level dependency graph generated from (gnu
@c packages base).

Bootstrapping is complete when we have a full tool chain that does not
depend on the pre-built bootstrap tools discussed above.  This
no-dependency requirement is verified by checking whether the files of
the final tool chain contain references to the @file{/nix/store}
directories of the bootstrap inputs.  The process that leads to this
``final'' tool chain is described by the package definitions found in
the @code{(gnu packages base)} module.

@c See <http://lists.gnu.org/archive/html/gnu-system-discuss/2012-10/msg00000.html>.
The first tool that gets built with the bootstrap binaries is
GNU Make, which is a prerequisite for all the following packages.
From there Findutils and Diffutils get built.

Then come the first-stage Binutils and GCC, built as pseudo cross
tools---i.e., with @code{--target} equal to @code{--host}.  They are
used to build libc.  Thanks to this cross-build trick, this libc is
guaranteed not to hold any reference to the initial tool chain.

From there the final Binutils and GCC are built.  GCC uses @code{ld}
from the final Binutils, and links programs against the just-built libc.
This tool chain is used to build the other packages used by Guix and by
the GNU Build System: Guile, Bash, Coreutils, etc.

And voilà!  At this point we have the complete set of build tools that
the GNU Build System expects.  These are in the @code{%final-inputs}
variables of the @code{(gnu packages base)} module, and are implicitly
used by any package that uses @code{gnu-build-system} (@pxref{Defining
Packages}).


@unnumberedsubsec Building the Bootstrap Binaries

Because the final tool chain does not depend on the bootstrap binaries,
those rarely need to be updated.  Nevertheless, it is useful to have an
automated way to produce them, should an update occur, and this is what
the @code{(gnu packages make-bootstrap)} module provides.

The following command builds the tarballs containing the bootstrap
binaries (Guile, Binutils, GCC, libc, and a tarball containing a mixture
of Coreutils and other basic command-line tools):

@example
guix build bootstrap-tarballs
@end example

The generated tarballs are those that should be referred to in the
@code{(gnu packages bootstrap)} module mentioned at the beginning of
this section.

Still here?  Then perhaps by now you've started to wonder: when do we
reach a fixed point?  That is an interesting question!  The answer is
unknown, but if you would like to investigate further (and have
significant computational and storage resources to do so), then let us
know.

@node Porting
@section Porting to a New Platform

As discussed above, the GNU distribution is self-contained, and
self-containment is achieved by relying on pre-built ``bootstrap
binaries'' (@pxref{Bootstrapping}).  These binaries are specific to an
operating system kernel, CPU architecture, and application binary
interface (ABI).  Thus, to port the distribution to a platform that is
not yet supported, one must build those bootstrap binaries, and update
the @code{(gnu packages bootstrap)} module to use them on that platform.

Fortunately, Guix can @emph{cross compile} those bootstrap binaries.
When everything goes well, and assuming the GNU tool chain supports the
target platform, this can be as simple as running a command like this
one:

@example
guix build --target=armv5tel-linux-gnueabi bootstrap-tarballs
@end example

Once these are built, the @code{(gnu packages bootstrap)} module needs
to be updated to refer to these binaries on the target platform.  In
addition, the @code{glibc-dynamic-linker} procedure in that module must
be augmented to return the right file name for libc's dynamic linker on
that platform; likewise, @code{system->linux-architecture} in @code{(gnu
packages linux)} must be taught about the new platform.

In practice, there may be some complications.  First, it may be that the
extended GNU triplet that specifies an ABI (like the @code{eabi} suffix
above) is not recognized by all the GNU tools.  Typically, glibc
recognizes some of these, whereas GCC uses an extra @code{--with-abi}
configure flag (see @code{gcc.scm} for examples of how to handle this).
Second, some of the required packages could fail to build for that
platform.  Lastly, the generated binaries could be broken for some
reason.


@node System Configuration
@section System Configuration

@emph{This section documents work-in-progress.  As such it may be
incomplete, outdated, or open to discussions.  Please discuss it on
@email{guix-devel@@gnu.org}.}

@cindex system configuration
The GNU system supports a consistent whole-system configuration
mechanism.  By that we mean that all aspects of the global system
configuration---such as the available system services, timezone and
locale settings, user accounts---are declared in a single place.  Such
a @dfn{system configuration} can be @dfn{instantiated}---i.e., effected.

One of the advantages of putting all the system configuration under the
control of Guix is that it supports transactional system upgrades, and
makes it possible to roll-back to a previous system instantiation,
should something go wrong with the new one (@pxref{Features}).  Another
one is that it makes it easy to replicate the exact same configuration
across different machines, or at different points in time, without
having to resort to additional administration tools layered on top of
the system's own tools.
@c Yes, we're talking of Puppet, Chef, & co. here.  ↑

This section describes this mechanism.  First we focus on the system
administrator's viewpoint---explaining how the system is configured and
instantiated.  Then we show how this mechanism can be extended, for
instance to support new system services.

@menu
* Using the Configuration System::      Customizing your GNU system.
* Defining Services::                   Adding new service definitions.
@end menu

@node Using the Configuration System
@subsection Using the Configuration System

The operating system is configured by filling in an
@code{operating-system} structure, as defined by the @code{(gnu system)}
module.  A simple setup, with the default system services, the default
Linux-Libre kernel, initial RAM disk, and boot loader looks like this:

@findex operating-system
@lisp
(use-modules (gnu system)
             (gnu system shadow)   ; for 'user-account'
             (gnu system service)  ; for 'lsh-service'
             (gnu packages base)   ; Coreutils, grep, etc.
             (gnu packages bash)   ; Bash
             (gnu packages system) ; dmd, Inetutils
             (gnu packages zile)   ; Zile
             (gnu packages less)   ; less
             (gnu packages guile)  ; Guile
             (gnu packages linux)) ; procps, psmisc

(define komputilo
  (operating-system
   (host-name "komputilo")
   (timezone "Europe/Paris")
   (locale "fr_FR.UTF-8")
   (users (list (user-account
                 (name "alice")
                 (password "")
                 (uid 1000) (gid 100)
                 (comment "Bob's sister")
                 (home-directory "/home/alice"))))
   (packages (list coreutils bash guile-2.0
                   guix dmd
                   inetutils
                   findutils grep sed
                   procps psmisc
                   zile less))
   (services (cons (lsh-service #:port 2222 #:allow-root-login? #t)
                   %standard-services))))
@end lisp

This example should be self-describing.  The @code{packages} field lists
packages provided by the various @code{(gnu packages ...)} modules above
(@pxref{Package Modules}).  These are the packages that will be globally
visible on the system, for all user accounts---i.e., in every user's
@code{PATH} environment variable---in addition to the per-user profiles
(@pxref{Invoking guix package}).

The @code{services} field lists @dfn{system services} to be made
available when the system starts.  The @var{%standard-services} list,
from the @code{(gnu system)} module, provides the basic services one
would expect from a GNU system: a login service (mingetty) on each tty,
syslogd, libc's name service cache daemon (nscd), etc.

The @code{operating-system} declaration above specifies that, in
addition to those services, we want the @command{lshd} secure shell
daemon listening on port 2222, and allowing remote @code{root} logins
(@pxref{Invoking lshd,,, lsh, GNU lsh Manual}).  Under the hood,
@code{lsh-service} arranges so that @code{lshd} is started with the
right command-line options, possibly with supporting configuration files
generated as needed (@pxref{Defining Services}).

@c TODO: update when that command exists
Assuming the above snippet is stored in the @file{my-system-config.scm}
file, the (yet unwritten!) @command{guix system --boot
my-system-config.scm} command instantiates that configuration, and makes
it the default GRUB boot entry.  The normal way to change the system's
configuration is by updating this file and re-running the @command{guix
system} command.

At the Scheme level, the bulk of an @code{operating-system} declaration
is instantiated with the following monadic procedure (@pxref{The Store
Monad}):

@deffn {Monadic Procedure} operating-system-derivation os
Return a derivation that builds @var{os}, an @code{operating-system}
object (@pxref{Derivations}).

The output of the derivation is a single directory that refers to all
the packages, configuration files, and other supporting files needed to
instantiate @var{os}.
@end deffn


@node Defining Services
@subsection Defining Services

The @code{(gnu system dmd)} module defines several procedures that allow
users to declare the operating system's services (@pxref{Using the
Configuration System}).  These procedures are @emph{monadic
procedures}---i.e., procedures that return a monadic value in the store
monad (@pxref{The Store Monad}).  Examples of such procedures include:

@table @code
@item mingetty-service
return the definition of a service that runs @command{mingetty} to
offer a login service on the given console tty;

@item nscd-service
return a definition for libc's name service cache daemon (nscd);

@item guix-service
return a definition for a service that runs @command{guix-daemon}
(@pxref{Invoking guix-daemon}).
@end table

@cindex service definition
The monadic value returned by those procedures is a @dfn{service
definition}---a structure as returned by the @code{service} form.
Service definitions specifies the inputs the service depends on, and an
expression to start and stop the service.  Behind the scenes, service
definitions are ``translated'' into the form suitable for the
configuration file of dmd, the init system (@pxref{Services,,, dmd, GNU
dmd Manual}).

As an example, here is what the @code{nscd-service} procedure looks
like:

@lisp
(define (nscd-service)
  (mlet %store-monad ((nscd (package-file glibc "sbin/nscd")))
    (return (service
             (documentation "Run libc's name service cache daemon.")
             (provision '(nscd))
             (start `(make-forkexec-constructor ,nscd "-f" "/dev/null"
                                                "--foreground"))
             (stop  `(make-kill-destructor))

             (respawn? #f)
             (inputs `(("glibc" ,glibc)))))))
@end lisp

@noindent
The @code{inputs} field specifies that this service depends on the
@var{glibc} package---the package that contains the @command{nscd}
program.  The @code{start} and @code{stop} fields are expressions that
make use of dmd's facilities to start and stop processes (@pxref{Service
De- and Constructors,,, dmd, GNU dmd Manual}).  The @code{provision}
field specifies the name under which this service is known to dmd, and
@code{documentation} specifies on-line documentation.  Thus, the
commands @command{deco start ncsd}, @command{deco stop nscd}, and
@command{deco doc nscd} will do what you would expect (@pxref{Invoking
deco,,, dmd, GNU dmd Manual}).


@c *********************************************************************
@node Contributing
@chapter Contributing

This project is a cooperative effort, and we need your help to make it
grow!  Please get in touch with us on @email{guix-devel@@gnu.org}.  We
welcome ideas, bug reports, patches, and anything that may be helpful to
the project.  We particularly welcome help on packaging
(@pxref{Packaging Guidelines}).

Please see the
@url{http://git.savannah.gnu.org/cgit/guix.git/tree/HACKING,
@file{HACKING} file} that comes with the Guix source code for practical
details about contributions.


@c *********************************************************************
@node Acknowledgments
@chapter Acknowledgments

Guix is based on the Nix package manager, which was designed and
implemented by Eelco Dolstra.  Nix pioneered functional package
management, and promoted unprecedented features, such as transactional
package upgrades and rollbacks, per-user profiles, and referentially
transparent build processes.  Without this work, Guix would not exist.

The Nix-based software distributions, Nixpkgs and NixOS, have also been
an inspiration for Guix.

@c *********************************************************************
@node GNU Free Documentation License
@appendix GNU Free Documentation License

@include fdl-1.3.texi

@c *********************************************************************
@node Concept Index
@unnumbered Concept Index
@printindex cp

@node Function Index
@unnumbered Function Index
@printindex fn

@bye

@c Local Variables:
@c ispell-local-dictionary: "american";
@c End: