Newer
Older
@item glib-or-gtk-wrap
This phase is the phase provided by @code{glib-or-gtk-build-system}, and it
is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}.
@item glib-or-gtk-compile-schemas
This phase is the phase provided by @code{glib-or-gtk-build-system}, and it
is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}.
@end table
@end defvr
@defvr {Scheme Variable} linux-module-build-system
@code{linux-module-build-system} allows building Linux kernel modules.
This build system is an extension of @code{gnu-build-system}, but with the
following phases changed:
@table @code
@item configure
This phase configures the environment so that the Linux kernel's Makefile
can be used to build the external kernel module.
@item build
This phase uses the Linux kernel's Makefile in order to build the external
kernel module.
@item install
This phase uses the Linux kernel's Makefile in order to install the external
kernel module.
@end table
It is possible and useful to specify the Linux kernel to use for building
the module (in the @code{arguments} form of a package using the
@code{linux-module-build-system}, use the key @code{#:linux} to specify it).
@defvr {Scheme Variable} node-build-system
This variable is exported by @code{(guix build-system node)}. It
implements the build procedure used by @uref{https://nodejs.org,
Node.js}, which implements an approximation of the @code{npm install}
command, followed by an @code{npm test} command.
Which Node.js package is used to interpret the @code{npm} commands can
be specified with the @code{#:node} parameter which defaults to
@code{node}.
@end defvr
Lastly, for packages that do not need anything as sophisticated, a
``trivial'' build system is provided. It is trivial in the sense that
it provides basically no support: it does not pull any implicit inputs,
and does not have a notion of build phases.
@defvr {Scheme Variable} trivial-build-system
This variable is exported by @code{(guix build-system trivial)}.
This build system requires a @code{#:builder} argument. This argument
must be a Scheme expression that builds the package output(s)---as
with @code{build-expression->derivation} (@pxref{Derivations,
@code{build-expression->derivation}}).
@end defvr
8063
8064
8065
8066
8067
8068
8069
8070
8071
8072
8073
8074
8075
8076
8077
8078
8079
8080
8081
8082
8083
8084
8085
8086
8087
8088
8089
8090
8091
8092
8093
8094
8095
8096
8097
8098
8099
8100
8101
8102
8103
8104
8105
8106
8107
8108
8109
8110
8111
8112
8113
8114
8115
8116
8117
8118
8119
8120
8121
8122
8123
8124
8125
8126
8127
8128
8129
8130
8131
8132
8133
8134
8135
8136
8137
8138
8139
8140
8141
8142
8143
8144
8145
8146
8147
8148
8149
8150
8151
8152
8153
8154
8155
8156
8157
8158
8159
8160
8161
8162
8163
8164
8165
8166
8167
8168
8169
8170
8171
8172
8173
8174
8175
8176
8177
8178
8179
8180
8181
8182
8183
@node Build Phases
@section Build Phases
@cindex build phases, for packages
Almost all package build systems implement a notion @dfn{build phases}:
a sequence of actions that the build system executes, when you build the
package, leading to the installed byproducts in the store. A notable
exception is the ``bare-bones'' @code{trivial-build-system}
(@pxref{Build Systems}).
As discussed in the previous section, those build systems provide a
standard list of phases. For @code{gnu-build-system}, the standard
phases include an @code{unpack} phase to unpack the source code tarball,
a @command{configure} phase to run @code{./configure}, a @code{build}
phase to run @command{make}, and (among others) an @code{install} phase
to run @command{make install}; @pxref{Build Systems}, for a more
detailed view of these phases. Likewise, @code{cmake-build-system}
inherits these phases, but its @code{configure} phase runs
@command{cmake} instead of @command{./configure}. Other build systems,
such as @code{python-build-system}, have a wholly different list of
standard phases. All this code runs on the @dfn{build side}: it is
evaluated when you actually build the package, in a dedicated build
process spawned by the build daemon (@pxref{Invoking guix-daemon}).
Build phases are represented as association lists or ``alists''
(@pxref{Association Lists,,, guile, GNU Guile Reference Manual}) where
each key is a symbol for the name of the phase and the associated value
is a procedure that accepts an arbitrary number of arguments. By
convention, those procedures receive information about the build in the
form of @dfn{keyword parameters}, which they can use or ignore.
For example, here is how @code{(guix build gnu-build-system)} defines
@code{%standard-phases}, the variable holding its alist of build
phases@footnote{We present a simplified view of those build phases, but
do take a look at @code{(guix build gnu-build-system)} to see all the
details!}:
@lisp
;; The build phases of 'gnu-build-system'.
(define* (unpack #:key source #:allow-other-keys)
;; Extract the source tarball.
(invoke "tar" "xvf" source))
(define* (configure #:key outputs #:allow-other-keys)
;; Run the 'configure' script. Install to output "out".
(let ((out (assoc-ref outputs "out")))
(invoke "./configure"
(string-append "--prefix=" out))))
(define* (build #:allow-other-keys)
;; Compile.
(invoke "make"))
(define* (check #:key (test-target "check") (tests? #true)
#:allow-other-keys)
;; Run the test suite.
(if tests?
(invoke "make" test-target)
(display "test suite not run\n")))
(define* (install #:allow-other-keys)
;; Install files to the prefix 'configure' specified.
(invoke "make" "install"))
(define %standard-phases
;; The list of standard phases (quite a few are omitted
;; for brevity). Each element is a symbol/procedure pair.
(list (cons 'unpack unpack)
(cons 'configure configure)
(cons 'build build)
(cons 'check check)
(cons 'install install)))
@end lisp
This shows how @code{%standard-phases} is defined as a list of
symbol/procedure pairs (@pxref{Pairs,,, guile, GNU Guile Reference
Manual}). The first pair associates the @code{unpack} procedure with
the @code{unpack} symbol---a name; the second pair defines the
@code{configure} phase similarly, and so on. When building a package
that uses @code{gnu-build-system} with its default list of phases, those
phases are executed sequentially. You can see the name of each phase
started and completed in the build log of packages that you build.
Let's now look at the procedures themselves. Each one is defined with
@code{define*}: @code{#:key} lists keyword parameters the procedure
accepts, possibly with a default value, and @code{#:allow-other-keys}
specifies that other keyword parameters are ignored (@pxref{Optional
Arguments,,, guile, GNU Guile Reference Manual}).
The @code{unpack} procedure honors the @code{source} parameter, which
the build system uses to pass the file name of the source tarball (or
version control checkout), and it ignores other parameters. The
@code{configure} phase only cares about the @code{outputs} parameter, an
alist mapping package output names to their store file name
(@pxref{Packages with Multiple Outputs}). It extracts the file name of
for @code{out}, the default output, and passes it to
@command{./configure} as the installation prefix, meaning that
@command{make install} will eventually copy all the files in that
directory (@pxref{Configuration, configuration and makefile
conventions,, standards, GNU Coding Standards}). @code{build} and
@code{install} ignore all their arguments. @code{check} honors the
@code{test-target} argument, which specifies the name of the Makefile
target to run tests; it prints a message and skips tests when
@code{tests?} is false.
@cindex build phases, customizing
The list of phases used for a particular package can be changed with the
@code{#:phases} parameter of the build system. Changing the set of
build phases boils down to building a new alist of phases based on the
@code{%standard-phases} alist described above. This can be done with
standard alist procedures such as @code{alist-delete} (@pxref{SRFI-1
Association Lists,,, guile, GNU Guile Reference Manual}); however, it is
more convenient to do so with @code{modify-phases} (@pxref{Build
Utilities, @code{modify-phases}}).
Here is an example of a package definition that removes the
@code{configure} phase of @code{%standard-phases} and inserts a new
phase before the @code{build} phase, called
@code{set-prefix-in-makefile}:
(define-public example
(package
(name "example")
;; other fields omitted
(build-system gnu-build-system)
(arguments
'(#:phases (modify-phases %standard-phases
(delete 'configure)
(add-before 'build 'set-prefix-in-makefile
(lambda* (#:key outputs #:allow-other-keys)
;; Modify the makefile so that its
;; 'PREFIX' variable points to "out".
(let ((out (assoc-ref outputs "out")))
(substitute* "Makefile"
(("PREFIX =.*")
(string-append "PREFIX = "
out "\n")))
#true))))))))
The new phase that is inserted is written as an anonymous procedure,
introduced with @code{lambda*}; it honors the @code{outputs} parameter
we have seen before. @xref{Build Utilities}, for more about the helpers
used by this phase, and for more examples of @code{modify-phases}.
@cindex code staging
@cindex staging, of code
Keep in mind that build phases are code evaluated at the time the
package is actually built. This explains why the whole
@code{modify-phases} expression above is quoted (it comes after the
@code{'} or apostrophe): it is @dfn{staged} for later execution.
@xref{G-Expressions}, for an explanation of code staging and the
@dfn{code strata} involved.
@node Build Utilities
@section Build Utilities
As soon as you start writing non-trivial package definitions
(@pxref{Defining Packages}) or other build actions
(@pxref{G-Expressions}), you will likely start looking for helpers for
``shell-like'' actions---creating directories, copying and deleting
files recursively, manipulating build phases, and so on. The
@code{(guix build utils)} module provides such utility procedures.
Most build systems load @code{(guix build utils)} (@pxref{Build
Systems}). Thus, when writing custom build phases for your package
definitions, you can usually assume those procedures are in scope.
When writing G-expressions, you can import @code{(guix build utils)} on
8234
8235
8236
8237
8238
8239
8240
8241
8242
8243
8244
8245
8246
8247
8248
8249
8250
8251
8252
8253
8254
8255
the ``build side'' using @code{with-imported-modules} and then put it in
scope with the @code{use-modules} form (@pxref{Using Guile Modules,,,
guile, GNU Guile Reference Manual}):
@lisp
(with-imported-modules '((guix build utils)) ;import it
(computed-file "empty-tree"
#~(begin
;; Put it in scope.
(use-modules (guix build utils))
;; Happily use its 'mkdir-p' procedure.
(mkdir-p (string-append #$output "/a/b/c")))))
@end lisp
The remainder of this section is the reference for most of the utility
procedures provided by @code{(guix build utils)}.
@c TODO Document what's missing.
@subsection Dealing with Store File Names
This section documents procedures that deal with store file names.
@deffn {Scheme Procedure} %store-directory
Return the directory name of the store.
@end deffn
@deffn {Scheme Procedure} store-file-name? @var{file}
Return true if @var{file} is in the store.
@end deffn
@deffn {Scheme Procedure} strip-store-file-name @var{file}
Strip the @file{/gnu/store} and hash from @var{file}, a store file name.
The result is typically a @code{"@var{package}-@var{version}"} string.
8269
8270
8271
8272
8273
8274
8275
8276
8277
8278
8279
8280
8281
8282
8283
8284
8285
8286
8287
8288
8289
8290
8291
8292
8293
8294
8295
8296
8297
8298
8299
8300
8301
8302
8303
8304
8305
8306
8307
8308
8309
8310
8311
8312
8313
8314
8315
8316
8317
8318
8319
8320
8321
8322
8323
8324
8325
8326
8327
8328
8329
8330
8331
8332
8333
8334
8335
8336
8337
8338
8339
8340
8341
8342
8343
8344
8345
8346
8347
8348
8349
8350
8351
8352
8353
8354
@end deffn
@deffn {Scheme Procedure} package-name->name+version @var{name}
Given @var{name}, a package name like @code{"foo-0.9.1b"}, return two
values: @code{"foo"} and @code{"0.9.1b"}. When the version part is
unavailable, @var{name} and @code{#f} are returned. The first hyphen
followed by a digit is considered to introduce the version part.
@end deffn
@subsection File Types
The procedures below deal with files and file types.
@deffn {Scheme Procedure} directory-exists? @var{dir}
Return @code{#t} if @var{dir} exists and is a directory.
@end deffn
@deffn {Scheme Procedure} executable-file? @var{file}
Return @code{#t} if @var{file} exists and is executable.
@end deffn
@deffn {Scheme Procedure} symbolic-link? @var{file}
Return @code{#t} if @var{file} is a symbolic link (aka. a ``symlink'').
@end deffn
@deffn {Scheme Procedure} elf-file? @var{file}
@deffnx {Scheme Procedure} ar-file? @var{file}
@deffnx {Scheme Procedure} gzip-file? @var{file}
Return @code{#t} if @var{file} is, respectively, an ELF file, an
@code{ar} archive (such as a @file{.a} static library), or a gzip file.
@end deffn
@deffn {Scheme Procedure} reset-gzip-timestamp @var{file} [#:keep-mtime? #t]
If @var{file} is a gzip file, reset its embedded timestamp (as with
@command{gzip --no-name}) and return true. Otherwise return @code{#f}.
When @var{keep-mtime?} is true, preserve @var{file}'s modification time.
@end deffn
@subsection File Manipulation
The following procedures and macros help create, modify, and delete
files. They provide functionality comparable to common shell utilities
such as @command{mkdir -p}, @command{cp -r}, @command{rm -r}, and
@command{sed}. They complement Guile's extensive, but low-level, file
system interface (@pxref{POSIX,,, guile, GNU Guile Reference Manual}).
@deffn {Scheme Syntax} with-directory-excursion @var{directory} @var{body}@dots{}
Run @var{body} with @var{directory} as the process's current directory.
Essentially, this macro changes the current directory to @var{directory}
before evaluating @var{body}, using @code{chdir} (@pxref{Processes,,,
guile, GNU Guile Reference Manual}). It changes back to the initial
directory when the dynamic extent of @var{body} is left, be it @i{via}
normal procedure return or @i{via} a non-local exit such as an
exception.
@end deffn
@deffn {Scheme Procedure} mkdir-p @var{dir}
Create directory @var{dir} and all its ancestors.
@end deffn
@deffn {Scheme Procedure} install-file @var{file} @var{directory}
Create @var{directory} if it does not exist and copy @var{file} in there
under the same name.
@end deffn
@deffn {Scheme Procedure} make-file-writable @var{file}
Make @var{file} writable for its owner.
@end deffn
@deffn {Scheme Procedure} copy-recursively @var{source} @var{destination} @
[#:log (current-output-port)] [#:follow-symlinks? #f] [#:keep-mtime? #f]
Copy @var{source} directory to @var{destination}. Follow symlinks if
@var{follow-symlinks?} is true; otherwise, just preserve them. When
@var{keep-mtime?} is true, keep the modification time of the files in
@var{source} on those of @var{destination}. Write verbose output to the
@var{log} port.
@end deffn
@deffn {Scheme Procedure} delete-file-recursively @var{dir} @
[#:follow-mounts? #f]
Delete @var{dir} recursively, like @command{rm -rf}, without following
symlinks. Don't follow mount points either, unless @var{follow-mounts?}
is true. Report but ignore errors.
@end deffn
@deffn {Scheme Syntax} substitute* @var{file} @
((@var{regexp} @var{match-var}@dots{}) @var{body}@dots{}) @dots{}
Substitute @var{regexp} in @var{file} by the string returned by
@var{body}. @var{body} is evaluated with each @var{match-var} bound to
the corresponding positional regexp sub-expression. For example:
@lisp
(substitute* file
(("hello")
"good morning\n")
(("foo([a-z]+)bar(.*)$" all letters end)
(string-append "baz" letter end)))
@end lisp
Here, anytime a line of @var{file} contains @code{hello}, it is replaced
by @code{good morning}. Anytime a line of @var{file} matches the second
regexp, @code{all} is bound to the complete match, @code{letters} is bound
to the first sub-expression, and @code{end} is bound to the last one.
8373
8374
8375
8376
8377
8378
8379
8380
8381
8382
8383
8384
8385
8386
8387
8388
8389
8390
8391
8392
8393
8394
8395
8396
8397
8398
8399
8400
8401
8402
8403
8404
8405
8406
8407
8408
8409
8410
8411
8412
8413
8414
8415
8416
8417
8418
8419
8420
8421
8422
8423
8424
8425
8426
8427
8428
8429
8430
8431
When one of the @var{match-var} is @code{_}, no variable is bound to the
corresponding match substring.
Alternatively, @var{file} may be a list of file names, in which case
they are all subject to the substitutions.
Be careful about using @code{$} to match the end of a line; by itself it
won't match the terminating newline of a line.
@end deffn
@subsection File Search
@cindex file, searching
This section documents procedures to search and filter files.
@deffn {Scheme Procedure} file-name-predicate @var{regexp}
Return a predicate that returns true when passed a file name whose base
name matches @var{regexp}.
@end deffn
@deffn {Scheme Procedure} find-files @var{dir} [@var{pred}] @
[#:stat lstat] [#:directories? #f] [#:fail-on-error? #f]
Return the lexicographically sorted list of files under @var{dir} for
which @var{pred} returns true. @var{pred} is passed two arguments: the
absolute file name, and its stat buffer; the default predicate always
returns true. @var{pred} can also be a regular expression, in which
case it is equivalent to @code{(file-name-predicate @var{pred})}.
@var{stat} is used to obtain file information; using @code{lstat} means
that symlinks are not followed. If @var{directories?} is true, then
directories will also be included. If @var{fail-on-error?} is true,
raise an exception upon error.
@end deffn
Here are a few examples where we assume that the current directory is
the root of the Guix source tree:
@lisp
;; List all the regular files in the current directory.
(find-files ".")
@result{} ("./.dir-locals.el" "./.gitignore" @dots{})
;; List all the .scm files under gnu/services.
(find-files "gnu/services" "\\.scm$")
@result{} ("gnu/services/admin.scm" "gnu/services/audio.scm" @dots{})
;; List ar files in the current directory.
(find-files "." (lambda (file stat) (ar-file? file)))
@result{} ("./libformat.a" "./libstore.a" @dots{})
@end lisp
@deffn {Scheme Procedure} which @var{program}
Return the complete file name for @var{program} as found in
@code{$PATH}, or @code{#f} if @var{program} could not be found.
@end deffn
@subsection Build Phases
@cindex build phases
The @code{(guix build utils)} also contains tools to manipulate build
phases as used by build systems (@pxref{Build Systems}). Build phases
are represented as association lists or ``alists'' (@pxref{Association
Lists,,, guile, GNU Guile Reference Manual}) where each key is a symbol
naming the phase and the associated value is a procedure (@pxref{Build
Phases}).
8438
8439
8440
8441
8442
8443
8444
8445
8446
8447
8448
8449
8450
8451
8452
8453
8454
8455
8456
8457
8458
8459
8460
8461
8462
8463
8464
8465
8466
8467
8468
8469
8470
8471
8472
8473
8474
8475
8476
8477
8478
8479
8480
8481
8482
8483
Guile core and the @code{(srfi srfi-1)} module both provide tools to
manipulate alists. The @code{(guix build utils)} module complements
those with tools written with build phases in mind.
@cindex build phases, modifying
@deffn {Scheme Syntax} modify-phases @var{phases} @var{clause}@dots{}
Modify @var{phases} sequentially as per each @var{clause}, which may
have one of the following forms:
@lisp
(delete @var{old-phase-name})
(replace @var{old-phase-name} @var{new-phase})
(add-before @var{old-phase-name} @var{new-phase-name} @var{new-phase})
(add-after @var{old-phase-name} @var{new-phase-name} @var{new-phase})
@end lisp
Where every @var{phase-name} above is an expression evaluating to a
symbol, and @var{new-phase} an expression evaluating to a procedure.
@end deffn
The example below is taken from the definition of the @code{grep}
package. It adds a phase to run after the @code{install} phase, called
@code{fix-egrep-and-fgrep}. That phase is a procedure (@code{lambda*}
is for anonymous procedures) that takes a @code{#:outputs} keyword
argument and ignores extra keyword arguments (@pxref{Optional
Arguments,,, guile, GNU Guile Reference Manual}, for more on
@code{lambda*} and optional and keyword arguments.) The phase uses
@code{substitute*} to modify the installed @file{egrep} and @file{fgrep}
scripts so that they refer to @code{grep} by its absolute file name:
@lisp
(modify-phases %standard-phases
(add-after 'install 'fix-egrep-and-fgrep
;; Patch 'egrep' and 'fgrep' to execute 'grep' via its
;; absolute file name instead of searching for it in $PATH.
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))
(bin (string-append out "/bin")))
(substitute* (list (string-append bin "/egrep")
(string-append bin "/fgrep"))
(("^exec grep")
(string-append "exec " bin "/grep")))
#t))))
@end lisp
In the example below, phases are modified in two ways: the standard
@code{configure} phase is deleted, presumably because the package does
not have a @file{configure} script or anything similar, and the default
@code{install} phase is replaced by one that manually copies the
executable files to be installed:
@lisp
(modify-phases %standard-phases
(delete 'configure) ;no 'configure' script
(replace 'install
(lambda* (#:key outputs #:allow-other-keys)
;; The package's Makefile doesn't provide an "install"
;; rule so do it by ourselves.
(let ((bin (string-append (assoc-ref outputs "out")
"/bin")))
(install-file "footswitch" bin)
(install-file "scythe" bin)
#t))))
@end lisp
@node The Store
@section The Store
@cindex store
@cindex store items
@cindex store paths
Conceptually, the @dfn{store} is the place where derivations that have
been built successfully are stored---by default, @file{/gnu/store}.
Sub-directories in the store are referred to as @dfn{store items} or
sometimes @dfn{store paths}. The store has an associated database that
contains information such as the store paths referred to by each store
path, and the list of @emph{valid} store items---results of successful
builds. This database resides in @file{@var{localstatedir}/guix/db},
where @var{localstatedir} is the state directory specified @i{via}
@option{--localstatedir} at configure time, usually @file{/var}.
The store is @emph{always} accessed by the daemon on behalf of its clients
(@pxref{Invoking guix-daemon}). To manipulate the store, clients
connect to the daemon over a Unix-domain socket, send requests to it,
and read the result---these are remote procedure calls, or RPCs.
@quotation Note
Users must @emph{never} modify files under @file{/gnu/store} directly.
This would lead to inconsistencies and break the immutability
assumptions of Guix's functional model (@pxref{Introduction}).
@xref{Invoking guix gc, @command{guix gc --verify}}, for information on
how to check the integrity of the store and attempt recovery from
accidental modifications.
@end quotation
The @code{(guix store)} module provides procedures to connect to the
daemon, and to perform RPCs. These are described below. By default,
@code{open-connection}, and thus all the @command{guix} commands,
connect to the local daemon or to the URI specified by the
@env{GUIX_DAEMON_SOCKET} environment variable.
@defvr {Environment Variable} GUIX_DAEMON_SOCKET
When set, the value of this variable should be a file name or a URI
designating the daemon endpoint. When it is a file name, it denotes a
Unix-domain socket to connect to. In addition to file names, the
supported URI schemes are:
@table @code
@item file
@itemx unix
These are for Unix-domain sockets.
@code{file:///var/guix/daemon-socket/socket} is equivalent to
@file{/var/guix/daemon-socket/socket}.
@item guix
@cindex daemon, remote access
@cindex remote access to the daemon
@cindex daemon, cluster setup
@cindex clusters, daemon setup
These URIs denote connections over TCP/IP, without encryption nor
authentication of the remote host. The URI must specify the host name
and optionally a port number (by default port 44146 is used):
guix://master.guix.example.org:1234
This setup is suitable on local networks, such as clusters, where only
trusted nodes may connect to the build daemon at
@code{master.guix.example.org}.
The @option{--listen} option of @command{guix-daemon} can be used to
instruct it to listen for TCP connections (@pxref{Invoking guix-daemon,
@item ssh
@cindex SSH access to build daemons
These URIs allow you to connect to a remote daemon over SSH. This
feature requires Guile-SSH (@pxref{Requirements}) and a working
@command{guile} binary in @env{PATH} on the destination machine. It
supports public key and GSSAPI authentication. A typical URL might look
like this:
ssh://charlie@@guix.example.org:22
As for @command{guix copy}, the usual OpenSSH client configuration files
are honored (@pxref{Invoking guix copy}).
@end table
Additional URI schemes may be supported in the future.
@c XXX: Remove this note when the protocol incurs fewer round trips
@c and when (guix derivations) no longer relies on file system access.
@quotation Note
The ability to connect to remote build daemons is considered
experimental as of @value{VERSION}. Please get in touch with us to
share any problems or suggestions you may have (@pxref{Contributing}).
@end quotation
@end defvr
@deffn {Scheme Procedure} open-connection [@var{uri}] [#:reserve-space? #t]
Connect to the daemon over the Unix-domain socket at @var{uri} (a string). When
@var{reserve-space?} is true, instruct it to reserve a little bit of
extra space on the file system so that the garbage collector can still
operate should the disk become full. Return a server object.
@var{file} defaults to @code{%default-socket-path}, which is the normal
location given the options that were passed to @command{configure}.
@deffn {Scheme Procedure} close-connection @var{server}
Close the connection to @var{server}.
@defvr {Scheme Variable} current-build-output-port
This variable is bound to a SRFI-39 parameter, which refers to the port
where build and error logs sent by the daemon should be written.
@end defvr
Procedures that make RPCs all take a server object as their first
argument.
@deffn {Scheme Procedure} valid-path? @var{server} @var{path}
@cindex invalid store items
Return @code{#t} when @var{path} designates a valid store item and
@code{#f} otherwise (an invalid item may exist on disk but still be
invalid, for instance because it is the result of an aborted or failed
A @code{&store-protocol-error} condition is raised if @var{path} is not
prefixed by the store directory (@file{/gnu/store}).
@end deffn
@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}]
Add @var{text} under file @var{name} in the store, and return its store
path. @var{references} is the list of store paths referred to by the
resulting store path.
@end deffn
@deffn {Scheme Procedure} build-derivations @var{store} @var{derivations} @
[@var{mode}]
Build @var{derivations}, a list of @code{<derivation>} objects, @file{.drv}
file names, or derivation/output pairs, using the specified
@var{mode}---@code{(build-mode normal)} by default.
Note that the @code{(guix monads)} module provides a monad as well as
monadic versions of the above procedures, with the goal of making it
more convenient to work with code that accesses the store (@pxref{The
Store Monad}).
@c FIXME
@i{This section is currently incomplete.}
@node Derivations
@section Derivations
@cindex derivations
Low-level build actions and the environment in which they are performed
are represented by @dfn{derivations}. A derivation contains the
following pieces of information:
@itemize
@item
The outputs of the derivation---derivations produce at least one file or
directory in the store, but may produce more.
@cindex build-time dependencies
@cindex dependencies, build-time
The inputs of the derivations---i.e., its build-time dependencies---which may
be other derivations or plain files in the store (patches, build scripts,
@item
The system type targeted by the derivation---e.g., @code{x86_64-linux}.
@item
The file name of a build script in the store, along with the arguments
to be passed.
@item
A list of environment variables to be defined.
@end itemize
@cindex derivation path
Derivations allow clients of the daemon to communicate build actions to
the store. They exist in two forms: as an in-memory representation,
both on the client- and daemon-side, and as files in the store whose
name end in @file{.drv}---these files are referred to as @dfn{derivation
paths}. Derivations paths can be passed to the @code{build-derivations}
procedure to perform the build actions they prescribe (@pxref{The
Store}).
@cindex fixed-output derivations
Operations such as file downloads and version-control checkouts for
which the expected content hash is known in advance are modeled as
@dfn{fixed-output derivations}. Unlike regular derivations, the outputs
of a fixed-output derivation are independent of its inputs---e.g., a
source code download produces the same result regardless of the download
method and tools being used.
@cindex references
@cindex run-time dependencies
@cindex dependencies, run-time
The outputs of derivations---i.e., the build results---have a set of
@dfn{references}, as reported by the @code{references} RPC or the
@command{guix gc --references} command (@pxref{Invoking guix gc}). References
are the set of run-time dependencies of the build results. References are a
subset of the inputs of the derivation; this subset is automatically computed
by the build daemon by scanning all the files in the outputs.
The @code{(guix derivations)} module provides a representation of
derivations as Scheme objects, along with procedures to create and
otherwise manipulate derivations. The lowest-level primitive to create
a derivation is the @code{derivation} procedure:
@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @
@var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:inputs '()] [#:env-vars '()] @
[#:system (%current-system)] [#:references-graphs #f] @
[#:allowed-references #f] [#:disallowed-references #f] @
[#:leaked-env-vars #f] [#:local-build? #f] @
[#:substitutable? #t] [#:properties '()]
Build a derivation with the given arguments, and return the resulting
@code{<derivation>} object.
When @var{hash} and @var{hash-algo} are given, a
@dfn{fixed-output derivation} is created---i.e., one whose result is
known in advance, such as a file download. If, in addition,
@var{recursive?} is true, then that fixed output may be an executable
file or a directory and @var{hash} must be the hash of an archive
containing this output.
When @var{references-graphs} is true, it must be a list of file
name/store path pairs. In that case, the reference graph of each store
path is exported in the build environment in the corresponding file, in
a simple text format.
When @var{allowed-references} is true, it must be a list of store items
or outputs that the derivation's output may refer to. Likewise,
@var{disallowed-references}, if true, must be a list of things the
outputs may @emph{not} refer to.
When @var{leaked-env-vars} is true, it must be a list of strings
denoting environment variables that are allowed to ``leak'' from the
daemon's environment to the build environment. This is only applicable
to fixed-output derivations---i.e., when @var{hash} is true. The main
use is to allow variables such as @code{http_proxy} to be passed to
derivations that download files.
When @var{local-build?} is true, declare that the derivation is not a
good candidate for offloading and should rather be built locally
(@pxref{Daemon Offload Setup}). This is the case for small derivations
where the costs of data transfers would outweigh the benefits.
When @var{substitutable?} is false, declare that substitutes of the
derivation's output should not be used (@pxref{Substitutes}). This is
useful, for instance, when building packages that capture details of the
host CPU instruction set.
@var{properties} must be an association list describing ``properties'' of the
derivation. It is kept as-is, uninterpreted, in the derivation.
@end deffn
@noindent
Here's an example with a shell script as its builder, assuming
@var{store} is an open connection to the daemon, and @var{bash} points
to a Bash executable in the store:
@lisp
(use-modules (guix utils)
(guix store)
(guix derivations))
(let ((builder ; add the Bash script to the store
(add-text-to-store store "my-builder.sh"
"echo hello world > $out\n" '())))
(derivation store "foo"
bash `("-e" ,builder)
#:inputs `((,bash) (,builder))
#:env-vars '(("HOME" . "/homeless"))))
@result{} #<derivation /gnu/store/@dots{}-foo.drv => /gnu/store/@dots{}-foo>
@end lisp
As can be guessed, this primitive is cumbersome to use directly. A
better approach is to write build scripts in Scheme, of course! The
best course of action for that is to write the build code as a
``G-expression'', and to pass it to @code{gexp->derivation}. For more
information, @pxref{G-Expressions}.
Once upon a time, @code{gexp->derivation} did not exist and constructing
derivations with build code written in Scheme was achieved with
@code{build-expression->derivation}, documented below. This procedure
is now deprecated in favor of the much nicer @code{gexp->derivation}.
@deffn {Scheme Procedure} build-expression->derivation @var{store} @
@var{name} @var{exp} @
[#:system (%current-system)] [#:inputs '()] @
[#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:env-vars '()] [#:modules '()] @
[#:references-graphs #f] [#:allowed-references #f] @
[#:disallowed-references #f] @
[#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f]
Return a derivation that executes Scheme expression @var{exp} as a
builder for derivation @var{name}. @var{inputs} must be a list of
@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
@code{"out"} is assumed. @var{modules} is a list of names of Guile
modules from the current search path to be copied in the store,
compiled, and made available in the load path during the execution of
@var{exp}---e.g., @code{((guix build utils) (guix build
gnu-build-system))}.
@var{exp} is evaluated in an environment where @code{%outputs} is bound
to a list of output/path pairs, and where @code{%build-inputs} is bound
to a list of string/output-path pairs made from @var{inputs}.
Optionally, @var{env-vars} is a list of string pairs specifying the name
and value of environment variables visible to the builder. The builder
terminates by passing the result of @var{exp} to @code{exit}; thus, when
@var{exp} returns @code{#f}, the build is considered to have failed.
@var{exp} is built using @var{guile-for-build} (a derivation). When
@var{guile-for-build} is omitted or is @code{#f}, the value of the
@code{%guile-for-build} fluid is used instead.
See the @code{derivation} procedure for the meaning of
@var{references-graphs}, @var{allowed-references},
@var{disallowed-references}, @var{local-build?}, and
@var{substitutable?}.
@noindent
Here's an example of a single-output derivation that creates a directory
containing one file:
@lisp
(let ((builder '(let ((out (assoc-ref %outputs "out")))
(mkdir out) ; create /gnu/store/@dots{}-goo
(call-with-output-file (string-append out "/test")
(lambda (p)
(display '(hello guix) p))))))
(build-expression->derivation store "goo" builder))
@result{} #<derivation /gnu/store/@dots{}-goo.drv => @dots{}>
@end lisp
@node The Store Monad
@section The Store Monad
@cindex monad
The procedures that operate on the store described in the previous
sections all take an open connection to the build daemon as their first
argument. Although the underlying model is functional, they either have
side effects or depend on the current state of the store.
The former is inconvenient: the connection to the build daemon has to be
carried around in all those functions, making it impossible to compose
functions that do not take that parameter with functions that do. The
latter can be problematic: since store operations have side effects
and/or depend on external state, they have to be properly sequenced.
@cindex monadic values
@cindex monadic functions
This is where the @code{(guix monads)} module comes in. This module
provides a framework for working with @dfn{monads}, and a particularly
useful monad for our uses, the @dfn{store monad}. Monads are a
construct that allows two things: associating ``context'' with values
(in our case, the context is the store), and building sequences of
computations (here computations include accesses to the store). Values
in a monad---values that carry this additional context---are called
@dfn{monadic values}; procedures that return such values are called
@dfn{monadic procedures}.
Consider this ``normal'' procedure:
(define (sh-symlink store)
;; Return a derivation that symlinks the 'bash' executable.
(let* ((drv (package-derivation store bash))
(out (derivation->output-path drv))
(sh (string-append out "/bin/bash")))
(build-expression->derivation store "sh"
`(symlink ,sh %output))))
Using @code{(guix monads)} and @code{(guix gexp)}, it may be rewritten
as a monadic function:
(define (sh-symlink)
;; Same, but return a monadic value.
(mlet %store-monad ((drv (package->derivation bash)))
(gexp->derivation "sh"
#~(symlink (string-append #$drv "/bin/bash")
#$output))))
There are several things to note in the second version: the @code{store}
parameter is now implicit and is ``threaded'' in the calls to the
@code{package->derivation} and @code{gexp->derivation} monadic
procedures, and the monadic value returned by @code{package->derivation}
is @dfn{bound} using @code{mlet} instead of plain @code{let}.
As it turns out, the call to @code{package->derivation} can even be
omitted since it will take place implicitly, as we will see later
(@pxref{G-Expressions}):
(define (sh-symlink)
(gexp->derivation "sh"
#~(symlink (string-append #$bash "/bin/bash")
#$output)))
@c See
@c <https://syntaxexclamation.wordpress.com/2014/06/26/escaping-continuations/>
@c for the funny quote.
Calling the monadic @code{sh-symlink} has no effect. As someone once
said, ``you exit a monad like you exit a building on fire: by running''.
So, to exit the monad and get the desired effect, one must use
@code{run-with-store}:
(run-with-store (open-connection) (sh-symlink))
@result{} /gnu/store/...-sh-symlink
Note that the @code{(guix monad-repl)} module extends the Guile REPL with
new ``meta-commands'' to make it easier to deal with monadic procedures:
@code{run-in-store}, and @code{enter-store-monad}. The former is used
to ``run'' a single monadic value through the store:
scheme@@(guile-user)> ,run-in-store (package->derivation hello)
$1 = #<derivation /gnu/store/@dots{}-hello-2.9.drv => @dots{}>
The latter enters a recursive REPL, where all the return values are
automatically run through the store:
@example
scheme@@(guile-user)> ,enter-store-monad
store-monad@@(guile-user) [1]> (package->derivation hello)
$2 = #<derivation /gnu/store/@dots{}-hello-2.9.drv => @dots{}>
store-monad@@(guile-user) [1]> (text-file "foo" "Hello!")
$3 = "/gnu/store/@dots{}-foo"
store-monad@@(guile-user) [1]> ,q
scheme@@(guile-user)>
@end example
@noindent
Note that non-monadic values cannot be returned in the
@code{store-monad} REPL.
The main syntactic forms to deal with monads in general are provided by
the @code{(guix monads)} module and are described below.
@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ...
Evaluate any @code{>>=} or @code{return} forms in @var{body} as being
in @var{monad}.
@end deffn
@deffn {Scheme Syntax} return @var{val}
Return a monadic value that encapsulates @var{val}.
@end deffn
@deffn {Scheme Syntax} >>= @var{mval} @var{mproc} ...
@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic
procedures @var{mproc}@dots{}@footnote{This operation is commonly
referred to as ``bind'', but that name denotes an unrelated procedure in
Guile. Thus we use this somewhat cryptic symbol inherited from the
Haskell language.}. There can be one @var{mproc} or several of them, as
in this example:
(run-with-state
(with-monad %state-monad
(>>= (return 1)
(lambda (x) (return (+ 1 x)))
(lambda (x) (return (* 2 x)))))
'some-state)
@result{} 4
@result{} some-state
@end deffn
@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @
@var{body} ...
@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @
@var{body} ...
Bind the variables @var{var} to the monadic values @var{mval} in
@var{body}, which is a sequence of expressions. As with the bind