* guix/build/union.scm: Rewrite; only 'file=?' remains unchanged.  Remove
  'tree-union' and 'delete-duplicate-leaves' exports.  Merge inputs in a
  breadth-first fashion.  Follow symlinks for purposes of making decisions
  about the merge.

* tests/union.scm: Remove tests of 'tree-union' and 'delete-duplicate-leaves'.

Based on a patch by Dan Kennedy <danielk1977@gmail.com>.

* gnu/packages/patches/sqlite-large-page-size-fix.patch: New file.
* gnu/packages/sqlite.scm (sqlite): Add it.
* gnu-system.am (dist_patch_DATA): Add it.

* gnu/packages/sqlite.scm (sqlite): Upgrade to 3.8.4.2.

* gnu/packages/sqlite.scm (sqlite): Compute the numeric-version from the
  traditional version string, and use it to compute the source URL.

* gnu/packages/mail.scm (mutt): Upgrade to 1.5.23.  Add bitbucket
  source URL and try it first.

* gnu/packages/version-control.scm (bazaar): Add a phase that
  fixes setup.py to install man pages in share/man.

* gnu/packages/ocaml.scm (ocaml): Pass -mandir to configure.

* gnu/packages/pth.scm (pth): Add --mandir to configure flags.

* gnu/packages/tcl.scm (expect): Add --mandir configure flag.
  Use 'let' to simplify configure flags generation.

* gnu/packages/zip.scm (zip): Eliminate custom configure phase which copied
  and patched the Makefile.  Instead add "-f" and "prefix" to make-flags, as
  well as "MANDIR".  Remove "generic_gcc" from the make-flags, so that it will
  not be passed during other phases involving 'make'.  Instead replace the
  'build' phase with one that specifies the "generic_gcc" target.

* gnu/packages/lua.scm (lua): Use 'system*' instead of 'system'.
  Add INSTALL_MAN argument to 'make'.

* gnu/packages/indent.scm (indent): Add a phase to fix the docdir
  setting in doc/Makefile.in.

* guix/scripts/package.scm (search-path-environment-variables)[manifest-entry->package]:
  Use 'find-best-packages-by-name' instead of 'find-packages-by-name'.
  On a profile with 140 packages, this reduces execution time of this
  procedure from 5.8 seconds to 2.9 seconds (50% improvement.)

* guix/scripts/package.scm (maybe-register-gc-root): New procedure.
* tests/guix-package.sh (profile): Grep the output of "guix gc
  --list-live" in a couple of places.

* gnu/packages/linux.scm (net-tools): Add 'INSTALLNLSDIR' and 'mandir'
  variables to make flags.

* gnu/packages/ssh.scm (libssh): Upgrade to 0.6.3.

* gnu/packages/pulseaudio.scm: Set the 'CK_DEFAULT_TIMEOUT' environment
  variable to 120 in the 'pre-check' phase.  Suggested by Ludovic Courtès.

* gnu/packages/file.scm (file): Upgrade to 5.18.

* gnu/packages/guile.scm (guile-json)[source]: Add 'snippet'.

* guix/pki.scm (acl-entry-sexp, acl-sexp): Remove.
  (public-keys->acl, current-acl): Return a native sexp.
  (acl->public-keys, authorized-key?): Expect ACL to be a native sexp.
* guix/scripts/archive.scm (authorize-key): Convert ACL to
  canonical-sexp when writing it.

* guix/scripts/substitute-binary.scm (regexp-exec): Change formals
  to (rx str . rest).

* guix/scripts/substitute-binary.scm (guix-substitute-binary) <--query>:
  Cache the result of (current-acl); pass it to 'valid-narinfo?' calls.
  This saves 12% wall-clock time for "guix build emacs -n".

* gnu/packages/mail.scm (bogofilter): New variable.

* gnu/packages/guile.scm (guile-json): New variable.

* guix/nar.scm (restore-file-set)[assert-valid-signature]: Rewrite in
  terms of 'signature-case'.
* guix/scripts/substitute-binary.scm (narinfo-signature->canonical-sexp):
  Call 'leave' instead of 'raise' when SIGNATURE is invalid.
  (&nar-signature-error, &nar-invalid-hash-error): Remove.
  (assert-valid-signature): Add 'narinfo' parameter; remove 'port'.
  Rewrite in terms of 'signature-case' and 'leave'.  Mention NARINFO's
  URI in error messages.  Adjust caller.
  (narinfo-sha256): New procedure.
  (assert-valid-narinfo): Use it.
  (valid-narinfo?): Rewrite using 'narinfo-sha256' and
  'signature-case'.
* tests/substitute-binary.scm (assert-valid-signature,
  test-error-condition): Remove.
  ("corrupt signature data", "unauthorized public key", "invalid
  signature"): Remove.

* guix/pki.scm (%signature-status): New procedure.
  (signature-case): New macro.
* tests/pki.scm (%secret-key, %alternate-secret-key): New variables.
  ("signature-case valid-signature", "signature-case invalid-signature",
  "signature-case hash-mismatch", "signature-case unauthorized-key",
  "signature-case corrupt-signature"): New tests.

* guix/scripts/substitute-binary.scm (assert-valid-narinfo): Add
  #:verbose? parameter; when true, write "found valid signature".
  (valid-narinfo?): Pass #:verbose? #f.

* guix/scripts/archive.scm (authorize-key): Add 'mkdir-p' call.
  Reported by Alex Sassmannshausen <alex.sassmannshausen@gmail.com>.

* doc/guix.texi (Substitutes): add a missing 's'.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>

augmented phases.

* gnu/packages/compression.scm (sharutils) [arguments]: Remove unused
  lambda args.
* gnu/packages/gl.scm (mesa) [arguments]: Same
* gnu/packages/fltk.scm [arguments]: Same.  Substitute const check
  phase with #:tests? #f.  Add pre-configure phase.
* gnu/packages/ghostscript.scm (ghostscript) [arguments]: Put makefile
  patches in separate phase.  Put so steps oinseparate phases.
* gnu/packages/glib.scm (gobject-introspection) [arguments]: Remove
  unused lambda args.  Put patches in pre-configure phase.
* gnu/packages/gnupg.scm (gnupg) [arguments]: Put patch in
  pre-configure phase.
  (pius) [arguments]: Delete const #t configure and build phases.
  (signing-party) [arguments]: Factor patches into post-unpack phase.
  (paperkey) [arguments]: Remove unused lambda args.  Factor out
  patches into pre-check phase.
* gnu/packages/icu4c.scm [arguments]: Change source dir after standard
  unpack phase.  Factor configure patches into pre-configure phase.
* gnu/packages/lsof.scm [arguments]: Remove unused lambda args.
  Remove unnecessary apply.
* gnu/packages/lvm.scm (lvm2) [arguments]: Factor out patches into
  pre-configure phase.
* gnu/packages/libtiff.scm [arguments]: Same
* gnu/packages/maths.scm (hdf5) [arguments]: Same
* gnu/packages/gtk.scm (gtk+-2) [arguments]: Same
* gnu/packages/mp3.scm (libmad) [arguments]: Same
  (id3lib) [arguments]: Same
* gnu/packages/python.scm (python-2) [arguments]: Same
* gnu/packages/texlive.scm (texlive) [arguments]: Same
* gnu/packages/pretty-print.scm (a2ps) [arguments]: Same
  (trueprint) [arguments]: Same
  (source-highlight) [arguments]: Same.  Remove unused lambda args.
* gnu/packages/netpbm.scm [arguments]: Remove unused lambda args.
  Factor out test patches into pre-check phase.  Condense calls to
  substitute*
* gnu/packages/openldap.scm [arguments]: Factor out libtool copy into
  post-configure phases. [synopsis] Remove package name.
* gnu/packages/ssh.scm (openssh) [arguments]: Factor out patches into
  separate phases.
* gnu/packages/tcsh.scm [arguments]: Factor out test patches into
  pre-check phase.
* gnu/packages/version-control.scm (git) [arguments]: Factor out
  patches into post-configure phase.
* gnu/packages/vim.scm [arguments]: Same. [synopsis] Remove package
  name.
* gnu/packages/vpn.scm (openconnect) [arguments]: Put configure flags
  in #:configure-flags

* guix/scripts/offload.scm (transfer-and-offload): Exit with code 100
  upon build failure.

* doc/guix.texi (Invoking guix-daemon): Add cross-reference to
  "Substitutes".
  (Features): Remove footnote about unauthenticated substitutes.  Link
  to "Substitutes".
  (Substitutes): New node.
  (Invoking guix build): Link to "Substitutes".
  (Packaging Guidelines): Likewise.

* hydra.gnu.org.pub: New file.
* Makefile.am (pkgdata_DATA): New variable.

* tests/substitute-binary.scm (test-error*): Rename to...
  (test-quit): ... this.  Add 'error-rx' parameter and honor it.
  ("not a number", "wrong version number", "substitute, no signature",
  "substitute, invalid hash", "substitute, unauthorized key"): Adjust
  accordingly.

* tests/substitute-binary.scm (%narinfo): Set 'URL', 'Compression', and
  'NarHash'.
  (call-with-narinfo): Create 'example.out' and 'example.nar'.
  ("query narinfo without signature", "substitute, no signature",
  "substitute, authorized key"): New tests.

* tests/substitute-binary.scm (signature-body): Change 'str' parameter
  to 'bv', and expect it to be a bytevector.
  (%signature-body, %wrong-signature, %acl): Remove.
  (signature): Rename to...
  (signature-field): ... this.  Add 'bv-or-str' parameter.  Change 'str'
  parameter to #:version.  Add #:public-key parameter.  Call
  'signature-body' directly.  Change domain part of the signature to
  'example.gnu.org'.
  ("not a number", "wrong version number", "valid
  narinfo-signature->canonical-sexp"): Use 'signature-field' instead of
  'signature' or %SIGNATURE.
  (test-error-condition): Add 'message-rx' parameter and honor it.
  ("corrupt signature data", "unauthorized public key", "invalid
  signature"): Adjust accordingly.
  (narinfo, %signed-narinfo): Remove.
  ("query narinfo with invalid hash"): Use '%narinfo' and
  'signature-field' instead of 'narinfo' and '%signature'.
  ("query narinfo signed with authorized key", "query narinfo signed
  with unauthorized key", "substitute, invalid hash", "substitute,
  unauthorized key"): Likewise.

* tests/store.scm ("substitute, corrupt output hash"): New test.

* guix/scripts/substitute-binary.scm (narinfo-signature->canonical-sexp):
  Catch 'gcry-error' around 'string->canonical-sexp' call, and re-raise
  as a SRFI-35 &message and &nar-signature-error.
  (narinfo-maker): Handle when SIGNATURE is #f or an invalid canonical
  sexp.
  (&nar-signature-error, &nar-invalid-hash-error): New variables.
  (assert-valid-signature): Use them.  Expect 'signature' to be a
  canonical sexp.
  (read-narinfo): Remove authentication and authorization checks.
  (%signature-line-rx): New variable.
  (assert-valid-narinfo, valid-narinfo?): New procedures.
  (guix-substitute-binary): Wrap body in 'with-error-handling'.
  [valid?]: New procedure.
  <--query>: Show only store items of narinfos that match
  'valid-narinfo?'.
  <--substitute>: Call 'assert-valid-narinfo'.
* tests/substitute-binary.scm (test-error*): Use 'test-equal'.
  (%keypair): Remove.
  (%public-key, %private-key): Load from signing-key.{pub,sec}.
  (signature-body): Add #:public-key parameter.
  (call-with-narinfo): New procedure.
  (with-narinfo): New macro.
  ("corrupt signature data", "unauthorized public key", "invalid
  signature"): Make the first argument to 'assert-valid-signature' a
  canonical sexp.
  ("invalid hash", "valid read-narinfo", "valid write-narinfo"):
  Remove.
  ("query narinfo with invalid hash", "query narinfo signed with
  authorized key", "query narinfo signed with unauthorized key",
  "substitute, invalid hash", "substitute, unauthorized key"): New
  tests.

* guix/scripts/substitute-binary.scm (<narinfo>)[uri-base]: New field.
  (narinfo-maker): Pass CACHE-URL as the 'uri-base' value.
  (string->narinfo): Add 'cache-uri' parameter.
  (lookup-narinfo)[cache-entry]: Switch to version 1.  Add 'cache-uri'
  field.  Adjust body accordingly.
  (remove-expired-cached-narinfos): Switch to version 1 by default.

* guix/scripts/substitute-binary.scm (<narinfo>): Add the 'signature'
  and 'contents' fields.
  (narinfo-signature->canonical-sexp): New function.
  (narinfo-maker): Add the 'signature' argument and use it.
  (assert-valid-signature): New function.
  (read-narinfo): Support the Signature field.
  (write-narinfo): Use 'narinfo-contents'.
  (%allow-unauthenticated-substitutes?): New variable.
* guix/base64.scm, tests/base64.scm, tests/substitute-binary.scm: New files.
* Makefile.am (SCM_TESTS): Add tests/base64.scm and
  tests/substitute-binary.scm.
  (MODULES): Add guix/base64.scm.
* test-env.in: Set 'GUIX_ALLOW_UNAUTHENTICATED_SUBSTITUTES'.