From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56041) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eDq6V-0000qz-Td for guix-patches@gnu.org; Sun, 12 Nov 2017 06:07:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eDq6Q-0007M2-Pb for guix-patches@gnu.org; Sun, 12 Nov 2017 06:07:07 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:55598) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eDq6Q-0007Lw-MU for guix-patches@gnu.org; Sun, 12 Nov 2017 06:07:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eDq6Q-0003Fw-GG for guix-patches@gnu.org; Sun, 12 Nov 2017 06:07:02 -0500 Subject: [bug#29271] [PATCH 1/1] doc: Split the "Substitutes" section into subsections. Resent-Message-ID: Received: from eggs.gnu.org ([2001:4830:134:3::10]:55983) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eDq5k-0000hC-Iw for guix-patches@gnu.org; Sun, 12 Nov 2017 06:06:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eDq5f-00075P-HG for guix-patches@gnu.org; Sun, 12 Nov 2017 06:06:20 -0500 Received: from mail-pf0-x22d.google.com ([2607:f8b0:400e:c00::22d]:44335) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eDq5f-000752-8g for guix-patches@gnu.org; Sun, 12 Nov 2017 06:06:15 -0500 Received: by mail-pf0-x22d.google.com with SMTP id x7so9871601pfa.1 for ; Sun, 12 Nov 2017 03:06:15 -0800 (PST) From: Chris Marusich Date: Sun, 12 Nov 2017 03:05:58 -0800 Message-Id: <20171112110558.1447-1-cmmarusich@gmail.com> In-Reply-To: <20171112110233.1384-1-cmmarusich@gmail.com> References: <20171112110233.1384-1-cmmarusich@gmail.com> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 29271@debbugs.gnu.org Cc: myglc2@gmail.com, Chris Marusich * doc/guix.texi (Substitutes): Move information into sections, and create one new subsection regarding substitution failure. (Top): In the detailed listing, add entries for the new subsections. (Hydra): Move info about Hydra here. Add information that explains when substitutes are enabled by default. (Substitute Server Authorization): Move information about how to enable or disable substitutes here. (Substitute Authentication): Move information about how Guix authenticates substitutes here. (Proxy Settings): Move information about how to download substitutes via proxy here. (Substitution Failure): New section. Add information about how Guix handles substitution failure. Be extra clear about behavior concerning --fallback. (On Trusting Binaries): Move information about trusting binaries here. (Invoking guix-daemon): Remove an incorrect statement about when substitutes are enabled by default. (Common Build Options): Clarify the behavior of --fallback. Co-authored-by: myglc2 --- doc/guix.texi | 126 +++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 94 insertions(+), 32 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 11a9de689..404ea2406 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -130,6 +130,15 @@ Package Management * Invoking guix pack:: Creating software bundles. * Invoking guix archive:: Exporting and importing store files. +Substitutes + +* Hydra:: One particular source of substitutes. +* Substitute Server Authorization:: How to enable or disable substitutes. +* Substitute Authentication:: How Guix verifies substitutes. +* Proxy Settings:: How to get substitutes via proxy. +* Substitution Failure:: What happens when substitution fails. +* On Trusting Binaries:: How can you trust that binary blob? + Programming Interface * Defining Packages:: Defining new packages. @@ -1108,10 +1117,6 @@ Do not use substitutes for build products. That is, always build things locally instead of allowing downloads of pre-built binaries (@pxref{Substitutes}). -By default substitutes are used, unless the client---such as the -@command{guix package} command---is explicitly invoked with -@code{--no-substitutes}. - When the daemon runs with @code{--no-substitutes}, clients can still explicitly enable substitution @i{via} the @code{set-build-options} remote procedure call (@pxref{The Store}). @@ -2108,7 +2113,6 @@ preserve transformations across upgrades, you should define your own package variant in a Guile module and add it to @code{GUIX_PACKAGE_PATH} (@pxref{Defining Packages}). - @node Substitutes @section Substitutes @@ -2116,8 +2120,8 @@ package variant in a Guile module and add it to @code{GUIX_PACKAGE_PATH} @cindex pre-built binaries Guix supports transparent source/binary deployment, which means that it can either build things locally, or download pre-built items from a -server. We call these pre-built items @dfn{substitutes}---they are -substitutes for local build results. In many cases, downloading a +server, or both. We call these pre-built items @dfn{substitutes}---they +are substitutes for local build results. In many cases, downloading a substitute is much faster than building things locally. Substitutes can be anything resulting from a derivation build @@ -2125,6 +2129,20 @@ Substitutes can be anything resulting from a derivation build pre-built package binaries, but source tarballs, for instance, which also result from derivation builds, can be available as substitutes. +@menu +* Hydra:: One particular source of substitutes. +* Substitute Server Authorization:: How to enable or disable substitutes. +* Substitute Authentication:: How Guix verifies substitutes. +* Proxy Settings:: How to get substitutes via proxy. +* Substitution Failure:: What happens when substitution fails. +* On Trusting Binaries:: How can you trust that binary blob? +@end menu + +@node Hydra +@subsection Hydra + +@cindex hydra +@cindex build farm The @code{hydra.gnu.org} server is a front-end to a build farm that builds packages from the GNU distribution continuously for some architectures, and makes them available as substitutes. This is the @@ -2141,9 +2159,19 @@ using HTTP makes all communications visible to an eavesdropper, who could use the information gathered to determine, for instance, whether your system has unpatched security vulnerabilities. +Substitutes from Hydra are enabled by default when using the Guix System +Distribution (@pxref{GNU Distribution}). However, substitutes are +disabled by default when using Guix on a foreign distribution +(@pxref{Installation}). The following paragraphs describe how to enable +or disable substitutes for Hydra; the same procedure can also be used to +enable substitutes for any other substitute server. + + +@node Substitute Server Authorization +@subsection Substitute Server Authorization + @cindex security -@cindex digital signatures -@cindex substitutes, authorization thereof +@cindex substitutes, how to enable @cindex access control list (ACL), for substitutes @cindex ACL (access control list), for substitutes To allow Guix to download substitutes from @code{hydra.gnu.org} or a @@ -2153,7 +2181,7 @@ imports, using the @command{guix archive} command (@pxref{Invoking guix archive}). Doing so implies that you trust @code{hydra.gnu.org} to not be compromised and to serve genuine substitutes. -This public key is installed along with Guix, in +Hydra's public key is installed along with Guix, in @code{@var{prefix}/share/guix/hydra.gnu.org.pub}, where @var{prefix} is the installation prefix of Guix. If you installed Guix from source, make sure you checked the GPG signature of @@ -2194,6 +2222,17 @@ The following files would be downloaded: This indicates that substitutes from @code{hydra.gnu.org} are usable and will be downloaded, when possible, for future builds. +@cindex substitutes, how to disable +The substitute mechanism can be disabled globally by running +@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking +guix-daemon}). It can also be disabled temporarily by passing the +@code{--no-substitutes} option to @command{guix package}, @command{guix +build}, and other command-line tools. + +@node Substitute Authentication +@subsection Substitute Authentication + +@cindex digital signatures Guix detects and raises an error when attempting to use a substitute that has been tampered with. Likewise, it ignores substitutes that are not signed, or that are not signed by one of the keys listed in the ACL. @@ -2218,15 +2257,6 @@ comes first in the list and can be considered a mirror of produce the same binaries, thanks to bit-reproducible builds (see below). -@vindex http_proxy -Substitutes are downloaded over HTTP or HTTPS. -The @code{http_proxy} environment -variable can be set in the environment of @command{guix-daemon} and is -honored for downloads of substitutes. Note that the value of -@code{http_proxy} in the environment where @command{guix build}, -@command{guix package}, and other client commands are run has -@emph{absolutely no effect}. - When using HTTPS, the server's X.509 certificate is @emph{not} validated (in other words, the server is not authenticated), contrary to what HTTPS clients such as Web browsers usually do. This is because Guix @@ -2234,18 +2264,49 @@ authenticates substitute information itself, as explained above, which is what we care about (whereas X.509 certificates are about authenticating bindings between domain names and public keys.) -You can get statistics on the substitutes provided by a server using the -@command{guix weather} command (@pxref{Invoking guix weather}). - -The substitute mechanism can be disabled globally by running -@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking -guix-daemon}). It can also be disabled temporarily by passing the -@code{--no-substitutes} option to @command{guix package}, @command{guix -build}, and other command-line tools. - +@node Proxy Settings +@subsection Proxy Settings -@unnumberedsubsec On Trusting Binaries +@vindex http_proxy +Substitutes are downloaded over HTTP or HTTPS. +The @code{http_proxy} environment +variable can be set in the environment of @command{guix-daemon} and is +honored for downloads of substitutes. Note that the value of +@code{http_proxy} in the environment where @command{guix build}, +@command{guix package}, and other client commands are run has +@emph{absolutely no effect}. +@node Substitution Failure +@subsection Substitution Failure + +Even when a substitute for a derivation is available, sometimes the +substitution attempt will fail. This can happen for a variety of +reasons: the substitute server might be offline, the substitute may +recently have been deleted, the connection might have been interrupted, +etc. + +When substitutes are enabled and a substitute for a derivation is +available, but the substitution attempt fails, Guix will attempt to +build the derivation locally depending on whether or not +@code{--fallback} was given (@pxref{fallback-option}). Specifically, if +@code{--fallback} was omitted, then no local build will be performed, +and the derivation is considered to have failed. However, if +@code{--fallback} was given, then Guix will attempt to build the +derivation locally, and the success or failure of the derivation depends +on the success or failure of the local build. Note that when +substitutes are disabled or no substitute is available for the +derivation in question, a local build will @emph{always} be performed, +regardless of whether or not @code{--fallback} was given. + +To get an idea of how many substitutes are available right now, you can +try running the @command{guix weather} command (@pxref{Invoking guix +weather}). This command provides statistics on the substitutes provided +by a server. + +@node On Trusting Binaries +@subsection On Trusting Binaries + +@cindex trust, of pre-built binaries Today, each individual's control over their own computing is at the mercy of institutions, corporations, and groups with enough power and determination to subvert the computing infrastructure and exploit its @@ -2272,7 +2333,6 @@ In the future, we want Guix to have support to publish and retrieve binaries to/from other users, in a peer-to-peer fashion. If you would like to discuss this project, join us on @email{guix-devel@@gnu.org}. - @node Packages with Multiple Outputs @section Packages with Multiple Outputs @@ -5191,9 +5251,11 @@ derivations has failed. @itemx -n Do not build the derivations. +@anchor{fallback-option} @item --fallback -When substituting a pre-built binary fails, fall back to building -packages locally. +Attempt to build locally instead of issuing an error when substitutes +are enabled and a substitute is available, but the substitution attempt +fails (@pxref{Substitution Failure}). @item --substitute-urls=@var{urls} @anchor{client-substitute-urls} -- 2.14.2