From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55962) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eyVN4-0002rS-CL for guix-patches@gnu.org; Wed, 21 Mar 2018 00:29:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eyVN0-0006eh-Lt for guix-patches@gnu.org; Wed, 21 Mar 2018 00:29:06 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:36819) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eyVN0-0006ec-GW for guix-patches@gnu.org; Wed, 21 Mar 2018 00:29:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eyVN0-0007c7-AJ for guix-patches@gnu.org; Wed, 21 Mar 2018 00:29:02 -0400 Subject: [bug#30572] [PATCH 2/7] tests: Add tests for "guix pack". Resent-Message-ID: From: Chris Marusich References: <20180222102933.4978-1-cmmarusich@gmail.com> <20180315040915.5556-1-cmmarusich@gmail.com> <20180315040915.5556-3-cmmarusich@gmail.com> <20180316220732.19c2375a@scratchpost.org> <87a7v6pnmm.fsf@gnu.org> <87y3im5b7u.fsf@gmail.com> Date: Wed, 21 Mar 2018 05:28:00 +0100 In-Reply-To: <87y3im5b7u.fsf@gmail.com> (Chris Marusich's message of "Wed, 21 Mar 2018 05:00:53 +0100") Message-ID: <87h8pa59yn.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 30572@debbugs.gnu.org --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Chris Marusich writes: > Sure thing. I'll include this change when pushing to origin. For the record, here's a new version of the patch with the two suggested improvements (put "warning:" in the warning message, and redirect superfluous output to /dev/null). =2D-=20 Chris --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0002-tests-Add-tests-for-guix-pack.patch Content-Transfer-Encoding: quoted-printable From=20dcb928d92bf92ef5ceff94501b91f98ed28a4226 Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Tue, 20 Feb 2018 02:17:54 +0100 Subject: [PATCH 2/7] tests: Add tests for "guix pack". * guix/scripts/pack.scm (%compressors) : New compressor. (%options) <--bootstrap>: New option. (show-help): Document the new --bootstrap option. (guix-pack): When --bootstrap is specified, use the bootstrap Guile, tar, and xz to build the pack, and do not use any profile hooks or locales. * doc/guix.texi (Invoking guix pull): Document the new --bootstrap option. * tests/guix-pack.sh: New file. * Makefile.am (SH_TESTS): Add guix-pack.sh. * gnu/packages/package-management.scm (guix) : Add util-linux. =2D-- Makefile.am | 1 + doc/guix.texi | 6 ++- gnu/packages/package-management.scm | 2 + guix/scripts/pack.scm | 61 +++++++++++++++++++++--------- tests/guix-pack.sh | 74 +++++++++++++++++++++++++++++++++= ++++ 5 files changed, 126 insertions(+), 18 deletions(-) create mode 100644 tests/guix-pack.sh diff --git a/Makefile.am b/Makefile.am index 6556799e6..637c934ed 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -374,6 +374,7 @@ SH_TESTS =3D \ tests/guix-download.sh \ tests/guix-gc.sh \ tests/guix-hash.sh \ + tests/guix-pack.sh \ tests/guix-package.sh \ tests/guix-package-net.sh \ tests/guix-system.sh \ diff --git a/doc/guix.texi b/doc/guix.texi index d3a7908f9..792539a12 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -23,7 +23,7 @@ Copyright @copyright{} 2015 Taylan Ulrich Bay=C4=B1rl=C4= =B1/Kammer@* Copyright @copyright{} 2015, 2016, 2017 Leo Famulari@* Copyright @copyright{} 2015, 2016, 2017, 2018 Ricardo Wurmus@* Copyright @copyright{} 2016 Ben Woodcroft@* =2DCopyright @copyright{} 2016, 2017 Chris Marusich@* +Copyright @copyright{} 2016, 2017, 2018 Chris Marusich@* Copyright @copyright{} 2016, 2017, 2018 Efraim Flashner@* Copyright @copyright{} 2016 John Darrington@* Copyright @copyright{} 2016, 2017 ng0@* @@ -2899,6 +2899,10 @@ added to it or removed from it after extraction of t= he pack. =20 One use case for this is the Guix self-contained binary tarball (@pxref{Binary Installation}). + +@item --bootstrap +Use the bootstrap binaries to build the pack. This option is only +useful to Guix developers. @end table =20 In addition, @command{guix pack} supports all the common build options diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-man= agement.scm index 709cdfd0f..a90ba7a21 100644 =2D-- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -257,6 +257,8 @@ ;; Many tests rely on the 'guile-bootstrap' package, which is why= we ;; have it here. ("boot-guile" ,(bootstrap-guile-origin (%current-system))) + ;; Some of the tests use "unshare" when it is available. + ("util-linux" ,util-linux) ,@(if (and (not (%current-target-system)) (string=3D? (%current-system) "x86_64-linux")) `(("boot-guile/i686" ,(bootstrap-guile-origin "i686-linux")= )) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 59dd117ed..64ed44460 100644 =2D-- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -3,6 +3,7 @@ ;;; Copyright =C2=A9 2017 Efraim Flashner ;;; Copyright =C2=A9 2017 Ricardo Wurmus ;;; Copyright =C2=A9 2018 Konrad Hinsen +;;; Copyright =C2=A9 2018 Chris Marusich ;;; ;;; This file is part of GNU Guix. ;;; @@ -33,7 +34,9 @@ #:use-module (guix derivations) #:use-module (guix scripts build) #:use-module (gnu packages) + #:use-module (gnu packages bootstrap) #:use-module (gnu packages compression) + #:use-module (gnu packages guile) #:autoload (gnu packages base) (tar) #:autoload (gnu packages package-management) (guix) #:autoload (gnu packages gnupg) (libgcrypt) @@ -63,6 +66,8 @@ #~(#+(file-append lzip "/bin/lzip") "-9")) (compressor "xz" ".xz" #~(#+(file-append xz "/bin/xz") "-e -T0")) + (compressor "bootstrap-xz" ".xz" + #~(#+(file-append %bootstrap-xz "/bin/xz") "-e -T0")) (compressor "bzip2" ".bz2" #~(#+(file-append bzip2 "/bin/bzip2") "-9")) (compressor "none" "" #f))) @@ -325,6 +330,9 @@ the image." (option '("localstatedir") #f #f (lambda (opt name arg result) (alist-cons 'localstatedir? #t result))) + (option '("bootstrap") #f #f + (lambda (opt name arg result) + (alist-cons 'bootstrap? #t result))) =20 (append %transformation-options %standard-build-options))) @@ -352,6 +360,8 @@ Create a bundle of PACKAGE.\n")) -m, --manifest=3DFILE create a pack with the manifest from FILE")) (display (G_ " --localstatedir include /var/guix in the resulting pack")) + (display (G_ " + --bootstrap use the bootstrap binaries to build the pack")) (newline) (display (G_ " -h, --help display this help and exit")) @@ -393,28 +403,43 @@ Create a bundle of PACKAGE.\n")) (else (packages->manifest packages))))) =20 (with-error-handling =2D (parameterize ((%graft? (assoc-ref opts 'graft?))) =2D (let* ((dry-run? (assoc-ref opts 'dry-run?)) =2D (manifest (manifest-from-args opts)) =2D (pack-format (assoc-ref opts 'format)) =2D (name (string-append (symbol->string pack-format) =2D "-pack")) =2D (target (assoc-ref opts 'target)) =2D (compressor (assoc-ref opts 'compressor)) =2D (symlinks (assoc-ref opts 'symlinks)) =2D (build-image (match (assq-ref %formats pack-format) =2D ((? procedure? proc) proc) =2D (#f =2D (leave (G_ "~a: unknown pack format") =2D format)))) =2D (localstatedir? (assoc-ref opts 'localstatedir?))) =2D (with-store store + (let* ((dry-run? (assoc-ref opts 'dry-run?)) + (manifest (manifest-from-args opts)) + (pack-format (assoc-ref opts 'format)) + (name (string-append (symbol->string pack-format) + "-pack")) + (target (assoc-ref opts 'target)) + (bootstrap? (assoc-ref opts 'bootstrap?)) + (compressor (if bootstrap? + (lookup-compressor "bootstrap-xz") + (assoc-ref opts 'compressor))) + (tar (if bootstrap? + %bootstrap-tar + tar)) + (symlinks (assoc-ref opts 'symlinks)) + (build-image (match (assq-ref %formats pack-format) + ((? procedure? proc) proc) + (#f + (leave (G_ "~a: unknown pack format") + format)))) + (localstatedir? (assoc-ref opts 'localstatedir?))) + (with-store store + (parameterize ((%graft? (assoc-ref opts 'graft?)) + (%guile-for-build (package-derivation + store + (if (assoc-ref opts 'bootstrap?) + %bootstrap-guile + (canonical-package guile-2.2= ))))) ;; Set the build options before we do anything else. (set-build-options-from-command-line store opts) =20 (run-with-store store (mlet* %store-monad ((profile (profile-derivation manifest + #:hooks (if bootstrap? + '() + %default-profile-ho= oks) + #:locales? (not bootstrap?) #:target target)) (drv (build-image name profile #:target @@ -424,7 +449,9 @@ Create a bundle of PACKAGE.\n")) #:symlinks symlinks #:localstatedir? =2D localstatedir?))) + localstatedir? + #:tar + tar))) (mbegin %store-monad (show-what-to-build* (list drv) #:use-substitutes? diff --git a/tests/guix-pack.sh b/tests/guix-pack.sh new file mode 100644 index 000000000..34692888b =2D-- /dev/null +++ b/tests/guix-pack.sh @@ -0,0 +1,74 @@ +# GNU Guix --- Functional package management for GNU +# Copyright =C2=A9 2018 Chris Marusich +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +# +# Test the `guix pack' command-line utility. +# + +guix pack --version + +# Use --no-substitutes because we need to verify we can do this ourselves. +GUIX_BUILD_OPTIONS=3D"--no-substitutes" +export GUIX_BUILD_OPTIONS + +# Build a tarball with no compression. +guix pack --compression=3Dnone --bootstrap guile-bootstrap + +# Build a tarball (with compression). +guix pack --bootstrap guile-bootstrap + +# Build a tarball with a symlink. +the_pack=3D"`guix pack --bootstrap -S /opt/gnu/bin=3Dbin guile-bootstrap`" + +is_available () { + # Use the "type" shell builtin to see if the program is on PATH. + type "$1" > /dev/null +} + +if is_available chroot && is_available unshare; then + # Verify we can extract and use it. + test_directory=3D"`mktemp -d`" + trap 'rm -rf "$test_directory"' EXIT + cd "$test_directory" + tar -xf "$the_pack" + unshare -r chroot . /opt/gnu/bin/guile --version + cd - +else + echo "warning: skipping pack verification because chroot or unshare is= unavailable" >&2 +fi + +# For the tests that build Docker images below, we currently have to use +# --dry-run because if we don't, there are only two possible cases: +# +# Case 1: We do not use --bootstrap, and the build takes hours to fini= sh +# because it needs to build tar etc. +# +# Case 2: We use --bootstrap, and the build fails because the bootstrap +# Guile cannot dlopen shared libraries. Not to mention the fa= ct +# that we would still have to build many non-bootstrap inputs +# (e.g., guile-json) in order to create the Docker image. + +# Build a Docker image. +guix pack --dry-run --bootstrap -f docker guile-bootstrap + +# Build a Docker image with a symlink. +guix pack --dry-run --bootstrap -f docker -S /opt/gnu=3D/ guile-bootstrap + +# Build a tarball pack of cross-compiled software. Use coreutils because +# guile-bootstrap is not intended to be cross-compiled. +guix pack --dry-run --bootstrap --target=3Darm-unknown-linux-gnueabihf cor= eutils =2D-=20 2.15.1 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlqx31AACgkQ3UCaFdgi Rp1wfRAA2NMKTRX2lXIYv7v0uhMQUmYp9e+AwqrVbOsJODpk2afXqonfQCQfA26n lPbxvFY+oQjpxReF3Q6qxUlb4b8CYlCXSrhlQahkKAUa3HQ3/IKcq77q2JiHenQK z5azTjGJVJwrVZAmO1FmHxSK5cu+0A0mdBGdFD7maqyxQzN4vVXU+gSvVV/3Mx47 XkLmaKm2MJ7hk21kGP8o0PjFZaICiUZa1aLq/MaTPd3Z9ewOtr+hch2P98Ad+Fz8 V2L2VoYSfDfwyb22bbN8YJaes+CRbGWjKwe5g2tw2xMcQmVbJuM19HRol1DgO9mf 2q2onNJNqpoCL62Vndac8DbCu587iyRx6c90EV79eRQAsXVz6kKHxlIbGHm5ba0v 30A0DkN04FBa5ty0xmbYFh4zErNKoBmZN5iNgiHvRgHkNMCbrefoaw0k2dLkVoEh 3pUGolluwzqTPHy4hQJupZddwO5fzP65mf+6bd3eKXqifomYD9HcfMJ72zMQQyue XPYD4S+Y2lAzmSsUVXvXGd97Gt/yJWsA6C4AkS8QntGPYkNxEeIhR1YA7gXeVpBD vBo581+bkTe9amM+EslxL5MRAMxDmDHvirU5MkkvbhDMU2I1vG/MVqJk4hGhAwY+ 0OOWy+JtLLa26wiGUw5kOab+WqGUi0Z52I87dGevuRu8w6gMAsM= =GVwB -----END PGP SIGNATURE----- --==-=-=--