Introducing ‘guix pack’

Introducing ‘guix pack’

[Ludovic Courtès]

[-- Attachment #1.1: Type: text/plain, Size: 4115 bytes --]

Hello Guix!

I had it on my to-do list and Andy said he’d like to have something like
that to publish Guile 2.2 binaries: the ‘guix pack’ command below is a
generalization of the code that builds the Guix binary tarball¹.  It
creates a bundle of the closure of the given packages, with a profile
containing all the packages.

Examples:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix pack guile-next
/gnu/store/c2brwnrx0066lvf1d9hg82lvb1wkmd78-tarball.tar.lz
$ ./pre-inst-env guix pack guile-next guix
/gnu/store/kc6bxqwrf9z2vvxwndhw8s39s39h5mz6-tarball.tar.lz
$ tar tvf /gnu/store/kc6bxqwrf9z2vvxwndhw8s39s39h5mz6-tarball.tar.lz|grep /bin/gui[lx]
-r-xr-xr-x root/root      2974 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guild
-r-xr-xr-x root/root     13472 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile
-r-xr-xr-x root/root      7008 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile-config
-r-xr-xr-x root/root      3050 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile-snarf
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile-tools -> guild
-r-xr-xr-x root/root      2978 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guild
-r-xr-xr-x root/root     12784 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile
-r-xr-xr-x root/root      7022 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-config
-r-xr-xr-x root/root      3072 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-snarf
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-tools -> guild
-r-xr-xr-x root/root       744 1970-01-01 01:00 ./gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix
-r-xr-xr-x root/root    933120 1970-01-01 01:00 ./gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix-daemon
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guild -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guild
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile-config -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-config
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile-snarf -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-snarf
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile-tools -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-tools
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guix -> /gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix
lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guix-daemon -> /gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix-daemon
--8<---------------cut here---------------end--------------->8---

Andy, does this correspond to what you have in mind?

I’d like to move support for Docker (currently in ‘guix archive’) to
this new command because I think it’s more appropriate: ‘guix archive’
is supposed to be rather low-level so it would not create a profile, for
instance.

Ricardo, WDYT?

If there are no objections, I’d like to commit it soonish.

Thanks!

Ludo’.

¹ https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/install.scm#n52


[-- Attachment #1.2: guix pack --]
[-- Type: text/plain, Size: 6909 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015, 2017 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix scripts pack)
  #:use-module (guix scripts)
  #:use-module (guix ui)
  #:use-module (guix gexp)
  #:use-module (guix utils)
  #:use-module (guix store)
  #:use-module (guix grafts)
  #:use-module (guix monads)
  #:use-module (guix profiles)
  #:use-module (guix derivations)
  #:use-module (guix scripts build)
  #:use-module (gnu packages)
  #:autoload   (gnu packages compression) (lzip)
  #:autoload   (gnu packages base) (tar)
  #:autoload   (gnu packages package-management) (guix)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-37)
  #:use-module (ice-9 match)
  #:export (guix-pack))

(define* (self-contained-tarball name profile
                                 #:key deduplicate?)
  "Return a self-contained tarball containing a store initialized with the
closure of PROFILE, a derivation.  The tarball contains /gnu/store, /var/guix,
and PROFILE is available as /root/.guix-profile."
  (define build
    (with-imported-modules '((guix build utils)
                             (guix build store-copy)
                             (gnu build install))
      #~(begin
          (use-modules (guix build utils)
                       (gnu build install))

          (define %root "root")

          ;; We need Guix here for 'guix-register'.
          (setenv "PATH"
                  (string-append #$guix "/sbin:" #$tar "/bin:" #$lzip "/bin"))

          ;; Note: there is not much to gain here with deduplication and
          ;; there is the overhead of the '.links' directory, so turn it
          ;; off.
          (populate-single-profile-directory %root
                                             #:profile #$profile
                                             #:closure "profile"
                                             #:deduplicate? #f)

          ;; Create the tarball.  Use GNU format so there's no file name
          ;; length limitation.
          (with-directory-excursion %root
            (zero? (system* "tar" "--lzip" "--format=gnu"

                            ;; Avoid non-determinism in the archive.  Use
                            ;; mtime = 1, not zero, because that is what the
                            ;; daemon does for files in the store (see the
                            ;; 'mtimeStore' constant in local-store.cc.)
                            "--sort=name"
                            "--mtime=@1"          ;for files in /var/guix
                            "--owner=root:0"
                            "--group=root:0"

                            "--check-links"
                            "-cvf" #$output
                            ;; Avoid adding / and /var to the tarball, so
                            ;; that the ownership and permissions of those
                            ;; directories will not be overwritten when
                            ;; extracting the archive.  Do not include /root
                            ;; because the root account might have a
                            ;; different home directory.
                            "./var/guix"
                            (string-append "." (%store-directory))))))))

  (gexp->derivation name build
                    #:references-graphs `(("profile" ,profile))))


\f
;;;
;;; Command-line options.
;;;

(define %default-options
  ;; Alist of default option values.
  `((system . ,(%current-system))
    (substitutes? . #t)
    (graft? . #t)
    (max-silent-time . 3600)
    (verbosity . 0)))

(define %options
  ;; Specifications of the command-line options.
  (cons* (option '(#\h "help") #f #f
                 (lambda args
                   (show-help)
                   (exit 0)))
         (option '(#\V "version") #f #f
                 (lambda args
                   (show-version-and-exit "guix pack")))

         (option '(#\n "dry-run") #f #f
                 (lambda (opt name arg result)
                   (alist-cons 'dry-run? #t (alist-cons 'graft? #f result))))

         %standard-build-options))

(define (show-help)
  (display (_ "Usage: guix pack [OPTION]... PACKAGE...
Create a bundle of PACKAGE.\n"))
  (show-build-options-help)

  (newline)
  (display (_ "
  -h, --help             display this help and exit"))
  (display (_ "
  -V, --version          display version information and exit"))
  (newline)
  (show-bug-report-information))

\f
;;;
;;; Entry point.
;;;

(define (guix-pack . args)
  (define opts
    (parse-command-line args %options (list %default-options)))

  (with-error-handling
    (parameterize ((%graft? (assoc-ref opts 'graft?)))
      (let* ((dry-run? (assoc-ref opts 'dry-run?))
             (specs    (filter-map (match-lambda
                                     (('argument . name)
                                      name)
                                     (x #f))
                                   opts))
             (packages (map (lambda (spec)
                              (call-with-values
                                  (lambda ()
                                    (specification->package+output spec))
                                list))
                            specs)))
        (with-store store
          (run-with-store store
            (mlet* %store-monad ((profile (profile-derivation
                                           (packages->manifest packages)))
                                 (drv (self-contained-tarball "tarball.tar.lz"
                                                              profile)))
              (mbegin %store-monad
                (show-what-to-build* (list drv)
                                     #:use-substitutes?
                                     (assoc-ref opts 'substitutes?)
                                     #:dry-run? dry-run?)
                (munless dry-run?
                  (built-derivations (list drv))
                  (return (format #t "~a~%"
                                  (derivation->output-path drv))))))))))))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Introducing ‘guix pack’

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> I had it on my to-do list and Andy said he’d like to have something like
> that to publish Guile 2.2 binaries: the ‘guix pack’ command below is a
> generalization of the code that builds the Guix binary tarball¹.  It
> creates a bundle of the closure of the given packages, with a profile
> containing all the packages.
>
> Examples:
>
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix pack guile-next
> /gnu/store/c2brwnrx0066lvf1d9hg82lvb1wkmd78-tarball.tar.lz
> $ ./pre-inst-env guix pack guile-next guix
> /gnu/store/kc6bxqwrf9z2vvxwndhw8s39s39h5mz6-tarball.tar.lz
> $ tar tvf /gnu/store/kc6bxqwrf9z2vvxwndhw8s39s39h5mz6-tarball.tar.lz|grep /bin/gui[lx]
> -r-xr-xr-x root/root      2974 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guild
> -r-xr-xr-x root/root     13472 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile
> -r-xr-xr-x root/root      7008 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile-config
> -r-xr-xr-x root/root      3050 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile-snarf
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/060piiiz4nmb51jc3wk01bgikajrnfjd-guile-2.0.13/bin/guile-tools -> guild
> -r-xr-xr-x root/root      2978 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guild
> -r-xr-xr-x root/root     12784 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile
> -r-xr-xr-x root/root      7022 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-config
> -r-xr-xr-x root/root      3072 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-snarf
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-tools -> guild
> -r-xr-xr-x root/root       744 1970-01-01 01:00 ./gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix
> -r-xr-xr-x root/root    933120 1970-01-01 01:00 ./gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix-daemon
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guild -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guild
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile-config -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-config
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile-snarf -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-snarf
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guile-tools -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guile-tools
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guix -> /gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix
> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guix-daemon -> /gnu/store/6xlybn33znrp4nyvy39wbrwjrl8l9qnd-guix-0.12.0-5.1162/bin/guix-daemon
> --8<---------------cut here---------------end--------------->8---

Very nice!

> I’d like to move support for Docker (currently in ‘guix archive’) to
> this new command because I think it’s more appropriate: ‘guix archive’
> is supposed to be rather low-level so it would not create a profile, for
> instance.
>
> Ricardo, WDYT?

Sounds good to me!

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Introducing ‘guix pack’

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3203 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello Guix!
>
> I had it on my to-do list and Andy said he’d like to have something like
> that to publish Guile 2.2 binaries: the ‘guix pack’ command below is a
> generalization of the code that builds the Guix binary tarball¹.  It
> creates a bundle of the closure of the given packages, with a profile
> containing all the packages.

Very cool!

> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guild -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guild

Why does a relative path ./gnu/store/... point to an absolute path
/gnu/store/...?  I would have thought it would be "relative to
relative", or "absolute to absolute", not "relative to absolute".
Perhaps I'm missing something.

What is the expected method for extracting the tarball?  Can you 'cd' to
any directory and run a command like 'tar -xf the_file', or does the
current working directory have to be '/'?

> I’d like to move support for Docker (currently in ‘guix archive’) to
> this new command because I think it’s more appropriate: ‘guix archive’
> is supposed to be rather low-level so it would not create a profile, for
> instance.

My understanding is that one use case for this command is to make it
easy to deploy software (and the closure of its dependencies) using
Docker, in the case (maybe always? I'm not too familiar with Docker)
where you cannot run Guix inside of the Docker instance.  If you could
run Guix, I imagine you would not need this solution, since you could
just install it via the usual Guix mechanisms (e.g., 'guix package -i
foo').

Another use case seems to be specifically the creation of a tarball
containing Guix to enable a binary installation of it (you mentioned
that was the inspiration for this feature).

What other use cases do you imagine?  Is the intent to make it easy to
deploy software (and the closure of its dependencies) to any place where
you either can't or don't want to install Guix first?

> (define* (self-contained-tarball name profile
>                                  #:key deduplicate?)
>   "Return a self-contained tarball containing a store initialized with the
> closure of PROFILE, a derivation.  The tarball contains /gnu/store, /var/guix,
> and PROFILE is available as /root/.guix-profile."

Why is it necessary to include /var/guix?  I'm thinking about the case
where you're using this to package something other than Guix, e.g., some
other piece of software for deployment in a Docker instance.

>           ;; Note: there is not much to gain here with deduplication and
>           ;; there is the overhead of the '.links' directory, so turn it
>           ;; off.
>           (populate-single-profile-directory %root
>                                              #:profile #$profile
>                                              #:closure "profile"
>                                              #:deduplicate? #f)

When you say "there is not much to gain here," do you mean "it is
unlikely that duplication will occur"?  If so, why is that true?

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi Chris,

Chris Marusich <cmmarusich@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

>> lrwxrwxrwx root/root         0 1970-01-01 01:00 ./gnu/store/ynafk7v924xil993dqbx4mxxnm9ifsi6-profile/bin/guild -> /gnu/store/62hqgi4cac0f70v1ycsvv985fl3l1hzr-guile-next-2.1.7/bin/guild
>
> Why does a relative path ./gnu/store/... point to an absolute path
> /gnu/store/...?  I would have thought it would be "relative to
> relative", or "absolute to absolute", not "relative to absolute".
> Perhaps I'm missing something.
>
> What is the expected method for extracting the tarball?  Can you 'cd' to
> any directory and run a command like 'tar -xf the_file', or does the
> current working directory have to be '/'?

The extraction method is supposed to be the same as for the Guix binary
tarball, since it’s exactly the same code:

  https://gnu.org/software/guix/manual/html_node/Binary-Installation.html

>> I’d like to move support for Docker (currently in ‘guix archive’) to
>> this new command because I think it’s more appropriate: ‘guix archive’
>> is supposed to be rather low-level so it would not create a profile, for
>> instance.
>
> My understanding is that one use case for this command is to make it
> easy to deploy software (and the closure of its dependencies) using
> Docker, in the case (maybe always? I'm not too familiar with Docker)
> where you cannot run Guix inside of the Docker instance.  If you could
> run Guix, I imagine you would not need this solution, since you could
> just install it via the usual Guix mechanisms (e.g., 'guix package -i
> foo').
>
> Another use case seems to be specifically the creation of a tarball
> containing Guix to enable a binary installation of it (you mentioned
> that was the inspiration for this feature).
>
> What other use cases do you imagine?  Is the intent to make it easy to
> deploy software (and the closure of its dependencies) to any place where
> you either can't or don't want to install Guix first?

The use case for ‘guix archive -f docker’ (which already exists; I’m
just proposing to move it to ‘guix pack’) is when you’d like to pass
software to someone who has Docker but not Guix:

  https://lists.gnu.org/archive/html/guix-devel/2017-01/msg00048.html

I think the impetus for Ricardo was when the Emacs folks were discussing
of building Docker images of Emacs.  :-)

But I guess it’s not uncommon for people to have Docker and not Guix, as
sad as it may sound.  ;-)

>> (define* (self-contained-tarball name profile
>>                                  #:key deduplicate?)
>>   "Return a self-contained tarball containing a store initialized with the
>> closure of PROFILE, a derivation.  The tarball contains /gnu/store, /var/guix,
>> and PROFILE is available as /root/.guix-profile."
>
> Why is it necessary to include /var/guix?  I'm thinking about the case
> where you're using this to package something other than Guix, e.g., some
> other piece of software for deployment in a Docker instance.

FTR, this code was in (gnu system install) until now where it was used
to create the Guix standalone tarball.

For the Guix binary tarball, it’s important to have /var/guix with the
store database properly initialized and so on.

For other cases, it’s probably less useful.  We could make it optional.

>>           ;; Note: there is not much to gain here with deduplication and
>>           ;; there is the overhead of the '.links' directory, so turn it
>>           ;; off.
>>           (populate-single-profile-directory %root
>>                                              #:profile #$profile
>>                                              #:closure "profile"
>>                                              #:deduplicate? #f)
>
> When you say "there is not much to gain here," do you mean "it is
> unlikely that duplication will occur"?  If so, why is that true?

In the Guix binary tarball, there are few files present several times,
hence “not much to be gained.”  Back then I compared both and the
deduplicated version was about the same size as the other one, IIRC.

But again, for the more general case, I agree it would make sense to
have an option for this.

I’ve pushed ‘guix pack’ in commit
239c22663ac928618028c4ec03cefc77de788e9d (slightly improved version of
what I posted.)  Still many ways in which we can improve it!

The Docker backend requires a bit of thought because of the dependency
on libgcrypt and guile-json, which ‘gexp->derivation’ doesn’t yet handle
nicely.

Thanks for your feedback!

Ludo’.

Re: Introducing ‘guix pack’

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

>> What other use cases do you imagine?  Is the intent to make it easy to
>> deploy software (and the closure of its dependencies) to any place where
>> you either can't or don't want to install Guix first?
>
> The use case for ‘guix archive -f docker’ (which already exists; I’m
> just proposing to move it to ‘guix pack’) is when you’d like to pass
> software to someone who has Docker but not Guix:
>
>   https://lists.gnu.org/archive/html/guix-devel/2017-01/msg00048.html
>
> I think the impetus for Ricardo was when the Emacs folks were discussing
> of building Docker images of Emacs.  :-)

That’s right.

Another use-case would be to create executable bundles *without*
involving Docker.  All that’s needed is a tiny stand-alone wrapper that
takes care of spawning a container, mounting the pack in some way, and
running a specified application in the container.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Introducing ‘guix pack’

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 143 bytes --]

Hi Ludo,

ludo@gnu.org (Ludovic Courtès) writes:

> Thanks for your feedback!

Thank you for taking the time to explain!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Introducing ‘guix pack’

[Andy Wingo]

On Fri 10 Mar 2017 22:50, ludo@gnu.org (Ludovic Courtès) writes:

> I had it on my to-do list and Andy said he’d like to have something like
> that to publish Guile 2.2 binaries: the ‘guix pack’ command below is a
> generalization of the code that builds the Guix binary tarball¹.  It
> creates a bundle of the closure of the given packages, with a profile
> containing all the packages.

Very cool, thank you!!!

> Andy, does this correspond to what you have in mind?

Yes, though I hadn't thought everything out.  I guess my mail question
is about user experience -- this is going to be a gateway for people to
get Guix and Guile and we should make sure there are no rough edges.  I
guess in particular I have a concern about users overwriting their
/var/guix.  (No worries about overwriting /gnu/store of course.)

Particularly if a user installs one "guix pack" then installs another
"guix pack", what happens?  Does the /var/guix tarball include the
sqlite db?  Could it be overwritten?  What if the user had already
installed Guix already; does this silently trash their Guix install?

A bug report: I just tried it but it seems guix pack doesn't respect
--no-build-hook for some reason, and also for some reason on this
machine my Guix daemon fails with "offload: command not found", which I
was getting around via --no-build-hook.

I guess what would be ideal would be:

  cd /
  sudo tar xvf aasdfafasdfjasdaldfhasdfh-guile.tar.xz

and then telling the user to run via
/gnu/store/asasdfadfgsadfa-profile/bin/guile.  That way there is very
little risk of trashing the user's system.

Of course we could also provide them a README of sorts for all the load
paths, but in the end this is a gateway to the real experience :)

I guess for binary installs you would of course want /var/guix, the
database, and the profiles.  I think in that case it makes sense to add
an option to --pack about including them, and have it default to off
(given the potential to trample a user's store).

WDYT?  Any of this make sense? :)

Andy

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi!

Andy Wingo <wingo@igalia.com> skribis:

> Yes, though I hadn't thought everything out.  I guess my mail question
> is about user experience -- this is going to be a gateway for people to
> get Guix and Guile and we should make sure there are no rough edges.  I
> guess in particular I have a concern about users overwriting their
> /var/guix.  (No worries about overwriting /gnu/store of course.)
>
> Particularly if a user installs one "guix pack" then installs another
> "guix pack", what happens?  Does the /var/guix tarball include the
> sqlite db?  Could it be overwritten?  What if the user had already
> installed Guix already; does this silently trash their Guix install?
>
> A bug report: I just tried it but it seems guix pack doesn't respect
> --no-build-hook for some reason, and also for some reason on this
> machine my Guix daemon fails with "offload: command not found", which I
> was getting around via --no-build-hook.
>
> I guess what would be ideal would be:
>
>   cd /
>   sudo tar xvf aasdfafasdfjasdaldfhasdfh-guile.tar.xz
>
> and then telling the user to run via
> /gnu/store/asasdfadfgsadfa-profile/bin/guile.  That way there is very
> little risk of trashing the user's system.

Yes, something like that.

> Of course we could also provide them a README of sorts for all the load
> paths, but in the end this is a gateway to the real experience :)
>
> I guess for binary installs you would of course want /var/guix, the
> database, and the profiles.  I think in that case it makes sense to add
> an option to --pack about including them, and have it default to off
> (given the potential to trample a user's store).
>
> WDYT?  Any of this make sense? :)

Yes!  I thought about making at least /var/guix/db optional and
off by default.

If we remove /var/guix/profiles, users will have to actually type
/gnu/store/asasdfadfgsadfa-profile/bin/guile.  This is not great, but I
don’t know what else could be done.  We could profile a
/bin/guile → /gnu/store/asasdfadfgsadfa-profile/bin/guile symlink, but
perhaps that’s risky too.

Thoughts?

Ludo’.

Re: Introducing ‘guix pack’

[Andy Wingo]

Hey :)

On Tue 14 Mar 2017 14:42, ludo@gnu.org (Ludovic Courtès) writes:

> If we remove /var/guix/profiles, users will have to actually type
> /gnu/store/asasdfadfgsadfa-profile/bin/guile.  This is not great, but I
> don’t know what else could be done.  We could profile a
> /bin/guile → /gnu/store/asasdfadfgsadfa-profile/bin/guile symlink, but
> perhaps that’s risky too.

As we were discussing in the channel, maybe it's OK for these binary
installs to claim "/opt/gnu".  Then we expose the profile in /opt/gnu,
so you would run Guile as /opt/gnu/bin/guile.  Additionally you could
actually build against that Guile, which would be pretty neat.  If the
user untars multiple guix packs, /gnu/store easily absorbs the union,
and /opt/gnu will adjoin any new profile directories/files and replace
any overwritten links.

We would have to make sure the union directory in /opt/gnu has all real
directories and only symlink files, as per the recent patch on
guix-devel.

Andy

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi!

Andy Wingo <wingo@igalia.com> skribis:

> On Tue 14 Mar 2017 14:42, ludo@gnu.org (Ludovic Courtès) writes:
>
>> If we remove /var/guix/profiles, users will have to actually type
>> /gnu/store/asasdfadfgsadfa-profile/bin/guile.  This is not great, but I
>> don’t know what else could be done.  We could profile a
>> /bin/guile → /gnu/store/asasdfadfgsadfa-profile/bin/guile symlink, but
>> perhaps that’s risky too.
>
> As we were discussing in the channel, maybe it's OK for these binary
> installs to claim "/opt/gnu".  Then we expose the profile in /opt/gnu,
> so you would run Guile as /opt/gnu/bin/guile.  Additionally you could
> actually build against that Guile, which would be pretty neat.  If the
> user untars multiple guix packs, /gnu/store easily absorbs the union,
> and /opt/gnu will adjoin any new profile directories/files and replace
> any overwritten links.
>
> We would have to make sure the union directory in /opt/gnu has all real
> directories and only symlink files, as per the recent patch on
> guix-devel.

Commit 5895ec8aa234ec9a4ce68ab8f94e795807630168 takes a slightly
different approach (it doesn’t use the union thing).

You can run:

  guix pack guile-next \
    --symlink=/opt/gnu/bin/guile=bin/guile \
    --symlink=/opt/gnu/bin/guild=bin/guild

and that does what you would expect.

In addition, /var/guix is no longer included by default.

Let me know what you think!

Ludo’.

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hello Guix!  Hello 2.2 Guilers!

ludo@gnu.org (Ludovic Courtès) skribis:

> I’d like to move support for Docker (currently in ‘guix archive’) to
> this new command because I think it’s more appropriate: ‘guix archive’
> is supposed to be rather low-level so it would not create a profile, for
> instance.

As of right now, ‘guix pack’ can create Docker image (‘guix archive’
can no longer do that):

  guix pack guile-next -S /opt/guile-2.2.0= -f docker

I’d be grateful if someone could check with Docker that I didn’t break
anything though.  Any takers?  :-)

The archive build process is bit-reproducible too, which is pretty cool
(Docker images created in January, 1970!).

That’s it!

Ludo’.

Re: Introducing ‘guix pack’

[Pjotr Prins]

On Thu, Mar 16, 2017 at 11:28:05PM +0100, Ludovic Courtès wrote:
> As of right now, ‘guix pack’ can create Docker image (‘guix archive’
> can no longer do that):
> 
>   guix pack guile-next -S /opt/guile-2.2.0= -f docker

If this works we should advertise it widely.

Pj.

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Thu, Mar 16, 2017 at 11:28:05PM +0100, Ludovic Courtès wrote:
>> As of right now, ‘guix pack’ can create Docker image (‘guix archive’
>> can no longer do that):
>> 
>>   guix pack guile-next -S /opt/guile-2.2.0= -f docker
>
> If this works we should advertise it widely.

I’d like to write a blog entry next week.

Ludo’.

Re: Introducing ‘guix pack’

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> Hello Guix!  Hello 2.2 Guilers!
>
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> I’d like to move support for Docker (currently in ‘guix archive’) to
>> this new command because I think it’s more appropriate: ‘guix archive’
>> is supposed to be rather low-level so it would not create a profile, for
>> instance.
>
> As of right now, ‘guix pack’ can create Docker image (‘guix archive’
> can no longer do that):
>
>   guix pack guile-next -S /opt/guile-2.2.0= -f docker

And now you can do:

  guix pack grep --target=i686-w64-mingw32

or:

  guix pack guile@2.0 --target=arm-linux-gnueabihf -f docker

Granted, this is not optimal space-wise because cross-compiled binaries
end up pulling the whole cross toolchain (due to cross-gcc not having a
separate “lib” output), so for instance the cross-built grep tarball
above weighs in at 91 MiB (330 MiB uncompressed :-)).  We’ll fix that.

Anyway, pretty cool!

Ludo’.

Re: Introducing ‘guix pack’

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> And now you can do:
>
>   guix pack grep --target=i686-w64-mingw32
>
> or:
>
>   guix pack guile@2.0 --target=arm-linux-gnueabihf -f docker

...exactly what has been missing: awesome, thanks!

> Granted, this is not optimal space-wise because cross-compiled binaries
> end up pulling the whole cross toolchain (due to cross-gcc not having a
> separate “lib” output), so for instance the cross-built grep tarball
> above weighs in at 91 MiB (330 MiB uncompressed :-)).  We’ll fix that.

:-)  Oh well.

> Anyway, pretty cool!

Pretty cool!
Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: Introducing ‘guix pack’

[Federico Beffa]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi!
>
> Andy Wingo <wingo@igalia.com> skribis:
>
>> On Tue 14 Mar 2017 14:42, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> If we remove /var/guix/profiles, users will have to actually type
>>> /gnu/store/asasdfadfgsadfa-profile/bin/guile.  This is not great, but I
>>> don’t know what else could be done.  We could profile a
>>> /bin/guile → /gnu/store/asasdfadfgsadfa-profile/bin/guile symlink, but
>>> perhaps that’s risky too.
>>
>> As we were discussing in the channel, maybe it's OK for these binary
>> installs to claim "/opt/gnu".  Then we expose the profile in /opt/gnu,
>> so you would run Guile as /opt/gnu/bin/guile.  Additionally you could
>> actually build against that Guile, which would be pretty neat.  If the
>> user untars multiple guix packs, /gnu/store easily absorbs the union,
>> and /opt/gnu will adjoin any new profile directories/files and replace
>> any overwritten links.
>>
>> We would have to make sure the union directory in /opt/gnu has all real
>> directories and only symlink files, as per the recent patch on
>> guix-devel.
>
> Commit 5895ec8aa234ec9a4ce68ab8f94e795807630168 takes a slightly
> different approach (it doesn’t use the union thing).
>
> You can run:
>
>   guix pack guile-next \
>     --symlink=/opt/gnu/bin/guile=bin/guile \
>     --symlink=/opt/gnu/bin/guild=bin/guild
>
> and that does what you would expect.
>
> In addition, /var/guix is no longer included by default.
>
> Let me know what you think!

Say, developer A distributes such an archive A and developer B
distributes archive B (a different program/library) and someone C
installs both.

Now developer A fixes a security hole and produces a new archive.  How
can C remove the library with the security hole from his system?  If he
just overlays the new version, the library with the security problem
stays on the system and could be exploited.  Deleting everything is also
less than ideal.

This seems to me similar to encouraging the much criticized practice of
bundling required libraries with your program.

Maybe 'pack' could at least include a 'remove-myself' thing.  Or have
you thought about the hole program life-cycle?

Fede

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi Federico,

Federico Beffa <beffa@ieee.org> skribis:

> Say, developer A distributes such an archive A and developer B
> distributes archive B (a different program/library) and someone C
> installs both.

Interestingly composability (what happens when you unpack both A and B
on the same system) is better than what you’d get with Docker: the
unpacked items that are identical are shared, and those parts that
differ don’t collide.

> Now developer A fixes a security hole and produces a new archive.  How
> can C remove the library with the security hole from his system?  If he
> just overlays the new version, the library with the security problem
> stays on the system and could be exploited.  Deleting everything is also
> less than ideal.
>
> This seems to me similar to encouraging the much criticized practice of
> bundling required libraries with your program.
>
> Maybe 'pack' could at least include a 'remove-myself' thing.  Or have
> you thought about the hole program life-cycle?

Good question.  There’s a fine line here.  In Guix circles we’re very
good at explaining why “app bundles” are a bad thing (composability- and
security-wise notably), and here that’s precisely what we’re producing.

The intended use case is mostly “one-off” packs where you just want
people to easily test something, as opposed to putting it in
production.  This was the case for the Guile 2.2.0 release.  In those
cases, people would essentially “rm -rf /gnu” when they’re done.

For code that is meant to be kept over time, I would recommend to either
use Guix, or to include Guix in the pack so that people can eventually
upgrade.

Thoughts?

Ludo’.

Re: Introducing ‘guix pack’

[Federico Beffa]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Federico,
>
> Federico Beffa <beffa@ieee.org> skribis:
>
>> Say, developer A distributes such an archive A and developer B
>> distributes archive B (a different program/library) and someone C
>> installs both.
>
> Interestingly composability (what happens when you unpack both A and B
> on the same system) is better than what you’d get with Docker: the
> unpacked items that are identical are shared, and those parts that
> differ don’t collide.

Packs share identical items, but it becomes essentially impossible to
remove one component out of many.

>> Now developer A fixes a security hole and produces a new archive.  How
>> can C remove the library with the security hole from his system?  If he
>> just overlays the new version, the library with the security problem
>> stays on the system and could be exploited.  Deleting everything is also
>> less than ideal.
>>
>> This seems to me similar to encouraging the much criticized practice of
>> bundling required libraries with your program.
>>
>> Maybe 'pack' could at least include a 'remove-myself' thing.  Or have
>> you thought about the hole program life-cycle?
>
> Good question.  There’s a fine line here.  In Guix circles we’re very
> good at explaining why “app bundles” are a bad thing (composability- and
> security-wise notably), and here that’s precisely what we’re producing.
>
> The intended use case is mostly “one-off” packs where you just want
> people to easily test something, as opposed to putting it in
> production.  This was the case for the Guile 2.2.0 release.  In those
> cases, people would essentially “rm -rf /gnu” when they’re done.

If you provide an archive such as
'guile-2.2.0-pack-x86_64-linux-gnu.tar.lz' reachable from the main
project page (especially without any warning about its intended
purpose), I bet that many peoples will install it and keep it.  If more
projects follow this example, we land to the above scenario where "rm
-rf /gnu" is not practical at all.

> For code that is meant to be kept over time, I would recommend to either
> use Guix, or to include Guix in the pack so that people can eventually
> upgrade.

This is clear to me, but there are many peoples who do not know about
Guix, or just don't want it.  They may still be interested in, say,
Guile 2.2.

With the 'pack' command it seems to me that Guix is being promoted as a
convenient development environment where at the end you can produce
binary bundles for distribution on any system that it supports.  But,
without providing at least a way to remove things, it seems to be
heading toward a dangerous direction.

Fede

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi,

Federico Beffa <beffa@ieee.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Hi Federico,
>>
>> Federico Beffa <beffa@ieee.org> skribis:
>>
>>> Say, developer A distributes such an archive A and developer B
>>> distributes archive B (a different program/library) and someone C
>>> installs both.
>>
>> Interestingly composability (what happens when you unpack both A and B
>> on the same system) is better than what you’d get with Docker: the
>> unpacked items that are identical are shared, and those parts that
>> differ don’t collide.
>
> Packs share identical items, but it becomes essentially impossible to
> remove one component out of many.

Yes, of course.  It’s really a frozen system snapshot.

>> The intended use case is mostly “one-off” packs where you just want
>> people to easily test something, as opposed to putting it in
>> production.  This was the case for the Guile 2.2.0 release.  In those
>> cases, people would essentially “rm -rf /gnu” when they’re done.
>
> If you provide an archive such as
> 'guile-2.2.0-pack-x86_64-linux-gnu.tar.lz' reachable from the main
> project page (especially without any warning about its intended
> purpose), I bet that many peoples will install it and keep it.  If more
> projects follow this example, we land to the above scenario where "rm
> -rf /gnu" is not practical at all.

I agree, there’s always a risk.  I think what we can do is communicate
about these risks, and avoid using distributing packs in situations that
make it too likely that people will keep the pack without ever
upgrading.

>> For code that is meant to be kept over time, I would recommend to either
>> use Guix, or to include Guix in the pack so that people can eventually
>> upgrade.
>
> This is clear to me, but there are many peoples who do not know about
> Guix, or just don't want it.  They may still be interested in, say,
> Guile 2.2.
>
> With the 'pack' command it seems to me that Guix is being promoted as a
> convenient development environment where at the end you can produce
> binary bundles for distribution on any system that it supports.  But,
> without providing at least a way to remove things, it seems to be
> heading toward a dangerous direction.

Sure.  I think it’s a handy but potentially dangerous tool.  In my view
the goal is not to promote Guix as a tool to make bundles, but rather
allow people to solve problems in specific circumstances, with the above
caveats.  Those caveats largely apply to other “app bundle” solutions,
and I’d argue that this one is “less bad” that most other bundling
solutions.

Thanks for your feedback!

Ludo’.

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi again!

I’ve written about ‘guix pack’ here:

  https://gnu.org/s/guix/news/creating-bundles-with-guix-pack.html

Thanks to rekado & jonsger on IRC for the quick review!

Ludo’.

Re: Introducing ‘guix pack’

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello Guix!  Hello 2.2 Guilers!
>
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> I’d like to move support for Docker (currently in ‘guix archive’) to
>> this new command because I think it’s more appropriate: ‘guix archive’
>> is supposed to be rather low-level so it would not create a profile, for
>> instance.
>
> As of right now, ‘guix pack’ can create Docker image (‘guix archive’
> can no longer do that):
>
>   guix pack guile-next -S /opt/guile-2.2.0= -f docker
>
> I’d be grateful if someone could check with Docker that I didn’t break
> anything though.  Any takers?  :-)

I just tested it (although I'm probably not the first) and it works
(with "docker load").  Thanks!

> The archive build process is bit-reproducible too, which is pretty cool
> (Docker images created in January, 1970!).
>
> That’s it!
>
> Ludo’.

Re: Introducing ‘guix pack’

[Ricardo Wurmus]

Clément Lassieur <clement@lassieur.org> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello Guix!  Hello 2.2 Guilers!
>>
>> ludo@gnu.org (Ludovic Courtès) skribis:
>>
>>> I’d like to move support for Docker (currently in ‘guix archive’) to
>>> this new command because I think it’s more appropriate: ‘guix archive’
>>> is supposed to be rather low-level so it would not create a profile, for
>>> instance.
>>
>> As of right now, ‘guix pack’ can create Docker image (‘guix archive’
>> can no longer do that):
>>
>>   guix pack guile-next -S /opt/guile-2.2.0= -f docker
>>
>> I’d be grateful if someone could check with Docker that I didn’t break
>> anything though.  Any takers?  :-)
>
> I just tested it (although I'm probably not the first) and it works
> (with "docker load").  Thanks!

I also tested it successfully:

    image=$(guix pack -f docker -S /opt/gnu/bin=bin r)
    docker load < $image
    id=$(docker images -q profile | tail -n 1)
    docker run --rm --tmpfs /tmp:rw,size=787448k,mode=1777 -ti $id /opt/gnu/bin/R

This creates an image for R, loads it, starts (and eventually removes)
a container in which R runs.

For people who find this cumbersome: use containers with Guix instead.
Here’s how:

    guix environment --container --ad-hoc r -- R

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Introducing ‘guix pack’

[Alex Sassmannshausen]

Excellent article, thanks for writing and sharing!

Alex


Ludovic Courtès writes:

> Hi again!
>
> I’ve written about ‘guix pack’ here:
>
>   https://gnu.org/s/guix/news/creating-bundles-with-guix-pack.html
>
> Thanks to rekado & jonsger on IRC for the quick review!
>
> Ludo’.

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Clément Lassieur <clement@lassieur.org> writes:
>
>> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>>> As of right now, ‘guix pack’ can create Docker image (‘guix archive’
>>> can no longer do that):
>>>
>>>   guix pack guile-next -S /opt/guile-2.2.0= -f docker
>>>
>>> I’d be grateful if someone could check with Docker that I didn’t break
>>> anything though.  Any takers?  :-)
>>
>> I just tested it (although I'm probably not the first) and it works
>> (with "docker load").  Thanks!
>
> I also tested it successfully:
>
>     image=$(guix pack -f docker -S /opt/gnu/bin=bin r)
>     docker load < $image
>     id=$(docker images -q profile | tail -n 1)
>     docker run --rm --tmpfs /tmp:rw,size=787448k,mode=1777 -ti $id /opt/gnu/bin/R
>
> This creates an image for R, loads it, starts (and eventually removes)
> a container in which R runs.

Awesome, thank you both for testing!

> For people who find this cumbersome: use containers with Guix instead.
> Here’s how:
>
>     guix environment --container --ad-hoc r -- R

Indeed!  :-)

Ludo’.

Re: Introducing ‘guix pack’

[Andy Wingo]

On Mon 20 Mar 2017 15:14, ludo@gnu.org (Ludovic Courtès) writes:

> Federico Beffa <beffa@ieee.org> skribis:
>
>> If you provide an archive such as
>> 'guile-2.2.0-pack-x86_64-linux-gnu.tar.lz' reachable from the main
>> project page (especially without any warning about its intended
>> purpose), I bet that many peoples will install it and keep it.  If more
>> projects follow this example, we land to the above scenario where "rm
>> -rf /gnu" is not practical at all.

Replying to Federico: These are the same considerations as with Guix
fwiw, unless you remove old profiles and "guix gc".

Another solution to this concern is to remove /gnu/store and re-unpack
the tarballs that you still want.

Generally though I think we shouldn't expect people to access the store
directly; they'd only use /opt/gnu or whatever.  In the case that you
have upgraded the software, surely the problem is fixed (though of
course you may fix the problem for pack A but not pack B).

The natural solution is to use a package manager of course, as you note
:)

> I agree, there’s always a risk.  I think what we can do is communicate
> about these risks, and avoid using distributing packs in situations that
> make it too likely that people will keep the pack without ever
> upgrading.

Agreed, though I wouldn't over-stress the risks to be honest -- Guix
gives both users and distributors the ability to generate a new pack
easily.  A user can decide not to upgrade even in a system that is
managed by Guix.  User freedom is all of this :)

Andy

Re: Introducing ‘guix pack’

[Federico Beffa]

Andy Wingo <wingo@igalia.com> writes:

> On Mon 20 Mar 2017 15:14, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Federico Beffa <beffa@ieee.org> skribis:
>>
>>> If you provide an archive such as
>>> 'guile-2.2.0-pack-x86_64-linux-gnu.tar.lz' reachable from the main
>>> project page (especially without any warning about its intended
>>> purpose), I bet that many peoples will install it and keep it.  If more
>>> projects follow this example, we land to the above scenario where "rm
>>> -rf /gnu" is not practical at all.
>
> Replying to Federico: These are the same considerations as with Guix
> fwiw, unless you remove old profiles and "guix gc".

There is a very big difference.  The Guix binary installation pack
does include the 'guix' command which allows you to remove stuff from
the store.  Any other pack not including 'guix' does not.

Suppose that Guix pack bundles become popular and compare them to,
say, Mac style archives.  Let's go through Ludovic's analysis:

1. Composability: With Mac bundles you extract the archive in a
   directory.  With Guix packs it's essentially the same.
   
   i. Sharing of store items: What are the chances that two
   independent projects will generate packs from the same git checkout
   (or guix pull)?  Pretty low.  Therefore the amount of sharing
   between different packs will be pretty negligible.

   ii. Adding a program. Mac style: you just extract it.  With Guix
   pack it's essentially the same, but it creates a manually
   unmanageable network of links which entangle all packs.

   iii. Remove an item: Mac style: delete a directory.  With Guix pack
   the choice is: delete everything or keep everything.  That is, you
   keep obsolete programs/libraries with security holes on your system
   ready for exploitation and unnecessarily filling your disk, or
   ... start from scratch.  Is this composability?

2. Security: Mac style bundles are problematic, but at least you can
   easily delete old stuff and replace them with updated versions.
   Guix packs are worse: delete everything or keep it all.

3. Reproducibility: As long as you carefully take note from which git
   checkout you generate a Guix pack, Guix packs seems to be superior.
   Oh, don't you also depend on upsteam published archives of every
   single package in Guix?  They sometimes disappear or are replaced
   in place with different archives and so, after some time, your
   carefully noted git checkout will not build anymore.

4. Experimentation: Guix is great for that, but packs?  Are they
   useful for testing on other GNU/Linux systems?  Maybe.  But aren't
   all Guix packages built in isolated environments anyway?  So, do
   you really need packs to test on other systems?  Maybe, but
   probably not.

Don't get me wrong, I find that Guix proper has many great features,
but pack is not one of them.  

I find very disturbing when peoples advertise things hiding half of
the story to make them appear better than what they really are.

Fede

Re: Introducing ‘guix pack’

[Ludovic Courtès]

Hi,

Federico Beffa <beffa@ieee.org> skribis:

> Suppose that Guix pack bundles become popular and compare them to,
> say, Mac style archives.  Let's go through Ludovic's analysis:
>
> 1. Composability: With Mac bundles you extract the archive in a
>    directory.  With Guix packs it's essentially the same.
>    
>    i. Sharing of store items: What are the chances that two
>    independent projects will generate packs from the same git checkout
>    (or guix pull)?  Pretty low.  Therefore the amount of sharing
>    between different packs will be pretty negligible.

That’s not true; you’d be likely to share glibc, gcc:lib, maybe GLib,
GTK+, etc.

>    ii. Adding a program. Mac style: you just extract it.  With Guix
>    pack it's essentially the same, but it creates a manually
>    unmanageable network of links which entangle all packs.
>
>    iii. Remove an item: Mac style: delete a directory.  With Guix pack
>    the choice is: delete everything or keep everything.  That is, you
>    keep obsolete programs/libraries with security holes on your system
>    ready for exploitation and unnecessarily filling your disk, or
>    ... start from scratch.  Is this composability?
>
> 2. Security: Mac style bundles are problematic, but at least you can
>    easily delete old stuff and replace them with updated versions.
>    Guix packs are worse: delete everything or keep it all.
>
> 3. Reproducibility: As long as you carefully take note from which git
>    checkout you generate a Guix pack, Guix packs seems to be superior.
>    Oh, don't you also depend on upsteam published archives of every
>    single package in Guix?  They sometimes disappear or are replaced
>    in place with different archives and so, after some time, your
>    carefully noted git checkout will not build anymore.
>
> 4. Experimentation: Guix is great for that, but packs?  Are they
>    useful for testing on other GNU/Linux systems?  Maybe.  But aren't
>    all Guix packages built in isolated environments anyway?  So, do
>    you really need packs to test on other systems?  Maybe, but
>    probably not.
>
> Don't get me wrong, I find that Guix proper has many great features,
> but pack is not one of them.  

Don’t get me wrong, I agree!  :-)

Again, I think packs are useful in some cases where the other options
are even worse, but I’m not advocating it as a general “solution.”

In the news entry online I tried to take into account the very
legitimate criticisms you made, but perhaps the end result didn’t make
it sufficiently clear that packs aren’t a general solution.

Ludo’.