unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#31592] Add Singularity and squashfs image support to guix pack.
@ 2018-05-25 14:28 Ricardo Wurmus
  2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
  0 siblings, 1 reply; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-25 14:28 UTC (permalink / raw)
  To: 31592

This patch series adds the Singularity package, and it also adds
squashfs image support for “guix pack”.  The squashfs images work with
this Singularity package.

--
Ricardo

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next.
  2018-05-25 14:28 [bug#31592] Add Singularity and squashfs image support to guix pack Ricardo Wurmus
@ 2018-05-25 15:47 ` Ricardo Wurmus
  2018-05-25 15:47   ` [bug#31592] [PATCH 2/4] gnu: Add singularity Ricardo Wurmus
                     ` (4 more replies)
  0 siblings, 5 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-25 15:47 UTC (permalink / raw)
  To: 31592; +Cc: Ricardo Wurmus

* gnu/packages/compression.scm (squashfs-tools-next): New variable.
---
 gnu/packages/compression.scm | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 1cb0d208a..f53b817c2 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -4,7 +4,7 @@
 ;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org>
 ;;; Copyright © 2015 Taylan Ulrich Bayırlı/Kammer <taylanbayirli@gmail.com>
 ;;; Copyright © 2015, 2016 Eric Bavier <bavier@member.fsf.org>
-;;; Copyright © 2015, 2016, 2017 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2015, 2016, 2017, 2018 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2015, 2017, 2018 Leo Famulari <leo@famulari.name>
 ;;; Copyright © 2015 Jeff Mickey <j@codemac.net>
 ;;; Copyright © 2015, 2016, 2017 Efraim Flashner <efraim@flashner.co.il>
@@ -843,6 +843,23 @@ systems where low overhead is needed.  This package allows you to create and
 extract such file systems.")
     (license license:gpl2+)))
 
+;; We need this for building squashfs images with symlinks.
+(define-public squashfs-tools-next
+  (let ((commit "fb33dfc32b131a1162dcf0e35bd88254ae10e265")
+        (revision "1"))
+    (package (inherit squashfs-tools)
+      (name "squashfs-tools-next")
+      (version (string-append "4.3-" revision (string-take commit 7)))
+      (source (origin
+                (method git-fetch)
+                (uri (git-reference
+                      (url "https://github.com/plougher/squashfs-tools.git")
+                      (commit commit)))
+                (file-name (git-file-name name version))
+                (sha256
+                 (base32
+                  "1x2skf8hxzfch978nzx5mh46d4hhi6gl22270hiarjszsjk3bnsx")))))))
+
 (define-public pigz
   (package
     (name "pigz")
-- 
2.15.1

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 2/4] gnu: Add singularity.
  2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
@ 2018-05-25 15:47   ` Ricardo Wurmus
  2018-05-27 12:52     ` Ludovic Courtès
  2018-05-25 15:47   ` [bug#31592] [PATCH 3/4] pack: Rename "tar" to "archiver" Ricardo Wurmus
                     ` (3 subsequent siblings)
  4 siblings, 1 reply; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-25 15:47 UTC (permalink / raw)
  To: 31592; +Cc: Ricardo Wurmus

* gnu/packages/linux.scm (singularity): New variable.
---
 gnu/packages/linux.scm | 40 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 70e612995..5670bf193 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -12,7 +12,7 @@
 ;;; Copyright © 2016 Raymond Nicholson <rain1@openmailbox.org>
 ;;; Copyright © 2016 Mathieu Lirzin <mthl@gnu.org>
 ;;; Copyright © 2016, 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr>
-;;; Copyright © 2016 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2016, 2018 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2016 John Darrington <jmd@gnu.org>
 ;;; Copyright © 2016, 2017, 2018 Marius Bakke <mbakke@fastmail.com>
@@ -80,6 +80,7 @@
   #:use-module (gnu packages multiprecision)
   #:use-module (gnu packages ncurses)
   #:use-module (gnu packages netpbm)
+  #:use-module (gnu packages nettle)
   #:use-module (gnu packages networking)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages perl)
@@ -2566,6 +2567,43 @@ similar in functionality to chroot, although pflask provides better isolation
 thanks to the use of namespaces.")
     (license license:bsd-2)))
 
+(define-public singularity
+  (package
+    (name "singularity")
+    (version "2.5.1")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/singularityware/singularity/releases/download/"
+                                  version "/singularity-" version ".tar.gz"))
+              (sha256
+               (base32
+                "0f28dgf2qcy8ljjfix7p9q36q12j7rxyicfzzi4n0fl8zr8ab88g"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:configure-flags
+       (list "--disable-suid"
+             "--localstatedir=/var")
+       #:phases
+       (modify-phases %standard-phases
+         ;; Do not create directories in /var.
+         (add-after 'unpack 'disable-install-hook
+           (lambda _
+             (substitute* "Makefile.in"
+               (("\\$\\(MAKE\\) .*install-data-hook") ""))
+             #t)))))
+    (inputs
+     `(("libarchive" ,libarchive)
+       ("python" ,python-wrapper)
+       ("nettle" ,nettle)
+       ("zlib" ,zlib)))
+    (propagated-inputs
+     `(("squashfs-tools" ,squashfs-tools)))
+    (home-page "https://singularity.lbl.gov/")
+    (synopsis "Container platform")
+    (description "Singularity is a container platform supporting a number of
+container image formats.")
+    (license license:bsd-3)))
+
 (define-public hdparm
   (package
     (name "hdparm")
-- 
2.15.1

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 3/4] pack: Rename "tar" to "archiver".
  2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
  2018-05-25 15:47   ` [bug#31592] [PATCH 2/4] gnu: Add singularity Ricardo Wurmus
@ 2018-05-25 15:47   ` Ricardo Wurmus
  2018-05-25 15:47   ` [bug#31592] [PATCH 4/4] pack: Add support for squashfs images Ricardo Wurmus
                     ` (2 subsequent siblings)
  4 siblings, 0 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-25 15:47 UTC (permalink / raw)
  To: 31592; +Cc: Ricardo Wurmus

* guix/scripts/pack.scm (self-contained-tarball, docker-image): Accept
"archiver" argument; remove "tar" argument.
(guix-pack): Invoke "build-image" with "archiver" argument.
---
 guix/scripts/pack.scm | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 1e84459e7..980aef0ed 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -93,7 +93,7 @@ found."
                                  (compressor (first %compressors))
                                  localstatedir?
                                  (symlinks '())
-                                 (tar tar))
+                                 (archiver tar))
   "Return a self-contained tarball containing a store initialized with the
 closure of PROFILE, a derivation.  The tarball contains /gnu/store; if
 LOCALSTATEDIR? is true, it also contains /var/guix, including /var/guix/db
@@ -142,7 +142,7 @@ added to the pack."
           ;; 2014-07-28.  For testing, we use the bootstrap tar, which is
           ;; older and doesn't support it.
           (define tar-supports-sort?
-            (zero? (system* (string-append #+tar "/bin/tar")
+            (zero? (system* (string-append #+archiver "/bin/tar")
                             "cf" "/dev/null" "--files-from=/dev/null"
                             "--sort=name")))
 
@@ -151,7 +151,7 @@ added to the pack."
                   (string-append #$(if localstatedir?
                                        (file-append guix "/sbin:")
                                        "")
-                                 #$tar "/bin"))
+                                 #$archiver "/bin"))
 
           ;; Note: there is not much to gain here with deduplication and there
           ;; is the overhead of the '.links' directory, so turn it off.
@@ -220,7 +220,7 @@ added to the pack."
                        (compressor (first %compressors))
                        localstatedir?
                        (symlinks '())
-                       (tar tar))
+                       (archiver tar))
   "Return a derivation to construct a Docker image of PROFILE.  The
 image is a tarball conforming to the Docker Image Specification, compressed
 with COMPRESSOR.  It can be passed to 'docker load'.  If TARGET is true, it
@@ -268,7 +268,7 @@ the image."
 
           (use-modules (guix docker) (srfi srfi-19) (guix build store-copy))
 
-          (setenv "PATH" (string-append #$tar "/bin"))
+          (setenv "PATH" (string-append #$archiver "/bin"))
 
           (build-docker-image #$output
                               (call-with-input-file "profile"
@@ -626,7 +626,7 @@ Create a bundle of PACKAGE.\n"))
                (compressor  (if bootstrap?
                                 bootstrap-xz
                                 (assoc-ref opts 'compressor)))
-               (tar         (if bootstrap?
+               (archiver    (if bootstrap?
                                 %bootstrap-coreutils&co
                                 tar))
                (symlinks    (assoc-ref opts 'symlinks))
@@ -654,8 +654,8 @@ Create a bundle of PACKAGE.\n"))
                                                    symlinks
                                                    #:localstatedir?
                                                    localstatedir?
-                                                   #:tar
-                                                   tar)))
+                                                   #:archiver
+                                                   archiver)))
               (mbegin %store-monad
                 (show-what-to-build* (list drv)
                                      #:use-substitutes?
-- 
2.15.1

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 4/4] pack: Add support for squashfs images.
  2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
  2018-05-25 15:47   ` [bug#31592] [PATCH 2/4] gnu: Add singularity Ricardo Wurmus
  2018-05-25 15:47   ` [bug#31592] [PATCH 3/4] pack: Rename "tar" to "archiver" Ricardo Wurmus
@ 2018-05-25 15:47   ` Ricardo Wurmus
  2018-05-27 12:57     ` Ludovic Courtès
  2018-05-27 12:50   ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ludovic Courtès
  2018-05-28 15:37   ` bug#31592: " Ricardo Wurmus
  4 siblings, 1 reply; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-25 15:47 UTC (permalink / raw)
  To: 31592; +Cc: Ricardo Wurmus

* guix/scripts/pack.scm (%formats): Add "squashfs" format.
(guix-pack): Adjust "archiver" dependent on pack-format.
(squashfs-image): New procedure.
---
 guix/scripts/pack.scm | 96 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 92 insertions(+), 4 deletions(-)

diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 980aef0ed..88a2495c8 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -1,7 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015, 2017, 2018 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2017 Efraim Flashner <efraim@flashner.co.il>
-;;; Copyright © 2017 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2017, 2018 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2018 Konrad Hinsen <konrad.hinsen@fastmail.net>
 ;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com>
 ;;;
@@ -214,6 +214,91 @@ added to the pack."
                     build
                     #:references-graphs `(("profile" ,profile))))
 
+(define* (squashfs-image name profile
+                         #:key target
+                         deduplicate?
+                         (compressor (first %compressors))
+                         localstatedir?
+                         (symlinks '())
+                         (archiver squashfs-tools-next))
+  "Return a squashfs image containing a store initialized with the closure of
+PROFILE, a derivation.  The image contains a subset of /gnu/store and .
+
+SYMLINKS must be a list of (SOURCE -> TARGET) tuples denoting symlinks to be
+added to the pack."
+  (define build
+    (with-imported-modules '((guix build utils)
+                             (guix build store-copy)
+                             (gnu build install))
+      #~(begin
+          (use-modules (guix build utils)
+                       (gnu build install)
+                       (guix build store-copy)
+                       (srfi srfi-1)
+                       (srfi srfi-26)
+                       (ice-9 match))
+
+          (setenv "PATH" (string-append #$archiver "/bin"))
+
+          ;; We need an empty file in order to have a valid file argument when
+          ;; we reparent the root file system.  Read on for why that's
+          ;; necessary.
+          (with-output-to-file ".empty" (lambda () (display "")))
+
+          ;; Create the squashfs image in several steps.
+          (exit
+           (and
+            ;; Add all store items.  Unfortunately mksquashfs throws away all
+            ;; ancestor directories and only keeps the basename.  We fix this
+            ;; in the following invocations of mksquashfs.
+            (zero? (apply system* "mksquashfs"
+                          `(,@(call-with-input-file "profile"
+                                read-reference-graph)
+                            ,#$output
+
+                            ;; Do not perform duplicate checking because we
+                            ;; don't have any dupes.
+                            "-no-duplicates"
+                            "-comp"
+                            ,#+(compressor-name compressor))))
+
+            ;; Here we reparent the store items.  For each sub-directory of
+            ;; the store prefix we need one invocation of "mksquashfs".
+            (every (lambda (dir)
+                     (zero? (apply system* "mksquashfs"
+                                   `(".empty"
+                                     ,#$output
+                                     "-root-becomes" ,dir))))
+                   (reverse (filter (negate string-null?)
+                                    (string-split (%store-directory) #\/))))
+
+            ;; Add symlinks and mount points.
+            (zero? (apply system* "mksquashfs"
+                          `(".empty"
+                            ,#$output
+                            ;; Create SYMLINKS via pseudo file definitions.
+                            ,@(append-map
+                               (match-lambda
+                                 ((source '-> target)
+                                  (list "-p"
+                                        (string-join
+                                         ;; name s mode uid gid symlink
+                                         (list source
+                                               "s" "777" "0" "0"
+                                               (string-append #$profile "/" target))))))
+                               '#$symlinks)
+
+                            ;; Create empty mount points.
+                            "-p" "/proc d 555 0 0"
+                            "-p" "/sys d 555 0 0"
+                            "-p" "/dev d 555 0 0"))))))))
+
+  (gexp->derivation (string-append name
+                                   (compressor-extension compressor)
+                                   ".squashfs")
+                    build
+                    #:references-graphs `(("profile" ,profile))))
+
 (define* (docker-image name profile
                        #:key target
                        deduplicate?
@@ -462,6 +547,7 @@ please email '~a'~%")
 (define %formats
   ;; Supported pack formats.
   `((tarball . ,self-contained-tarball)
+    (squashfs . ,squashfs-image)
     (docker  . ,docker-image)))
 
 (define %options
@@ -626,9 +712,11 @@ Create a bundle of PACKAGE.\n"))
                (compressor  (if bootstrap?
                                 bootstrap-xz
                                 (assoc-ref opts 'compressor)))
-               (archiver    (if bootstrap?
-                                %bootstrap-coreutils&co
-                                tar))
+               (archiver    (if (equal? pack-format 'squashfs)
+                                squashfs-tools-next
+                                (if bootstrap?
+                                    %bootstrap-coreutils&co
+                                    tar)))
                (symlinks    (assoc-ref opts 'symlinks))
                (build-image (match (assq-ref %formats pack-format)
                               ((? procedure? proc) proc)
-- 
2.15.1

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next.
  2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
                     ` (2 preceding siblings ...)
  2018-05-25 15:47   ` [bug#31592] [PATCH 4/4] pack: Add support for squashfs images Ricardo Wurmus
@ 2018-05-27 12:50   ` Ludovic Courtès
  2018-05-27 18:11     ` Ricardo Wurmus
  2018-05-28 15:37   ` bug#31592: " Ricardo Wurmus
  4 siblings, 1 reply; 15+ messages in thread
From: Ludovic Courtès @ 2018-05-27 12:50 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: 31592

Hello!

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> * gnu/packages/compression.scm (squashfs-tools-next): New variable.

[...]

> +;; We need this for building squashfs images with symlinks.
> +(define-public squashfs-tools-next
> +  (let ((commit "fb33dfc32b131a1162dcf0e35bd88254ae10e265")
> +        (revision "1"))

Does it mean that the current version does not support symlinks in the
image?

> +    (package (inherit squashfs-tools)
> +      (name "squashfs-tools-next")
> +      (version (string-append "4.3-" revision (string-take commit 7)))

Nitpick: you can use (git-version …) here.

Otherwise LGTM, thanks!

Ludo’.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 2/4] gnu: Add singularity.
  2018-05-25 15:47   ` [bug#31592] [PATCH 2/4] gnu: Add singularity Ricardo Wurmus
@ 2018-05-27 12:52     ` Ludovic Courtès
  2018-05-27 18:14       ` Ricardo Wurmus
  0 siblings, 1 reply; 15+ messages in thread
From: Ludovic Courtès @ 2018-05-27 12:52 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: 31592

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> * gnu/packages/linux.scm (singularity): New variable.

[...]

> +    (propagated-inputs
> +     `(("squashfs-tools" ,squashfs-tools)))

Could we patch the code that invokes mksquashfs to use an absolute file
name?  That way we wouldn’t need to propagate it.

> +    (home-page "https://singularity.lbl.gov/")
> +    (synopsis "Container platform")
> +    (description "Singularity is a container platform supporting a number of
> +container image formats.")

Would be nice to expound a little bit, like mentioning that it can
be used to build images, to run code from Docker images, etc.

Thanks!  :-)

Ludo’.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 4/4] pack: Add support for squashfs images.
  2018-05-25 15:47   ` [bug#31592] [PATCH 4/4] pack: Add support for squashfs images Ricardo Wurmus
@ 2018-05-27 12:57     ` Ludovic Courtès
  2018-05-27 18:17       ` Ricardo Wurmus
  0 siblings, 1 reply; 15+ messages in thread
From: Ludovic Courtès @ 2018-05-27 12:57 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: 31592

Woohoo!

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> * guix/scripts/pack.scm (%formats): Add "squashfs" format.
> (guix-pack): Adjust "archiver" dependent on pack-format.
> (squashfs-image): New procedure.

[...]

> +          (exit
> +           (and
> +            ;; Add all store items.  Unfortunately mksquashfs throws away all
> +            ;; ancestor directories and only keeps the basename.  We fix this
> +            ;; in the following invocations of mksquashfs.
> +            (zero? (apply system* "mksquashfs"

This code should use ‘invoke’ everywhere, which would alleviate the need
for the clumsy (exit (and …)) idiom.

> +                   (reverse (filter (negate string-null?)
> +                                    (string-split (%store-directory) #\/))))

I think this is equivalent to:

  (reverse (string-tokenize (%store-directory)
                            (char-set-complement (char-set #\/))))

Alternately, you can use (remove …) instead of (filter (negate …) …).

:-)

One last thing: could you update guix.texi and the ‘--help’ output?

I suppose using the squashfs format allows us to bypass the bug
described at
<https://github.com/singularityware/singularity/issues/1487>, right?

Thanks a lot for working on this!

Ludo’.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next.
  2018-05-27 12:50   ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ludovic Courtès
@ 2018-05-27 18:11     ` Ricardo Wurmus
  0 siblings, 0 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-27 18:11 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 31592


Ludovic Courtès <ludovic.courtes@inria.fr> writes:

>> +;; We need this for building squashfs images with symlinks.
>> +(define-public squashfs-tools-next
>> +  (let ((commit "fb33dfc32b131a1162dcf0e35bd88254ae10e265")
>> +        (revision "1"))
>
> Does it mean that the current version does not support symlinks in the
> image?

Yes, the latest release does not support pseudo file specs for symbolic
links.

>> +    (package (inherit squashfs-tools)
>> +      (name "squashfs-tools-next")
>> +      (version (string-append "4.3-" revision (string-take commit 7)))
>
> Nitpick: you can use (git-version …) here.

Right!  I keep forgetting this.

--
Ricardo

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 2/4] gnu: Add singularity.
  2018-05-27 12:52     ` Ludovic Courtès
@ 2018-05-27 18:14       ` Ricardo Wurmus
  2018-06-06  8:31         ` Danny Milosavljevic
  0 siblings, 1 reply; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-27 18:14 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 31592


Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> * gnu/packages/linux.scm (singularity): New variable.
>
> [...]
>
>> +    (propagated-inputs
>> +     `(("squashfs-tools" ,squashfs-tools)))
>
> Could we patch the code that invokes mksquashfs to use an absolute file
> name?  That way we wouldn’t need to propagate it.

Maybe.  It uses a shell function “singularity_which” to find it in the
PATH at runtime.  I don’t know in how many places this is done, but I
can try to patch all instances to use the absolute file name.

>> +    (home-page "https://singularity.lbl.gov/")
>> +    (synopsis "Container platform")
>> +    (description "Singularity is a container platform supporting a number of
>> +container image formats.")
>
> Would be nice to expound a little bit, like mentioning that it can
> be used to build images, to run code from Docker images, etc.

True.  I tried to get information on features from the Singularity
website, but it’s all very vague and uses too many marketing phrases.
I’ll try to come up with a better description.

--
Ricardo

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 4/4] pack: Add support for squashfs images.
  2018-05-27 12:57     ` Ludovic Courtès
@ 2018-05-27 18:17       ` Ricardo Wurmus
  2018-05-28  7:49         ` Ludovic Courtès
  0 siblings, 1 reply; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-27 18:17 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 31592


Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Woohoo!
>
> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> * guix/scripts/pack.scm (%formats): Add "squashfs" format.
>> (guix-pack): Adjust "archiver" dependent on pack-format.
>> (squashfs-image): New procedure.
>
> [...]
>
>> +          (exit
>> +           (and
>> +            ;; Add all store items.  Unfortunately mksquashfs throws away all
>> +            ;; ancestor directories and only keeps the basename.  We fix this
>> +            ;; in the following invocations of mksquashfs.
>> +            (zero? (apply system* "mksquashfs"
>
> This code should use ‘invoke’ everywhere, which would alleviate the need
> for the clumsy (exit (and …)) idiom.

Okay.

>> +                   (reverse (filter (negate string-null?)
>> +                                    (string-split (%store-directory) #\/))))
>
> I think this is equivalent to:
>
>   (reverse (string-tokenize (%store-directory)
>                             (char-set-complement (char-set #\/))))
>
> Alternately, you can use (remove …) instead of (filter (negate …) …).

Good points.

> One last thing: could you update guix.texi and the ‘--help’ output?

Absolutely!

> I suppose using the squashfs format allows us to bypass the bug
> described at
> <https://github.com/singularityware/singularity/issues/1487>, right?

I generated an image containing “emacs”, “bash”, and “coreutils” and
both “singularity shell” and “singularity exec” worked fine, so I think
this problem does not affect squashfs images.

Thanks for the review!

I’ll update the patches, write some documentation, and push them in the
coming days.

-- 
Ricardo

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 4/4] pack: Add support for squashfs images.
  2018-05-27 18:17       ` Ricardo Wurmus
@ 2018-05-28  7:49         ` Ludovic Courtès
  0 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2018-05-28  7:49 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: 31592

Heya,

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Ludovic Courtès <ludovic.courtes@inria.fr> writes:

[...]

>> I suppose using the squashfs format allows us to bypass the bug
>> described at
>> <https://github.com/singularityware/singularity/issues/1487>, right?
>
> I generated an image containing “emacs”, “bash”, and “coreutils” and
> both “singularity shell” and “singularity exec” worked fine, so I think
> this problem does not affect squashfs images.

Good.

> I’ll update the patches, write some documentation, and push them in the
> coming days.

Awesome, thank you!

Ludo’.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#31592: [PATCH 1/4] gnu: Add squashfs-tools-next.
  2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
                     ` (3 preceding siblings ...)
  2018-05-27 12:50   ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ludovic Courtès
@ 2018-05-28 15:37   ` Ricardo Wurmus
  4 siblings, 0 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2018-05-28 15:37 UTC (permalink / raw)
  To: 31592-done


I just pushed all four patches to the “master” branch.

-- 
Ricardo

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 2/4] gnu: Add singularity.
  2018-05-27 18:14       ` Ricardo Wurmus
@ 2018-06-06  8:31         ` Danny Milosavljevic
  2018-06-06  9:57           ` Ricardo Wurmus
  0 siblings, 1 reply; 15+ messages in thread
From: Danny Milosavljevic @ 2018-06-06  8:31 UTC (permalink / raw)
  To: Ricardo Wurmus, 31592; +Cc: Ludovic Courtès

Hi Ricardo,

I've tested this by installing it into my user profile and then invoking

  singularity

and I get:

/gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 48: cat: command not found

GENERAL COMMANDS:
/gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 62: egrep: command not found

CONTAINER USAGE COMMANDS:
/gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 73: egrep: command not found

CONTAINER MANAGEMENT COMMANDS:
/gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 84: egrep: command not found

COMMAND GROUPS:
/gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 95: egrep: command not found


/gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 106: cat: command not found


That's because there's a PATH="/bin..." inside /home/dannym/.guix-profile/bin/singularity .

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [bug#31592] [PATCH 2/4] gnu: Add singularity.
  2018-06-06  8:31         ` Danny Milosavljevic
@ 2018-06-06  9:57           ` Ricardo Wurmus
  0 siblings, 0 replies; 15+ messages in thread
From: Ricardo Wurmus @ 2018-06-06  9:57 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: Ludovic Courtès, 31592


Hi Danny,

> /gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 48: cat: command not found
>
> GENERAL COMMANDS:
> /gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 62: egrep: command not found
>
> CONTAINER USAGE COMMANDS:
> /gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 73: egrep: command not found
>
> CONTAINER MANAGEMENT COMMANDS:
> /gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 84: egrep: command not found
>
> COMMAND GROUPS:
> /gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 95: egrep: command not found
>
>
> /gnu/store/18xq9sc79j81zlzhdm7kamq71vlivi59-singularity-2.5.1/libexec/singularity/cli/help.exec: line 106: cat: command not found

Oh, sorry about that.  I only tried this on a Fedora workstation, where
PATH="/bin…" would be no problem.  Thanks for reporting this!

-- 
Ricardo

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2018-06-06  9:58 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-05-25 14:28 [bug#31592] Add Singularity and squashfs image support to guix pack Ricardo Wurmus
2018-05-25 15:47 ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ricardo Wurmus
2018-05-25 15:47   ` [bug#31592] [PATCH 2/4] gnu: Add singularity Ricardo Wurmus
2018-05-27 12:52     ` Ludovic Courtès
2018-05-27 18:14       ` Ricardo Wurmus
2018-06-06  8:31         ` Danny Milosavljevic
2018-06-06  9:57           ` Ricardo Wurmus
2018-05-25 15:47   ` [bug#31592] [PATCH 3/4] pack: Rename "tar" to "archiver" Ricardo Wurmus
2018-05-25 15:47   ` [bug#31592] [PATCH 4/4] pack: Add support for squashfs images Ricardo Wurmus
2018-05-27 12:57     ` Ludovic Courtès
2018-05-27 18:17       ` Ricardo Wurmus
2018-05-28  7:49         ` Ludovic Courtès
2018-05-27 12:50   ` [bug#31592] [PATCH 1/4] gnu: Add squashfs-tools-next Ludovic Courtès
2018-05-27 18:11     ` Ricardo Wurmus
2018-05-28 15:37   ` bug#31592: " Ricardo Wurmus

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).