[bug#32116] Allow bytevector as content of plain-file for binary data.

[bug#32116] Allow bytevector as content of plain-file for binary data.

[Jan Nieuwenhuizen]

Currently, plain-file does not handle binary content correctly.  As discussed
on IRC[0] I added bytevector support to plain-file.

This allows for using a package source directly from git, doing something like

--8<---------------cut here---------------start------------->8---
(define (command->bytevector command)
  (let ((port (apply open-pipe* OPEN_READ command)))
    (let ((output (get-bytevector-all port)))
      (close-port port)
      output)))

(define-public hello-git
  (package
    (name "hello")
    (version "git")
    (source (let* ((commit "stable-2.0")
                   (content (command->bytevector
                             `("git" "archive" "--format" "tar" "--prefix"
                               ,(string-append commit "/") ,commit)))
                   (file-name (string-append "hello-" commit)))
              (plain-file file-name content)))
    ...
    ))
--8<---------------cut here---------------end--------------->8---

Greetings,
janneke

[0] https://gnunet.org/bot/log/guix/2018-07-10#T1763807

[bug#32117] [PATCH 1/2] store: Add `binary-file'.

[Jan Nieuwenhuizen]

* guix/store.scm (binary-file): New function.
* doc/guix.texi (G-Expressions): Describe binary-file*.
---
 doc/guix.texi  |  7 ++++++-
 guix/store.scm | 15 ++++++++++++++-
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index a8e53a530..04d7a79ac 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -27,7 +27,7 @@ Copyright @copyright{} 2016, 2017, 2018 Chris Marusich@*
 Copyright @copyright{} 2016, 2017, 2018 Efraim Flashner@*
 Copyright @copyright{} 2016 John Darrington@*
 Copyright @copyright{} 2016, 2017 Nils Gillmann@*
-Copyright @copyright{} 2016, 2017 Jan Nieuwenhuizen@*
+Copyright @copyright{} 2016, 2017, 2018 Jan Nieuwenhuizen@*
 Copyright @copyright{} 2016 Julien Lepiller@*
 Copyright @copyright{} 2016 Alex ter Weele@*
 Copyright @copyright{} 2017, 2018 Clément Lassieur@*
@@ -5463,6 +5463,11 @@ as in:
 This is the declarative counterpart of @code{text-file*}.
 @end deffn
 
+@deffn {Monadic Procedure} binary-file* @var{name} @var{data} @dots{}
+Return as a monadic value a derivation that builds a text file
+containing @var{data}.  @var{data} is a bytevector.
+@end deffn
+
 @deffn {Scheme Procedure} file-union @var{name} @var{files}
 Return a @code{<computed-file>} that builds a directory containing all of @var{files}.
 Each item in @var{files} must be a two-element list where the first element is the
diff --git a/guix/store.scm b/guix/store.scm
index bac42f273..cc5c24a77 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2018 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -77,6 +78,7 @@
             add-data-to-store
             add-text-to-store
             add-to-store
+            binary-file
             build-things
             build
             query-failed-paths
@@ -1362,7 +1364,18 @@ taking the store as its first argument."
 ;; Store monad operators.
 ;;
 
-(define* (text-file name text
+(define* (binary-file name
+                      data ;bytevector
+                      #:optional (references '()))
+  "Return as a monadic value the absolute file name in the store of the file
+containing DATA, a bytevector.  REFERENCES is a list of store items that the
+resulting text file refers to; it defaults to the empty list."
+  (lambda (store)
+    (values (add-data-to-store store name data references)
+            store)))
+
+(define* (text-file name
+                    text ;string
                     #:optional (references '()))
   "Return as a monadic value the absolute file name in the store of the file
 containing TEXT, a string.  REFERENCES is a list of store items that the
-- 
2.18.0

[bug#32118] [PATCH 2/2] gexp: Allow bytevector as content of `plain-file'.

[Jan Nieuwenhuizen]

This allows for using a package source directly from git, doing something like

    (define (command->bytevector command)
      (let ((port (apply open-pipe* OPEN_READ command)))
        (let ((output (get-bytevector-all port)))
          (close-port port)
          output)))

    (define-public hello-git
      (package
        (name "hello")
        (version "git")
        (source (let* ((commit "stable-2.0")
                       (content (command->bytevector
                                 `("git" "archive" "--format" "tar" "--prefix"
                                   ,(string-append commit "/") ,commit)))
                       (file-name (string-append "hello-" commit)))
                  (plain-file file-name content)))
        ...
        ))

* guix/gexp.scm (<plain-file>): Also allow bytevector content.
(plain-file-compiler): Handle bytevector content.
* doc/guix.texi (G-Expressions): Describe plain-file now also taking
bytevectors.
---
 doc/guix.texi |  2 +-
 guix/gexp.scm | 10 +++++++---
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 04d7a79ac..4d80f3e19 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5349,7 +5349,7 @@ procedure (@pxref{The Store Monad, @code{interned-file}}).
 
 @deffn {Scheme Procedure} plain-file @var{name} @var{content}
 Return an object representing a text file called @var{name} with the given
-@var{content} (a string) to be added to the store.
+@var{content} (a string or a bytevector) to be added to the store.
 
 This is the declarative counterpart of @code{text-file}.
 @end deffn
diff --git a/guix/gexp.scm b/guix/gexp.scm
index 153b29bd4..cc3613f6f 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015, 2016, 2017, 2018 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
+;;; Copyright © 2018 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -23,6 +24,7 @@
   #:use-module (guix derivations)
   #:use-module (guix grafts)
   #:use-module (guix utils)
+  #:use-module (rnrs bytevectors)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-9 gnu)
@@ -334,7 +336,7 @@ appears."
   (%plain-file name content references)
   plain-file?
   (name        plain-file-name)                   ;string
-  (content     plain-file-content)                ;string
+  (content     plain-file-content)                ;string or bytevector
   (references  plain-file-references))            ;list (currently unused)
 
 (define (plain-file name content)
@@ -349,8 +351,10 @@ This is the declarative counterpart of 'text-file'."
 (define-gexp-compiler (plain-file-compiler (file <plain-file>) system target)
   ;; "Compile" FILE by adding it to the store.
   (match file
-    (($ <plain-file> name content references)
-     (text-file name content references))))
+    (($ <plain-file> name (and (? string?) content) references)
+     (text-file name content references))
+    (($ <plain-file> name (and (? bytevector?) content) references)
+     (binary-file name content references))))
 
 (define-record-type <computed-file>
   (%computed-file name gexp guile options)
-- 
2.18.0

[bug#32116] [bug#32117] [PATCH 1/2] store: Add `binary-file'.

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> * guix/store.scm (binary-file): New function.
> * doc/guix.texi (G-Expressions): Describe binary-file*.

[...]

> +@deffn {Monadic Procedure} binary-file* @var{name} @var{data} @dots{}

There’s no ‘*’ in the actual procedure name.

Also, could you move this @deffn to “The Store Monad”, right after
‘text-file’?

Apart from that it LGTM, thanks!

Ludo’.

[bug#32117] [PATCH 1/2] store: Add `binary-file'.

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> * guix/store.scm (binary-file): New function.
> * doc/guix.texi (G-Expressions): Describe binary-file*.

[...]

> +@deffn {Monadic Procedure} binary-file* @var{name} @var{data} @dots{}

There’s no ‘*’ in the actual procedure name.

Also, could you move this @deffn to “The Store Monad”, right after
‘text-file’?

Apart from that it LGTM, thanks!

Ludo’.

[bug#32116] [bug#32118] [PATCH 2/2] gexp: Allow bytevector as content of `plain-file'.

[Ludovic Courtès]

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> This allows for using a package source directly from git, doing something like
>
>     (define (command->bytevector command)
>       (let ((port (apply open-pipe* OPEN_READ command)))
>         (let ((output (get-bytevector-all port)))
>           (close-port port)
>           output)))
>
>     (define-public hello-git
>       (package
>         (name "hello")
>         (version "git")
>         (source (let* ((commit "stable-2.0")
>                        (content (command->bytevector
>                                  `("git" "archive" "--format" "tar" "--prefix"
>                                    ,(string-append commit "/") ,commit)))
>                        (file-name (string-append "hello-" commit)))
>                   (plain-file file-name content)))
>         ...
>         ))
>
> * guix/gexp.scm (<plain-file>): Also allow bytevector content.
> (plain-file-compiler): Handle bytevector content.
> * doc/guix.texi (G-Expressions): Describe plain-file now also taking
> bytevectors.

LGTM, thanks!

Ludo'.

bug#32116: [bug#32117] [PATCH 1/2] store: Add `binary-file'.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> +@deffn {Monadic Procedure} binary-file* @var{name} @var{data} @dots{}
>
> There’s no ‘*’ in the actual procedure name.
>
> Also, could you move this @deffn to “The Store Monad”, right after
> ‘text-file’?

Ah, I based this on the `text-file*' description!  Moved, and rewritten
as

@deffn {Monadic Procedure} binary-file @var{name} @var{data} [@var{references}]
Return as a monadic value the absolute file name in the store of the file
containing @var{data}, a bytevector.  @var{references} is a list of store
items that the resulting binary file refers to; it defaults to the empty list.
@end deffn

> Apart from that it LGTM, thanks!

Pushed to master as  f3a422511f793fb6c6cfeec2bb8735965a03294a

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[bug#32116] [bug#32118] [PATCH 2/2] gexp: Allow bytevector as content of `plain-file'.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> * guix/gexp.scm (<plain-file>): Also allow bytevector content.
>> (plain-file-compiler): Handle bytevector content.
>> * doc/guix.texi (G-Expressions): Describe plain-file now also taking
>> bytevectors.
>
> LGTM, thanks!

Pushed to master as e8e1f295f15fa56660a2c460d422795b1a31bed8

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com