Toward 0.2

Toward 0.2

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 943 bytes --]

Hi!

I think we should plan for 0.2 within a month or so.  Here are the
important features I’d like to see in:

  • ‘guix pull’ (see TODO; I’ve started working on that);

  • a simple “binary substituter”, which would allow pre-built binaries
    to be transparently downloaded from trusted source, such as
    hydra.gnu.org (I also have unfinished code locally);

  • core updates

    - install glibc locale data;
    - install glibc timezone data;
    - merge mips64 branch;
    - items under “guix build utils” in TODO;
    - core software updates;

Optionally (beware of priority inversion!):

  • cross-compilation support (again, I have unfinished stuff around!);

  • Xorg & co. packaged (Andreas? :-));

  • auto-updater for GNU packages (based on my gnupdate in Nixpkgs);

  • package synopses synchronization with the Womb.

Anything else?  Comments?

Thanks,
Ludo’.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 154 bytes --]

I guess that I can implement this one:

** Add equivalent to Nixpkgs's ‘wrapProgram’

What's the purpose of 'wrapProgram'?  Could you elaborate?

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

> I guess that I can implement this one:
>
> ** Add equivalent to Nixpkgs's ‘wrapProgram’
>
> What's the purpose of 'wrapProgram'?  Could you elaborate?

Here’s an example from the Guile expression in Nixpkgs:

  wrapProgram $out/bin/guile-snarf --prefix PATH : "${gawk}/bin"

What it does is that it renames guile-snarf to .guile-snarf-real, and
makes guile-snarf a wrapper around the real one, like this:

  #!/bin/sh
  export PATH="/nix/store/...-gawk/bin:$PATH"
  exec ./.guile-snarf-real "$@"

This is useful for scripts that expect particular programs to be in
$PATH, for programs that expect particular shared libraries to be in
$LD_LIBRARY_PATH, or modules in $GUILE_LOAD_PATH, etc.

The source for Nixpkgs’s ‘wrapProgram’ is at:

  https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/setup-hooks/make-wrapper.sh

In our case, the equivalent would be a procedure in (guix build utils).
The generated wrapper would probably still be a Bash script as above.

Thanks!

Ludo’.

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

>  wrapProgram $out/bin/guile-snarf --prefix PATH : "${gawk}/bin"

How can I get the output of a package?  There is 'package-outputs':

scheme@(guile-user)> ,use (gnu packages wget) (guix packages)
scheme@(guile-user)> (package-outputs wget)
$1 = ("out")

But how can I get the value (like in '#:phases')?

>   #!/bin/sh
>   export PATH="/nix/store/...-gawk/bin:$PATH"
>   exec ./.guile-snarf-real "$@"

Should it be saved as 'guile-snarf'?  And I have to use rename(2),
right?

What should I use to handle prefixes (e.g., a keyword, a simple
argument)?  Should they be case insensitive?

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>>  wrapProgram $out/bin/guile-snarf --prefix PATH : "${gawk}/bin"
>
> How can I get the output of a package?

With the recently-added ‘package-output’ (singular):

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (package-output s binutils "out")
$2 = "/nix/store/bfh5c2a4is27jdmc811fp6g0jfac7fiw-binutils-2.22"
scheme@(guile-user)> (package-output s binutils "lib")
$3 = "/nix/store/n29brzqlhjkzww51labk9anx493gl4d3-binutils-2.22-lib"
--8<---------------cut here---------------end--------------->8---

Under the hood it calls ‘package-derivation’, which does the actual
output path computation.

>>   #!/bin/sh
>>   export PATH="/nix/store/...-gawk/bin:$PATH"
>>   exec ./.guile-snarf-real "$@"
>
> Should it be saved as 'guile-snarf'?  And I have to use rename(2),
> right?

You first rename the wrapped program, then create the wrapper as above,
and finally chmod it so that it’s executable.

> What should I use to handle prefixes (e.g., a keyword, a simple
> argument)?  Should they be case insensitive?

Good question.  The thing is that sometimes you want to affect several
environment variables.  Maybe something like:

  (define* (wrap-program file #:rest variables)
     ...)

Where each rest argument has a form like this:

  ("PATH" ":" prefix ("/nix/.../foo/bin" "/nix/.../bar/bin"))

Instead of ‘prefix’, users could ask for ‘prefix’ (prepend to the search
path) or ‘=’ (set the search path to exactly that value.)

WDYT?

Ludo’.

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

> With the recently-added ‘package-output’ (singular):

> scheme@(guile-user)> (package-output s binutils "out")
> $2 = "/nix/store/bfh5c2a4is27jdmc811fp6g0jfac7fiw-binutils-2.22"
> scheme@(guile-user)> (package-output s binutils "lib")
> $3 = "/nix/store/n29brzqlhjkzww51labk9anx493gl4d3-binutils-2.22-lib"

What's the value of 's' here?  There is no explanation in the docstrings
of 'package-derivation', 'package-cross-derivation', and
'package-output'.

Are you sure that it's OK to have 'package-output' and 'package-outputs'
in the same module?  Can we rename one?

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>> With the recently-added ‘package-output’ (singular):
>
>> scheme@(guile-user)> (package-output s binutils "out")
>> $2 = "/nix/store/bfh5c2a4is27jdmc811fp6g0jfac7fiw-binutils-2.22"
>> scheme@(guile-user)> (package-output s binutils "lib")
>> $3 = "/nix/store/n29brzqlhjkzww51labk9anx493gl4d3-binutils-2.22-lib"
>
> What's the value of 's' here?

(define s (open-connection))

> There is no explanation in the docstrings of 'package-derivation',
> 'package-cross-derivation', and 'package-output'.

For the ‘store’ parameter you mean?  We could add one, but this is a
convention shared by (guix derivations) and (guix packages), so I’m not
sure docstrings should be “cluttered” this way.

> Are you sure that it's OK to have 'package-output' and 'package-outputs'
> in the same module?  Can we rename one?

I agree we should be caution with this sort of names, but I’ve
considered it OK.  There’s a tension between the length and expressivity
of a name.

Ludo’.

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1.1: Type: text/plain, Size: 2716 bytes --]

I'm attaching a patch.

It's a separate file; if I add these lines

   #:use-module (guix packages)
   #:use-module (guix store)

to 'guix/build/utils.scm', they will raise the following error on
'make'.

Backtrace:
In ice-9/eval.scm:
 400: 19 [eval # ()]
In ice-9/boot-9.scm:
2681: 18 [define-module* (guix utils) #:filename ...]
2656: 17 [resolve-imports (((guix config)) ((guix packages)) ((guix store)) ...)]
2594: 16 [resolve-interface (guix packages) #:select ...]
2519: 15 [#<procedure 8fee7a0 at ice-9/boot-9.scm:2507:4 (name #:optional autoload version #:key ensure)> # ...]
2786: 14 [try-module-autoload (guix packages) #f]
2131: 13 [save-module-excursion #<procedure 92a4eb8 at ice-9/boot-9.scm:2787:17 ()>]
2797: 12 [#<procedure 92a4eb8 at ice-9/boot-9.scm:2787:17 ()>]
In unknown file:
   ?: 11 [primitive-load-path "guix/packages" #f]
In guix/packages.scm:
  19: 10 [#<procedure 92cece0 ()>]
In ice-9/boot-9.scm:
2681: 9 [define-module* (guix packages) #:filename ...]
2656: 8 [resolve-imports (((guix utils)) ((guix store)) ((guix base32)) ...)]
2594: 7 [resolve-interface (guix derivations) #:select ...]
2519: 6 [#<procedure 8fee7a0 at ice-9/boot-9.scm:2507:4 (name #:optional autoload version #:key ensure)> # ...]
2786: 5 [try-module-autoload (guix derivations) #f]
2131: 4 [save-module-excursion #<procedure 93f87c8 at ice-9/boot-9.scm:2787:17 ()>]
2797: 3 [#<procedure 93f87c8 at ice-9/boot-9.scm:2787:17 ()>]
In unknown file:
   ?: 2 [primitive-load-path "guix/derivations" #f]
In guix/derivations.scm:
 317: 1 [#<procedure 9439920 ()>]
In ice-9/boot-9.scm:
 106: 0 [#<procedure 913a8c0 at ice-9/boot-9.scm:97:6 (thrown-k . args)> unbound-variable ...]

ice-9/boot-9.scm:106:20: In procedure #<procedure 913a8c0 at ice-9/boot-9.scm:97:6 (thrown-k . args)>:
ice-9/boot-9.scm:106:20: In procedure module-lookup: Unbound variable: memoize
make[2]: *** [guix/utils.go] Error 1
make[2]: Leaving directory `/home/guix-savannah'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/guix-savannah'
make: *** [all] Error 2

I don't think that 'store-location' is really needed.  I added it
because I hadn't found a procedure that can return something like
'/nix/store/*-wget-1.14/bin/wget'.

Also, 'bin-location' and 'chmod' are not safe.  It can be solved with
'file-exists?', but I decided not to use it because I want to rewrite
the whole thing.  Any suggestions?

Example:

scheme@(guile-user)> ,use (gnu packages wget) (gnu packages gawk)
scheme@(guile-user)> ,use (wrap-program)
scheme@(guile-user)> (wrap-program wget #t "PATH" (store-location gawk "out" "/bin"))

#!/bin/sh
export PATH="/nix/store/l5gkkxbjrlhddpxyxl6glhyczvh0gggw-gawk-4.0.0/bin:$PATH"
exec ./.wget-real "$@"


[-- Attachment #1.2: wrap-program.scm --]
[-- Type: text/plain, Size: 2122 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (wrap-program)
  #:use-module (guix packages)
  #:use-module (guix store)
  #:use-module (srfi srfi-26)
  #:export (store-location
            wrap-program))

(define (store-location package output rest)
  "Return a PACKAGE-related location."
  (string-append (package-output (open-connection) package output)
                 rest))

(define (wrap-program program prefix? variable var-dir)
  "Copy PROGRAM to .PROGRAM-real and make PROGRAM a wrapper."
  (let* ((bin-location (store-location program "out"
                                       (string-append "/bin"))) ; not safe
         (program-name (package-name program))
         (old (string-append bin-location "/" program-name))
         (new (string-append bin-location "/." program-name "-real"))
         (tmp (string-append bin-location "/." program-name "-tmp")))

    (define (change-variable)
      ;; Prepend VAR-DIR to VARIABLE or return VAR-DIR.
      (if prefix?
          (string-append var-dir ":$" variable)
          var-dir))

    (copy-file old new)

    (call-with-output-file
        tmp
      (cut format <> "#!/bin/sh~%export ~a=\"~a\"~%exec ./~a \"$@\"~%"
           variable (change-variable) (basename new)))

    (chmod tmp #o755)
    (rename-file tmp old)))

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Mark H Weaver]

Nikita Karetnikov <nikita@karetnikov.org> writes:

> I'm attaching a patch.
>
> It's a separate file; if I add these lines
>
>    #:use-module (guix packages)
>    #:use-module (guix store)
>
> to 'guix/build/utils.scm', they will raise the following error on
> 'make'.

Based on the backtrace below, it looks like you tried to add those to
guix/utils.scm, not guix/build/utils.scm.  It failed because it resulted
in a circular dependency between modules.

  (guix utils) -> (guix packages) -> (guix derivations) -> (guix utils)

Circular dependencies are not always fatal in Guile, but they can be
when one of the modules depends on a *macro* from another module in the
cycle.  In this case, (guix derivations) uses the 'memoize' macro from
(guix utils).

[Adds to Guile TODO list: better error messages in cases like this]

> Also, 'bin-location' and 'chmod' are not safe.  It can be solved with
> 'file-exists?'

I don't know what the issue is here (and don't have time now to figure
it out), but the idea that 'file-exists?' can make anything safe is
setting off warning bells in my head, because it sounds like a
TOCTTOU[1] race condition: someone else may change the filesystem
between your 'file-exists?' check and whatever you do that depends on
that check for safety.

     Regards,
       Mark

[1] http://en.wikipedia.org/wiki/Time-of-check-to-time-of-use

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

> ice-9/boot-9.scm:106:20: In procedure #<procedure 913a8c0 at ice-9/boot-9.scm:97:6 (thrown-k . args)>:
> ice-9/boot-9.scm:106:20: In procedure module-lookup: Unbound variable: memoize

As Mark noted, that’s because of a circular dependency.

> (define-module (wrap-program)
>   #:use-module (guix packages)
>   #:use-module (guix store)
>   #:use-module (srfi srfi-26)
>   #:export (store-location
>             wrap-program))
>
> (define (store-location package output rest)
>   "Return a PACKAGE-related location."
>   (string-append (package-output (open-connection) package output)
>                  rest))

There’s a problem here: this is “host” code, whereas we want code to run
in the builder–i.e., a procedure in (guix build utils).

Remember that (guix build ...) modules must not use Guix modules other
than (guix build ...).  This is a convention to distinguish between code
for the “builder stratum”, and code for the “host stratum”.

> (define (wrap-program program prefix? variable var-dir)
>   "Copy PROGRAM to .PROGRAM-real and make PROGRAM a wrapper."

So here PROGRAM would be a file name (not necessarily absolute).

>   (let* ((bin-location (store-location program "out"
>                                        (string-append "/bin"))) ; not safe
>          (program-name (package-name program))
>          (old (string-append bin-location "/" program-name))
>          (new (string-append bin-location "/." program-name "-real"))
>          (tmp (string-append bin-location "/." program-name "-tmp")))

You just need to keep ‘old’ and ‘new’ (the installation of the wrapper
doesn’t need to be atomic.)

>     (define (change-variable)
>       ;; Prepend VAR-DIR to VARIABLE or return VAR-DIR.
>       (if prefix?
>           (string-append var-dir ":$" variable)
>           var-dir))

This should add a colon before $VARIABLE if and only if VARIABLE is
non-empty, like this:

  export VARIABLE=xxx:${VARIABLE:+:}$VARIABLE

>     (copy-file old new)

Just ‘rename-file’.

>     (call-with-output-file
>         tmp
>       (cut format <> "#!/bin/sh~%export ~a=\"~a\"~%exec ./~a \"$@\"~%"
>            variable (change-variable) (basename new)))

Use (which "bash") instead of /bin/sh.  Avoid ‘cut’ because it’s not
helpful here.

Also it’s better to use the absolute file name for ‘exec’, so just
replace (basename new) by (canonicalize-path new).

>     (chmod tmp #o755)

OK.

However, this only allows for prefix, and of one variable only.  I was
instead suggesting this API:

    (define* (wrap-program file #:rest variables)
       ...)

  Where each rest argument has a form like this:

    ("PATH" ":" prefix ("/nix/.../foo/bin" "/nix/.../bar/bin"))

  Instead of ‘prefix’, users could ask for ‘prefix’ (prepend to the search
  path) or ‘=’ (set the search path to exactly that value.)

Examples:

  (wrap-program "wget" '(("PATH" ":" prefix ("/nix/.../gawk/bin"))))
  =>
  #!/.../sh
  export PATH=/nix/.../gawk/bin:${PATH:+:}$PATH
  exec /.../bin/.wget-real


  (wrap-program "wget" '(("PATH" ":" = ("/nix/.../gawk/bin"))
                         ("CERT_PATH" ":" suffix '("/nix/.../share/certs"))))
  =>
  #!/.../sh
  export PATH=/nix/.../gawk/bin
  export CERT_PATH=$CERT_PATH${CERT_PATH:+:}:/nix/.../share/certs
  exec /.../bin/.wget-real

Perhaps the separator string could be optional.

WDYT?

Let me know if this is unclear.

Thanks,
Ludo’.

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1.1: Type: text/plain, Size: 824 bytes --]

> Remember that (guix build ...) modules must not use Guix modules other
> than (guix build ...).  This is a convention to distinguish between code
> for the “builder stratum”, and code for the “host stratum”.

OK.  But how can I get the location of a package?  The needed procedure
(i.e., 'package-output') is in the host code.

> (define* (wrap-program file #:rest variables)
>    ...)

Why do you want to use '#:rest' here?  The purpose of 'wrap-program' is
to export some variables.

> export CERT_PATH=$CERT_PATH${CERT_PATH:+:}:/nix/.../share/certs

Could you explain the "$CERT_PATH${CERT_PATH:+:}" part?

> Perhaps the separator string could be optional.

Is it possible to make it optional without creating a separate clause?

I'm attaching a patch.  It lacks the 'rename-file' part.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-utils-Add-wrap-program.patch --]
[-- Type: text/x-diff, Size: 2491 bytes --]

From 0c84fdb879c78a129c9d77bcdf9a5e3135825ad9 Mon Sep 17 00:00:00 2001
From: Nikita Karetnikov <nikita@karetnikov.org>
Date: Thu, 28 Feb 2013 20:27:36 +0000
Subject: [PATCH] utils: Add 'wrap-program'.

* guix/build/utils.scm (wrap-program): New procedure.
---
 guix/build/utils.scm |   41 ++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 40 insertions(+), 1 deletions(-)

diff --git a/guix/build/utils.scm b/guix/build/utils.scm
index 6921e31..63012f0 100644
--- a/guix/build/utils.scm
+++ b/guix/build/utils.scm
@@ -49,7 +49,8 @@
             patch-shebang
             patch-makefile-SHELL
             fold-port-matches
-            remove-store-references))
+            remove-store-references
+            wrap-program))
 
 \f
 ;;;
@@ -605,6 +606,44 @@ known as `nuke-refs' in Nixpkgs."
                              (put-u8 out (char->integer char))
                              result))))))
 
+(define (wrap-program prog varlst)
+  "Copy PROG to .PROG-real and make PROG a wrapper."
+  (define (export-envvar lst)
+    ;; Return a string that exports an environment variable.
+    (define (separate lst delim)
+      ;; Return a string of directories separated by DELIM.
+      (fold-right (lambda (x acc)
+                    (if (string-null? acc)
+                        x
+                        (string-append x delim acc)))
+                  ""
+                  lst))
+
+    ;; TODO: Make SEP optional.
+    (match lst
+           ((var sep '= rest)
+            (format #f "export ~a=\"~a\""
+                    var (separate rest sep)))
+
+           ((var sep 'prefix rest)
+            (format #f "export ~a=\"~a~a${~a~a+~a}$~a\""
+                    var (separate rest sep) sep var sep sep var))
+
+           ((var sep 'suffix rest)
+            (format #f "export ~a=\"$~a${~a~a+~a}~a~a\""
+                    var var var sep sep sep (separate rest sep)))))
+
+  ;; XXX: Use an absolute filename; remove '-real'.
+  (format #f "#!~a~%~aexec ./.~a-real~%"
+          (which "bash")
+
+          (fold-right (lambda (x acc)
+                        (string-append x "\n" acc))
+                      ""
+                      (map export-envvar varlst))
+
+          prog)) ; XXX: use a real program instead
+
 ;;; Local Variables:
 ;;; eval: (put 'call-with-output-file/atomic 'scheme-indent-function 1)
 ;;; eval: (put 'with-throw-handler 'scheme-indent-function 1)
-- 
1.7.5.4


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>> Remember that (guix build ...) modules must not use Guix modules other
>> than (guix build ...).  This is a convention to distinguish between code
>> for the “builder stratum”, and code for the “host stratum”.
>
> OK.  But how can I get the location of a package?  The needed procedure
> (i.e., 'package-output') is in the host code.

It’s not needed.  All that needed is the name of the executable file to wrap.

>> (define* (wrap-program file #:rest variables)
>>    ...)
>
> Why do you want to use '#:rest' here?  The purpose of 'wrap-program' is
> to export some variables.

For the case where there are have several variables you want to set.

>> export CERT_PATH=$CERT_PATH${CERT_PATH:+:}:/nix/.../share/certs
>
> Could you explain the "$CERT_PATH${CERT_PATH:+:}" part?

The :+ part is to add a colon if and only if $CERT_PATH is non-empty
(info "(bash) Shell Parameter Expansion").

Otherwise you could end up with a value like “:/foo”, but the first
empty element (before the colon) is usually equivalent to “.”, and we
don’t want that.

>> Perhaps the separator string could be optional.
>
> Is it possible to make it optional without creating a separate clause?

No, you’d need an additional clause.

> +(define (wrap-program prog varlst)
> +  "Copy PROG to .PROG-real and make PROG a wrapper."

Could you document what the wrapper does, and what the grammar of
‘varlst’ (rather call it ‘vars’) is?

> +  (define (export-envvar lst)

‘export-variable’, rather.

> +    (define (separate lst delim)

You ‘string-join’ instead:

  (string-join '("a" "b") ":")
  => "a:b"

> +    ;; TODO: Make SEP optional.
> +    (match lst
> +           ((var sep '= rest)
> +            (format #f "export ~a=\"~a\""
> +                    var (separate rest sep)))
> +
> +           ((var sep 'prefix rest)
> +            (format #f "export ~a=\"~a~a${~a~a+~a}$~a\""
> +                    var (separate rest sep) sep var sep sep var))
> +
> +           ((var sep 'suffix rest)
> +            (format #f "export ~a=\"$~a${~a~a+~a}~a~a\""
> +                    var var var sep sep sep (separate rest sep)))))

Good (please indent like other uses of ‘match’ in this file).

> +  ;; XXX: Use an absolute filename; remove '-real'.
> +  (format #f "#!~a~%~aexec ./.~a-real~%"
> +          (which "bash")
> +
> +          (fold-right (lambda (x acc)
> +                        (string-append x "\n" acc))
> +                      ""
> +                      (map export-envvar varlst))

string-join

> +          prog)) ; XXX: use a real program instead

Just (canonicalize-path prog).

Thanks,
Ludo’.

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

> It’s not needed.  All that needed is the name of the executable file
> to wrap.

I don't understand.  How will 'rename-file' get the location of the
executable?  For example:

(wrap-program "wget" [...])

I guess that it will only work if you invoke 'wrap-program' from the
same directory.

> For the case where there are have several variables you want to set.

Could you provide an example?  The following works without '#:rest':

(display (wrap-program "wget" '(("PATH" ":" = ("/nix/.../gawk/bin"))
                                ("CERT_PATH" ":" suffix ("/nix/.../share/certs"
                                                         "/nix/.../foo/certs")))))

#!/bin/bash
export PATH="/nix/.../gawk/bin"
export CERT_PATH="$CERT_PATH${CERT_PATH:+:}:/nix/.../share/certs:/nix/.../foo/certs"
exec ./.wget-real

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>> It’s not needed.  All that needed is the name of the executable file
>> to wrap.
>
> I don't understand.  How will 'rename-file' get the location of the
> executable?  For example:
>
> (wrap-program "wget" [...])
>
> I guess that it will only work if you invoke 'wrap-program' from the
> same directory.

Yes, and that’s what we want.

However, the ‘exec’ line in the wrapper needs the absolute file name.
For that it can do along the lines of:

  (string-append "exec " (canonicalize-path file))

>> For the case where there are have several variables you want to set.
>
> Could you provide an example?  The following works without '#:rest':
>
> (display (wrap-program "wget" '(("PATH" ":" = ("/nix/.../gawk/bin"))
>                                 ("CERT_PATH" ":" suffix ("/nix/.../share/certs"
>                                                          "/nix/.../foo/certs")))))

Yeah there are two choices: two required arguments (like above), where
the second one is a list, or one required argument and one rest
argument.

In the latter case, you would instead do:

  (wrap-program "wget" '("PATH" ":" = ("/nix/.../gawk/bin"))
                       '("CERT_PATH" ":" suffix ("/nix/.../share/certs"
                                                 "/nix/.../foo/certs")))

I tend to prefer this form because it may be more concise in common
cases.

HTH,
Ludo’.

Re: Toward 0.2

[Nikita Karetnikov]

[-- Attachment #1.1: Type: text/plain, Size: 589 bytes --]

What about this patch?

Is there a way to compose the following part somehow to prevent a
possible race condition?

    (rename-file prog prog-real)

    (with-output-to-file prog (lambda ()
                                (format #t
                                        "#!~a~%~a~%exec ~a~%"
                                        (which "bash")
                                        (string-join (map export-variable vars)
                                                     "\n")
                                        (canonicalize-path prog-real))))
    (chmod prog #o755)))


[-- Attachment #1.2: 0001-utils-Add-wrap-program.patch --]
[-- Type: text/x-diff, Size: 2954 bytes --]

From 6bd8c169c320ffe9dd13a85a6f63a60b8508bc3c Mon Sep 17 00:00:00 2001
From: Nikita Karetnikov <nikita@karetnikov.org>
Date: Sat, 2 Mar 2013 20:08:39 +0000
Subject: [PATCH] utils: Add 'wrap-program'.

* guix/build/utils.scm (wrap-program): New procedure.
---
 guix/build/utils.scm |   40 +++++++++++++++++++++++++++++++++++++++-
 1 files changed, 39 insertions(+), 1 deletions(-)

diff --git a/guix/build/utils.scm b/guix/build/utils.scm
index 6921e31..2bff789 100644
--- a/guix/build/utils.scm
+++ b/guix/build/utils.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2012, 2013 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -49,7 +50,8 @@
             patch-shebang
             patch-makefile-SHELL
             fold-port-matches
-            remove-store-references))
+            remove-store-references
+            wrap-program))
 
 \f
 ;;;
@@ -605,6 +607,42 @@ known as `nuke-refs' in Nixpkgs."
                              (put-u8 out (char->integer char))
                              result))))))
 
+(define* (wrap-program prog #:rest vars)
+  "Rename PROG to .PROG-real and make PROG a wrapper."
+  (let ((prog-real (string-append "." prog "-real")))
+    (define (export-variable lst)
+      ;; Return a string that exports an environment variable.
+      (match lst
+        ((var sep '= rest)
+         (format #f "export ~a=\"~a\""
+                 var (string-join rest sep)))
+        ((var sep 'prefix rest)
+         (format #f "export ~a=\"~a${~a~a+~a}$~a\""
+                 var (string-join rest sep) var sep sep var))
+        ((var sep 'suffix rest)
+         (format #f "export ~a=\"$~a${~a~a+~a}~a\""
+                 var var var sep sep (string-join rest sep)))
+        ((var '= rest)
+         (format #f "export ~a=\"~a\""
+                 var (string-join rest ":")))
+        ((var 'prefix rest)
+         (format #f "export ~a=\"~a${~a:+:}$~a\""
+                 var (string-join rest ":") var var))
+        ((var 'suffix rest)
+         (format #f "export ~a=\"$~a${~a:+:}~a\""
+                 var var var (string-join rest ":")))))
+
+    (rename-file prog prog-real)
+
+    (with-output-to-file prog (lambda ()
+                                (format #t
+                                        "#!~a~%~a~%exec ~a~%"
+                                        (which "bash")
+                                        (string-join (map export-variable vars)
+                                                     "\n")
+                                        (canonicalize-path prog-real))))
+    (chmod prog #o755)))
+
 ;;; Local Variables:
 ;;; eval: (put 'call-with-output-file/atomic 'scheme-indent-function 1)
 ;;; eval: (put 'with-throw-handler 'scheme-indent-function 1)
-- 
1.7.5.4


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Toward 0.2

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

> Is there a way to compose the following part somehow to prevent a
> possible race condition?
>
>     (rename-file prog prog-real)
>
>     (with-output-to-file prog (lambda ()
>                                 (format #t
>                                         "#!~a~%~a~%exec ~a~%"
>                                         (which "bash")
>                                         (string-join (map export-variable vars)
>                                                      "\n")
>                                         (canonicalize-path prog-real))))
>     (chmod prog #o755)))

If you’re really concerned, you could do (pseudocode):

  cp prog prog-real
  echo wrapper code > prog.tmp
  chmod +x prog.tmp
  rename prog.tmp prog

I wouldn’t bother, though, because it will be used in contexts where
there’s no risk of ‘prog’ being used while we’re fiddling with it
(single-threaded, after ‘make install’).

> From 6bd8c169c320ffe9dd13a85a6f63a60b8508bc3c Mon Sep 17 00:00:00 2001
> From: Nikita Karetnikov <nikita@karetnikov.org>
> Date: Sat, 2 Mar 2013 20:08:39 +0000
> Subject: [PATCH] utils: Add 'wrap-program'.
>
> * guix/build/utils.scm (wrap-program): New procedure.
> ---
>  guix/build/utils.scm |   40 +++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 39 insertions(+), 1 deletions(-)

Looks good to me.

> +(define* (wrap-program prog #:rest vars)
> +  "Rename PROG to .PROG-real and make PROG a wrapper."

Can you just expound the docstring before pushing?

Thanks!

Ludo’.

[PATCH] utils: Add 'wrap-program'. (was: Toward 0.2)

[Nikita Karetnikov]

[-- Attachment #1.1: Type: text/plain, Size: 335 bytes --]

> I wouldn’t bother, though, because it will be used in contexts where
> there’s no risk of ‘prog’ being used while we’re fiddling with it
> (single-threaded, after ‘make install’).

Better safe than sorry.  So I added '.PROG-tmp'.

> Can you just expound the docstring before pushing?

Is it OK?  Can I push?


[-- Attachment #1.2: 0001-utils-Add-wrap-program.patch --]
[-- Type: text/x-diff, Size: 3945 bytes --]

From 0ffa1b39556c9f7b0b18a864080df2d18651ed1e Mon Sep 17 00:00:00 2001
From: Nikita Karetnikov <nikita@karetnikov.org>
Date: Sun, 3 Mar 2013 12:40:49 +0000
Subject: [PATCH] utils: Add 'wrap-program'.

* guix/build/utils.scm (wrap-program): New procedure.
---
 guix/build/utils.scm |   68 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 67 insertions(+), 1 deletions(-)

diff --git a/guix/build/utils.scm b/guix/build/utils.scm
index 6921e31..3395f02 100644
--- a/guix/build/utils.scm
+++ b/guix/build/utils.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2012, 2013 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -49,7 +50,8 @@
             patch-shebang
             patch-makefile-SHELL
             fold-port-matches
-            remove-store-references))
+            remove-store-references
+            wrap-program))
 
 \f
 ;;;
@@ -605,6 +607,70 @@ known as `nuke-refs' in Nixpkgs."
                              (put-u8 out (char->integer char))
                              result))))))
 
+(define* (wrap-program prog #:rest vars)
+  "Rename PROG to .PROG-real and make PROG a wrapper.  VARS should look like
+this:
+
+  '(VARIABLE DELIMITER POSITION LIST-OF-DIRECTORIES)
+
+where DELIMITER is optional.  ':' will be used if DELIMITER is not given.
+
+For example, this command:
+
+  (wrap-program \"foo\"
+                '(\"PATH\" \":\" = (\"/nix/.../bar/bin\"))
+                '(\"CERT_PATH\" suffix (\"/nix/.../baz/certs\"
+                                        \"/qux/certs\")))
+
+will copy 'foo' to '.foo-real' and create the file 'foo' with the following
+contents:
+
+  #!location/of/bin/bash
+  export PATH=\"/nix/.../bar/bin\"
+  export CERT_PATH=\"$CERT_PATH${CERT_PATH:+:}/nix/.../baz/certs:/qux/certs\"
+  exec location/of/.foo-real
+
+This is useful for scripts that expect particular programs to be in $PATH, for
+programs that expect particular shared libraries to be in $LD_LIBRARY_PATH, or
+modules in $GUILE_LOAD_PATH, etc."
+  (let ((prog-real (string-append "." prog "-real"))
+        (prog-tmp  (string-append "." prog "-tmp")))
+    (define (export-variable lst)
+      ;; Return a string that exports an environment variable.
+      (match lst
+        ((var sep '= rest)
+         (format #f "export ~a=\"~a\""
+                 var (string-join rest sep)))
+        ((var sep 'prefix rest)
+         (format #f "export ~a=\"~a${~a~a+~a}$~a\""
+                 var (string-join rest sep) var sep sep var))
+        ((var sep 'suffix rest)
+         (format #f "export ~a=\"$~a${~a~a+~a}~a\""
+                 var var var sep sep (string-join rest sep)))
+        ((var '= rest)
+         (format #f "export ~a=\"~a\""
+                 var (string-join rest ":")))
+        ((var 'prefix rest)
+         (format #f "export ~a=\"~a${~a:+:}$~a\""
+                 var (string-join rest ":") var var))
+        ((var 'suffix rest)
+         (format #f "export ~a=\"$~a${~a:+:}~a\""
+                 var var var (string-join rest ":")))))
+
+    (copy-file prog prog-real)
+
+    (with-output-to-file prog-tmp
+                         (lambda ()
+                           (format #t
+                                   "#!~a~%~a~%exec ~a~%"
+                                   (which "bash")
+                                   (string-join (map export-variable vars)
+                                                "\n")
+                                   (canonicalize-path prog-real))))
+
+    (chmod prog-tmp #o755)
+    (rename-file prog-tmp prog)))
+
 ;;; Local Variables:
 ;;; eval: (put 'call-with-output-file/atomic 'scheme-indent-function 1)
 ;;; eval: (put 'with-throw-handler 'scheme-indent-function 1)
-- 
1.7.5.4


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] utils: Add 'wrap-program'.

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>> I wouldn’t bother, though, because it will be used in contexts where
>> there’s no risk of ‘prog’ being used while we’re fiddling with it
>> (single-threaded, after ‘make install’).
>
> Better safe than sorry.  So I added '.PROG-tmp'.

Good.  :-)

> +    (copy-file prog prog-real)

You lack a (chmod prog-real #o755), I think.

> +    (with-output-to-file prog-tmp
> +                         (lambda ()

Opening bracket aligned with the ‘i’, please.  :-)

Other than that it looks perfect, and ready to be pushed.

Thanks!

Ludo’.

Re: [PATCH] utils: Add 'wrap-program'.

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

>> +    (copy-file prog prog-real)

> You lack a (chmod prog-real #o755), I think.

Are you sure?

If 'prog' is an executable, 'prog-real' will be an executable too.

scheme@(guile-user)> (open-file "foo" "w")
$1 = #<output: foo 9>
scheme@(guile-user)> (chmod "foo" #o755)
scheme@(guile-user)> (copy-file "foo" "bar")
scheme@(guile-user)> (format #f "~o" (stat:perms (stat "bar")))
$2 = "755"

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] utils: Add 'wrap-program'.

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>>> +    (copy-file prog prog-real)
>
>> You lack a (chmod prog-real #o755), I think.
>
> Are you sure?
>
> If 'prog' is an executable, 'prog-real' will be an executable too.
>
> scheme@(guile-user)> (open-file "foo" "w")
> $1 = #<output: foo 9>
> scheme@(guile-user)> (chmod "foo" #o755)
> scheme@(guile-user)> (copy-file "foo" "bar")
> scheme@(guile-user)> (format #f "~o" (stat:perms (stat "bar")))
> $2 = "755"

Oh you’re right (I thought permissions were not preserved.)

So you can go ahead.  Sorry for the noise!

Ludo’.

Re: Toward 0.2

[Andreas Enge]

Am Sonntag, 17. Februar 2013 schrieb Ludovic Courtès:
>   • Xorg & co. packaged (Andreas? :-));

Ok, I will have a look during the weekend.

Andreas

Re: Toward 0.2

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> I think we should plan for 0.2 within a month or so.  Here are the
> important features I’d like to see in:
>
>   • ‘guix pull’ (see TODO; I’ve started working on that);
>
>   • a simple “binary substituter”, which would allow pre-built binaries
>     to be transparently downloaded from trusted source, such as
>     hydra.gnu.org (I also have unfinished code locally);
>
>   • core updates
>
>     - install glibc locale data;
>     - install glibc timezone data;
>     - merge mips64 branch;
>     - items under “guix build utils” in TODO;
>     - core software updates;

So!  Of these items, the remaining blocker is “core software updates”,
that is upgrading GCC, Binutils, etc.  I’m happy to look into it, but
even happier if somebody else does.  :-)

I think mips64 support (and cross-compilation) will be for 0.3, right?

I’ll look into tzdata, but that should be simple.

Comments?

Ludo’.

Re: Toward 0.2

[Ludovic Courtès]

We’re getting there!  I propose releasing sometime between May 7th and
May 12th.

There’s room for improvement to the manual and website, please share
your ideas and patches!

Ludo’.

Re: Toward 0.2

[Ludovic Courtès]

Hello!

I’ve been doing some polishing (and adding a few packages), but I think
I’ll tag the new version tomorrow (Sunday 12th).

Ludo’.