[bug#48277] [PATCH 0/1] New wrap-in-search-paths function

[bug#48277] [PATCH 0/1] New wrap-in-search-paths function

[Edouard Klein]

This patch adds the wrap-in-search-paths function.

This function takes an executable and a list of packages as arguments, and wrap the executable in the search-paths needed by the list of packages.

Two use-cases have pushed me to create this function, but I suspect it may be useful in other cases.

First, when running on a foreign distro, guix packages (especially python packages) can break the foreign distribution by putting Guix's python interpreter before the host's in the PATH. Scripts that rely on a #!/usr/bin/env python shebang then breaks. This for example breaks gdm on the latest Ubuntu when you install any package for which python is a propagated input.

This new function solves this problem by allowing one to write a G-exp that wraps the needed execs in the search paths for their packages, without putting them in the default profile, therefore avoiding masking the host's command.

A second use case is when defining operating-system-services, the system profile is not available to the environment where the command is launched, so if this command has any dynamically loaded part (as most executable today do), they won't be found despite being installed and present in the system profile.

This new function solves the problem by allowing one to wrap the service executable with an activation-service, to that the sheperd-service can launch it wihtout having to source the system profile.

See this thread on help-guix to see an example of the problem:
https://lists.gnu.org/archive/html/help-guix/2021-04/msg00100.html

Here is an example that can be built with guix build -f and demonstrate the use of the function. The (quite useless) resulting script will output the current version of flask, despite the flask binary not being in the current profile's PATH:


(use-modules (gnu packages python-web)
             (gnu packages bash)
             (guix gexp)
             (guix modules)
             (guix search-paths)
             (gnu packages guile)
             (gnu packages gnupg))


(with-extensions
 (list guile-zlib guile-gcrypt)
 (with-imported-modules
     (source-module-closure
      '((guix build utils)
        (guix search-paths)
        ))
   #~(begin
       (use-modules (guix build utils)
                    (guix search-paths))
       (mkdir-p (string-append #$output "/bin/"))
       (with-output-to-file (string-append #$output "/bin/flask-version")
         (lambda _
           (display (string-append "#!" #$bash "/bin/bash\n"))
           (display "flask --version\n")))
       (chmod  (string-append #$output "/bin/flask-version") #o755)
       (set-path-environment-variable "PATH" '("bin") (list #$bash))
       #$(wrap-in-search-paths #~(string-append #$output "/bin/flask-version") (list python-flask)))))


Edouard Klein (1):
  guix: search-paths: Add wrap-in-search-paths

 guix/search-paths.scm | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

-- 
2.31.1

[bug#48277] [PATCH 1/1] guix: search-paths: Add wrap-in-search-paths

[Edouard Klein]

From: edk@beaver-labs.com

---
 guix/search-paths.scm | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/guix/search-paths.scm b/guix/search-paths.scm
index 002e6342bb..34a632077c 100644
--- a/guix/search-paths.scm
+++ b/guix/search-paths.scm
@@ -18,6 +18,8 @@
 
 (define-module (guix search-paths)
   #:use-module (guix records)
+  #:use-module (guix profiles)
+  #:use-module (guix gexp)
   #:use-module (guix build utils)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
@@ -35,6 +37,8 @@
 
             search-path-specification->sexp
             sexp->search-path-specification
+            esps->wrap-sexp
+            wrap-in-search-paths
             string-tokenize*
             evaluate-search-paths
             environment-variable-definition
@@ -96,6 +100,37 @@ a <search-path-specification> object."
   (with-error-to-port (%make-void-port "w")
     (lambda () exp)))
 
+(define (esps->wrap-sexp esps)
+  "Return a list '(VARIABLE POSITION (STRING)) as expected by wrap-program, converted from the evaluated search-path-specification ESPS.
+
+     An evaluated search-path-specification is the type of things returned in a list by evaluate-search-paths: (sps . string) couples.
+
+     We do abuse wrap-program a bit, because it expects a list of directories, and the string we return is already a concatenation of the relevant directories. There would be no point in splitting it again and then having wrap-program joining it again, so we just pass it as is."
+  (match esps
+         ((sps . str) `(,(search-path-specification-variable sps) = (,str)))))
+
+(define (wrap-in-search-paths exec packages)
+  "Wrap EXEC in a script that will set the search paths to the values needed by the list of package PACKAGES."
+  (define (reconstruct-sps sps)
+    "Return a G-exp that evaluates, on the build strata, to the search-path-specification SPS."
+    #~(search-path-specification
+       (variable     #$(search-path-specification-variable sps))
+       (files        (list #$@(search-path-specification-files sps)))
+       (separator    #$(search-path-specification-separator sps))
+       (file-type    (quote #$(search-path-specification-file-type sps)))
+       (file-pattern #$(search-path-specification-file-pattern sps))))
+
+  (define (reconstruct-sps-list spsl)
+    "Return a G-exp that evaluates, on the build strata, to the list of search-path-specifications SPSL."
+    #~(list #$@(map reconstruct-sps spsl)))
+
+  (let ((manifest (packages->manifest packages)))
+    #~(apply wrap-program #$exec
+             (map esps->wrap-sexp
+                  (evaluate-search-paths
+                   #$(reconstruct-sps-list (manifest-search-paths manifest))
+                   (list #$@(map manifest-entry-item (manifest-transitive-entries manifest))))))))
+
 ;; XXX: This procedure used to be in (guix utils) but since we want to be able
 ;; to use (guix search-paths) on the build side, we want to avoid the
 ;; dependency on (guix utils), and so this procedure is back here for now.
-- 
2.31.1

[bug#48277] [PATCH 0/1] New wrap-in-search-paths function

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1140 bytes --]

Edouard Klein schreef op vr 07-05-2021 om 17:42 [+0200]:
> This patch adds the wrap-in-search-paths function [...].
> [... text about shepherd services, foreign distros, propagated-inputs,
>      gexps ...]

I don't see any obvious problems with the patch, though I haven't tested.

It would be easier to review if you modified one or two packages
and services in guix itself to use this wrap-in-search-paths procedure
though. (Preferably services with a system test in gnu/tests/*.scm.)
Otherwise, guix would have a procedure that is not called from anywhere
and isn't tested either, which is a bit of a hard sell.

Not sure how this helps with

> First, when running on a foreign distro, guix packages (especially python packages)
> can break the foreign distribution by putting Guix's python interpreter before the
> host's in the PATH. Scripts that rely on a #!/usr/bin/env python shebang then breaks.
> This for example breaks gdm on the latest Ubuntu when you install any package for
> which python is a propagated input.

, but this seems useful for shepherd services (as you mentioned).

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#48277] [PATCH 0/1] New wrap-in-search-paths function

[Edouard Klein]

Hi,

Thanks for reviewing the patch,

Good point about the tests. I'll try to send a follow-up patch on this
thread with the requested changes to a tested service.

Cheers,

Edouard.
Maxime Devos writes:

> Edouard Klein schreef op vr 07-05-2021 om 17:42 [+0200]:
>> This patch adds the wrap-in-search-paths function [...].
>> [... text about shepherd services, foreign distros, propagated-inputs,
>>      gexps ...]
>
> I don't see any obvious problems with the patch, though I haven't tested.
>
> It would be easier to review if you modified one or two packages
> and services in guix itself to use this wrap-in-search-paths procedure
> though. (Preferably services with a system test in gnu/tests/*.scm.)
> Otherwise, guix would have a procedure that is not called from anywhere
> and isn't tested either, which is a bit of a hard sell.
>
> Not sure how this helps with
>
>> First, when running on a foreign distro, guix packages (especially python packages)
>> can break the foreign distribution by putting Guix's python interpreter before the
>> host's in the PATH. Scripts that rely on a #!/usr/bin/env python shebang then breaks.
>> This for example breaks gdm on the latest Ubuntu when you install any package for
>> which python is a propagated input.
>
> , but this seems useful for shepherd services (as you mentioned).
>
> Greetings,
> Maxime.