[bug#30351] [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

[bug#30351] [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

Hello!

As a follow up to my previous patch to mcron2, this completes the wrap
phases and fix a small issue where the mcron modules were installed to
share/guile/site/2.0 instead of 2.2.

For those using dyndns services, a job like the following should now
work without having to propagate Guile or GnuTLS in your user/system
profile[0]:

--8<---------------cut here---------------start------------->8---
;;; Secrets defined as variables prefixed by "sec-".
(include "/abs/path/to/your/secrets.scm")

(define duckdns-job
  ;; Update personal domain IP every 5 minutes.
  #~(job '(next-minute (range 0 60 5))
	 (lambda ()
	   (use-modules (web client))
	   (http-get
	    (string-append
	     "https://www.duckdns.org/update?domains=apteryx&token="
	     #$sec-dyndns-token "&ip=")))))

(operating-system
 ...
 (services
  (cons*
   (mcron-service (list duckdns-job))
   ...
--8<---------------cut here---------------end--------------->8---

I had tested my previous patch as a user (where Guile and GnuTLS were
installed), so it was working, but after testing from the "real" herd
service (which runs as root), it was failing, not finding the (web
client) and GnuTLS modules.

The attached patch fixes that.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-mcron2-Correct-and-enhance-the-wrap-phase.patch --]
[-- Type: text/x-patch, Size: 4186 bytes --]

From 385343b1370d87e6104ebe2ef473bf2d1e31f2f2 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Sun, 4 Feb 2018 14:05:40 -0500
Subject: [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

* gnu/packages/guile.scm (mcron2)[inputs]: Rename "guile-2.2" to just "guile".
[phases]: Install mcron2 modules to guile/site/2.2 instead of guile/site/2.0.
Add Guile 2.2 and GnuTLS modules to the wrap phase.
---
 gnu/packages/guile.scm | 43 ++++++++++++++++++++++++++-----------------
 1 file changed, 26 insertions(+), 17 deletions(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 3bc18ffdd..37e6703fa 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -64,6 +64,7 @@
   #:use-module (gnu packages sdl)
   #:use-module (gnu packages maths)
   #:use-module (gnu packages image)
+  #:use-module (gnu packages tls)
   #:use-module (gnu packages version-control)
   #:use-module (gnu packages xdisorg)
   #:use-module (gnu packages xorg)
@@ -632,7 +633,8 @@ format is also supported.")
          ("texinfo" ,texinfo)
          ("help2man" ,help2man)))
       (inputs
-       `(("guile-2.2" ,guile-2.2)
+       `(("guile" ,guile-2.2)
+         ("gnutls" ,gnutls)             ;for the wrap-mcron phase
          ,@(srfi-1:alist-delete "guile" (package-inputs mcron))))
       (arguments
        `(#:modules ((ice-9 match) (ice-9 ftw)
@@ -645,6 +647,7 @@ format is also supported.")
            (add-after 'unpack 'use-guile-2.2
              (lambda _
                (substitute* "configure.ac"
+                 (("guile/site/2.0") "guile/site/2.2")
                  (("PKG_CHECK_MODULES\\(\\[GUILE\\],.*$")
                   "PKG_CHECK_MODULES([GUILE], [guile-2.2])\n"))
                #t))
@@ -652,22 +655,28 @@ format is also supported.")
              (lambda _
                (invoke "autoreconf" "-vfi")))
            (add-after 'install 'wrap-mcron
-             (lambda* (#:key outputs #:allow-other-keys)
-               ;; Wrap the 'mcron' command to refer to the right
-               ;; modules.
-               (let* ((out  (assoc-ref outputs "out"))
-                      (bin  (string-append out "/bin"))
-                      (site (string-append
-                             out "/share/guile/site")))
-                 (match (scandir site)
-                   (("." ".." version)
-                    (let ((modules (string-append site "/" version)))
-                      (wrap-program (string-append bin "/mcron")
-                        `("GUILE_LOAD_PATH" ":" prefix
-                          (,modules))
-                        `("GUILE_LOAD_COMPILED_PATH" ":" prefix
-                          (,modules)))
-                      #t))))))))))))
+             (lambda* (#:key inputs outputs #:allow-other-keys)
+               ;; Wrap the 'mcron' command to refer to the right modules. We
+               ;; also include Guile 2.2 modules and GnuTLS, so that Guile
+               ;; libraries can be used in mcron jobs without having to
+               ;; propagate those in a user profile.
+               (let* ((site-dir "/share/guile/site/2.2")
+                      (ccache-dir "/lib/guile/2.2/ccache")
+                      (mcron  (assoc-ref outputs "out"))
+                      (mcron-bin (string-append mcron "/bin/mcron"))
+                      (mcron-modules (string-append mcron site-dir))
+                      (guile (assoc-ref inputs "guile"))
+                      (guile-modules (string-append guile site-dir))
+                      (guile-ccache (string-append guile ccache-dir))
+                      (gnutls (assoc-ref inputs "gnutls"))
+                      (gnutls-modules (string-append gnutls site-dir))
+                      (gnutls-ccache (string-append gnutls ccache-dir)))
+                 (wrap-program mcron-bin
+                   `("GUILE_LOAD_PATH" ":" prefix
+                     (,mcron-modules ,guile-modules ,gnutls-modules))
+                   `("GUILE_LOAD_COMPILED_PATH" ":" prefix
+                     (,mcron-modules ,guile-ccache ,gnutls-ccache)))
+                 #t)))))))))
 
 (define-public guile-ics
   (package
-- 
2.16.0


[-- Attachment #3: Type: text/plain, Size: 252 bytes --]


Thank you,

Maxim

[0]  A note of caution: secrets managed that way are not that secret:
they are visible in the derived job script (world readable under
/gnu/store). You might want to prefer keeping your mcron jobs in your
$HOME for better security.

[bug#30351] [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

[ng0]

On Sun, 04 Feb 2018, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
> Hello!
>
> As a follow up to my previous patch to mcron2, this completes the wrap
> phases and fix a small issue where the mcron modules were installed to
> share/guile/site/2.0 instead of 2.2.

Looks promising, this could fix the issue I still have.
I reconfigured the system and it looks good to me. At least the
no errors are being displayed.

> For those using dyndns services, a job like the following should now
> work without having to propagate Guile or GnuTLS in your user/system
> profile[0]:
>
> --8<---------------cut here---------------start------------->8---
> ;;; Secrets defined as variables prefixed by "sec-".
> (include "/abs/path/to/your/secrets.scm")
>
> (define duckdns-job
>   ;; Update personal domain IP every 5 minutes.
>   #~(job '(next-minute (range 0 60 5))
> 	 (lambda ()
> 	   (use-modules (web client))
> 	   (http-get
> 	    (string-append
> 	     "https://www.duckdns.org/update?domains=apteryx&token="
> 	     #$sec-dyndns-token "&ip=")))))
>
> (operating-system
>  ...
>  (services
>   (cons*
>    (mcron-service (list duckdns-job))
>    ...
> --8<---------------cut here---------------end--------------->8---
>
> I had tested my previous patch as a user (where Guile and GnuTLS were
> installed), so it was working, but after testing from the "real" herd
> service (which runs as root), it was failing, not finding the (web
> client) and GnuTLS modules.
>
> The attached patch fixes that.
>
> From 385343b1370d87e6104ebe2ef473bf2d1e31f2f2 Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Sun, 4 Feb 2018 14:05:40 -0500
> Subject: [PATCH] gnu: mcron2: Correct and enhance the wrap phase.
>
> * gnu/packages/guile.scm (mcron2)[inputs]: Rename "guile-2.2" to just "guile".
> [phases]: Install mcron2 modules to guile/site/2.2 instead of guile/site/2.0.
> Add Guile 2.2 and GnuTLS modules to the wrap phase.
> ---
>  gnu/packages/guile.scm | 43 ++++++++++++++++++++++++++-----------------
>  1 file changed, 26 insertions(+), 17 deletions(-)
>
> diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
> index 3bc18ffdd..37e6703fa 100644
> --- a/gnu/packages/guile.scm
> +++ b/gnu/packages/guile.scm
> @@ -64,6 +64,7 @@
>    #:use-module (gnu packages sdl)
>    #:use-module (gnu packages maths)
>    #:use-module (gnu packages image)
> +  #:use-module (gnu packages tls)
>    #:use-module (gnu packages version-control)
>    #:use-module (gnu packages xdisorg)
>    #:use-module (gnu packages xorg)
> @@ -632,7 +633,8 @@ format is also supported.")
>           ("texinfo" ,texinfo)
>           ("help2man" ,help2man)))
>        (inputs
> -       `(("guile-2.2" ,guile-2.2)
> +       `(("guile" ,guile-2.2)
> +         ("gnutls" ,gnutls)             ;for the wrap-mcron phase
>           ,@(srfi-1:alist-delete "guile" (package-inputs mcron))))
>        (arguments
>         `(#:modules ((ice-9 match) (ice-9 ftw)
> @@ -645,6 +647,7 @@ format is also supported.")
>             (add-after 'unpack 'use-guile-2.2
>               (lambda _
>                 (substitute* "configure.ac"
> +                 (("guile/site/2.0") "guile/site/2.2")
>                   (("PKG_CHECK_MODULES\\(\\[GUILE\\],.*$")
>                    "PKG_CHECK_MODULES([GUILE], [guile-2.2])\n"))
>                 #t))
> @@ -652,22 +655,28 @@ format is also supported.")
>               (lambda _
>                 (invoke "autoreconf" "-vfi")))
>             (add-after 'install 'wrap-mcron
> -             (lambda* (#:key outputs #:allow-other-keys)
> -               ;; Wrap the 'mcron' command to refer to the right
> -               ;; modules.
> -               (let* ((out  (assoc-ref outputs "out"))
> -                      (bin  (string-append out "/bin"))
> -                      (site (string-append
> -                             out "/share/guile/site")))
> -                 (match (scandir site)
> -                   (("." ".." version)
> -                    (let ((modules (string-append site "/" version)))
> -                      (wrap-program (string-append bin "/mcron")
> -                        `("GUILE_LOAD_PATH" ":" prefix
> -                          (,modules))
> -                        `("GUILE_LOAD_COMPILED_PATH" ":" prefix
> -                          (,modules)))
> -                      #t))))))))))))
> +             (lambda* (#:key inputs outputs #:allow-other-keys)
> +               ;; Wrap the 'mcron' command to refer to the right modules. We
> +               ;; also include Guile 2.2 modules and GnuTLS, so that Guile
> +               ;; libraries can be used in mcron jobs without having to
> +               ;; propagate those in a user profile.
> +               (let* ((site-dir "/share/guile/site/2.2")
> +                      (ccache-dir "/lib/guile/2.2/ccache")
> +                      (mcron  (assoc-ref outputs "out"))
> +                      (mcron-bin (string-append mcron "/bin/mcron"))
> +                      (mcron-modules (string-append mcron site-dir))
> +                      (guile (assoc-ref inputs "guile"))
> +                      (guile-modules (string-append guile site-dir))
> +                      (guile-ccache (string-append guile ccache-dir))
> +                      (gnutls (assoc-ref inputs "gnutls"))
> +                      (gnutls-modules (string-append gnutls site-dir))
> +                      (gnutls-ccache (string-append gnutls ccache-dir)))
> +                 (wrap-program mcron-bin
> +                   `("GUILE_LOAD_PATH" ":" prefix
> +                     (,mcron-modules ,guile-modules ,gnutls-modules))
> +                   `("GUILE_LOAD_COMPILED_PATH" ":" prefix
> +                     (,mcron-modules ,guile-ccache ,gnutls-ccache)))
> +                 #t)))))))))
>  
>  (define-public guile-ics
>    (package

-- 
ng0 :: https://ea.n0.is
A88C8ADD129828D7EAC02E52E22F9BBFEE348588 :: https://ea.n0.is/keys/

[bug#30351] [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> As a follow up to my previous patch to mcron2, this completes the wrap
> phases and fix a small issue where the mcron modules were installed to
> share/guile/site/2.0 instead of 2.2.

Good catch.  I’ve committed that change separately.

BTW, I encourage you to submit changes to bug-mcron@gnu.org.  Mcron is
now maintained by Mathieu Lirzin, who is familiar with current Guile and
certainly interested in receiving improvements like this.

(Another thing that should be done would be install .go files in
$libdir/guile/2.2/site-ccache.)

> For those using dyndns services, a job like the following should now
> work without having to propagate Guile or GnuTLS in your user/system
> profile[0]:

[...]

>>From 385343b1370d87e6104ebe2ef473bf2d1e31f2f2 Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Sun, 4 Feb 2018 14:05:40 -0500
> Subject: [PATCH] gnu: mcron2: Correct and enhance the wrap phase.
>
> * gnu/packages/guile.scm (mcron2)[inputs]: Rename "guile-2.2" to just "guile".
> [phases]: Install mcron2 modules to guile/site/2.2 instead of guile/site/2.0.
> Add Guile 2.2 and GnuTLS modules to the wrap phase.

[...]

>             (add-after 'install 'wrap-mcron
> -             (lambda* (#:key outputs #:allow-other-keys)
> -               ;; Wrap the 'mcron' command to refer to the right
> -               ;; modules.
> -               (let* ((out  (assoc-ref outputs "out"))
> -                      (bin  (string-append out "/bin"))
> -                      (site (string-append
> -                             out "/share/guile/site")))
> -                 (match (scandir site)
> -                   (("." ".." version)
> -                    (let ((modules (string-append site "/" version)))
> -                      (wrap-program (string-append bin "/mcron")
> -                        `("GUILE_LOAD_PATH" ":" prefix
> -                          (,modules))
> -                        `("GUILE_LOAD_COMPILED_PATH" ":" prefix
> -                          (,modules)))
> -                      #t))))))))))))
> +             (lambda* (#:key inputs outputs #:allow-other-keys)
> +               ;; Wrap the 'mcron' command to refer to the right modules. We
> +               ;; also include Guile 2.2 modules and GnuTLS, so that Guile
> +               ;; libraries can be used in mcron jobs without having to
> +               ;; propagate those in a user profile.
> +               (let* ((site-dir "/share/guile/site/2.2")
> +                      (ccache-dir "/lib/guile/2.2/ccache")
> +                      (mcron  (assoc-ref outputs "out"))
> +                      (mcron-bin (string-append mcron "/bin/mcron"))
> +                      (mcron-modules (string-append mcron site-dir))
> +                      (guile (assoc-ref inputs "guile"))
> +                      (guile-modules (string-append guile site-dir))
> +                      (guile-ccache (string-append guile ccache-dir))
> +                      (gnutls (assoc-ref inputs "gnutls"))
> +                      (gnutls-modules (string-append gnutls site-dir))
> +                      (gnutls-ccache (string-append gnutls ccache-dir)))
> +                 (wrap-program mcron-bin
> +                   `("GUILE_LOAD_PATH" ":" prefix
> +                     (,mcron-modules ,guile-modules ,gnutls-modules))
> +                   `("GUILE_LOAD_COMPILED_PATH" ":" prefix
> +                     (,mcron-modules ,guile-ccache ,gnutls-ccache)))
> +                 #t)))))))))

A couple of issues here: the ‘scandir’ trick above allowed us to not
hard-code “2.2”.  I think it’d be nice to try to preserve such things;
it’ll be less pain down the road.

Second issue is about adding GnuTLS to the search path: it’s not the
right place for that.  What if someone wants Guile-JSON?  And Guile-Git?
And…  You get the idea.  :-)

Instead I’d suggest writing your mcron job along these lines:

  #~(begin
      (add-to-load-path (string-append #+gnutls "/share/guile/site/"
                                       (effective-version)))
      (use-modules (web client))
      …)

(See (guix download) for an example of this hack.)

The extra boilerplate is admittedly not great, so I’d like to add a
‘with-extensions’ or ‘with-imported-packages’ form that would be like
’with-imported-modules’ but for Guile “extensions” like GnuTLS.

Last thing: it’s not necessary to put Guile’s own module directories in
the search path.  They’re already there by default.

So overall I think I’m arguing for the status quo.  Would that work for
you?

Thanks,
Ludo’.

bug#30351: [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

[Maxim Cournoyer]

Hello!

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> As a follow up to my previous patch to mcron2, this completes the wrap
>> phases and fix a small issue where the mcron modules were installed to
>> share/guile/site/2.0 instead of 2.2.
>
> Good catch.  I’ve committed that change separately.
>
> BTW, I encourage you to submit changes to bug-mcron@gnu.org.  Mcron is
> now maintained by Mathieu Lirzin, who is familiar with current Guile and
> certainly interested in receiving improvements like this.
>
> (Another thing that should be done would be install .go files in
> $libdir/guile/2.2/site-ccache.)

It's already all fixed upstream! I verified myself. I guess we could
poke Mathieu for a new release ;).

>> For those using dyndns services, a job like the following should now
>> work without having to propagate Guile or GnuTLS in your user/system
>> profile[0]:
>
> [...]
>
>>>From 385343b1370d87e6104ebe2ef473bf2d1e31f2f2 Mon Sep 17 00:00:00 2001
>> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>> Date: Sun, 4 Feb 2018 14:05:40 -0500
>> Subject: [PATCH] gnu: mcron2: Correct and enhance the wrap phase.
>>
>> * gnu/packages/guile.scm (mcron2)[inputs]: Rename "guile-2.2" to just "guile".
>> [phases]: Install mcron2 modules to guile/site/2.2 instead of guile/site/2.0.
>> Add Guile 2.2 and GnuTLS modules to the wrap phase.
>
> [...]
>
>>             (add-after 'install 'wrap-mcron
>> -             (lambda* (#:key outputs #:allow-other-keys)
>> -               ;; Wrap the 'mcron' command to refer to the right
>> -               ;; modules.
>> -               (let* ((out  (assoc-ref outputs "out"))
>> -                      (bin  (string-append out "/bin"))
>> -                      (site (string-append
>> -                             out "/share/guile/site")))
>> -                 (match (scandir site)
>> -                   (("." ".." version)
>> -                    (let ((modules (string-append site "/" version)))
>> -                      (wrap-program (string-append bin "/mcron")
>> -                        `("GUILE_LOAD_PATH" ":" prefix
>> -                          (,modules))
>> -                        `("GUILE_LOAD_COMPILED_PATH" ":" prefix
>> -                          (,modules)))
>> -                      #t))))))))))))
>> +             (lambda* (#:key inputs outputs #:allow-other-keys)
>> +               ;; Wrap the 'mcron' command to refer to the right modules. We
>> +               ;; also include Guile 2.2 modules and GnuTLS, so that Guile
>> +               ;; libraries can be used in mcron jobs without having to
>> +               ;; propagate those in a user profile.
>> +               (let* ((site-dir "/share/guile/site/2.2")
>> +                      (ccache-dir "/lib/guile/2.2/ccache")
>> +                      (mcron  (assoc-ref outputs "out"))
>> +                      (mcron-bin (string-append mcron "/bin/mcron"))
>> +                      (mcron-modules (string-append mcron site-dir))
>> +                      (guile (assoc-ref inputs "guile"))
>> +                      (guile-modules (string-append guile site-dir))
>> +                      (guile-ccache (string-append guile ccache-dir))
>> +                      (gnutls (assoc-ref inputs "gnutls"))
>> +                      (gnutls-modules (string-append gnutls site-dir))
>> +                      (gnutls-ccache (string-append gnutls ccache-dir)))
>> +                 (wrap-program mcron-bin
>> +                   `("GUILE_LOAD_PATH" ":" prefix
>> +                     (,mcron-modules ,guile-modules ,gnutls-modules))
>> +                   `("GUILE_LOAD_COMPILED_PATH" ":" prefix
>> +                     (,mcron-modules ,guile-ccache ,gnutls-ccache)))
>> +                 #t)))))))))
>
> A couple of issues here: the ‘scandir’ trick above allowed us to not
> hard-code “2.2”.  I think it’d be nice to try to preserve such things;
> it’ll be less pain down the road.
>
> Second issue is about adding GnuTLS to the search path: it’s not the
> right place for that.  What if someone wants Guile-JSON?  And Guile-Git?
> And…  You get the idea.  :-)
>
> Instead I’d suggest writing your mcron job along these lines:
>
>   #~(begin
>       (add-to-load-path (string-append #+gnutls "/share/guile/site/"
>                                        (effective-version)))
>       (use-modules (web client))
>       …)
>
> (See (guix download) for an example of this hack.)

Thanks for sharing this idea; it makes sense. I hadn't thought about
manipulating the load-path at run time.

> The extra boilerplate is admittedly not great, so I’d like to add a
> ‘with-extensions’ or ‘with-imported-packages’ form that would be like
> ’with-imported-modules’ but for Guile “extensions” like GnuTLS.

I don't think it's that bad as it is. Any enhancements welcome, of
course :).

> Last thing: it’s not necessary to put Guile’s own module directories in
> the search path.  They’re already there by default.

Tried, and indeed it works without it. Thanks. I must have gotten
confused somewhere.

> So overall I think I’m arguing for the status quo.  Would that work for
> you?

I'm happy with status quo as well. Thanks for lending your sharp eyes to
this review!

Maxim

[bug#30351] [PATCH] gnu: mcron2: Correct and enhance the wrap phase.

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Hi Maxim,
>>
>> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>>
>>> As a follow up to my previous patch to mcron2, this completes the wrap
>>> phases and fix a small issue where the mcron modules were installed to
>>> share/guile/site/2.0 instead of 2.2.
>>
>> Good catch.  I’ve committed that change separately.
>>
>> BTW, I encourage you to submit changes to bug-mcron@gnu.org.  Mcron is
>> now maintained by Mathieu Lirzin, who is familiar with current Guile and
>> certainly interested in receiving improvements like this.
>>
>> (Another thing that should be done would be install .go files in
>> $libdir/guile/2.2/site-ccache.)
>
> It's already all fixed upstream! I verified myself. I guess we could
> poke Mathieu for a new release ;).

Definitely!  It’s great that Mathieu has taken care of all this.

>> So overall I think I’m arguing for the status quo.  Would that work for
>> you?
>
> I'm happy with status quo as well. Thanks for lending your sharp eyes to
> this review!

Heheh, you’re welcome.  I hope I’ll get around to adding that
‘with-extensions’ thing soonish, because ‘add-to-load-path’ is not as
convenient as we’d like.

Thanks,
Ludo’.