bug#47172: GNU Shepherd 0.8.1 fails on core-updates

bug#47172: GNU Shepherd 0.8.1 fails on core-updates

[Léo Le Bouter via Bug reports for GNU Guix]

[-- Attachment #1.1: Type: text/plain, Size: 418 bytes --]

Some tests fail:

FAIL: tests/no-home.sh
FAIL: tests/status-sexp.sh
PASS: tests/misbehaved-client.sh
FAIL: tests/replacement.sh
PASS: tests/file-creation-mask.sh
PASS: tests/restart.sh
PASS: tests/one-shot.sh
FAIL: tests/basic.sh
PASS: tests/respawn-throttling.sh
PASS: tests/signals.sh
PASS: tests/respawn.sh
PASS: tests/forking-service.sh
PASS: tests/pid-file.sh

Attached bzip2 compressed full log

[-- Attachment #1.2: 3qhfxagkqrx96xci8ylc4lj3s4zvd2-shepherd-0.8.1.drv.bz2 --]
[-- Type: application/x-bzip, Size: 7688 bytes --]

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#47172: GNU Shepherd 0.8.1 fails on core-updates

[Léo Le Bouter via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

It seems this is due to guile 3.0.5, GNU Shepherd 0.8.1 does not work
with it, it works with guile 3.0.2 however.

Thanks to Efraim on IRC for hints.

It would be great if people knowledgeable with Scheme, GNU Shepherd and
Guile could fix it, it blocks GNOME upgrade work on top of core-
updates.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#47172: Shepherd 0.8.1 tests fail on core-updates

[Ludovic Courtès]

Hi there,

Léo Le Bouter <lle-bout@zaclys.net> skribis:

> Some tests fail:
>
> FAIL: tests/no-home.sh
> FAIL: tests/status-sexp.sh
> PASS: tests/misbehaved-client.sh

[...]

> It seems this is due to guile 3.0.5, GNU Shepherd 0.8.1 does not work
> with it, it works with guile 3.0.2 however.

This turns out to be due to a… miscompilation bug.

In (shepherd scripts herd), ‘run-command’ has this code:

  (let ((sock    (open-connection socket-file))
        (action* (if (and (eq? action 'detailed-status)
                          (memq service '(root shepherd)))
                     'status
                     action)))
    …)

Problem is that everything works as if (eq? action 'detailed-status)
was omitted, such that ‘herd stop root’ is interpreted as ‘herd status
root’.

Simply wrapping the condition in (pk …) “fixes” the problem.

The peval output looks good (it contains the 'detailed-status
comparison), but the assembly seems to lack the 'detailed-status
comparison altogether:

--8<---------------cut here---------------start------------->8---
Disassembly of <unnamed function> at #x29e0:

   0    (instrument-entry 15700)                              at shepherd/scripts/herd.scm:127:2
   2    (assert-nargs-ee/locals 1 11)   ;; 12 slots (0 args)
   3    (static-ref 10 15369)           ;; #f                 at shepherd/scripts/herd.scm:128:19
   5    (immediate-tag=? 10 7 0)        ;; heap-object?
   7    (je 9)                          ;; -> L1
   8    (static-ref 10 14166)           ;; #f
  10    (static-ref 9 15372)            ;; open-connection
  12    (call-scm<-scm-scm 10 10 9 111) 
  14    (static-set! 10 15358)          ;; #f
L1:
  16    (scm-ref/immediate 7 10 1)      
  17    (scm-ref/immediate 6 11 2)      
  18    (handle-interrupts)                                   at shepherd/scripts/herd.scm:128:18
  19    (call 4 2)                      
  21    (receive 1 4 12)                
  23    (scm-ref/immediate 9 11 3)      
  24    (static-ref 8 15360)            ;; #f                 at shepherd/scripts/herd.scm:134:6
  26    (immediate-tag=? 8 7 0)         ;; heap-object?
  28    (je 9)                          ;; -> L2
  29    (static-ref 8 14145)            ;; #f
  31    (static-ref 7 15363)            ;; write-command
  33    (call-scm<-scm-scm 8 8 7 111)   
  35    (static-set! 8 15349)           ;; #f
L2:
  37    (scm-ref/immediate 8 8 1)       
  38    (static-ref 7 15358)            ;; #f                 at shepherd/scripts/herd.scm:134:21
  40    (immediate-tag=? 7 7 0)         ;; heap-object?
  42    (je 9)                          ;; -> L3
  43    (static-ref 7 14131)            ;; #f
  45    (static-ref 6 15361)            ;; shepherd-command
  47    (call-scm<-scm-scm 7 7 6 111)   
  49    (static-set! 7 15347)           ;; #f
L3:
  51    (scm-ref/immediate 7 7 1)       
  52    (scm-ref/immediate 6 11 4)      
  53    (static-ref 5 15363)            ;; root
  55    (eq? 6 5)                       
  56    (je 5)                          ;; -> L4
  57    (static-ref 5 13655)            ;; shepherd
  59    (eq? 6 5)                       
  60    (jne 3)                         ;; -> L5
L4:
  61    (static-ref 9 15365)            ;; status             at shepherd/scripts/herd.scm:131:22
L5:
  63    (static-ref 1 15375)            ;; #:arguments        at shepherd/scripts/herd.scm:134:54
  65    (scm-ref/immediate 0 11 5)      
  66    (mov 4 7)                                             at shepherd/scripts/herd.scm:134:20
--8<---------------cut here---------------end--------------->8---

(This is compiled with 3.0.7 and the default optimizations, so -O2.)

To be continued…

Ludo’.

bug#47172: Shepherd 0.8.1 tests fail on core-updates

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

Ludovic Courtès <ludo@gnu.org> skribis:

> This turns out to be due to a… miscompilation bug.
>
> In (shepherd scripts herd), ‘run-command’ has this code:
>
>   (let ((sock    (open-connection socket-file))
>         (action* (if (and (eq? action 'detailed-status)
>                           (memq service '(root shepherd)))
>                      'status
>                      action)))
>     …)
>
> Problem is that everything works as if (eq? action 'detailed-status)
> was omitted, such that ‘herd stop root’ is interpreted as ‘herd status
> root’.

A workaround that works with 3.0.7 is swapping the two ‘and’
sub-expressions:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 664 bytes --]

diff --git a/modules/shepherd/scripts/herd.scm b/modules/shepherd/scripts/herd.scm
index 106de1e..39d2e34 100644
--- a/modules/shepherd/scripts/herd.scm
+++ b/modules/shepherd/scripts/herd.scm
@@ -126,8 +126,8 @@ of pairs."
 the daemon via SOCKET-FILE."
   (with-system-error-handling
    (let ((sock    (open-connection socket-file))
-         (action* (if (and (eq? action 'detailed-status)
-                           (memq service '(root shepherd)))
+         (action* (if (and (memq service '(root shepherd))
+                           (eq? action 'detailed-status))
                       'status
                       action)))
      ;; Send the command.

[-- Attachment #3: Type: text/plain, Size: 12 bytes --]


Ludo’.

bug#47172: Shepherd 0.8.1 tests fail on core-updates

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1652 bytes --]

Ludovic Courtès <ludo@gnu.org> skriver:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> This turns out to be due to a… miscompilation bug.
>>
>> In (shepherd scripts herd), ‘run-command’ has this code:
>>
>>   (let ((sock    (open-connection socket-file))
>>         (action* (if (and (eq? action 'detailed-status)
>>                           (memq service '(root shepherd)))
>>                      'status
>>                      action)))
>>     …)
>>
>> Problem is that everything works as if (eq? action 'detailed-status)
>> was omitted, such that ‘herd stop root’ is interpreted as ‘herd status
>> root’.
>
> A workaround that works with 3.0.7 is swapping the two ‘and’
> sub-expressions:
>
> diff --git a/modules/shepherd/scripts/herd.scm b/modules/shepherd/scripts/herd.scm
> index 106de1e..39d2e34 100644
> --- a/modules/shepherd/scripts/herd.scm
> +++ b/modules/shepherd/scripts/herd.scm
> @@ -126,8 +126,8 @@ of pairs."
>  the daemon via SOCKET-FILE."
>    (with-system-error-handling
>     (let ((sock    (open-connection socket-file))
> -         (action* (if (and (eq? action 'detailed-status)
> -                           (memq service '(root shepherd)))
> +         (action* (if (and (memq service '(root shepherd))
> +                           (eq? action 'detailed-status))
>                        'status
>                        action)))
>       ;; Send the command.

Cc'ing the relevant Guile bug:

  https://bugs.gnu.org/48368

See also commit 79be6a985799adc6d663890250f4fb7c12f015b4 on
'core-updates' that builds with -O1 as a less satisfactory workaround.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#47172: Shepherd 0.8.1 tests fail on core-updates

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

Hello,

Marius Bakke <marius@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skriver:

[...]

>> A workaround that works with 3.0.7 is swapping the two ‘and’
>> sub-expressions:
>>
>> diff --git a/modules/shepherd/scripts/herd.scm b/modules/shepherd/scripts/herd.scm
>> index 106de1e..39d2e34 100644
>> --- a/modules/shepherd/scripts/herd.scm
>> +++ b/modules/shepherd/scripts/herd.scm
>> @@ -126,8 +126,8 @@ of pairs."
>>  the daemon via SOCKET-FILE."
>>    (with-system-error-handling
>>     (let ((sock    (open-connection socket-file))
>> -         (action* (if (and (eq? action 'detailed-status)
>> -                           (memq service '(root shepherd)))
>> +         (action* (if (and (memq service '(root shepherd))
>> +                           (eq? action 'detailed-status))
>>                        'status
>>                        action)))
>>       ;; Send the command.
>
> Cc'ing the relevant Guile bug:
>
>   https://bugs.gnu.org/48368

Oh nice!  (It would have saved me a bit of time to catch up on email
beforehand.  :-))

> See also commit 79be6a985799adc6d663890250f4fb7c12f015b4 on
> 'core-updates' that builds with -O1 as a less satisfactory workaround.

I found that ‘-O2 -Ono-resolve-primitives’ also does the trick.

If we manually replace ‘memq’ by two ‘eq?’ tests (which is what the
compiler does), the same problem is exhibited:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 643 bytes --]

diff --git a/modules/shepherd/scripts/herd.scm b/modules/shepherd/scripts/herd.scm
index 106de1e..513508f 100644
--- a/modules/shepherd/scripts/herd.scm
+++ b/modules/shepherd/scripts/herd.scm
@@ -127,7 +127,8 @@ the daemon via SOCKET-FILE."
   (with-system-error-handling
    (let ((sock    (open-connection socket-file))
          (action* (if (and (eq? action 'detailed-status)
-                           (memq service '(root shepherd)))
+                           (or (eq? service 'root)
+                               (eq? service 'shepherd)))
                       'status
                       action)))
      ;; Send the command.

[-- Attachment #3: Type: text/plain, Size: 192 bytes --]


‘-Ono-resolve-primitives’ also helps in this case.

‘-Ono-optimize-branch-chains’ has no effect.

So, not much progress, but at least we have a workaround.

Thanks,
Ludo’.

bug#47172: Shepherd 0.8.1 tests fail on core-updates

[Ludovic Courtès]

Hi,

Marius Bakke <marius@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skriver:

[...]

>> A workaround that works with 3.0.7 is swapping the two ‘and’
>> sub-expressions:
>>
>> diff --git a/modules/shepherd/scripts/herd.scm b/modules/shepherd/scripts/herd.scm
>> index 106de1e..39d2e34 100644
>> --- a/modules/shepherd/scripts/herd.scm
>> +++ b/modules/shepherd/scripts/herd.scm
>> @@ -126,8 +126,8 @@ of pairs."
>>  the daemon via SOCKET-FILE."
>>    (with-system-error-handling
>>     (let ((sock    (open-connection socket-file))
>> -         (action* (if (and (eq? action 'detailed-status)
>> -                           (memq service '(root shepherd)))
>> +         (action* (if (and (memq service '(root shepherd))
>> +                           (eq? action 'detailed-status))
>>                        'status
>>                        action)))
>>       ;; Send the command.
>
> Cc'ing the relevant Guile bug:
>
>   https://bugs.gnu.org/48368
>
> See also commit 79be6a985799adc6d663890250f4fb7c12f015b4 on
> 'core-updates' that builds with -O1 as a less satisfactory workaround.

The bug has been fixed in Guile (will be in 3.0.8), worked around in
Shepherd commit a066c5ac05037a6ffad8e4ea3e8de8150869aa8b, and worked
around in Guix on ‘core-updates’, so I think we can close it now.

Thanks,
Ludo’.