bug#73181: guix-daemon fails when invoking ‘guix authenticate’ on the Hurd

[Janneke Nieuwenhuizen <janneke@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 4539 bytes --]

Ludovic Courtès writes:

Hello!

> <janneke@gnu.org> skribis:
>
>> Anyway, using this patch 0001 it seems that suppressing the warnings
>> works, I no longer get
>>
>> "GC Warning: Repeated allocation of very large block (appr. size 112
>> KiB):\n\tMay lead to memory leak and poor performance\n"
>>
>>
>> but still get
>>
>> unexpected build daemon error: stoi
>
> Damnit.  Could you check with rpctrace what the daemon receives?
>
> I wonder if I misunderstood what the root cause is.

TL;DR: Found it, attached is a patch to fix it.

Today I spent some time looking into this again, instrumented both
daemons with the attached patch for local-store.cc; a "guix copy
root@childhurd hello" gives:

host:
--8<---------------cut here---------------start------------->8---
accepted connection from pid 21474, user janneke
00 nondigit: ` ' [32]
01 read until now: >>>0<<<
00 nondigit: `:' [58]
01 read until now: >>>430<<<
--8<---------------cut here---------------end--------------->8---

childhurd:
--8<---------------cut here---------------start------------->8---
5 operations
00 nondigit: `G' [71]
01 read until now: >>><<<
02 nondigit: `G' [71]
03 nondigit: >>>GC Warning: Repeated allocation of very large block (appr. size 112 KiB):<<<
0 operations
--8<---------------cut here---------------end--------------->8---

...you already knew that from the rpctrace log.

So, the problem is that our patch doesrn't disable the warnings after all.
The guile-launcher has

guile-launcher.c:
--8<---------------cut here---------------start------------->8---
#if defined __GNU__
  /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
     warnings that are annoying and interfere with communications between
     'guix-daemon' and 'guix authenticate':
     <https://issues.guix.gnu.org/73181>.  Silence them.  */
  std::cerr << "silencing libgc warnings" << std::endl;
  GC_set_warn_proc (no_warnings);
#endif
  ..
  scm_boot_guile (argc, argv, inner_main, 0);
--8<---------------cut here---------------end--------------->8---

and then guile's gc.c just undoes that

init.c:
--8<---------------cut here---------------start------------->8---
scm_i_init_guile (void *base)
{
  ..
  scm_init_gc ();		/* Requires hooks and `get_internal_run_time' */
--8<---------------cut here---------------end--------------->8---

gc.c:
--8<---------------cut here---------------start------------->8---
void
scm_init_gc ()
{
  ..
  GC_set_warn_proc (scm_gc_warn_proc);
--8<---------------cut here---------------end--------------->8---

Doh' So, attached is also a patch for Guix (that I made for Guile) that
fixed offloading again for me.  Weirdly, I had to use
(overload-threshold #f) because my childhurd never falls below 1.0:

--8<---------------cut here---------------start------------->8---
root@guixygnu ~# uptime
 7:45:28 PM  up 35 minutes,  0 users,  load averages: 1.00, 1.01, 1.00
--8<---------------cut here---------------end--------------->8---


--8<---------------cut here---------------start------------->8---
19:25:48 janneke@glimdal:~/src/guix/hurd-team
$ ./pre-inst-env guix build --system=i586-gnu -e '(@@ (gnu packages commencement) gnu-make-boot0)' --with-configure-flag=make-boot0=foo=bar
The following derivation will be built:
  /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv
process 4454 acquired build slot '/var/guix/offload/localhost:11022/0'
normalized load on machine 'localhost' is 1.00
building /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv...
guix offload: sending 41 store items (259 MiB) to 'localhost'...
[..]
@ build-started /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv - i586-gnu /var/log/guix/drvs/cp//1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv.gz 360
[..]
@ build-succeeded /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv -
retrieving 1 store item from 'localhost'...
importing file or directory '/gnu/store/j83d3mzcjw83lcbvyd8hrs6i6ymdqbmc-make-boot0-4.4.1'...
found valid signature for '/gnu/store/j83d3mzcjw83lcbvyd8hrs6i6ymdqbmc-make-boot0-4.4.1'
registering 1 items
done with offloaded '/gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv'
successfully built /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv
/gnu/store/j83d3mzcjw83lcbvyd8hrs6i6ymdqbmc-make-boot0-4.4.1
--8<---------------cut here---------------end--------------->8---

WDYT?

Greetings,
Janneke


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-REMOVEME-stoi-add-some-debugging.patch --]
[-- Type: text/x-patch, Size: 1639 bytes --]

From 9695d17df1538460aad11500f56071500d9e4c80 Mon Sep 17 00:00:00 2001
Message-ID: <9695d17df1538460aad11500f56071500d9e4c80.1733510219.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Mon, 18 Nov 2024 00:16:56 +0100
Subject: [PATCH] REMOVEME: stoi: add some debugging.

Change-Id: I114025c43cd2404a00dc65af7def06a312a31ca3
---
 nix/libstore/local-store.cc | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/nix/libstore/local-store.cc b/nix/libstore/local-store.cc
index 0883a4bbce..e98bf04392 100644
--- a/nix/libstore/local-store.cc
+++ b/nix/libstore/local-store.cc
@@ -1164,9 +1164,9 @@ static std::shared_ptr<Agent> authenticationAgent()
 static int readInteger(int fd)
 {
     string str;
+    char ch;
 
     while (1) {
-        char ch;
         ssize_t rd = read(fd, &ch, 1);
         if (rd == -1) {
             if (errno != EINTR)
@@ -1177,11 +1177,20 @@ static int readInteger(int fd)
 	    if (isdigit(ch)) {
 		str += ch;
 	    } else {
-		break;
+              std::cerr << "00 nondigit: `" << ch << "' [" << (int) ch << "]" << std::endl;
+              std::cerr << "01 read until now: >>>" << str << "<<<" << std::endl;
+              break;
 	    }
         }
     }
 
+    if (str.empty()) {
+      std::cerr << "02 nondigit: `" << ch << "' [" << (int) ch << "]" << std::endl;
+      str += ch;
+      str += readLine(fd);
+      std::cerr << "03 nondigit: >>>" << str << "<<<" << std::endl;
+      throw EndOfFile("unexpected non-digit reading an integer");
+    }
     return stoi(str);
 }
 

base-commit: 889c396cc5fd948d2c874c2d4a51115f82591964
-- 
2.46.0


[-- Attachment #3: 0001-gnu-guile-3.0-Silence-GC-warnings-on-the-Hurd.patch --]
[-- Type: text/x-patch, Size: 4782 bytes --]

From c555134310728360cb3b12472e8d487421d57a7d Mon Sep 17 00:00:00 2001
Message-ID: <c555134310728360cb3b12472e8d487421d57a7d.1733509997.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 6 Dec 2024 16:25:54 +0100
Subject: [PATCH] gnu: guile-3.0: Silence GC warnings on the Hurd.

* gnu/packages/patches/guile-hurd-silence-gc-warnings.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
* gnu/packages/guile.scm (guile-3.0): When building for the Hurd, use it in
new stage "patch-silence-gc-warnings".

Change-Id: I48f2641a162c3fab15655293e10c4aa2200d5843
---
 gnu/local.mk                                  |  1 +
 gnu/packages/guile.scm                        | 11 ++++
 .../guile-hurd-silence-gc-warnings.patch      | 58 +++++++++++++++++++
 3 files changed, 70 insertions(+)
 create mode 100644 gnu/packages/patches/guile-hurd-silence-gc-warnings.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index a7bd32453b..d848ceddc4 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1520,6 +1520,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/guile-fix-invalid-unicode-handling.patch \
   %D%/packages/patches/guile-gdbm-ffi-support-gdbm-1.14.patch	\
   %D%/packages/patches/guile-hurd-posix-spawn.patch		\
+  %D%/packages/patches/guile-hurd-silence-gc-warnings.patch	\
   %D%/packages/patches/guile-lzlib-hurd64.patch			\
   %D%/packages/patches/guile-present-coding.patch		\
   %D%/packages/patches/guile-rsvg-pkgconfig.patch		\
diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 69dff9211f..41fbd895ed 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -413,6 +413,17 @@ (define-public guile-3.0
                                (search-patch "guile-hurd-posix-spawn.patch")))
                           (invoke "patch" "--force" "-p1" "-i" patch))))
                    #~())
+            #$@(if (target-hurd?)
+                   #~((add-before 'build 'patch-silence-gc-warnings
+                        (lambda _
+                          ;; TODO: Move patch to 'source' on next rebuild
+                          ;; cycle.
+                          (define patch
+                            #$(local-file
+                               (search-patch
+                                "guile-hurd-silence-gc-warnings.patch")))
+                          (invoke "patch" "--force" "-p1" "-i" patch))))
+                   #~())
             #$@(if (system-hurd?)
                    #~((add-after 'unpack 'disable-popen.test-no-duplicate
                         ;; This test hangs on the Hurd.
diff --git a/gnu/packages/patches/guile-hurd-silence-gc-warnings.patch b/gnu/packages/patches/guile-hurd-silence-gc-warnings.patch
new file mode 100644
index 0000000000..a006d93361
--- /dev/null
+++ b/gnu/packages/patches/guile-hurd-silence-gc-warnings.patch
@@ -0,0 +1,58 @@
+From 1256fb0925c5ff7c94249f53e0fb47d1ec280b3f Mon Sep 17 00:00:00 2001
+From: Janneke Nieuwenhuizen <janneke@gnu.org>
+Date: Fri, 6 Dec 2024 16:18:13 +0100
+Subject: [PATCH] Silence GC warnings on the Hurd.
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
+warnings that are annoying and interfere with communications between
+'guix-daemon' and 'guix authenticate':
+<https://issues.guix.gnu.org/73181>.
+
+* libguile/gc.c (scm_gc_no_warnings)[__GNU__]: New procedure.
+(scm_init_gc)[__GNU__]: Use it to silence GC warnings.
+
+Co-authored-by: Ludovic Courtès <ludo@gnu.org>
+---
+ libguile/gc.c | 15 +++++++++++++++
+ 1 file changed, 15 insertions(+)
+
+diff --git a/libguile/gc.c b/libguile/gc.c
+index 3cbe43ec5..1ea4abcf8 100644
+--- a/libguile/gc.c
++++ b/libguile/gc.c
+@@ -84,6 +84,13 @@ int scm_debug_cells_gc_interval = 0;
+    garbage collection.  */
+ static SCM scm_protects;
+ 
++#if defined __GNU__
++#include <gc.h>
++static void
++scm_gc_no_warnings (char *message, GC_word arg)
++{
++}
++#endif
+ 
+ \f
+ 
+@@ -616,7 +623,15 @@ scm_init_gc ()
+   scm_c_hook_add (&scm_after_gc_c_hook, accumulate_gc_timer, NULL, 0);
+ 
+   GC_set_oom_fn (scm_oom_fn);
++#if __GNU__
++  /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
++     warnings that are annoying and interfere with communications between
++     'guix-daemon' and 'guix authenticate':
++     <https://issues.guix.gnu.org/73181>.  Silence them.  */
++  GC_set_warn_proc (scm_gc_no_warnings);
++#else
+   GC_set_warn_proc (scm_gc_warn_proc);
++#endif
+   GC_set_start_callback (run_before_gc_c_hook);
+ 
+ #include "gc.x"
+-- 
+2.46.0
+

base-commit: 84fa76db424df8962ee21315ffd680c083edbf00
-- 
2.46.0


[-- Attachment #4: Type: text/plain, Size: 164 bytes --]


-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com