Re: Utf8 error

[ludo@gnu.org (Ludovic Courtès)]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

Andreas Enge <andreas@enge.fr> skribis:

>  385: 2 [process-stderr #]
>  170: 1 [read-string #<input-output: socket 5>]
> In unknown file:
>    ?: 0 [utf8->string #vu8(115 97 109 112 108 101 95 114 97 116 101 95 105 

That’s because the build log contains a non-UTF-8 sequence, and
store.scm expects UTF-8 (for no good reason).

The attached patch removes that UTF-8 assumption.  Can you test whether
it fixes the problem?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2029 bytes --]

diff --git a/guix/store.scm b/guix/store.scm
index 668bc9a..560e567 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -175,6 +175,14 @@
         (get-bytevector-n p (- 8 m)))
     str))
 
+(define (read-latin1-string p)
+  (let* ((len (read-int p))
+         (m   (modulo len 8))
+         (str (get-string-n p len)))
+    (or (zero? m)
+        (get-bytevector-n p (- 8 m)))
+    str))
+
 (define (write-string-list l p)
   (write-int (length l) p)
   (for-each (cut write-string <> p) l))
@@ -362,7 +370,11 @@ operate, should the disk become full.  Return a server object."
   "Read standard output and standard error from SERVER, writing it to
 CURRENT-BUILD-OUTPUT-PORT.  Return #t when SERVER is done sending data, and
 #f otherwise; in the latter case, the caller should call `process-stderr'
-again until #t is returned or an error is raised."
+again until #t is returned or an error is raised.
+
+Since the build process's output cannot be assumed to be UTF-8, we
+conservatively consider it to be Latin-1, thereby avoiding possible
+encoding conversion errors."
   (define p
     (nix-server-socket server))
 
@@ -375,18 +387,18 @@ again until #t is returned or an error is raised."
 
   (let ((k (read-int p)))
     (cond ((= k %stderr-write)
-           (read-string p)
+           (read-latin1-string p)
            #f)
           ((= k %stderr-read)
            (let ((len (read-int p)))
-             (read-string p)                      ; FIXME: what to do?
+             (read-latin1-string p)               ; FIXME: what to do?
              #f))
           ((= k %stderr-next)
-           (let ((s (read-string p)))
+           (let ((s (read-latin1-string p)))
              (display s (current-build-output-port))
              #f))
           ((= k %stderr-error)
-           (let ((error  (read-string p))
+           (let ((error  (read-latin1-string p))
                  (status (if (>= (nix-server-minor-version server) 8)
                              (read-int p)
                              1)))

[-- Attachment #3: Type: text/plain, Size: 21 bytes --]


Thanks,
Ludo’.