unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Guillaume Le Vaillant <glv@posteo.net>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: Simon Streit <simon@netpanic.org>,
	54723@debbugs.gnu.org,
	Maxim Cournoyer <maxim.cournoyer@gmail.com>
Subject: [bug#54723] [PATCH] Check URI when verifying narinfo validity.
Date: Thu, 14 Apr 2022 12:18:51 +0000	[thread overview]
Message-ID: <87sfqfc0fq.fsf@kitej> (raw)
In-Reply-To: <87o816wt36.fsf@kitej>

[-- Attachment #1: Type: text/plain, Size: 4328 bytes --]

Guillaume Le Vaillant <glv@posteo.net> skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Woow.  How do you build and run ‘guix publish’?  Is it a distro package
>> or is it coming straight from Guix?  What command-line options are you
>> passing?
>>
>> I’ve never seen this, although we have it running on several servers,
>> notably ci.guix.  I wonder what could cause this.
>>
>> Thanks,
>> Ludo’.
>
> I'm using guix-publish-service-type in the operating-system definition
> to manage the "guix publish" server, using the on-the-fly mode and
> fast Zstandard compression:
>
> (service guix-publish-service-type
>          (guix-publish-configuration
>           (host "0.0.0.0")
>           (port 8080)
>           (compression '(("zstd" 3)))
>           (advertise? #t)))
>
>
> When booting the machine, shepherd starts the server with the following
> command-line options:
>
> /gnu/store/059svbd32i4s0l9s5i7z0krcnl666bjy-guix-1.3.0-24.2fb4304/libexec/guix/guile \ /gnu/store/059svbd32i4s0l9s5i7z0krcnl666bjy-guix-1.3.0-24.2fb4304/bin/guix publish -u guix-publish -p 8080 -C zstd:3 --nar-path=nar --listen=0.0.0.0 --advertise
>
> There's another report about this at <https://issues.guix.gnu.org/53668>
> I had forgotten about, where Simon Streit and Maxim Cournoyer indicate
> that they have seen this issue too.

I tested a few things, and it looks like there is a concurrency issue in
the 'http-write' function of the publish server.
After removing the 'call-with-new-thread' in the part of the function
sending narinfos, I can't reproduce the "broken pipe" error anymore.

--8<---------------cut here---------------start------------->8---
diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm
index 870dfc11e9..e9fad5845a 100644
--- a/guix/scripts/publish.scm
+++ b/guix/scripts/publish.scm
@@ -1002,28 +1002,25 @@ (define compression
            (begin
              (when (keep-alive? response)
                (keep-alive client))
-             (call-with-new-thread
-              (lambda ()
-                (set-thread-name "publish narinfo")
-                (let* ((narinfo
-                        (with-store store
-                          (narinfo-string store (utf8->string body)
-                                          #:nar-path nar-path
-                                          #:compressions compressions)))
-                       (narinfo-bv (string->bytevector narinfo "UTF-8"))
-                       (narinfo-length
-                        (bytevector-length narinfo-bv))
-                       (response (write-response
-                                  (with-content-length response
-                                                       narinfo-length)
-                                  client))
-                       (output (response-port response)))
-                  (configure-socket client)
-                  (put-bytevector output narinfo-bv)
-                  (force-output output)
-                  (unless (keep-alive? response)
-                    (close-port output))
-                  (values)))))
+             (let* ((narinfo
+                     (with-store store
+                       (narinfo-string store (utf8->string body)
+                                       #:nar-path nar-path
+                                       #:compressions compressions)))
+                    (narinfo-bv (string->bytevector narinfo "UTF-8"))
+                    (narinfo-length
+                     (bytevector-length narinfo-bv))
+                    (response (write-response
+                               (with-content-length response
+                                                    narinfo-length)
+                               client))
+                    (output (response-port response)))
+               (configure-socket client)
+               (put-bytevector output narinfo-bv)
+               (force-output output)
+               (unless (keep-alive? response)
+                 (close-port output))
+               (values)))
            (%http-write server client response body))))
     (_
      (match (assoc-ref (response-headers response) 'x-raw-file)
--8<---------------cut here---------------end--------------->8---

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

  reply	other threads:[~2022-04-14 12:31 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-05  9:58 [bug#54723] [PATCH] Check URI when verifying narinfo validity Guillaume Le Vaillant
2022-04-05 17:08 ` Ludovic Courtès
2022-04-05 17:51   ` Guillaume Le Vaillant
2022-04-09 20:32     ` Ludovic Courtès
2022-04-09 21:06       ` Guillaume Le Vaillant
2022-04-11 13:31         ` Guillaume Le Vaillant
2022-04-12  7:47           ` Ludovic Courtès
2022-04-12  8:54             ` Guillaume Le Vaillant
2022-04-14 12:18               ` Guillaume Le Vaillant [this message]
2022-04-18 19:39                 ` Ludovic Courtès
2022-04-20 14:10                   ` Guillaume Le Vaillant
2022-04-29 16:20                     ` Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87sfqfc0fq.fsf@kitej \
    --to=glv@posteo.net \
    --cc=54723@debbugs.gnu.org \
    --cc=ludo@gnu.org \
    --cc=maxim.cournoyer@gmail.com \
    --cc=simon@netpanic.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).