From: "Ludovic Courtès" <ludo@gnu.org>
To: Guillaume Le Vaillant <glv@posteo.net>
Cc: Mathieu Othacehe <othacehe@gnu.org>, 54723@debbugs.gnu.org
Subject: [bug#54723] [PATCH] Check URI when verifying narinfo validity.
Date: Fri, 29 Apr 2022 18:20:14 +0200 [thread overview]
Message-ID: <87y1zn6ev5.fsf_-_@gnu.org> (raw)
In-Reply-To: <87v8v3ddz5.fsf@kitej> (Guillaume Le Vaillant's message of "Wed, 20 Apr 2022 14:10:43 +0000")
Hi,
(+Cc: Mathieu.)
Guillaume Le Vaillant <glv@posteo.net> skribis:
> With master at 101edbe63a887678722bc26cd85a7b7f5637428f, I reproduce the
> issue very often when trying to upgrade a profile with around 400
> packages. The logs of the publish server show hundreds of narinfo
> requests and tens of "broken pipe" errors.
>
> With an additional commit reverting
> f743f2046be2c5a338ab871ae8666d8f6de7440b, I can't reproduce the issue
> anymore. The logs still show hundreds of narinfo requests, but no
> errors.
For the record, a simple way to reproduce it is to start ‘guix publish’:
sudo -E ./pre-inst-env guix publish -u nobody -p 9999
and to spawn ‘guix weather’, for example with:
guix weather $(guix package -I. -p ~/.guix-home/profile |cut -f1) \
--substitute-urls=http://localhost:9999
which crashes with the dreaded backtrace:
--8<---------------cut here---------------start------------->8---
Backtrace:
11 (primitive-load "/home/ludo/.config/guix/current/bin/guix")
In guix/ui.scm:
2230:7 10 (run-guix . _)
2193:10 9 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 8 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In guix/scripts/weather.scm:
595:9 7 (_)
In guix/build/utils.scm:
677:23 6 (every* #<procedure 7f4af9621de0 at guix/scripts/weather.scm:595:17 (server)> _)
In guix/scripts/weather.scm:
597:21 5 (_ "http://localhost:9999")
120:17 4 (report-server-coverage _ ("/gnu/store/428bzp9325mfapyr4ywzwsz4ic7ssx2b-shepherd-0.9.0" "/gnu/store/sll9nfmqk8lkrraqbkyp3y…" …) …)
In unknown file:
3 (_ #<procedure 7f4af896ca40 at guix/scripts/weather.scm:201:2 ()> #<procedure list _> . #w())
In guix/substitutes.scm:
322:31 2 (lookup-narinfos "http://localhost:9999" _ #:open-connection _ #:make-progress-reporter _)
245:26 1 (fetch-narinfos _ _ #:open-connection _ #:make-progress-reporter _)
In ice-9/boot-9.scm:
1685:16 0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Wrong type (expecting exact integer): #f
--8<---------------cut here---------------end--------------->8---
I looked more closely at f743f2046be2c5a338ab871ae8666d8f6de7440b and I
think it had a logical flaw: if you pipeline 100 GET requests, then,
with this commit, ‘guix publish’ would spawn 100 threads that would all
reply concurrently (more or less). This is clearly wrong: replies
should be sent sequentially.
So in commit c1719a0adf3fa7611b56ca4d75b3ac8cf5c9c8ac I went ahead and
reverted it that commit. I also added a test that reproduces the issue
above.
Now, commit f743f2046be2c5a338ab871ae8666d8f6de7440b was itself an
attempt to fix a bug whereby ‘narinfo-string’ would take too long,
thereby preventing ‘guix publish’ from accepting connections (since it’s
single-threaded).
I think the only reasonable way to fix it is by using Fibers to make
‘guix publish’ concurrent (using a fiberized server like in Cuirass).
We should do that at some point. That’ll also allow us to remove some
of the hacks we’ve accumulated.
Now, the only way ‘narinfo-string’ can take too long these days is (I
think) if the store GC lock is held (we should check that hypothesis,
but I believe that if the GC lock is held, then the ‘query-path-info’
RPC made from ‘narinfo-string’ might block until the lock is released).
The GC lock is no longer held for hours on berlin, so there’s less
pressure to address that.
To summarize: I think ‘guix publish’ is okay as-is but we should
fiberize it sometime.
Thoughts?
Ludo’.
prev parent reply other threads:[~2022-04-29 16:32 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-05 9:58 [bug#54723] [PATCH] Check URI when verifying narinfo validity Guillaume Le Vaillant
2022-04-05 17:08 ` Ludovic Courtès
2022-04-05 17:51 ` Guillaume Le Vaillant
2022-04-09 20:32 ` Ludovic Courtès
2022-04-09 21:06 ` Guillaume Le Vaillant
2022-04-11 13:31 ` Guillaume Le Vaillant
2022-04-12 7:47 ` Ludovic Courtès
2022-04-12 8:54 ` Guillaume Le Vaillant
2022-04-14 12:18 ` Guillaume Le Vaillant
2022-04-18 19:39 ` Ludovic Courtès
2022-04-20 14:10 ` Guillaume Le Vaillant
2022-04-29 16:20 ` Ludovic Courtès [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y1zn6ev5.fsf_-_@gnu.org \
--to=ludo@gnu.org \
--cc=54723@debbugs.gnu.org \
--cc=glv@posteo.net \
--cc=othacehe@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).