From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id YBdxNSAdYGIoSwEAbAwnHQ (envelope-from ) for ; Wed, 20 Apr 2022 16:48:00 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id iOk/NSAdYGKwTwAA9RJhRA (envelope-from ) for ; Wed, 20 Apr 2022 16:48:00 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9B0772EB7E for ; Wed, 20 Apr 2022 16:48:00 +0200 (CEST) Received: from localhost ([::1]:60150 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nhBcd-00040r-Iy for larch@yhetil.org; Wed, 20 Apr 2022 10:47:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53752) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nhBME-0001bS-Sa for guix-patches@gnu.org; Wed, 20 Apr 2022 10:31:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53106) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nhBME-0006oj-J1 for guix-patches@gnu.org; Wed, 20 Apr 2022 10:31:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nhBME-0005MB-Gk for guix-patches@gnu.org; Wed, 20 Apr 2022 10:31:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#54723] [PATCH] Check URI when verifying narinfo validity. Resent-From: Guillaume Le Vaillant Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 20 Apr 2022 14:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54723 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 54723@debbugs.gnu.org Received: via spool by 54723-submit@debbugs.gnu.org id=B54723.165046503320532 (code B ref 54723); Wed, 20 Apr 2022 14:31:02 +0000 Received: (at 54723) by debbugs.gnu.org; 20 Apr 2022 14:30:33 +0000 Received: from localhost ([127.0.0.1]:47001 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nhBLk-0005L4-FH for submit@debbugs.gnu.org; Wed, 20 Apr 2022 10:30:33 -0400 Received: from mout01.posteo.de ([185.67.36.65]:57771) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nhBLi-0005Kb-OC for 54723@debbugs.gnu.org; Wed, 20 Apr 2022 10:30:31 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id AF3D8240027 for <54723@debbugs.gnu.org>; Wed, 20 Apr 2022 16:30:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1650465024; bh=OxGcjX23VntZnZIc50Om1EKYnVHXH5wVldlIVaySpm8=; h=From:To:Cc:Subject:Date:From; b=k8j/j3x6rgvURiThTKXh4DzV+k3+a0nwT2v+IHfUH6vbtOdT5Qn6ovddvnGpvIh9x vUVI0wGi7inGWPgO9vQy77PUrGoVzziU1m2WonbzEWhvVehIksoEMTsfF9WDKEylFU jvo7BVunlpO1XSitP3JE9M+AAa86L/CZnchrUmSSf5MoVsln835Bkr8y4HIpCFwWcM RNiWvLwzxdnMgWW+p5WsuiYobMe5CqqgNmhbB4SeLNmtE0JG2ULIoTDUZuUC59gSLh EhRIDcxfyVQsmzP4r9dfwMNwAL6uX7ChXiwiKTXCjXJxVnMvlhW8Clp9EIoy/BwXly cuSQT7Kat9Ebg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Kk33z428nz9rxP; Wed, 20 Apr 2022 16:30:23 +0200 (CEST) References: <87a6czbzvh.fsf@kitej> <877d83lapv.fsf@gnu.org> <875ynnbe65.fsf@kitej> <87pmlq6lqx.fsf_-_@gnu.org> <877d7ydjwk.fsf@kitej> <87wnfv90cl.fsf@kitej> <871qy2g2v3.fsf@gnu.org> <87o816wt36.fsf@kitej> <87sfqfc0fq.fsf@kitej> <87bkwyxjtp.fsf@gnu.org> From: Guillaume Le Vaillant Date: Wed, 20 Apr 2022 14:10:43 +0000 In-reply-to: <87bkwyxjtp.fsf@gnu.org> Message-ID: <87v8v3ddz5.fsf@kitej> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1650466080; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=DiolULjP7GpppSx48WTqgjEwNx10scjuOyj+oxVJ32A=; b=VsnfwAyHZ8YJH2n3ReFS2d2EP6q788uV3urgUoSQV8bBp9k8P/7FIoGeJ2U9Ts1S89boL5 dDQUot72k735SpJqO3R2ZE99/NXBec8ls0vFE0XPohMeZP2OBTXW7xpDnvEZTLHiQEYj9U oobol60PBbBPAUW7fFZ0bGC/gbjC0hOjPurAZUmgJxMFBUV9F45lT+sz2vQNeY/vIF34HC Y5rDdAV2LLEMrcQ597hzEiWSrvOJtmplWoU0EGlQc2b1GrZVKX18dUXZWs9m+weETNKFkp jrebmZM58Zi4GIGTk4kSlNVtAXF3B8RXdxZ/DLmD1l2hF780B4r6+cfb7qAy0Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1650466080; a=rsa-sha256; cv=none; b=UE/t4pcuhGjmSK7lVl06LjHRKE0w4v4PpZn4wMakqS8YsQA0UYWo0vvnO5WBRWC3r1Uxcl VJOlhO603bgEncvt1Xb7bwmI1QGLR6dmMm2CEbCt8YlWQSTPyRQCgEu5FhC3Jyc9g1BC+o 1aEeW9sFT+d3IqfAoufPwdjVriRPOSLkKYmmsw/OjKRnlMH9StncsujGB9bE27vb+KQa91 dI4BzT+ZyM+7nZHbpcBJ8f12f38U+scdufoFBG+g0eh7nsAgc3QZnNzXhec4VYAxur7bZB Jnd2aZs0ZQk+QufFc9t4CeQEsXQq6mTjMA9DrZqGu0m6LAuSQQiPxqHz+Xi0YA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b="k8j/j3x6"; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 4.16 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b="k8j/j3x6"; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 9B0772EB7E X-Spam-Score: 4.16 X-Migadu-Scanner: scn1.migadu.com X-TUID: 3DhzFDdTumSg --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s skribis: > Hi! > > Guillaume Le Vaillant skribis: > >> I tested a few things, and it looks like there is a concurrency issue in >> the 'http-write' function of the publish server. >> After removing the 'call-with-new-thread' in the part of the function >> sending narinfos, I can't reproduce the "broken pipe" error anymore. > > Ah ha! Little has changed in this code, so I think the culprit must be > f743f2046be2c5a338ab871ae8666d8f6de7440b. > > Could you try running it from master with this patch reverted? > > You can spawn it from the build tree with: > > sudo -E ./pre-inst-env guix publish -u nobody > > Bonus points if you have a script to reproduce the issue. I guess that > involves making pipelined keep-alive narinfo requests the way =E2=80=98gu= ix > substitute=E2=80=99 and =E2=80=98guix weather=E2=80=99 do. > > Thanks, > Ludo=E2=80=99. Hi, With master at 101edbe63a887678722bc26cd85a7b7f5637428f, I reproduce the issue very often when trying to upgrade a profile with around 400 packages. The logs of the publish server show hundreds of narinfo requests and tens of "broken pipe" errors. With an additional commit reverting f743f2046be2c5a338ab871ae8666d8f6de7440b, I can't reproduce the issue anymore. The logs still show hundreds of narinfo requests, but no errors. I tried to make a script making many narinfo requests to reproduce the issue more easily, but I didn't get it to work yet. I thought accessing the 'fetch-narinfos' function from "guix/substitutes.scm" with: =2D-8<---------------cut here---------------start------------->8--- (use-modules (guix) (guix substitutes)) (define fetch-narinfos (@@ (guix substitutes) fetch-narinfos)) =2D-8<---------------cut here---------------end--------------->8--- would work, but it doesn't: =2D-8<---------------cut here---------------start------------->8--- $ guix repl fetch-narinfos.scm Backtrace: 4 (primitive-load "/home/guillaume/fetch-narinfos.scm") In ice-9/eval.scm: 626:19 3 (_ #) 223:20 2 (proc #) In unknown file: 1 (%resolve-variable (5 (guix substitutes) # . #f) #) In ice-9/boot-9.scm: 1685:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1685:16: In procedure raise-exception: erreur : fetch-narinfos : variable non li=C3=A9e =2D-8<---------------cut here---------------end--------------->8--- --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYmAY/g8cZ2x2QHBvc3Rl by5uZXQACgkQa+ggit8h/j+sLQEAk6QV8vHv5acWW6Q6qX3nw/gfMDMYXL/zpByM DoiM7vkA+wbpvXYmS4UcQrjoHzDfQZyHy2EOG4DPjb8vi1balYQB =vEZs -----END PGP SIGNATURE----- --=-=-=--