From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:c151::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id cM65JTAUWGKiRgEAgWs5BA (envelope-from ) for ; Thu, 14 Apr 2022 14:31:44 +0200 Received: from aspmx2.migadu.com ([2001:41d0:2:c151::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id qJ+PHjAUWGI1ZAAAG6o9tA (envelope-from ) for ; Thu, 14 Apr 2022 14:31:44 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx2.migadu.com (Postfix) with ESMTPS id D2BA428139 for ; Thu, 14 Apr 2022 14:31:40 +0200 (CEST) Received: from localhost ([::1]:58952 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1neydK-0006Sz-5G for larch@yhetil.org; Thu, 14 Apr 2022 08:31:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1neyco-0006RU-Qm for guix-patches@gnu.org; Thu, 14 Apr 2022 08:31:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:33422) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1neyco-0006tq-I9 for guix-patches@gnu.org; Thu, 14 Apr 2022 08:31:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1neyco-0001cf-EI for guix-patches@gnu.org; Thu, 14 Apr 2022 08:31:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#54723] [PATCH] Check URI when verifying narinfo validity. Resent-From: Guillaume Le Vaillant Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 14 Apr 2022 12:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54723 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Simon Streit , 54723@debbugs.gnu.org, Maxim Cournoyer Received: via spool by 54723-submit@debbugs.gnu.org id=B54723.16499394146179 (code B ref 54723); Thu, 14 Apr 2022 12:31:02 +0000 Received: (at 54723) by debbugs.gnu.org; 14 Apr 2022 12:30:14 +0000 Received: from localhost ([127.0.0.1]:55552 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1neyc1-0001bb-Qg for submit@debbugs.gnu.org; Thu, 14 Apr 2022 08:30:14 -0400 Received: from mout01.posteo.de ([185.67.36.65]:56173) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1neybz-0001aD-5U for 54723@debbugs.gnu.org; Thu, 14 Apr 2022 08:30:12 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id E68DC240028 for <54723@debbugs.gnu.org>; Thu, 14 Apr 2022 14:30:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1649939404; bh=ISuXq+Qj+Xx8yHd10e00hZulWYB/o1MFjTzZUlh/f50=; h=From:To:Cc:Subject:Date:From; b=l2y323fCU1TqALOslMr3QIEiPUP+jrFOSZxgcfOv9a/fHsNxPwfbRVUCylY4rIJqo 4Q8Gsn7rjvfcHwUZeaObcGrWA3QBXzEWubBI0yfRFkemGokmo9T5xL37BrHq6ByO24 X2wTfMLiv8QM9EVXZrdMcvUw9MyhaSITV4agryl/DopG52YVMM3Q67y8YAh8gXckva sSfAwVJKv0K+ZTI6JFoP39/hiBSqx2ogCO/W3IuWSpx7OY3FDU1SrVZLGFUq5m1hMD puIFaR6tWbhcp+mA1HuK6R35nP8OU6uZcapAzUdphIy/aWBS0R5yMNzYHo6pQdI2OC zmO/Mv55W+9pg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4KfJgt39MYz9rxW; Thu, 14 Apr 2022 14:30:02 +0200 (CEST) References: <87a6czbzvh.fsf@kitej> <877d83lapv.fsf@gnu.org> <875ynnbe65.fsf@kitej> <87pmlq6lqx.fsf_-_@gnu.org> <877d7ydjwk.fsf@kitej> <87wnfv90cl.fsf@kitej> <871qy2g2v3.fsf@gnu.org> <87o816wt36.fsf@kitej> From: Guillaume Le Vaillant Date: Thu, 14 Apr 2022 12:18:51 +0000 In-reply-to: <87o816wt36.fsf@kitej> Message-ID: <87sfqfc0fq.fsf@kitej> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1649939504; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=ZBoTj9k1m7W7ajTl4lLGzQExwZBFmGgDewSDmOv0XFQ=; b=g1WyyfUTEdauwsURJDio0ABmGtoCSqS/wL8Qv2u+1pqngbFqcU/MhjJWnWGYJQ55SUJ7YE Rk2WSD78SBH8hnBQDNuZoUWEXv6eRUf+oSrv/0D+F46naCMYdAIa+24IabvYLluiAOjPYc ZY3FpIlmGvVTNYol6cEzHHW8LFJgeaTLakVLGU1DIjR3YWBRIfwAun164FFak9rWSYCUwy nOnzdwMPDBdJQsFHOJ3d4eUZJcSE7nRos4WTNQPO553d/zHVuOgGapoRZOyn7zs03a/8hY QV4dOT2uZzp93hqnjWpuZUJWLM/GrI10t7/bLCdaLOqSRr1klGWNqGgTPq0nUA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1649939504; a=rsa-sha256; cv=none; b=Qr6stuKP7hIuDl7IVo+6EzWF85PtMQr1rSzMj3eXjNIQTNpfx4ozYNvBwoTmn5uIIyh7VI IkGWOZpXQLqdXoSb4RnxZzsVX+zmDZhO3DiFJwkzlpNw+eB0QwnetPg0PDFtlgVWJ6NI3n 8DmXjgxvSYBHjVmsH2icsaN7Au9bszoMDWPPUSGKJgo5DbAVDiqlzu2cVi6HX2Y4sZPxQp qVHIIQreIHCOMp1Sh4W9tr7RsT0Y9+YfERrYosUdHkj/fVp7tZwRMfBPX+DPh3FSVXwv2f N66/fH3/95MXffEqo+karjtAjbNsbVuNevDm2EWlwJYa01pzT4sDthr6MBJMwQ== ARC-Authentication-Results: i=1; aspmx2.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=l2y323fC; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none); spf=pass (aspmx2.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 5.65 Authentication-Results: aspmx2.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=l2y323fC; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none); spf=pass (aspmx2.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D2BA428139 X-Spam-Score: 5.65 X-Migadu-Scanner: scn1.migadu.com X-TUID: sf77xf2LmAw4 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Guillaume Le Vaillant skribis: > Ludovic Court=C3=A8s skribis: > >> Woow. How do you build and run =E2=80=98guix publish=E2=80=99? Is it a= distro package >> or is it coming straight from Guix? What command-line options are you >> passing? >> >> I=E2=80=99ve never seen this, although we have it running on several ser= vers, >> notably ci.guix. I wonder what could cause this. >> >> Thanks, >> Ludo=E2=80=99. > > I'm using guix-publish-service-type in the operating-system definition > to manage the "guix publish" server, using the on-the-fly mode and > fast Zstandard compression: > > (service guix-publish-service-type > (guix-publish-configuration > (host "0.0.0.0") > (port 8080) > (compression '(("zstd" 3))) > (advertise? #t))) > > > When booting the machine, shepherd starts the server with the following > command-line options: > > /gnu/store/059svbd32i4s0l9s5i7z0krcnl666bjy-guix-1.3.0-24.2fb4304/libexec= /guix/guile \ /gnu/store/059svbd32i4s0l9s5i7z0krcnl666bjy-guix-1.3.0-24.2fb= 4304/bin/guix publish -u guix-publish -p 8080 -C zstd:3 --nar-path=3Dnar --= listen=3D0.0.0.0 --advertise > > There's another report about this at > I had forgotten about, where Simon Streit and Maxim Cournoyer indicate > that they have seen this issue too. I tested a few things, and it looks like there is a concurrency issue in the 'http-write' function of the publish server. After removing the 'call-with-new-thread' in the part of the function sending narinfos, I can't reproduce the "broken pipe" error anymore. =2D-8<---------------cut here---------------start------------->8--- diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 870dfc11e9..e9fad5845a 100644 =2D-- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -1002,28 +1002,25 @@ (define compression (begin (when (keep-alive? response) (keep-alive client)) =2D (call-with-new-thread =2D (lambda () =2D (set-thread-name "publish narinfo") =2D (let* ((narinfo =2D (with-store store =2D (narinfo-string store (utf8->string body) =2D #:nar-path nar-path =2D #:compressions compressions))) =2D (narinfo-bv (string->bytevector narinfo "UTF-8")) =2D (narinfo-length =2D (bytevector-length narinfo-bv)) =2D (response (write-response =2D (with-content-length response =2D narinfo-length) =2D client)) =2D (output (response-port response))) =2D (configure-socket client) =2D (put-bytevector output narinfo-bv) =2D (force-output output) =2D (unless (keep-alive? response) =2D (close-port output)) =2D (values))))) + (let* ((narinfo + (with-store store + (narinfo-string store (utf8->string body) + #:nar-path nar-path + #:compressions compressions))) + (narinfo-bv (string->bytevector narinfo "UTF-8")) + (narinfo-length + (bytevector-length narinfo-bv)) + (response (write-response + (with-content-length response + narinfo-length) + client)) + (output (response-port response))) + (configure-socket client) + (put-bytevector output narinfo-bv) + (force-output output) + (unless (keep-alive? response) + (close-port output)) + (values))) (%http-write server client response body)))) (_ (match (assoc-ref (response-headers response) 'x-raw-file) =2D-8<---------------cut here---------------end--------------->8--- --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYlgTyQ8cZ2x2QHBvc3Rl by5uZXQACgkQa+ggit8h/j+hKAD/TUAKe/Vv8UWCHTciK8KcpJCI6VeHvl+liDxd aNdrMjgA/0X0tqnhOfDNHEdgGZokZyHDKBl2gn1ICTPE5V0AppFw =v6M3 -----END PGP SIGNATURE----- --=-=-=--