From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id gDMPJE2MG2Q9SQAASxT56A (envelope-from ) for ; Thu, 23 Mar 2023 00:16:29 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id KAUtI02MG2QUZwAAG6o9tA (envelope-from ) for ; Thu, 23 Mar 2023 00:16:29 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5CDD2C57E for ; Thu, 23 Mar 2023 00:16:28 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pf7Ae-0007ao-CK; Wed, 22 Mar 2023 18:43:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pf7Ad-0007aN-0M for guix-patches@gnu.org; Wed, 22 Mar 2023 18:43:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pf7Ac-0000XF-N2 for guix-patches@gnu.org; Wed, 22 Mar 2023 18:43:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pf7Ac-0006JG-LG for guix-patches@gnu.org; Wed, 22 Mar 2023 18:43:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020 Resent-From: Timothy Sample Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 22 Mar 2023 22:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43442 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43442@debbugs.gnu.org, zimoun Received: via spool by 43442-submit@debbugs.gnu.org id=B43442.167952497224215 (code B ref 43442); Wed, 22 Mar 2023 22:43:02 +0000 Received: (at 43442) by debbugs.gnu.org; 22 Mar 2023 22:42:52 +0000 Received: from localhost ([127.0.0.1]:36853 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pf7AR-0006IV-U2 for submit@debbugs.gnu.org; Wed, 22 Mar 2023 18:42:52 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:48509) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pf7AP-0006I8-PM for 43442@debbugs.gnu.org; Wed, 22 Mar 2023 18:42:50 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id B270B3200312; Wed, 22 Mar 2023 18:42:42 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 22 Mar 2023 18:42:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1679524962; x=1679611362; bh=xQPUc74W/vXs1TFmZwpQsdK+9S4EUQVp1Ch QFeFj6I0=; b=F8Mr+P5NDVfoc6UIaMw3IlRGYl92UXBSZc29ugQFSRFFMv/Y4Np e6z93oYcb/uXQjH5FKwsbm0mpLqCqKNXwYM83urWc5eAadFNcTlzqO65jDkyg1vA lduBR2gRlQcJM4nCWB6zS8/taBy2u+mzeGGgg/mdFlEMkNZCf1hq3YS/scFfS/GV PIbzscXCIV0uDVnOLvRw9QnM+KUhBK6iP9MYcgRfqTrtVnybGAxsyttpGVEGHxF1 /2zKURIM8ETDtMpXyrFAqqN4eCy6GzjutDPoZwi4Fy7gr3odqf++CaZ9fZCHjQQQ sHp2V/I0Gu5qax4CznXMx0cUzqIAHFY6hHg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdegvddgudeihecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufhfffgjkfgfgggtgfesthhqredttderjeenucfhrhhomhepvfhi mhhothhhhicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhghihrohdrtghomheqnecugg ftrfgrthhtvghrnhepudfhvedugfdvhfejlefffeejgfeiffethfdtkeetgfejueeghedu geeitddukedvnecuffhomhgrihhnpehsohhfthifrghrvghhvghrihhtrghgvgdrohhrgh dpihhnrhhirgdrfhhrnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghi lhhfrhhomhepshgrmhhplhgvthesnhhghihrohdrtghomh X-ME-Proxy: Feedback-ID: i4721425c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 22 Mar 2023 18:42:41 -0400 (EDT) From: Timothy Sample References: <20200916081411.5801-1-zimon.toutoune@gmail.com> <20200916081642.6716-1-zimon.toutoune@gmail.com> <87o8lyj090.fsf@gnu.org> <86zh5iol1n.fsf@gmail.com> <87h7rocvkt.fsf@gnu.org> <877dsi44lu.fsf@gnu.org> <87y2knhei3.fsf@gnu.org> <87jzzbms54.fsf_-_@gnu.org> Date: Wed, 22 Mar 2023 16:42:39 -0600 In-Reply-To: <87jzzbms54.fsf_-_@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Mon, 20 Mar 2023 15:09:11 +0100") Message-ID: <87jzz8xva8.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1679526989; a=rsa-sha256; cv=none; b=dN7Jt2hbIxdcmXPPg8m7dajH8BTz064ViDyAxoc2nlw9htWH7F1KnTku+mOQTGJ9p9aq/K sRgK0b2tgjDURVea+a7rzUf44Qgc/I6N4jbvlx7DlgpguP/5rmNP1UCugDgn11jZKvP9PN g34M+O6bhTePFb2nbSzvHZW/e++n6eA9HDg1eMok2/FuClGZu/ZmvqbbdneYKf971D2LES zE//RpEI9rvSzfLuTZA+vla+O+WHNYwqz16lnt/WdGr+nzf0YJpSR6vnRbOUId1Hy9D9Kh 0CUlqUJYx+lQMxYJiUQNy6I4CJg9RoEGKstFj7OOGoWInwdnu7Wm/LiQiWGWSQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b=F8Mr+P5N; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1679526989; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=xQPUc74W/vXs1TFmZwpQsdK+9S4EUQVp1ChQFeFj6I0=; b=RYK/R+fiBRZG+dqdzhWvJTIkNpZxPz/50Vqkj/y4XzxSYwV66sAJx158j2OwTWSLhDfbZR npSNGn51+XZb6vP+yzFc65DXjP7QCLD5VYhmG7a7cFTb7x0n/8UV95vJUl1G106qft3Om9 gWicVOiYoddTvucwsvPtmP/44dbRHIUmp9uiW+pnWXPkN5qU26RjITxOeicFyKzhgVeHj+ OOIWan2jWjYOejrnx695Kxovx9oN6ty9poZ/aDr8gldthDCyvpYUzT4aBkgeRtZjMGyJUV LhLc8xGK9DotyV3OtojL2RdtP1MnNFY7ZJlxI4RkktHPYIz31C5UyVMdqNDXUw== Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b=F8Mr+P5N; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -0.85 X-Spam-Score: -0.85 X-Migadu-Queue-Id: 5CDD2C57E X-TUID: xwQ82ufnI4YM Hello, Ludovic Court=C3=A8s writes: > This whole hack looks like a dead end. > > It would be ideal if SWH would compute nar hashes, as you proposed: > > https://gitlab.softwareheritage.org/swh/meta/-/issues/4538 > > As a stopgap, I wonder if we could use =E2=80=9Cdouble hashing=E2=80=9D o= n our side, but > only for svn: we=E2=80=99d store both the nar sha256 as we currently do, = plus > the swhid. It still seems to me that it=E2=80=99d be hard to scale and to > maintain that over time, even if it=E2=80=99s limited to svn. Plus, ther= e=E2=80=99d > still be the problem of =E2=80=98svn-multi-fetch=E2=80=99, which is what = most TeX=C2=A0Live > packages use. > > Thoughts? Not too many, but I do have more bad news. Apologies if this is already known, but I=E2=80=99m just getting up to speed with how SWH handles Subver= sion (for coverage checking) and thought this seemed pretty significant. I was starting with doing a simple check for the =E2=80=9Ceasy=E2=80=9D Sub= version repositories. That is, no externals (=E2=80=98recursive?=E2=80=99) and no =E2=80=98svn-multi-fetch=E2=80=99 [1]. I immediately hit a problem. Guix = hashes the export of the repository with the keywords processed, while SWH hashes it with unprocessed keywords. For example, take =E2=80=98libsmpeg=E2=80=99. It has a file called =E2=80= =9Cmkinstalldirs=E2=80=9D, which has a keyword in it: =E2=80=9C$Id$=E2=80=9D. The SWH loader hashes t= his as $Id$ while we hash it as $Id: mkinstalldirs 9 1999-10-21 15:55:01Z hercules $ This is not a big issue in terms of coverage checking, but it will be an issue for automatic recovery. Even if you know the exact SWH directory ID, you won=E2=80=99t get a directory that satisfies the daemon=E2=80=99s h= ash check. I have no idea how hard it is to process the keywords with only data from SWH. In this case, you would have to walk revisions to find the last time =E2=80=9Cmkinstalldirs=E2=80=9D was modified, and then format its meta= data. However, I assume the Subversion properties are gone, so there might be edge cases like a file with =E2=80=9C$Id$=E2=80=9D (or whatever) that Subve= rsion wouldn=E2=80=99t processes. Again, apologies if this is old news. Actually apologies either way, =E2=80=99cause this is a bit of a downer! -- Tim [1] More precisely, I was going to process recursive =E2=80=98svn-fetch=E2= =80=99 origins because a lot of them are needlessly marked as recursive. In some (many?) cases, the repositories don=E2=80=99t actually have external refere= nces, so the flag does nothing. I was only going to skip the ones where it makes a difference.