From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ONxLJ14DfGHQYgEAgWs5BA (envelope-from ) for ; Fri, 29 Oct 2021 16:21:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id OLYWI14DfGFBEQAA1q6Kng (envelope-from ) for ; Fri, 29 Oct 2021 14:21:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6259C2C22B for ; Fri, 29 Oct 2021 16:21:18 +0200 (CEST) Received: from localhost ([::1]:41754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgSkv-0004j3-Il for larch@yhetil.org; Fri, 29 Oct 2021 10:21:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42732) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgSkH-00048s-Cs for guix-devel@gnu.org; Fri, 29 Oct 2021 10:20:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37366) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgSkG-0002na-Ty; Fri, 29 Oct 2021 10:20:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=iwTsWewDcO9xbe6FaPlTTJ9nKVSsXvEwW1mcHt25tvE=; b=bSVNqIIKT+NMVObWfY71 sq7WLF02aTDrnxCZc98MgpDtmBAcL7OaPRvpz0HTbWp0cSzLj1GORF9hTqjnIGDudqHrK4KLFz5ap mLce99jfS+jkJb0sTOuqVvwGzfqHLzil66ubpv2yP3W9pAmqZs1DKeaj4Q2P35XJnamZrq0M4HNzM 7pHTSh8ubp/xRl8s+BDGFq1RAquSP09Ng37PaVUUvtIkBH45/OD4JfitOKh7zZeEuD366FgoKOhj8 ENfo16+WOcucKSOajk4BJAnHEnfsR2apCDJ9be4cP3+FFKJk/fbydfyk2LbaFM9QTguzafBU04FpD bi/0dFxI1yZjKA==; Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=51480 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgSjx-0008DT-L5; Fri, 29 Oct 2021 10:20:28 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Timothy Sample Subject: Re: Preservation of Guix Report References: <87o87jjx54.fsf@ngyro.com> <87a6j2w1et.fsf@gnu.org> <87tuh9cfbu.fsf@ngyro.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 8 Brumaire an 230 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 29 Oct 2021 16:20:15 +0200 In-Reply-To: <87tuh9cfbu.fsf@ngyro.com> (Timothy Sample's message of "Fri, 22 Oct 2021 10:19:17 -0400") Message-ID: <87h7cz29r4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1635517278; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=iwTsWewDcO9xbe6FaPlTTJ9nKVSsXvEwW1mcHt25tvE=; b=g0fJK/9OYzLMpZbntTaGKk40DUamGMJw+APi957AVYpt6lpxedqK7yoOqQjVifurpPczSp dD8/lbNr95aIvnACBAThp3LvS59Mp575Raeo63CAbOL+9gmRgXK2P4usJl8t2yT+y2AYe4 Yq+b0p2aHV4GftRuUHCQXTiTajQhe3htDppgjSw5gz9Y4zL6TeTpoTuwJZjVFb8tkxc8S0 0MXrC7/LqwD9Y2yfu3argG7TSBq6P1jRD3GyowckktKlNWDv2g2e5/8WzLbk/MqgFQBXaT t6ysDnKzLmpZcmkn93Pdwkfs9qdU6tplc/fnqAy7ba0FJUPyp0kP8uGaDKjFjA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1635517278; a=rsa-sha256; cv=none; b=vC0gA1KEOs90cwhIuEsIuAxhJ0mEsm+2dL4TfS1i0nfxlpWy4Te9J3s0My5GMDHNua/na/ WT5hLdRFFu690uS4UKg6arVbxkceCcgFCeJ7E4yITHjzvt71Ohmb9Pt0r+YjGJemwIZPOX PG0jiOofSn2GrGKhsfThh6wRpY5p41BhwEEPyUilv99s9lqIO8B7AO+BVnhxnJVIe5p/F2 nbaVKrBIDTjvBHr+7eVrg0ccJz96tLm/r3/dN+l6230sHO9XQHSNHxe3XAYuJYadkSotnH YVgD+oC9W7vbZojgCztg7VBWmYJ9I0jENhvrNCX35kslJ2ZbSS1BHCXurj2yLA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gnu.org header.s=fencepost-gnu-org header.b=bSVNqIIK; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.12 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gnu.org header.s=fencepost-gnu-org header.b=bSVNqIIK; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 6259C2C22B X-Spam-Score: -3.12 X-Migadu-Scanner: scn0.migadu.com X-TUID: 03W6yMcWvsn6 Hello! Timothy Sample skribis: > Ludovic Court=C3=A8s writes: [...] >> This is truly awesome! (Did you manage to grab all that info with the >> default rate limit?!) > > Yes, but I have another trick. The =E2=80=9Cknown=E2=80=9D endpoint [1].= If you > already know the SWHIDs you want to check, you can check 1,000 per call. > With the anonymous rate limit, I can check 120,000 every hour, which is > plenty. > > [1] https://docs.softwareheritage.org/devel/swh-web/uri-scheme-api.html#g= et--api-1-content-known-(sha1)[,(sha1),%20...,(sha1)]- Oh, smart. >> Some of our refer to tags, not commits. How do you >> determine whether they=E2=80=99re saved? > > The short answer is =E2=80=9Celbow grease=E2=80=9D. Basically, I=E2=80= =99m taking a =E2=80=9Cwork > harder, not smarter=E2=80=9D approach. :p I go out and obtain the sourc= e, > verify it with Guix=E2=80=99s hash, and then compute the SWHID. This is = another > thing we could move to the CI infrastructure, but I think there might be > some hiccoughs. For git-references, I believe we can=E2=80=99t just comp= ute the > ID after the download derivation =E2=80=93 we would have to change the do= wnload > derivation itself. Maybe add an =E2=80=98swhid=E2=80=99 output? It=E2= =80=99s a little more > complicated than just throwing up some scripts, anyway. Just like we have =E2=80=98etc/disarchive-manifest.scm=E2=80=99, we could h= ave a thing that computes the SWHID of all the =E2=80=98git-fetch=E2=80=99 origins, for= instance, using the Disarchive code. Would that help? That would allow us to maintain a mapping from nar hash to swh:dir hash. >> =E2=80=98guix lint -c archival=E2=80=99 uses =E2=80=98lookup-origin-revi= sion=E2=80=99, which is a good >> approximation, but it=E2=80=99s not 100% reliable because tags can be mo= dified >> and that procedure only tells you that a same-named tag was found, not >> that it=E2=80=99s the commit you were expecting. (And really, we should= stop >> referring to tags.) > > Like zimoun said elsewhere in this thread, having an explicit mapping > from Guix hash to SHWID will improve reliability quite a bit. It=E2=80= =99s hard > to get to 100%, though! With the reports, we will eventually be able to > check everything. However, there=E2=80=99s still a small possibility of = bugs > and false positives. Ultimately, I=E2=80=99m hoping the reports will help > detect small problems (some specific source is missing) and guide our > efforts on big problems (xz support in Disarchive or support for more > version control systems, etc.). Definitely, thumbs up! Ludo=E2=80=99.