From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id iO6EGIMPLGS5bwEASxT56A (envelope-from ) for ; Tue, 04 Apr 2023 13:52:35 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id OPuRF4MPLGSyGAEAG6o9tA (envelope-from ) for ; Tue, 04 Apr 2023 13:52:35 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 26C0A16914 for ; Tue, 4 Apr 2023 13:52:34 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pjfCb-0006ek-Jz; Tue, 04 Apr 2023 07:51:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pjfCX-0006eL-TV for guix-devel@gnu.org; Tue, 04 Apr 2023 07:51:50 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pjfCT-000724-Bn for guix-devel@gnu.org; Tue, 04 Apr 2023 07:51:46 -0400 Received: by mail-wm1-x330.google.com with SMTP id n19so18845571wms.0 for ; Tue, 04 Apr 2023 04:51:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680609104; x=1683201104; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=vfLg4MX3qJgUty8NzWDKyvPofeSFcedg19gSZOy0nNY=; b=TxTRCbXa84AfmwJr5Q91Jx/s5LNZn+nHxt/ZJx4mF/8kgGP94tkWR5gDfJVd4hRrWm nMYTvHsb+5AjZU8Ts9x2FnEhe75DWtHLUtqAljY8otKWb5L47PuSz+g3wE3l2CyY91yK uw+vB7bdrHSXgFy6Kl1rVKp6DHNlsBDVMMTeIwy3h9m0YYoy/wOP6vibg+uPbgTVmXCs 8jZE+1dysuEAhqQbWqmntXRagjZ0s5txBlZ/aJYD4XmsjyXfWxmGxIzoCYYK8Pgs9TrO 6fFuOTNTncZNP392x581AljaBzQCXmQ8iX6R5ZL/dQkdeUPqjj/5/sqUv6d4JvwWP3RO 2b6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680609104; x=1683201104; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vfLg4MX3qJgUty8NzWDKyvPofeSFcedg19gSZOy0nNY=; b=t41l1RT7CP6ltWpRHoy2BG6yT77+tvRIa9tOi/lS55ivP/rHrIamfbDwaI8Shgk2Me JhVQ3IJFt18t15gMuCLgwFGXdpPevyMZZlfwxXmYiPkRSCnAcQlJ5J4exazBgv43IRiS 42t5Dczsra6kD75qMiDmGVXkeTqhgDdxK5NWAbA41HbePW/ToIpZpG9T0xuE+jf4g5iV EmrXOJs1DwgZFbEAj4KtAKUUcvbBkKSeyX62Yx7ld/sSdfg6v+ssr9EKYV7TyNx1E6Hm qE7EDEnSwMhtxALE/iQk2ZFxOMUuPpFAxelS+RCsr816cj17aTvSYkPntqONqwNtdAeG +lig== X-Gm-Message-State: AAQBX9exBtx44MPpUBGm9qr47gSHQf+oBMSwow7ldGfEK1Chln2Vn6KH hTssL8sa+LRQBQl3XAGT0ss= X-Google-Smtp-Source: AKy350Z16jlfm47Jy6FiNezDRScRlajrJaI09OH5aNW2BBh6vTqVdnpP4qX0Hhh2AdBEEz8EgByd9Q== X-Received: by 2002:a05:600c:1c82:b0:3ed:d2ae:9adb with SMTP id k2-20020a05600c1c8200b003edd2ae9adbmr2180618wms.0.1680609103723; Tue, 04 Apr 2023 04:51:43 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id fc11-20020a05600c524b00b003ebff290a52sm22289961wmb.28.2023.04.04.04.51.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Apr 2023 04:51:43 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , =?utf-8?Q?Bj=C3=B6rn_H=C3=B6fling?= Cc: guix-devel@gnu.org, Lars-Dominik Braun Subject: SWH: extend sources.json and Mercurial (or not Git and not tarball) In-Reply-To: <878rfwgbng.fsf@gnu.org> References: <20230312220021.22bfff4f@tangletp> <878rfwgbng.fsf@gnu.org> Date: Mon, 03 Apr 2023 23:42:26 +0200 Message-ID: <86lej8y75p.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=zimon.toutoune@gmail.com; helo=mail-wm1-x330.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_12_24=1.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1680609155; a=rsa-sha256; cv=none; b=oh2LnlLRT6H5vWzdil6IwBZz0dEtlwBy5YUxTHX+/iPt4U/Q5WlNrjZUM14VtbdI/CrBNg 8DAKfORqzc1fspAXutch+roWrNE05yNBAmRi1mC/wNilzmx4GSzJoBAoClb9g+hS1Gx4ov o6lbGBQ+yMjZWsQoKrUD8gEntMGcMTmrDKmdy+R82L2YqRz4ZMo1Bdbsw4glRzCCSYYt+m udTGCqlmLRS/vqUJvvBxIhk4m0xDbq9YOavRBaLLM+zC3hfIpUVbdx1vl/W0LHg1MVOukQ P5TsbBi7r0/z6mW2u8U23ety+darptXkZaU3fzFlUpEIDDa0VkNKvK2lsjyzlg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=TxTRCbXa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1680609155; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=vfLg4MX3qJgUty8NzWDKyvPofeSFcedg19gSZOy0nNY=; b=MOEyNBjR05BUYDSn6bM9GUIy5/LosRsvi3AhDh0goT7wGURpqTRegrpXLfLbsDo+XRbfxS ISHJLb2wS06dz9DcroSXxLB4iRmrfRCaWGB6rxJ/VNA0qYTDFjf0Wj49nsJdUkoWfR2sUC bnso64nWli51QE2kdWXFAAY5R0VaT8gVPk8QLhzfaKyKxanNqq40SPInmybEgJ5k48BBeN QSfWcz4OyDQru/RuTSoa+DGVhrgzWjszI2Xjo57GB7zXM/hvZPyyMKtbtTwLKUHFbe59Rg 9WTjfmpl2Ahx6o6oaj2jswg8pjgtWCIz9Qi14GeiUWgOoEu4vBi36h99VL2aKw== Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=TxTRCbXa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn0.migadu.com X-Migadu-Spam-Score: -6.45 X-Spam-Score: -6.45 X-Migadu-Queue-Id: 26C0A16914 X-TUID: 0vyVDZ4kgBZ0 Hi, On Thu, 16 Mar 2023 at 12:48, Ludovic Court=C3=A8s wrote: > 1. Reproducibility of past revisions. If we lose copies of the > auto-generated tarballs, then OpenJDK in past revisions of Guix is > irreparably lost. We should check whether/how to get them in > Disarchive + SWH. The file sources.json that SWH ingests only contains original upstream and not our copies. One step forward would be to also list the URL of our tarball substitutes as the last mirror in sources.json. Any taker? :-) > > 2. Mercurial/SWH bridge. While SWH has a one-to-one mapping with Git > (you can ask it for a specific Git commit ID), that=E2=80=99s not tr= ue for > hg. This is a more general problem, but as things are today, > there=E2=80=99s no automatic SWH fallback if the upstream hg server > vanishes. Since most git-fetch origins use label tags, the one-to-one mapping is not guarantee and we rely on SWH resolver using URL + label tag to get the content from SWH. For instance, if the label tag is changed in-place by upstream pointing then to one different commit, then SWH creates another snapshot but our fallback will fail (known issue: history of history, etc.) If we would have a list of identifiers instead of only NAR+SHA256, and we could have Git commit ID here (or SWHID or others), then it would ease the fallback machinery. SWH folk is currently adding NAR hashes; they store it as =E2=80=99ExtID=E2= =80=99 (see [1] and merge request [2]), but it is not clear yet how they would expose the API entry point or if they would do. Extending =E2=80=99origin=E2=80=99 with another optional field using other content-address keys would robustify the preservation of Guix. Yeah, indeed we could also build the X-to-SWH bridge with the Disarchive database (global bridge) but it would appear to me better to have some =E2=80=9Clocal=E2=80=9D origin-based bridge. 1: https://gitlab.softwareheritage.org/swh/meta/-/issues/4979 2: https://gitlab.softwareheritage.org/swh/devel/swh-loader-core/-/merge_re= quests/459 Cheers, simon