From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id oPfqGKY5uWXB5gAAqHPOHw:P1 (envelope-from ) for ; Tue, 30 Jan 2024 19:02:14 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id oPfqGKY5uWXB5gAAqHPOHw (envelope-from ) for ; Tue, 30 Jan 2024 19:02:14 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=ngyro.com header.s=fm1 header.b=5XpEQtiW; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="c lmJN8V"; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1706637734; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=KeSArjw08VHNeyqM2i//aq5JT3ZGxQkfqASoOjgy9XY=; b=O+ver9Q2zPsB/Axv5WiRxaVfh7AM9nkmy4ZnCKBupzpCxHpr28oCfTVxoulKBWVrz8GVh/ 12fjAeNlNll90/dSFeL/7GHg8OY0VFVjB5Dnbf/7k7j8tfwvml4K1AeQSWldi5ByNCjiYg CX92DZkEPQ0cfFhpk//g9nCO7TooK/LCvzBKRsV6bDA6DlVkB2sQB5KQPUURx9ApmnbJx3 Rsv0/5rRbDhFpJFilzHAY35XVj8hEpzlAWG4sjZVty8rh5cI3ORNMSDr8Vv4YVG8NDg/TL 7v9H89BQW3wfG2OjM8QHloouWA8f+GInRzIhMr4nFNnaq+3clnHMLPN2Io4aaQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=ngyro.com header.s=fm1 header.b=5XpEQtiW; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="c lmJN8V"; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1706637734; a=rsa-sha256; cv=none; b=pXF+29bUJA3y4dcTfk+g/H/kOs6eoQfDJJnlPHWQkFRwPuGqyExNvHn17S95wnU2KfQR/R IxB81a9L4jGQnWUbkkVG1/RB7fISUEMNGfYDWeBkgivWFa7GkkjwFP0X+BsUprFiRyXUsm qqcS/ey1a/QZpZj1e8m9JKWbXdIMfGxzEJajpReCL2g5xMkAldKgVWpS/RKfGNNubKaaAV YZesPGUdgxChe/iMKdPBncY5mbfH2nxahAJh3PpizYLa/L/RkiziqvmNZNxQo+vO8e1EZw Kan4RXW2auTakIqh6VCtlZaINORI4ShESiyVRNc8wTqTeuZv9kO8w4E2zpRu8w== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3CA531D28F for ; Tue, 30 Jan 2024 19:02:14 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rUsQR-00065x-EY; Tue, 30 Jan 2024 13:01:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rUsQO-00065b-NU for guix-devel@gnu.org; Tue, 30 Jan 2024 13:01:32 -0500 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rUsQM-0004sE-AI for guix-devel@gnu.org; Tue, 30 Jan 2024 13:01:32 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 74C673200BCE; Tue, 30 Jan 2024 13:01:26 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 30 Jan 2024 13:01:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ngyro.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1706637685; x=1706724085; bh=KeSArjw08VHNeyqM2i//aq5JT3ZGxQkfqASoOjgy9XY=; b= 5XpEQtiWd6vl3aUdDASTQ8+wdJZFXD7WUzvlhPAnpXG+zGXCsoztqL4tTUkofq+w lQyoNYKEIH5tTpJjYe8QSVURUUaQXq8KCBAgWEyD0xIVTaMu2bqMdNp7RkSwLaOn JRkWjKHazdGVdEyflU5aKa+H+vPT+olxD5kzzKgXqrO1ZZ0XoY5r3Z4N4VGl8YwY S5BOPOPFmKGZdVcnp7Mv9qNeZLM8vqB5acJJ2Y/vnXboCAaRwtGPVA1gAx8tGCcd sPd7QQkH8x16UgV40RvlnDsUUi1h9gGXL5Yab/lIfAydMn6/dZNc+Q8+T4rwN2q8 dIv85dhRuKi/eK3TwKk1rw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1706637685; x= 1706724085; bh=KeSArjw08VHNeyqM2i//aq5JT3ZGxQkfqASoOjgy9XY=; b=c lmJN8ViORMJoIAddSWpsboBf7TCg+YMeJJFYXC5woekGoDu0G+Ued8BUeOneleNJ AFl16qnRgWs/1D9iv0m/k762PGDeQIdlLFbjO1LAYb6XSNr+KiiW/O7qbVMucL1Z ba/4t4blkH6c80RkS9AROBSMLrjUYurvTfDTGhTP7msFuUxM4jYp6GDSyqYzpi0z 3EMF2kTC/cCkDJ1Fbw3nSuQ/AY45hQcSHnPfYNXNiDXZ6R2MLP0BknvmcW8Q6dKS Zdw7TYFsBky6G6+greYVLfpRxoZpDuAuQJZvB9SWKgb1+mF2j1PD+q5p+aY+N1P5 wCexSaKNtUsegxTmnmUtQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtjedggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefujghffffkfgggtgfgsehtqhertddtreejnecuhfhrohhmpefvihhm ohhthhihucfurghmphhlvgcuoehsrghmphhlvghtsehnghihrhhordgtohhmqeenucggtf frrghtthgvrhhnpeekfeejffeukeevffeufeevhfehgfdttedvteekveefffffgffgvdfh veegtdekheenucffohhmrghinhepghhnuhdrohhrghenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsrghmphhlvghtsehnghihrhhordgtohhm X-ME-Proxy: Feedback-ID: i4721425c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 30 Jan 2024 13:01:24 -0500 (EST) From: Timothy Sample To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org Subject: Re: Preservation of Guix report for 2024-01-26 In-Reply-To: <87zfwokqlg.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 29 Jan 2024 18:16:11 +0100") References: <87a5oq8esg.fsf@ngyro.com> <87zfwokqlg.fsf@gnu.org> Date: Tue, 30 Jan 2024 12:01:22 -0600 Message-ID: <87wmrqya31.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=64.147.123.20; envelope-from=samplet@ngyro.com; helo=wout4-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -10.30 X-Spam-Score: -10.30 X-Migadu-Queue-Id: 3CA531D28F X-Migadu-Scanner: mx12.migadu.com X-TUID: SugJSVNld8KC Ludovic Court=C3=A8s writes: > Thumbs up on bzip2 support! We should update Disarchive in Guix but > perhaps that=E2=80=99s already in your pipeline? I sent https://issues.guix.gnu.org/68769. Now I see that I didn=E2=80=99t = have the newest Git hooks installed, so no change ID and no email to the relevant team. Sorry! (I use worktrees so the Makefile didn=E2=80=99t fix= this for me automatically =E2=80=93 I should have double checked.) > We=E2=80=99ll also have to sync the disarchive.guix.gnu.org with ngyro.co= m. Hopefully our old system will work again, but I will have to consolidate my collection of Disarchive specifications, first. > How did you implement the Subversion check? The hard way: download, verify Guix hash, compute SWHID, check existence in SWHID. (That=E2=80=99s how everything works to date with PoG, but hopef= ully the hard way will become obsolete with all the recent support SWH has been providing us.) >> Some of these (I didn=E2=80=99t check them all) are in SWH as content = rather >> than directories. > > Back in the day, they told me that tarballs can sometimes be ingested, > for instance if they are committed to a VCS repo (that=E2=80=99s why our > fallback code tries that as well). Maybe that=E2=80=99s what happened? Probably, but I don=E2=80=99t quite understand the mechanism. The =E2=80= =9Cnixguix=E2=80=9D loader uses ExtIDs for deduplication. My assumption is that it will only skip unpacking a tarball if there=E2=80=99s an existing ExtID. It doe= sn=E2=80=99t look like there are ExtIDs for these tarballs, so I=E2=80=99m not sure. (I= =E2=80=99ve been fumbling a bit trying to use the ExtID API, so maybe it=E2=80=99s a mi= stake on my end.) >> The long-term road map is to make it work like an archive. It will run >> continuously and store *all* Guix sources. To make this easy data-wise, >> it will only store what=E2=80=99s not covered by SWH. > > By =E2=80=9Cit=E2=80=9D, you mean the Disarchive DB? I mean the PoG =E2=80=9Cproject=E2=80=9D. Instead of just testing and repo= rting, it will preserve. For instance, right now if it encounters a tarball that Disarchive can=E2=80=99t unpack, it shrugs and moves on. I want it to store those so that we are guaranteed to be able to revisit it in the future. Same thing for sources not (yet) in SWH. I want to store those so that SWH can ingest them later. Because we are doing so well working with SWH, the storage requirements for this will be manageable (10s of gigabytes). With that in place, the PoG report will simply explain what the archive needs to store and why. Our goal, then, will be for it to store nothing. -- Tim