From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id QLF1ICElDmKDDgAAgWs5BA (envelope-from ) for ; Thu, 17 Feb 2022 11:36:17 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id iCraHSElDmL1UQEA9RJhRA (envelope-from ) for ; Thu, 17 Feb 2022 11:36:17 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 02903138B1 for ; Thu, 17 Feb 2022 11:36:17 +0100 (CET) Received: from localhost ([::1]:34748 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nKe91-0004V2-Vj for larch@yhetil.org; Thu, 17 Feb 2022 05:36:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50750) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nKe8p-0004UB-CX for guix-patches@gnu.org; Thu, 17 Feb 2022 05:36:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:56347) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nKe8o-0005Cu-Ge for guix-patches@gnu.org; Thu, 17 Feb 2022 05:36:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nKe8o-0003Sq-Es for guix-patches@gnu.org; Thu, 17 Feb 2022 05:36:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53818] Improving updaters and =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?refresh=E2=80=99?= Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 17 Feb 2022 10:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53818 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Nicolas Goaziou Cc: 53818@debbugs.gnu.org, Xinglu Chen Received: via spool by 53818-submit@debbugs.gnu.org id=B53818.164509416013308 (code B ref 53818); Thu, 17 Feb 2022 10:36:02 +0000 Received: (at 53818) by debbugs.gnu.org; 17 Feb 2022 10:36:00 +0000 Received: from localhost ([127.0.0.1]:50244 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKe8m-0003SZ-5T for submit@debbugs.gnu.org; Thu, 17 Feb 2022 05:36:00 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:56830) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKe8k-0003SL-4R for 53818@debbugs.gnu.org; Thu, 17 Feb 2022 05:35:58 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 7575E8DE; Thu, 17 Feb 2022 11:35:51 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PuMFHHma5jqt; Thu, 17 Feb 2022 11:35:50 +0100 (CET) Received: from ribbon (unknown [IPv6:2001:660:6102:320:e120:2c8f:8909:cdfe]) by hera.aquilenet.fr (Postfix) with ESMTPSA id B83B44A; Thu, 17 Feb 2022 11:35:49 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87pmnx7ynw.fsf@gnu.org> <87y22kxkv3.fsf@yoctocell.xyz> <87leyi4fcv.fsf@gnu.org> <87pmnprasd.fsf@nicolasgoaziou.fr> <87fsolv109.fsf@gnu.org> <87leydqohm.fsf@nicolasgoaziou.fr> <87ee44pi4p.fsf_-_@gnu.org> <87a6eroubv.fsf@nicolasgoaziou.fr> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 29 =?UTF-8?Q?Pluvi=C3=B4se?= an 230 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 17 Feb 2022 11:35:48 +0100 In-Reply-To: <87a6eroubv.fsf@nicolasgoaziou.fr> (Nicolas Goaziou's message of "Wed, 16 Feb 2022 13:43:32 +0100") Message-ID: <87mtipok57.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Rspamd-Server: hera X-Rspamd-Queue-Id: 7575E8DE X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1645094177; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=zPGm4rHQ3PYa3kGV4Q/6xhGiN5ZiJgo1XsaZIwTO10w=; b=DyBuTj3Yhaa5nuii7uQoNhMM/fhYKOuEF0YPJ3JvzMUYiI3t9yHKt9StiE9Piru7TkksfR l0zkb3lQ3elceWcJNJvF6QmsOlJvOCEXc3z3Eglbu8ZNvXzEQdUWYWGEz/MyKDmo5ZP8ym FO6XORNbplKFEDCSY8k1jQyWHqX64jYrEfCNYJ8XrbW8WPCXTqEQvuVz8orewkEjIKnI0J C80zr+R1B6uWkbj0DdqqLRjpo2jlWAMKn2PvIJnjn6skPGq0A5sA3n+TyIxJaFR+13yeQE D4WNFXniWILOhxwP89U2FH7KYrY0A5FqkHrbmdddcyHgOw5pXIhr1WwtuUSi+A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1645094177; a=rsa-sha256; cv=none; b=np+2mBZrUC0LnQc9rn7B7vWPohPuZy/Zwvg+pX0EXcYUOVgK3dDOaoVsBvPT715Vp3SWMt KYcaj0Bm3aSQOlwQOt/7PTH1PsZZXTeAThU9j0qLPy1PJlMab9kXXssczSpQmJX7r8v7qE UKplDKj1gvCgVJUNee0MpRRi6A7o8Ehn6dmMw8THA/NUxb1R0XNdjPQLKlkqvUNJw2X/hb YiYljJMxXC4SndeZXFD84kkRmL8ntwjUd2UF1CAgnLQYAErZqMbGMHt78jGpEhVO5j/bLs oR9F+uCryUSJLTrk1WVSV225/t1SZy434KtGXI9v/yIzwoorkZSbSLaOYzn0HA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.63 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 02903138B1 X-Spam-Score: -3.63 X-Migadu-Scanner: scn1.migadu.com X-TUID: Yo28SPvHDE9K Hi, Nicolas Goaziou skribis: > Ludovic Court=C3=A8s writes: > >> Regarding =E2=80=9Cspecifying many packages=E2=80=9D, do examples like t= hese work for >> you: >> >> =E2=80=A2 guix refresh -t elpa > > I don't find it very useful in practice. As a user, the packages I'm > interested in probably rely on more than one updater. I'm not even > supposed to know what updater relates to a given package. Right, that=E2=80=99s more for packagers than for users. As a user, what works better is: guix refresh -r $(guix package -I |cut -f1) -s non-core =E2=80=A6 or simply =E2=80=98--with-latest=E2=80=99, if I=E2=80=99m not int= erested in updating package definitions. >> =E2=80=A2 guix refresh $(guix package -A ^emacs- | cut -f1) > > This one is interesting. This illustrates that the UI is, from my point > of view, a bit lacking. It would be a nice improvement to add a regexp > mechanism built-in, like in "guix search". Makes sense, we can do that. > In any case, this fails after reporting status of around 50 packages, > with this time: > > real 0m41,881s > user 0m12,155s > sys 0m0,726s How does it fail? If it=E2=80=99s the GitHub rate limit, then there=E2=80= =99s only one answer: you have to provide a token. > Assuming I don't get the "rate limit exceeded" error, at this rate, it > would take more than 15 minutes to check all the packages in > "emacs-xyz.scm". This is a bit long. > I don't see how this could reasonably be made faster without relying on > an external centralized service doing the checks regularly (e.g., once > a day) before the user actually requests them. Maybe you=E2=80=99re right, but before jumping to the conclusion, we have to investigate a bit. Like I wrote, the =E2=80=98gnu=E2=80=99 updater for ins= tance fetches a single file that remains in cache afterwards=E2=80=94the cost is constant. We should identify updaters that have linear cost and check what can be done. =E2=80=98github=E2=80=99, =E2=80=98generic-html=E2=80=99, and =E2=80= =98generic-git=E2=80=99 are of that kind. Now, the command I gave above looks at 1,134 packages, so is it even something you want to do as a packager? >> =E2=80=A2 guix refresh -r emacs-emms > > It also fails with the "rate limit exceeded". While this sounds > theoretically nice, I wouldn't know how to make use of it yet. > >> =E2=80=A2 guix refresh -s non-core -t generic-git > > See above about "-t elpa". > >> =E2=80=A2 guix refresh -m packages-i-care-about.scm > > Yes, obviously, this is a nice, too. However, it doesn't scale if you > need to specify 1000+ packages. You can use =E2=80=98fold-packages=E2=80=99 and have three lines that retur= n a manifest of 10K packages if you want it. Honestly, since I mostly rely on others these days :-), I=E2=80=99m no long= er sure what the packager=E2=80=99s workflow is. Also, the level of coupling varies greatly between, say, a C/C++ package and a set of Python/Emacs/Rust packages. I find that =E2=80=98guix refresh=E2=80=99 works fine for loosely-coupled C= /C++ packages where often you=E2=80=99d want to upgrade packages individually. But for Python and Emacs packages, what do we want? Do packagers always want to check 1K+ packages at once? Or are there other patterns? >> If not, what kind of selection mechanism could help? =E2=80=98-s=E2=80= =99 currently >> accepts only two values, but we could augment it. > > Besides regexp matching, it may be useful to filter packages per module, > or source file name. Package categories is a bit awkward, tho, and > probably not satisfying. We can add options to make it more convenient, but it=E2=80=99s already possible: guix refresh $(guix package -A | grep emacs-xyz.scm | cut -f1) >> I realize this is going off-topic, but let=E2=80=99s see if we can impro= ve the >> existing infrastructure to make it more convenient. > > Is it really off-topic? > > Anyway, all of this is only one data point, and, as a reminder, > I certainly don't want to disparage either Xinglu Chen's work, or > current "guix refresh" functionality. Yup, same here! I think we have nice infrastructure but you raise important shortcomings. What Xinglu Chen did might in fact be one way to address it, and there may also be purely UI issues that we could address. Thanks, Ludo=E2=80=99.