From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 5KF5HHIgr14ZGAAA0tVLHw (envelope-from ) for ; Sun, 03 May 2020 19:50:10 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id OLjtDXwgr164ZAAAB5/wlQ (envelope-from ) for ; Sun, 03 May 2020 19:50:20 +0000 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:470:142::17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 60CEE9400CD for ; Sun, 3 May 2020 19:50:09 +0000 (UTC) Received: from localhost ([::1]:41034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jVKcr-0006aS-Jh for larch@yhetil.org; Sun, 03 May 2020 15:50:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37756) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jVKck-0006ZR-2v for guix-patches@gnu.org; Sun, 03 May 2020 15:50:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46520) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jVKcj-0005Jb-PQ for guix-patches@gnu.org; Sun, 03 May 2020 15:50:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jVKcj-0006GW-J4 for guix-patches@gnu.org; Sun, 03 May 2020 15:50:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#39258] [PATCH v4 0/3] Faster cache generation (similar as v3) Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 03 May 2020 19:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39258 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: zimoun Cc: Arun Isaac , Pierre Neidhardt , 39258@debbugs.gnu.org Received: via spool by 39258-submit@debbugs.gnu.org id=B39258.158853535924018 (code B ref 39258); Sun, 03 May 2020 19:50:01 +0000 Received: (at 39258) by debbugs.gnu.org; 3 May 2020 19:49:19 +0000 Received: from localhost ([127.0.0.1]:58061 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jVKc3-0006FK-Fe for submit@debbugs.gnu.org; Sun, 03 May 2020 15:49:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jVKc2-0006F7-1r for 39258@debbugs.gnu.org; Sun, 03 May 2020 15:49:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58785) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jVKbu-00054M-6N; Sun, 03 May 2020 15:49:10 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=49782 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jVKbt-0004Up-Cg; Sun, 03 May 2020 15:49:09 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200503150154.26532-1-zimon.toutoune@gmail.com> <87r1w1ynnm.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 15 =?UTF-8?Q?Flor=C3=A9al?= an 228 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 03 May 2020 21:49:06 +0200 In-Reply-To: (zimoun's message of "Sun, 3 May 2020 20:10:10 +0200") Message-ID: <87tv0wyf2l.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 X-Spam-Score: 0.49 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 2001:470:142::17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Scan-Result: default: False [0.49 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.49385748702609]; FORGED_SENDER_MAILLIST(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:470:142::/48:c]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.14), country: US(-0.00), ip: 2001:470:142::17(-0.49)]; DWL_DNSWL_FAIL(0.00)[2001:470:142::17:server fail]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; MAILLIST(-0.20)[mailman]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_FAIL(0.00)[2001:470:142::17:server fail]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:22989, ipnet:2001:470:142::/48, country:US]; TAGGED_FROM(0.00)[larch=yhetil.org]; ARC_NA(0.00)[]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ludo@gnu.org,guix-patches-bounces@gnu.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; URIBL_BLOCKED(0.00)[gnu.org:email]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gnu.org]; HAS_LIST_UNSUB(-0.01)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_SEVEN(0.00)[9]; SUSPICIOUS_RECIPS(1.50)[] X-TUID: lJ4iTF/1Ic5v Hi, zimoun skribis: > On Sun, 3 May 2020 at 18:43, Ludovic Court=C3=A8s wrote: > >> > Therefore the cache '/lib/guix/package.cache' contains more >> > information. >> >> This breaks the binary interface, so we=E2=80=99ll have to analyze the i= mpact of >> such a change and devise a strategy. > > Interface between what and what? Guix revision N creates a cache that will be read by revision N+1, upon =E2=80=98guix pull=E2=80=99 completion. > Note that I have read your comment in v3 2/3 but I did not understand it.= Sorry. > > I realize the other cache also has that problem, but it would be nice to > add a version tag to the cache. Basically emit something like: > > (package-metadata-cache (version 0) VECTOR =E2=80=A6) > > instead of just: > > (VECTOR =E2=80=A6) Yes, it would be better. > For example, 'supported-systems' is saved as a list of strings, > whereas 'license' is expanded as 3 strings without be packed in a list > of strings. From my point of view, it is inconsistent and I do not > know what is the best (readibility, startup time, etc.). I guess both =E2=80=98license=E2=80=99 and =E2=80=98supported-systems=E2=80= =99 should be list of strings. It doesn=E2=80=99t really have an impact on startup time (I thoug= ht you were suggesting storing the cache as an sexp instead of an object file.) >> Something that must be done explicitly doesn=E2=80=99t seem great to me.= As a >> user, I=E2=80=99d rather not think about search indexes and all. But I = don=E2=80=99t >> know, maybe if it happened automatically on the first =E2=80=98guix sear= ch=E2=80=99 >> invocation that=E2=80=99d be fine. > > I do not think it is an option to build the BM25 the first time "guix > search" is called. Back-to-envelop estimation, it needs ~25 seconds > to Xapian* to do so. > > From my point of view, two options: > a) "guix pull" does this extra ~25 seconds (compared to 10 seconds to > build the v4 cache) > b) the user manually build the index (I agree it is awkward!) > > Well, the first question is to evaluate if it is worth -- I am using > the v2 version based on Xapian to have an idea. Please if you have > suggestions about query (terms an user could type) and results > (packages an user could expect), there are welcome. Yeah, dunno. Maybe an option would be to create the index in such a way that it is substitutable. [...] > Yes, it was already your suggestion in v3 3/3. Do you suggest to > refactor 'package->recutils'? For example, > > (define* (package->recutils name version > ... all-the-other-fields ... > port #:optional (width (%text-width)) > #:key > (hyperlinks? (supports-hyperlinks? port)) > (extra-fields '())) Yes. >> > 4. Impolite '@@' is used to access the private license construction. >> >> (guix licenses) could provide a =E2=80=98string->license=E2=80=99 proced= ure. > > Well, do you suggest: > > (define (string->license name) (license name #f #f)) No; rather, it would look up the license in a dictionary and return the corresponding object or #f if it=E2=80=99s not a known license. Thanks, Ludo=E2=80=99.