From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id iBCqA4xm+GW5sgAAqHPOHw:P1 (envelope-from ) for ; Mon, 18 Mar 2024 17:06:36 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id iBCqA4xm+GW5sgAAqHPOHw (envelope-from ) for ; Mon, 18 Mar 2024 17:06:36 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1710777996; a=rsa-sha256; cv=none; b=Tr9vH9de4uNwkdHowM0+9Pw+1b03jR7b558VNgtZRQTr+S6Z85sFYHYzxyDfyKn9G6L8Sq vVbKJHvngobDTmxAVpoyFuz3+szTyxUC5JF22ECVR7lAxi6d+IywW3Dwv2KiCTtv7i1IH7 f9gRg/cOm7IeAPd/SDGo6Cc4y2rTXOZvvyM017ZKriLsdJkNvaqjkW6h8xBdr81hJij51q +mQCaZastCkudZsbzO/4Z3y6p5/BMYDLyoRm+1CoFeBoD8gBYkRfEDokAN4EtOaMNykYIw TgyuCNXsXv0j6Okt5FEXnMBrQ5gfmSLUqJhABE5v9ZsLUu4KJF3Ge8dpsPImuQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1710777996; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=WIHCa8Yuum6aCuukLJ2J1pkwht8Jyt79sO++/l+xEiE=; b=c933b6BcDgPAnJQjOaMH3m+RuNCqLDy3L6K87NYLQt4zD2wgWMrxE4Yi18q3d/Deq7/0iw ivTtUtkFj6yKzGLXiHRQoMVF/gsNbmJcKt79/WQK5sF8d48GJegPCvOsv3eOdeL1oszvBr hCAkHb7N1z/LqkIlgfPfWGyVxttTAFjVoaKE4ThcIVqVmtBhvbWn+WQp5thXVQn97ILWbu bnfwgLgJPY3Oi3sbHQmtO8ziDYWEyhUMpyloOqb3BSZTR44P1lbaQoEI5eI1FefXehd9Mp q0VLAyX2T+If20x1YzXSojwXs6OlOfXbABX+CeLLlMGFUocD8pb6wx/z9yc2eA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6EA416CCA6 for ; Mon, 18 Mar 2024 17:06:35 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rmFUz-0002SW-2U; Mon, 18 Mar 2024 12:06:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rmFUY-0002OS-PI for guix-devel@gnu.org; Mon, 18 Mar 2024 12:05:39 -0400 Received: from vmi993448.contaboserver.net ([194.163.141.236] helo=mutix.org) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rmFUU-0002m8-E2 for guix-devel@gnu.org; Mon, 18 Mar 2024 12:05:37 -0400 Received: from [192.168.1.172] (host81-147-82-218.range81-147.btcentralplus.com [81.147.82.218]) (Authenticated sender: cdo) by mutix.org (Postfix) with ESMTPSA id 39EDBA634E5; Mon, 18 Mar 2024 17:05:29 +0100 (CET) Message-ID: <09733ef0-61a1-650b-c6d3-6f51ca91e9d5@mutix.org> Date: Mon, 18 Mar 2024 16:05:28 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Subject: Re: Mechanism for helping in multi-channels configuration (and Xapian index) Content-Language: en-US To: Simon Tournier Cc: guix-devel References: <87v875kf5u.fsf@gmail.com> <0109ba4a-f62f-b438-c0ba-f46e66395a7e@mutix.org> <87mss17opr.fsf@gmail.com> From: Christina O'Donnell In-Reply-To: <87mss17opr.fsf@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=194.163.141.236; envelope-from=cdo@mutix.org; helo=mutix.org X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, NICE_REPLY_A=-0.684, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -6.31 X-Spam-Score: -6.31 X-Migadu-Queue-Id: 6EA416CCA6 X-TUID: Y5lRmCTC80xZ Hi Simon, Sorry for the really long delay, I meant to reply after I'd had a good read through the conversation you linked, but I haven't had a chance to really get into it yet, but I have read enough to get a surface idea of the project. The project looks fun, and looks like it will help Guix users and developers so I'd be on board in principle. On 15/02/2024 15:05, Simon Tournier wrote: >> ... > I think you mean ’fold-packages’. > >>  2. Have a script that determines the symbols needed by each file. (Macros >>     make this more difficult, but.) > Well, this would be difficult, IMHO. Somehow, it is what the compiler > does. :-) I asked on guile-devel and Maxime suggested using `module-map` or `module-for-each` which map over all symbols in a module. Presumably this would know what's quotes literally and what's a proper symbol. >>  3. Have both scripts have an incremental version that runs on diffs (for >>     performance). >>  4. Run this for every commit on every branch on every channel caching the >>     result. >>  5. Have a CI script keep this updated for new commits. >>  6. Have a server track incompatibilities. > Here, I think the issue is that one server needs to track all the > channels. And that’s a too strong assumption, IMHO. > > I think the design should be something on channel maintainer side. > Somehow, the main Guix channel could be seen as a Git submodule from the > channel side and the issue is that information is not tracked. > > There is this ’.guix-channel’ file which allows to describe channel > dependencies. And the improvements could be to add more there. The > question is what to add and how to add it. Keeping in mind the > simplicity and the maintenance burden-free. :-) Okay, this makes sense. I'm thinking that you could have something like a sqlite index that can be generated by running a script on the code. The index could exist on the server separate to the channel repo, pointed at by the .guix-channel file. The commit hook could: (1) update the local index to include the latest commit; (2) update the hash inside .guix-channel. Then a push hook could also push the index to the server. It's a bit clunky because you've got this binary blob that you have to synchronize with the channel, and it's easy to get this wrong. Putting the index in the channel repo would bloat the channel with old versions of the index. Forcing users to generate the index from scratch is undesirable too. As an alternative to having the index referenced in the .guix-channel, we could use git-annex. This would take care of: Fetching the index, uploading the new index on push, and updating the hash. No extra steps would be /required/ by developers, as it won't be necessary to have the index 100% up to date. But developers could choose to regenerate the index and call `git annex sync`. I suspect that adding git-annex as a dependency would be resisted, but that's the way I think would work best. And could apply to existing indexes. It depends on how long it takes to generate the index from scratch. There was some talk of data.guix.gnu.org using PostgreSQL to index packages. I suppose it'd be worth figuring out what they do to see if they have anything sql or code that might be portable to sqlite. >> Full disclosure: I've got nothing lined up for the summer yet, so I'm on the >> prowl for GSoC projects :) > Cool! > > In that spirit, one tool that is missing is: search packages in all the > history. Somehow the need is described by this message [1]: how to find > which Guix revision provides which version of Foo? > > In addition, “guix search” is slow [2]. > > Well, I have started the embryo of an extension based on Guile-Xapian > for indexing and improving the search. Really an embryo. :-) > > I think this would fit some GSoC. ;-) As I said above, [2] is a fairly long thread, but I think I get the general idea. It seems that Xapian was implemented but didn't have the desired speedup. Am I getting the right impression there? It's certainly an interesting problem. I'll keep thinking about it. Kind regards, Christina > 1: Re: List available versions of package. > Philippe Veber > Tue, 11 Jun 2019 09:43:08 +0200 > id:CAOOOohSzUezKvm=RO0bXRGH3m0eo2x0cOTvd--vARxWoqtceaQ@mail.gmail.com > https://lists.gnu.org/archive/html/help-guix/2019-06 > https://yhetil.org/guix/CAOOOohSzUezKvm=RO0bXRGH3m0eo2x0cOTvd--vARxWoqtceaQ@mail.gmail.com > > 2: https://issues.guix.gnu.org/issue/39258