From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.devel Subject: Unboxed package manager Date: Sun, 19 Mar 2023 21:18:33 -0400 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32593"; mail-complaints-to="usenet@ciao.gmane.io" To: emacs-devel Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Mar 20 02:19:41 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pe4BZ-0008KN-1G for ged-emacs-devel@m.gmane-mx.org; Mon, 20 Mar 2023 02:19:41 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pe4Ai-0005NX-Ve; Sun, 19 Mar 2023 21:18:48 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pe4Ah-0005NI-IO for emacs-devel@gnu.org; Sun, 19 Mar 2023 21:18:47 -0400 Original-Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pe4Af-00041V-Rb for emacs-devel@gnu.org; Sun, 19 Mar 2023 21:18:47 -0400 Original-Received: by mail-pg1-x52c.google.com with SMTP id q189so5735035pga.9 for ; Sun, 19 Mar 2023 18:18:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679275123; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=8jUOboG2Ff/WcdVgbs0frxj+mibglRDjylYZQaGVULQ=; b=IqbsUcruty1Zc2Bk5tEDhSHDpx6EEA+59ep4B5cGvdquL8dVmIWQH8V1rRuq6MvgFP M3VNm95mCuCCVMKefm1aHHCnOwT4tI5RwZkvjgDmgUQ6Us7qyy9cSYig2OY70ddaCMpt PgZFWidU5sir3VWyCMugJmSu39wuS+9BQ0DRI/89mo0Y0Pi2THQHT/gdRldiAQ/ta+1r IeorNAixmc9fwEeIG/EuHPlTfcfVl9N/B0kS2ZiI04BYbZyrGgVyXIO9B3Gzz6GwNHyM RRih0RxMA9zMceoHZ7+g5tnpfeorac6pHGT8MO8PSC66/+ttjVY8h6jF3NGLdUusPYAE 4WjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679275123; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=8jUOboG2Ff/WcdVgbs0frxj+mibglRDjylYZQaGVULQ=; b=IZqjhzs5ia1jqscjj/T2XHD+gBTCy+5G4ayV7v+VERNxXzN0FQ1T5pxjhq/Iwr7ifv exfFWgrmVJXJ3Nr8i6ShckWsgT/799XnLmq1OE0HIGsz4BONWyjokhTvq5pS4faYxGkw KtMhImhimH1Vn6qC0IUDO+NFMzcvF27NsNgtBXcO9n9SZehtY0PJL68eRvQHQnCYCddp ycr+0710VPdUr3wEc6aRenSnWPmoha1MFl5hV7P8rcOdm5OX4Cy1ZnxCIAeBT6Y4R8kf gvFRcjlXSYtIHloUaUGH8WHqEjwdhiz8GYwXWgDC2A8HwqGXqTaoNAu/YHASVhFvkpqz Y7QQ== X-Gm-Message-State: AO0yUKVInFdmeDY4xTaQnxOwJ4Ym2ZAo+rMnGoxdoX7WA6jGEZphmrhM 2uQQVjYgxVh2YV6LlqGOF7Vyf4u7hsU7t01y6wPGnL8/Uks= X-Google-Smtp-Source: AK7set/4YfrCBiy4DF48J4gJOG7cpSCujqOYDv+zsdw0ed9IvpbnnYqOImwpqEkweGqd/N+asOLTtCLtQMNh8tWsYZY= X-Received: by 2002:a65:568c:0:b0:503:91ff:8dd8 with SMTP id v12-20020a65568c000000b0050391ff8dd8mr1364574pgs.4.1679275123094; Sun, 19 Mar 2023 18:18:43 -0700 (PDT) Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=owinebar@gmail.com; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:304586 Archived-At: Hi, I want to write a small package "unboxed" that extends the built-in package manager with packages whose elisp files are installed into a common packages directory instead of one directory per package. I've done this manually on various systems I use with a significant improvement in startup performance. Of 2403 packages I've installed on my current personal system, only 161 have a subdirectory. Ignoring "-pkg.el" and "-autoloads.el" files, 1971 of those 2403 packages have a single elisp file. I'm including a complete histogram table at the end of this mail for reference. I would like to just tweak the existing functionality so that, if a predicate is satisfied, a given package will be locally administered by "unboxed" rather than "package": * mapping package contents to local paths for installation and removal (e.g. info files go to a common info directory for unboxed packages, themes to a separate theme directory, etc) * creating autoloads for the common directory, preferably after an entire set of packages has been installed/removed rather than one-at-a-time * providing package-desc objects to the package initialization routine I can think of 3 predicates for determining when "unboxed" might be assigned, depending on how risk-adverse the user is 1) A package contains a single elisp library 2) A package has no subdirectories 3) A package has subdirectories but it is referenced through some variable set in one of the source files, which can be patched. Most packages are amenable to the 3rd option - with the exception of a small number of cases, of which none that I can recall have a subdirectory anyway, the variable in question contains the only reference to "load-file-name". There are some packages that simply cannot be managed this way without extraordinary effort, basically any package using "load-relative". Packages that attempt to facilitate active development even after installation can also do things that are problematic, like look for git information in the installation directory. An explicit list of these cases (as a customization variable) can be maintained to ensure they are handled the default way. >From my perspective, package.el already supports 2 package tracking mechanisms - one for "built-in" packages, and the usual one for external packages or updates to the built-in packages. However, if there is no appetite for facilitating a more general "wiring in" of additional local package management (which might be useful if you wanted to enable backends with an explicit database of installed packages), then my best option would appear to be to derive from the package-menu-mode as a replacement, and either define variants of the functions in the package-menu and package namespaces or use advice. The latter goes against my taste in code, but is probably the most expedient. Can any maintainers of package.el advise? The following is the promised histogram of the number of packages from my sample with a specified number of elisp libraries. 2403 package directories in load-path absolutely devastates locate-file. Even the most conservative approach above (option 1) would drastically reduce this effect. #libs #packs Percent 1 1971 82.022 2 148 88.181 3 57 90.553 4 37 92.093 5 17 92.801 6 49 94.84 8 11 95.298 9 17 96.005 10 8 96.338 11 5 96.546 12 8 96.879 13 7 97.17 15 7 97.462 16 4 97.628 17 5 97.836 18 6 98.086 19 2 98.169 20 2 98.252 21 2 98.335 22 6 98.585 23 1 98.627 24 3 98.752 26 2 98.835 27 9 99.209 29 1 99.251 30 1 99.293 31 4 99.459 32 2 99.542 35 1 99.584 36 2 99.667 37 1 99.709 41 2 99.792 45 2 99.875 66 1 99.917 77 1 99.958 95 1 100 For the curious, the last three are jdee, doom-themes, and lsp-mode respectively. They can be "installed" by simply copying them into a common directory, updating the directory autoloads, and byte-compiling without any issue. Thanks, Lynn