From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#41646: Startup in Windows is very slow when load-path contains many Date: Thu, 31 Oct 2024 11:04:43 -0400 Message-ID: References: <86r08luqsq.fsf@gnu.org> <86frp1unvu.fsf@gnu.org> <86y12stv24.fsf@gnu.org> <86set0th9d.fsf@gnu.org> <86iktwt49w.fsf@gnu.org> <86cyk4t2su.fsf@gnu.org> <86a5f8t0sf.fsf@gnu.org> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12377"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , acorallo@gnu.org, 41646@debbugs.gnu.org, stefankangas@gmail.com, monnier@gnu.org To: Lin Sun Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Oct 31 16:05:29 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1t6Wjo-00030i-UJ for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 31 Oct 2024 16:05:29 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6WjS-0005sK-1W; Thu, 31 Oct 2024 11:05:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6WjP-0005rv-Na for bug-gnu-emacs@gnu.org; Thu, 31 Oct 2024 11:05:03 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t6WjP-0000iR-EK for bug-gnu-emacs@gnu.org; Thu, 31 Oct 2024 11:05:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=Ex9qoEBsTJUJaXnHHEhK+p2vzBmJqZAJrww5tMHYmsI=; b=Mfx14L4zaxRrZw7lE7DuiphPjAXbaV/7FOgVrVa3EHzoxZbsglwjo/8PX6ciiBEEkGTTBzZvNu+78fZ+bqeI/O4uCjPndvn+XjZrXpPaMv23LyUNEPmpxJv10Hr7MA/r2BHQPGDPowaSEmXY2TWXRiLz3x8+tPQUruuMbP/6acZ4dAYmfTces15NysZm180yffaTxDw6X62KAvp9iAOHsDIumcWOgKSNjEHyrdcCfvOk77+uzyLptEB8cyNjUOfpxlLTGXvzGm9RmwuZRJkqMfAxk4lk24sfWIY/EOK/yqXGPSNdTgDUwPOcaewfyM+TkXR6sNdby/607VNax31yGA==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t6WjO-0003r0-6n for bug-gnu-emacs@gnu.org; Thu, 31 Oct 2024 11:05:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 31 Oct 2024 15:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41646 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch fixed Original-Received: via spool by 41646-submit@debbugs.gnu.org id=B41646.173038709814803 (code B ref 41646); Thu, 31 Oct 2024 15:05:02 +0000 Original-Received: (at 41646) by debbugs.gnu.org; 31 Oct 2024 15:04:58 +0000 Original-Received: from localhost ([127.0.0.1]:42512 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t6WjK-0003qe-4N for submit@debbugs.gnu.org; Thu, 31 Oct 2024 11:04:58 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:43661) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t6WjH-0003qY-Sf for 41646@debbugs.gnu.org; Thu, 31 Oct 2024 11:04:56 -0400 Original-Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 0638580964; Thu, 31 Oct 2024 11:04:50 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1730387088; bh=5H5b2Fzda8Sjex/KPCqQTgFBg8XLGIVTEJ+XYxqnKIY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=EI7WaUE0WsU4X3KK6piaGyt3nSFsHismJjGbnFvS5MtREOIXctjYebbW740qCrkV9 dLlHpThdjjBx1nEO4PW3g1AlUzMl5JEyAsWvavV9T1aYto6RigFMNBUsaZ3msITdMh O+jfN+wkEVWzfHrn7zJEfdn3j4+WD/DZ6E+wOSGL4lOHGnqvwcSh6sdz0B90Q8HETM TaM9/FeWalV7ZMLFgp0XWvx8H+Fp13TAPd/Qdnryk/+IUAoOBvl/aptOK4rU8AgGUh 4vU3SHLwVSjj9UhBKwdGTlCuftsct5REKainZmz7q4QpONsG1AVBIz7spXhmdWDyUi Dx2cI5WxDMctQ== Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id DCBB180187; Thu, 31 Oct 2024 11:04:48 -0400 (EDT) Original-Received: from alfajor (unknown [38.133.22.166]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9A3981204D5; Thu, 31 Oct 2024 11:04:48 -0400 (EDT) In-Reply-To: (Lin Sun's message of "Mon, 21 Oct 2024 17:11:17 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:294630 Archived-At: >> The downside is that it can break existing setups for users who use >> `package.el` but also modify their `load-path` "by hand" in the >> init file, and it doesn't help users who don't use `package.el`. > The `load-hints' do nothing with its default value nil. No, but if `package.el` adds elements to `load-hints` and then in your init file you add more elements to `load-path`, the `load-hints` may end up taking precedence over the new elements of `load-path` in some cases and thus change which file shadows which. > I had checked the radix-tree at the beginning, it's not user-friendly, > or it's not easy to dump the radix tree for an end user to understand > which is obviously matching the entry or not. You're not "supposed" to look at the representation of the radix tree, indeed (unless you're working on `radix-tree.el`, of course). Feel free to ask for help using it. > The `load-hints' in the list are easy to understand / maintain by the > end user. I don't think we need to burden end users with load hints. >> But your prefix idea makes me think maybe we can aim for a significantly >> smaller table, where we basically record only one entry per >> package/directory, like for "~/.emacs.d/elpa/helm-core-VERSION/" we just >> record "helm" because all the `.el` files share the "helm" prefix. >> I.e. keep for each dir the corresponding longest-common-prefix. >> If we're careful to consider only those files with a `.el` suffix, then >> I think we can reduce the hint to such a longest-common-prefix. I.e. an >> info which doesn't say just "you can find FOO* files here" but "you can >> find *only* FOO* files here". > > I had searched all 200+ packages in my test env, most of the packages > use their feature name as the prefix, only 11 packages have > exceptions. But I didn't understand how it works toward the > `load-hints'. Here's the idea: the `-autoloads.el` file can registers the longest common prefix of all the `.el` files for its own `load-path` entry, with say: (load-prefix-register ) where we'd define this function along the lines of (defconst load-prefix-directories (make-hash-table :test 'equal)) "Set of entries from `load-path` for which we have prefix info.") (defconst load-prefix-map radix-tree-empty "Table associating file prefixes to directories.") (defun load-prefix-register (dir prefix) (puthash dir t load-prefix-directories) (let ((dirs (radix-tree-lookup load-prefix-map prefix))) (unless (member dir dirs) (setq load-prefix-map (radix-tree-insert load-prefix-map prefix (cons dir dirs)))))) (defun load-prefix-trim-load-path (file) "Return a trimmed `load-path` to use for FILE." (if (file-name-directory file) ;; If there's a `/` in FILE, fallback on the safe default. load-path (let* ((prefixes (radix-tree-prefixes load-prefix-map file)) (dirs (apply #'append (mapcar #'cdr prefixes)))) ;; Remove from `load-path` the entries which can't possibly ;; have FILE because their prefixes doesn't match. (cl-remove-if (lambda (dir) (and (gethash dir load-prefix-directories) (not (member dir dirs)))) load-path)))) and then `load` can use `load-prefix-trim-load-path` to iterate on a much shorter `load-path`. I'm not completely sure if it's a good idea, tho: I'd really prefer a solution that doesn't require any change to any package management code, which instead uses a cache (updated/filled automatically) of all the files found in all the `load-path` directories. >> Is that with or without using `package-quickstart`? > The `package-quickstart' does not help in this scenario, AFAICT your scenario includes Emacs startup with packages installed, so `package-quickstart' can definitely make a difference. But maybe you're right that it will affect both cases equally. >> Similarly, I wasn't able to convince myself that your patch does the >> right thing when `require` or `load` is used such that MUST_SUFFIX is >> not specified. > The `load-hints` just put the matched paths on the top of `load-path`, > still following the `load-path' mechanism, and won't affect any other > features (Or someone already has some code to adjust the `load-path' > orders, can just ignore the `load-hints', everything work like before, > no break changes). But if FOO is in one dir and FOO.el is in another dir, adding entries to the load path can change which file we end up loading. Stefan