From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#41646: Startup in Windows is very slow when load-path contains many Date: Fri, 01 Nov 2024 09:11:24 -0400 Message-ID: References: <86r08luqsq.fsf@gnu.org> <86frp1unvu.fsf@gnu.org> <86y12stv24.fsf@gnu.org> <86set0th9d.fsf@gnu.org> <86iktwt49w.fsf@gnu.org> <86cyk4t2su.fsf@gnu.org> <86a5f8t0sf.fsf@gnu.org> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1740"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , acorallo@gnu.org, 41646@debbugs.gnu.org, stefankangas@gmail.com, monnier@gnu.org To: Lin Sun Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Nov 01 14:12:18 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1t6rRp-0000Ep-93 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 01 Nov 2024 14:12:17 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6rRe-0003f8-CQ; Fri, 01 Nov 2024 09:12:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6rRb-0003ez-4L for bug-gnu-emacs@gnu.org; Fri, 01 Nov 2024 09:12:03 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t6rRa-0004wj-54 for bug-gnu-emacs@gnu.org; Fri, 01 Nov 2024 09:12:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=trQ1CEYSNu63P7LdtHxyXs2TRDCqaiWZDsBOGWpggPI=; b=ca9cHhZ31N/qDMkY75E+ws2QDHVMt+SiqSxNhce6uqOhp+kyKqByrnXym0bnkfBsMUJkDBSLIvQHe3LTFWWDvSws5HZ3gm9xDC/z2WQCY2hUwj8kItFFiqjIuvTFkJGHPDZqEX72MqGNhOmB1DEFCcEVstHA2nBEixi2wiVWaskQC+CtLDT29Y0LeG6j7SJ3no5iLq9PBK/9Z5+LReBPp0ZMz5a2d/0Era2IvADwWoqghDd4/GZMvClHAQRXLR/po1XoRrQPf6xw4S3oo6b4Cate0ipZse7Rt8l9ziwYfYqWet299hgwcvx/DyM5xAZDa83Q2ZJ7ot7hp6HQdmXX5Q==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t6rRZ-0000Ph-VQ for bug-gnu-emacs@gnu.org; Fri, 01 Nov 2024 09:12:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 01 Nov 2024 13:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41646 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch fixed Original-Received: via spool by 41646-submit@debbugs.gnu.org id=B41646.17304666961567 (code B ref 41646); Fri, 01 Nov 2024 13:12:01 +0000 Original-Received: (at 41646) by debbugs.gnu.org; 1 Nov 2024 13:11:36 +0000 Original-Received: from localhost ([127.0.0.1]:49398 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t6rRA-0000PC-BA for submit@debbugs.gnu.org; Fri, 01 Nov 2024 09:11:36 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:36324) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t6rR8-0000P2-Hy for 41646@debbugs.gnu.org; Fri, 01 Nov 2024 09:11:35 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 91CAA442217; Fri, 1 Nov 2024 09:11:27 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1730466685; bh=1/HxO/ZqzBvO0kw1CfL4t/GeZ3oq5hWcTIOBEjrjzhw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=DfzfsW9HC9Qbecr+QQ1yZAlLz4ZKMlg7WEM5MrUszKMM+2gecGaTktsX9cfKvlGhs +PgLk/N5f/1wEjdb6F2fwvjq8RuMMWym0s7gcPsfMXlLX0Bj7mX2MllYj8y2ZWJpaR sP7yheZgrcOGLRjKyUHc4Rz12yf2tzlUvZQ+wG9+qI+nqKr2HYy+4Cxl7zkyovuCbJ g5pmwJLxiRa03MQNA18+eKpOYbR+vU+HOnGpA2XPHAHnG9V4eHojNH4U7Qh7yXjNLy e+XfA+2OdY1aAiSqnUDmCdGCxO7wQCwn/bH/m9KXimYRpeCw/B4A0zk6qm+7h9c+7U hv8ZIGHEdeWkg== Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id D02AE441968; Fri, 1 Nov 2024 09:11:25 -0400 (EDT) Original-Received: from pastel (104-195-225-43.cpe.teksavvy.com [104.195.225.43]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 8757E120403; Fri, 1 Nov 2024 09:11:25 -0400 (EDT) In-Reply-To: (Lin Sun's message of "Fri, 1 Nov 2024 07:18:11 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:294677 Archived-At: > Yes, introducing this new variable will increase the complexity to the > end user. Exactly, and I think we should first try to solve the problem without exposing the user to such complexity. >> Here's the idea: the `-autoloads.el` file can registers the longest >> common prefix of all the `.el` files for its own `load-path` entry, with >> say: >> >> (load-prefix-register ) >> >> where we'd define this function along the lines of >> >> (defconst load-prefix-directories (make-hash-table :test 'equal)) >> "Set of entries from `load-path` for which we have prefix info.") >> >> (defconst load-prefix-map radix-tree-empty >> "Table associating file prefixes to directories.") >> >> (defun load-prefix-register (dir prefix) >> (puthash dir t load-prefix-directories) >> (let ((dirs (radix-tree-lookup load-prefix-map prefix))) >> (unless (member dir dirs) >> (setq load-prefix-map (radix-tree-insert load-prefix-map prefix >> (cons dir dirs)))))) >> >> (defun load-prefix-trim-load-path (file) >> "Return a trimmed `load-path` to use for FILE." >> (if (file-name-directory file) >> ;; If there's a `/` in FILE, fallback on the safe default. >> load-path >> (let* ((prefixes (radix-tree-prefixes load-prefix-map file)) >> (dirs (apply #'append (mapcar #'cdr prefixes)))) >> ;; Remove from `load-path` the entries which can't possibly >> ;; have FILE because their prefixes doesn't match. >> (cl-remove-if (lambda (dir) >> (and (gethash dir load-prefix-directories) >> (not (member dir dirs)))) >> load-path)))) >> >> and then `load` can use `load-prefix-trim-load-path` to iterate on >> a much shorter `load-path`. Note that this above proposal should be transparent to the end user (tho it requires extra work on the `package.el` side): e.g. funny changes to `load-path` would be handled without fuss. >> I'm not completely sure if it's a good idea, tho: I'd really prefer >> a solution that doesn't require any change to any package management >> code, which instead uses a cache (updated/filled automatically) of all >> the files found in all the `load-path` directories. > If that, we have to track the file/path changes in each entry of > load-path, it may not be possible for all the supported OSs. We can easily detect changes to `load-path` itself, of course, but as for changes to the content of the directories in `load-path` that would be more difficult&costly, admittedly. My plan was to do nothing about it (i.e. allow the cache to go stale): if we use the cache only to tell `load` in which directory to look for the file, it should usually be safe because IME it's rare for files to be added/removed from directories such that it changes from which directory a given ELisp file is loaded. But of course we could also make efforts to try and keep our cache consistent, e.g. via OS-level notification infrastructure or by flushing the cache after a N seconds. Stefan