From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lin Sun Newsgroups: gmane.emacs.bugs Subject: bug#41646: Startup in Windows is very slow when load-path contains many Date: Fri, 1 Nov 2024 07:18:11 +0000 Message-ID: References: <86r08luqsq.fsf@gnu.org> <86frp1unvu.fsf@gnu.org> <86y12stv24.fsf@gnu.org> <86set0th9d.fsf@gnu.org> <86iktwt49w.fsf@gnu.org> <86cyk4t2su.fsf@gnu.org> <86a5f8t0sf.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37592"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , acorallo@gnu.org, 41646@debbugs.gnu.org, stefankangas@gmail.com, monnier@gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Nov 01 08:20:12 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1t6lx5-0009aP-0i for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 01 Nov 2024 08:20:11 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6lwx-0005C2-U6; Fri, 01 Nov 2024 03:20:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6lww-0005Bi-GK for bug-gnu-emacs@gnu.org; Fri, 01 Nov 2024 03:20:02 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t6lww-0001mm-7P for bug-gnu-emacs@gnu.org; Fri, 01 Nov 2024 03:20:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=Date:From:In-Reply-To:References:MIME-Version:To:Subject; bh=Hg5Ouhcus2xbD99hSCPDGXHUMKEnk1b9kCs8DDUb69c=; b=ZX5qcYqgmPN8t6l0QR7O1+4Ac9DbpHoZodXAtWy9xPTB2P96HAF89pojQorW//8FX81TbIdH1t45/5zVwu6Rzt4Ik5ylMkBtK1YWlON2ksdtEBTmNksz45ggBhAEm9kZ+uWpgRnJFn6D0GhM+vAlb+GtLJrm32Q2hYs9b41gVtMXBgP+VkGlj61kKKVr+3dQl0NnP1OkXp5gmmqCWBF5Vm8XtyQIB3S43hdPjuTTCLpxHyAsqCHKiBN/TGHZFpi7pM0wqaWcUKfACDEWV7KDs10vAxZ4n5VD0mglkTPLWlOifdryfgBTv+GGGkSvWbro2w0y/dOP3SlW4mvoc+2wxA==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t6lww-0005Z7-2B for bug-gnu-emacs@gnu.org; Fri, 01 Nov 2024 03:20:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Lin Sun Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 01 Nov 2024 07:20:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41646 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch fixed Original-Received: via spool by 41646-submit@debbugs.gnu.org id=B41646.173044557221338 (code B ref 41646); Fri, 01 Nov 2024 07:20:02 +0000 Original-Received: (at 41646) by debbugs.gnu.org; 1 Nov 2024 07:19:32 +0000 Original-Received: from localhost ([127.0.0.1]:47394 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t6lwR-0005Y6-Pj for submit@debbugs.gnu.org; Fri, 01 Nov 2024 03:19:32 -0400 Original-Received: from mail-ej1-f45.google.com ([209.85.218.45]:52557) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t6lwP-0005Xv-5n for 41646@debbugs.gnu.org; Fri, 01 Nov 2024 03:19:30 -0400 Original-Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a9a5f555cfbso113230366b.1 for <41646@debbugs.gnu.org>; Fri, 01 Nov 2024 00:19:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730445503; x=1731050303; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Hg5Ouhcus2xbD99hSCPDGXHUMKEnk1b9kCs8DDUb69c=; b=jpmAcGDbPbpII0pBdKEx0KSKG9aWujTVlZ7J3aiKV3ms9bdgsCOPjgjG5HIiz+oiBF kiWH5Tj7Ty8xFdjpA5ZjZfgqe0SQZ+ot7Tbo3Hrpr/gbjEWHMufScI49aSY/1m9XotVK ks+3c3GyuUWrNnqCMgjFQZ7N1rYuHY1VXIkbeAJFt9cVpNcfgga9zSe8/79OIsTZU5Bu /TftPa8h5yjSm7pQ3gQJn7fK7ZM9/cjVYzMyDXwdR/+wWztepo62f5/upVRyCrJBX5cx MTgaWUr19nbjeAUsJcJDcZmd33EUal4yz1K3klPwvcXoj1yFGeqIRuYxtiwNwjKeM3Il jcrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730445503; x=1731050303; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hg5Ouhcus2xbD99hSCPDGXHUMKEnk1b9kCs8DDUb69c=; b=C3oR+qFDGl/ngSZAr6hi1SJBgAkMW1+AxaI5GIolGUg02YeX1xX2aWgGjt2w4UqUf/ v1N/cSwu5Dz4hOSZrcpBVi3rDVWPKo1aOHq4qAFGc7p2CC7sZO2ffBLuNgUulOUUPvEZ rmAZFIXL9iZlHOoTGY3ZpaSVIpmtcULsq18HB2Ldt6n8scA7enOcQG25cIAISPZWMW3d CpyixF3+Kzp+YimjhmrGBXR0j1ytH6I4NDThtoQWuxMpMZw9nok01FeNzNgGkOfzJskC sEP9+56mJZDMky8SRx9At3c5rZVBE5kxw6fmF2KC8J+zTQTLcFDkzi3ROulLudUl6FrC m89w== X-Forwarded-Encrypted: i=1; AJvYcCX/jFYIxksxC3qjF8nYUolYLeSIcwTU0++BN7vYZghT8VFlKaWHGPmST9DDiJEEOoJU1yyuVQ==@debbugs.gnu.org X-Gm-Message-State: AOJu0Yz4r8aDdqolCen/ttOMktGJKMeIXT9NMS2Q0yQ82d7KENLx5Xc/ 7t+/tzp/pPo0PpOqgmPUMn9xzUkxd2DjMukeIrV5+Jn37Bmq060+FhzmtiP0C4Q388x/0e80+hT 6ZIbfzw1AGcymXRDgL5QQi5IMiaM= X-Google-Smtp-Source: AGHT+IEZ6uNr4317eF2SjSkZjEGqrvTGFXYlkeqN4t3MCSRSLtD0W1caFK2U+Kh5Ag/wCacXa8E7bA3GsWGA8zE5FTo= X-Received: by 2002:a05:6402:4308:b0:5c9:34b4:69a8 with SMTP id 4fb4d7f45d1cf-5ceb923ee52mr2165010a12.6.1730445502632; Fri, 01 Nov 2024 00:18:22 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:294660 Archived-At: On Thu, Oct 31, 2024 at 3:04=E2=80=AFPM Stefan Monnier wrote: > > >> The downside is that it can break existing setups for users who use > >> `package.el` but also modify their `load-path` "by hand" in the > >> init file, and it doesn't help users who don't use `package.el`. > > The `load-hints' do nothing with its default value nil. > > No, but if `package.el` adds elements to `load-hints` and then in your > init file you add more elements to `load-path`, the `load-hints` may end > up taking precedence over the new elements of `load-path` in some cases > and thus change which file shadows which. Yes, introducing this new variable will increase the complexity to the end = user. > > I had checked the radix-tree at the beginning, it's not user-friendly, > > or it's not easy to dump the radix tree for an end user to understand > > which is obviously matching the entry or not. > > You're not "supposed" to look at the representation of the radix tree, > indeed (unless you're working on `radix-tree.el`, of course). > Feel free to ask for help using it. > > > The `load-hints' in the list are easy to understand / maintain by the > > end user. > > I don't think we need to burden end users with load hints. > > >> But your prefix idea makes me think maybe we can aim for a significant= ly > >> smaller table, where we basically record only one entry per > >> package/directory, like for "~/.emacs.d/elpa/helm-core-VERSION/" we ju= st > >> record "helm" because all the `.el` files share the "helm" prefix. > >> I.e. keep for each dir the corresponding longest-common-prefix. > >> If we're careful to consider only those files with a `.el` suffix, the= n > >> I think we can reduce the hint to such a longest-common-prefix. I.e. = an > >> info which doesn't say just "you can find FOO* files here" but "you ca= n > >> find *only* FOO* files here". > > > > I had searched all 200+ packages in my test env, most of the packages > > use their feature name as the prefix, only 11 packages have > > exceptions. But I didn't understand how it works toward the > > `load-hints'. > > Here's the idea: the `-autoloads.el` file can registers the longest > common prefix of all the `.el` files for its own `load-path` entry, with > say: > > (load-prefix-register ) > > where we'd define this function along the lines of > > (defconst load-prefix-directories (make-hash-table :test 'equal)) > "Set of entries from `load-path` for which we have prefix info.") > > (defconst load-prefix-map radix-tree-empty > "Table associating file prefixes to directories.") > > (defun load-prefix-register (dir prefix) > (puthash dir t load-prefix-directories) > (let ((dirs (radix-tree-lookup load-prefix-map prefix))) > (unless (member dir dirs) > (setq load-prefix-map (radix-tree-insert load-prefix-map prefix > (cons dir dirs)))))) > > (defun load-prefix-trim-load-path (file) > "Return a trimmed `load-path` to use for FILE." > (if (file-name-directory file) > ;; If there's a `/` in FILE, fallback on the safe default. > load-path > (let* ((prefixes (radix-tree-prefixes load-prefix-map file)) > (dirs (apply #'append (mapcar #'cdr prefixes)))) > ;; Remove from `load-path` the entries which can't possibly > ;; have FILE because their prefixes doesn't match. > (cl-remove-if (lambda (dir) > (and (gethash dir load-prefix-directories) > (not (member dir dirs)))) > load-path)))) > > and then `load` can use `load-prefix-trim-load-path` to iterate on > a much shorter `load-path`. > > I'm not completely sure if it's a good idea, tho: I'd really prefer > a solution that doesn't require any change to any package management > code, which instead uses a cache (updated/filled automatically) of all > the files found in all the `load-path` directories. If that, we have to track the file/path changes in each entry of load-path, it may not be possible for all the supported OSs. > >> Is that with or without using `package-quickstart`? > > The `package-quickstart' does not help in this scenario, > > AFAICT your scenario includes Emacs startup with packages installed, so > `package-quickstart' can definitely make a difference. But maybe you're > right that it will affect both cases equally. I had tried the package-quickstart, it does NOT help on windows, like after load the quickstart.el, it will add all the packages directories into the load-path, the load-path may have ~300 entries, then a simple "(require 'org)" will trigger the emacs walk on the load-path one by one to try open (org.so, org.so.gz, org.elc, org.elc.gz, org.el, org.el.gz) for 6 times, while the org actually is on the bottom of load-path, so the emacs tried near 1800 (300x6) times to load one org file (its depends need similar counts), on windows systems, it's very slow. > >> Similarly, I wasn't able to convince myself that your patch does the > >> right thing when `require` or `load` is used such that MUST_SUFFIX is > >> not specified. > > The `load-hints` just put the matched paths on the top of `load-path`, > > still following the `load-path' mechanism, and won't affect any other > > features (Or someone already has some code to adjust the `load-path' > > orders, can just ignore the `load-hints', everything work like before, > > no break changes). > > But if FOO is in one dir and FOO.el is in another dir, adding entries > to the load path can change which file we end up loading. Yeh, that may happen, and the end user will get confused.