From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#41646: Startup in Windows is very slow when load-path contains many Date: Mon, 21 Oct 2024 10:34:24 -0400 Message-ID: References: <86jzedy84g.fsf@gnu.org> <86r08luqsq.fsf@gnu.org> <86frp1unvu.fsf@gnu.org> <86y12stv24.fsf@gnu.org> <86set0th9d.fsf@gnu.org> <86iktwt49w.fsf@gnu.org> <86cyk4t2su.fsf@gnu.org> <86a5f8t0sf.fsf@gnu.org> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37161"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , acorallo@gnu.org, 41646@debbugs.gnu.org, stefankangas@gmail.com, monnier@gnu.org To: Lin Sun Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Oct 21 16:35:50 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1t2tVe-0009Vw-0S for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 21 Oct 2024 16:35:50 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t2tVV-0004KV-0l; Mon, 21 Oct 2024 10:35:41 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t2tVS-0004K4-0l for bug-gnu-emacs@gnu.org; Mon, 21 Oct 2024 10:35:38 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t2tVP-0008VO-Oc for bug-gnu-emacs@gnu.org; Mon, 21 Oct 2024 10:35:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=Wvh4alkL+OnmCnBJYh7PKjlosPOz+2uyvy3PevEGR9M=; b=Hn5x2+RYkBWayj3diCNFMPy8LeR8cHuAjyW0MNXqErmQHCVuEJssEl5cuMOK1dxEpymEqyyMCAmxhoqFbJgRghxKcQ2mpGqCVZGtg+520lZWcBq+JQWVYFU8n83goCwQuzsSKc38b53aF5XASKPFAlAQM52HhfYSe/8BpsdfsgmdMruLmsX9Apwu3g5v60n2U0uq76eQQWRws0ozUfAXPbXq+3/rbsLJ13YnimE7bYbcUnqVBV/o22CWB/UeUeu/29QwS2xvODt6Mu7VyWPSRguqqX9wEuLVUh8L/TOtoYTYqa8IEJkzDnJoB+Q4UEl/W4/KEC2x++vf09TPJRoTRA==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t2tVq-0004xQ-9D for bug-gnu-emacs@gnu.org; Mon, 21 Oct 2024 10:36:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 21 Oct 2024 14:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41646 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch fixed Original-Received: via spool by 41646-submit@debbugs.gnu.org id=B41646.172952130218606 (code B ref 41646); Mon, 21 Oct 2024 14:36:02 +0000 Original-Received: (at 41646) by debbugs.gnu.org; 21 Oct 2024 14:35:02 +0000 Original-Received: from localhost ([127.0.0.1]:52786 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t2tUr-0004po-Kc for submit@debbugs.gnu.org; Mon, 21 Oct 2024 10:35:02 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:10032) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t2tUp-0004pN-8m for 41646@debbugs.gnu.org; Mon, 21 Oct 2024 10:35:00 -0400 Original-Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id CEB0A1000C3; Mon, 21 Oct 2024 10:34:26 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1729521266; bh=2VAvbJsgvoD6AqHji2DsDNZC3NkXFKERdZUfGF2wGlg=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=lYnwlMVP7xrPLvorr2EUiPj46/g7FFTQPW2ttKC6Tl69fRllusI+5uXtEBtXUpi8b pQldk45LXnbztylUFBNoejsNWFNdTGufJkeWS7Fq+7lofNWJpkImMwZpfj4hnwXDCZ HF/6ibl8dy9jfwjZsUEXIfIrczJQCpdXGW6RODnSHsfLMgaYLwOPqf7sEsni1WdX22 tIP9FheZ/xNebAzt0MXyYELlhGrubCWvjDhwv2raNI+4Cd+WSnhsh1gz10iKKYz7Y4 nALyO//o2x5/oqw7DnR/h2DIoyZwElRkvnQnIslo0opF2ImZyy4Q46omHPBiNmwSYb bxbp1Mzk9RhNg== Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 136EC100042; Mon, 21 Oct 2024 10:34:26 -0400 (EDT) Original-Received: from pastel (69-196-161-60.dsl.teksavvy.com [69.196.161.60]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id CBB801207F2; Mon, 21 Oct 2024 10:34:25 -0400 (EDT) In-Reply-To: (Lin Sun's message of "Mon, 21 Oct 2024 04:09:25 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:294064 Archived-At: > The `load-hints` can reduce searching attempts by putting the matched > paths on the top of `load-path`, it won't break the original > load-path; and the patch for the `package.el` will put the installed > files into the `load-hints`. The downside is that it can break existing setups for users who use `package.el` but also modify their `load-path` "by hand" in the init file, and it doesn't help users who don't use `package.el`. Note also that your `load-hints` could grow large, so scanning it could take a significant amount of time. Maybe it would make sense to turn it into a hash table for those entries that don't use the "*" special thingy (and maybe use a radix-tree for those entries using the "*" special thingy)? But your prefix idea makes me think maybe we can aim for a significantly smaller table, where we basically record only one entry per package/directory, like for "~/.emacs.d/elpa/helm-core-VERSION/" we just record "helm" because all the `.el` files share the "helm" prefix. I.e. keep for each dir the corresponding longest-common-prefix. If we're careful to consider only those files with a `.el` suffix, then I think we can reduce the hint to such a longest-common-prefix. I.e. an info which doesn't say just "you can find FOO* files here" but "you can find *only* FOO* files here". Then we should be able to create quickly (so it can be recomputed on the fly whenever `load-path` changes) a radix-tree that maps a relative file name to the list of directories from `load-path` where it is worthwhile to look (by filtering out those dirs whose longest-common-prefix doesn't match). We'd only do it for MUST_SUFFIX is specified, of course. > 1. On my local linux test env, disable load-hints on the package.el, > the test cli spends 6.327s; and enable load-hints then it spends > 5.392s. > > 2. On my local Windows test env, disable load-hints on the package.el, > the test cli spends 11.769s, and enable load-hints then it spends > 7.279s. Is that with or without using `package-quickstart`? BTW, in your patch, you change `locate-file-internal` which seems wrong, since that function is not specific to loading ELisp files, it's also used for $MANPATH, $PATH, and things like that. Similarly, I wasn't able to convince myself that your patch does the right thing when `require` or `load` is used such that MUST_SUFFIX is not specified. Stefan