From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lin Sun Newsgroups: gmane.emacs.bugs Subject: bug#41646: Startup in Windows is very slow when load-path contains many Date: Mon, 21 Oct 2024 17:11:17 +0000 Message-ID: References: <86jzedy84g.fsf@gnu.org> <86r08luqsq.fsf@gnu.org> <86frp1unvu.fsf@gnu.org> <86y12stv24.fsf@gnu.org> <86set0th9d.fsf@gnu.org> <86iktwt49w.fsf@gnu.org> <86cyk4t2su.fsf@gnu.org> <86a5f8t0sf.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27563"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , acorallo@gnu.org, 41646@debbugs.gnu.org, stefankangas@gmail.com, monnier@gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Oct 21 19:12:54 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1t2vxd-00071z-Gb for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 21 Oct 2024 19:12:53 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t2vxP-0004Fe-LH; Mon, 21 Oct 2024 13:12:39 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t2vxN-0004FV-7p for bug-gnu-emacs@gnu.org; Mon, 21 Oct 2024 13:12:37 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t2vxL-00020y-Op for bug-gnu-emacs@gnu.org; Mon, 21 Oct 2024 13:12:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=Date:From:In-Reply-To:References:MIME-Version:To:Subject; bh=rhl0ZYuOCk/FIPq37Oet58l9OBRhsnlPRXuHenETsXw=; b=k58Hx+xmsoBd9+KzkPbfj1gbfDHeSMgVcJoyV9nrtdoUpwmU/eMmXQryJ8OI9kU/JNuF+kfQ2hYbrdO/KZg8Pw4babrf1utqEP9Xcn7PXbUGY429/I391PKdAaOknpu369hXSdRu5Cn3TnFdA0riwjDfmEgSU5au9Mo5zjgQt3aXCSGPL+YFmeMvuxU6tP+cj5Xl09Xtt+862MRzFcRpzv8cT1UhZwt8OhMsiOVHbEPbEsFXQDDpd0m7kSQDvIXkANmVcmYThp/ozg6ySPYOCjZSLTBTlm4/xY1BJe/xXRLAJtcImCAsh4wEF/rdOkfYlvYgxqgIW7RmQGlIOfypQw==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t2vxm-0003mm-D9 for bug-gnu-emacs@gnu.org; Mon, 21 Oct 2024 13:13:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Lin Sun Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 21 Oct 2024 17:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41646 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch fixed Original-Received: via spool by 41646-submit@debbugs.gnu.org id=B41646.172953077814540 (code B ref 41646); Mon, 21 Oct 2024 17:13:02 +0000 Original-Received: (at 41646) by debbugs.gnu.org; 21 Oct 2024 17:12:58 +0000 Original-Received: from localhost ([127.0.0.1]:53151 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t2vxi-0003mR-5o for submit@debbugs.gnu.org; Mon, 21 Oct 2024 13:12:58 -0400 Original-Received: from mail-wm1-f42.google.com ([209.85.128.42]:51280) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t2vxg-0003mJ-0H for 41646@debbugs.gnu.org; Mon, 21 Oct 2024 13:12:56 -0400 Original-Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-4315abed18aso43774795e9.2 for <41646@debbugs.gnu.org>; Mon, 21 Oct 2024 10:12:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729530688; x=1730135488; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=rhl0ZYuOCk/FIPq37Oet58l9OBRhsnlPRXuHenETsXw=; b=Lgtqieh6IAHWkFxY3mK09K4ht/2kC1ygJ0p4IHqBdcJCJVJdiRt740W0wseX4ijbCh heU/r3tFutKLsLdlSnEq1ULDonB6J2ncH2oEpb552UndQ013XTesOcfPyiHarPyyIBU7 9hZSqJZILtNIx/mlLPrIbiOrG9QDR9tXeFxNGDqkMHr7PQc8Mg96ORRM2TC5NBvoVAdd 4NawXVNFgd7D8+O0Aj+cK/9qbo4p1uUlH6UAUWKCrJoOQtajSam18DVGb/eawr+UMaMd 4FLYLoCvrT3L0s3tFY6idmXKkRXTlSR3qabbPzQaoS/d6bbtBq7pwQiWInFKAwWjPdlM uuzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729530688; x=1730135488; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rhl0ZYuOCk/FIPq37Oet58l9OBRhsnlPRXuHenETsXw=; b=ukzenQ3Sg0ceXnskSr9cWPG/35qzsxZNv77+uKgK62eTZFkGO8M5Tri/Y3fhFeeN8q 4y02TBsjFhwaPx2HS1gpdZ3gjkxabKt4PxBA8bzG0jMocoWjYQO8yLfY/XKXxbddi80Y clt6vH0DZ25WifdLwaQuCdjhRhpv7uN0TKLI2KZCHBAwVRKzBI08kEYacOgp0tJgBCpv tSq0VvRiSDXzweBeedgV+jYCQ668An+w9ylhkZuj7com6uxSxqNkgbHsVB370ovk8Tvj PRgLtskcN/aPlIKjmDq20OfpEMCnVKOTZR8p78w2H286At+KS1xD2B5cbbxRuv8KmcY8 eDZg== X-Forwarded-Encrypted: i=1; AJvYcCVmdOcZLOG3CI1d7/Y9DAjN0VzZ4ZjyNGoVVJUxxGVsMS1rC4zeXp9XHekSq2ZcuBZ2ledVHA==@debbugs.gnu.org X-Gm-Message-State: AOJu0Yywi2ttEbVYW9vRHoHxeG1Mnue64L+tj+tK2hmZnmJ5yZdCBRrB q2QC79yrd7TgRn2ALzWZSCp983y183pzTOtVLpMHbLyj7mNGHsJaoQnSAEgKJT4uBpy8A9kR0M+ mgyfdqaMPk55NwO9FXVbwfMX7Kls= X-Google-Smtp-Source: AGHT+IG5VG5kaIc+l+R+fsSC/XX6VjuiJ608VieA/v9tUNArjN5lbJLokYcEwkWyG/9auahRPKawmL++OBwGeZE8B8U= X-Received: by 2002:a5d:4e4d:0:b0:37d:542a:7872 with SMTP id ffacd0b85a97d-37eab7281ecmr7852024f8f.49.1729530688344; Mon, 21 Oct 2024 10:11:28 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:294076 Archived-At: On Mon, Oct 21, 2024 at 2:34=E2=80=AFPM Stefan Monnier wrote: > > > The `load-hints` can reduce searching attempts by putting the matched > > paths on the top of `load-path`, it won't break the original > > load-path; and the patch for the `package.el` will put the installed > > files into the `load-hints`. > > The downside is that it can break existing setups for users who use > `package.el` but also modify their `load-path` "by hand" in the > init file, and it doesn't help users who don't use `package.el`. The `load-hints' do nothing with its default value nil. The package manager can use the `load-hints' or ignore it. The "package.el" is emacs builtin package manager, can support or ignore the `load-hints' by setting `package-enable-load-hints` to t or nil. Other package managers can continue without any change, or do some work to get performance benefits by supporting the `load-hints'. > Note also that your `load-hints` could grow large, so scanning it could > take a significant amount of time. Maybe it would make sense to turn it > into a hash table for those entries that don't use the "*" special > thingy (and maybe use a radix-tree for those entries using the "*" > special thingy)? The `load-hints` could grow as the `load-path', or larger than the `load-path', but it should not be too much. For the hash table, it can not support the '*', or the hash-table requires every file to have an entry explicitly, which costs a lot on building the table. I had checked the radix-tree at the beginning, it's not user-friendly, or it's not easy to dump the radix tree for an end user to understand which is obviously matching the entry or not. The `load-hints' in the list are easy to understand / maintain by the end u= ser. > But your prefix idea makes me think maybe we can aim for a significantly > smaller table, where we basically record only one entry per > package/directory, like for "~/.emacs.d/elpa/helm-core-VERSION/" we just > record "helm" because all the `.el` files share the "helm" prefix. > I.e. keep for each dir the corresponding longest-common-prefix. > If we're careful to consider only those files with a `.el` suffix, then > I think we can reduce the hint to such a longest-common-prefix. I.e. an > info which doesn't say just "you can find FOO* files here" but "you can > find *only* FOO* files here". I had searched all 200+ packages in my test env, most of the packages use their feature name as the prefix, only 11 packages have exceptions. But I didn't understand how it works toward the `load-hints'. > Then we should be able to create quickly (so it can be recomputed on the > fly whenever `load-path` changes) a radix-tree that maps a relative > file name to the list of directories from `load-path` where it is > worthwhile to look (by filtering out those dirs whose > longest-common-prefix doesn't match). We'd only do it for MUST_SUFFIX > is specified, of course. > > > 1. On my local linux test env, disable load-hints on the package.el, > > the test cli spends 6.327s; and enable load-hints then it spends > > 5.392s. > > > > 2. On my local Windows test env, disable load-hints on the package.el, > > the test cli spends 11.769s, and enable load-hints then it spends > > 7.279s. > > Is that with or without using `package-quickstart`? The `package-quickstart' does not help in this scenario, the key point is the `load-path` count will times the read attempt. A simple "(require 'X)" will lead the emacs walks through the `load-path' to attempt opening the "X.so, X.so.gz, X.elc, X.elc.gz, X.el, X.el.gz" one by one, if the `load-path' has 200 entries, emacs will try search 200x6=3D1200 times for the worst case. The `load-hints` will help put the matched path to the top of `load-path` then emacs can find the X on the top entries of `load-path' then returns shortly. > BTW, in your patch, you change `locate-file-internal` which seems wrong, > since that function is not specific to loading ELisp files, it's also > used for $MANPATH, $PATH, and things like that. > > Similarly, I wasn't able to convince myself that your patch does the > right thing when `require` or `load` is used such that MUST_SUFFIX is > not specified. I'm going to search the cases carefully. Thank you for all the comments, appreciate it.