From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: sbaugh@catern.com Newsgroups: gmane.emacs.bugs Subject: bug#69775: [PATCH] Use regexp-opt in dired-omit-regexp Date: Sat, 23 Mar 2024 13:29:06 +0000 (UTC) Message-ID: <87sf0h6pr2.fsf@catern.com> References: <86o7bh9iz3.fsf@gnu.org> <8734sqjdyz.fsf@catern.com> <864jcz3m57.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18849"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: sbaugh@janestreet.com, 69775@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Mar 23 14:29:46 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ro1RR-0004gm-FT for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 23 Mar 2024 14:29:46 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ro1R7-0007Zf-Ji; Sat, 23 Mar 2024 09:29:25 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ro1R4-0007ZS-8G for bug-gnu-emacs@gnu.org; Sat, 23 Mar 2024 09:29:24 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ro1R3-0007qT-Vv for bug-gnu-emacs@gnu.org; Sat, 23 Mar 2024 09:29:22 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ro1Ri-0005le-NS for bug-gnu-emacs@gnu.org; Sat, 23 Mar 2024 09:30:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: sbaugh@catern.com Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 23 Mar 2024 13:30:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69775 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 69775-submit@debbugs.gnu.org id=B69775.171120059922135 (code B ref 69775); Sat, 23 Mar 2024 13:30:02 +0000 Original-Received: (at 69775) by debbugs.gnu.org; 23 Mar 2024 13:29:59 +0000 Original-Received: from localhost ([127.0.0.1]:55482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ro1Re-0005kv-C1 for submit@debbugs.gnu.org; Sat, 23 Mar 2024 09:29:59 -0400 Original-Received: from s.wfbtzhsw.outbound-mail.sendgrid.net ([159.183.224.105]:12152) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ro1Ra-0005kQ-1g for 69775@debbugs.gnu.org; Sat, 23 Mar 2024 09:29:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=catern.com; h=from:subject:in-reply-to:references:mime-version:to:cc:content-type: cc:content-type:from:subject:to; s=s1; bh=m2Jx+hSymSiTiLpRXofnPBlKPW4c9i15BBC2cGfx3WU=; b=PV5P4kDXnLY59RW+pstQ8QJJhVv9Ag/p2L9gGOo2XYt4unazWw4mrZuUKLEsDogZtOyc syFPsPP0dLaWlMwdzTIEIt3HAqnth29RLhsRT1YzPcnXC1ix7gfXJJ8RGny/cYPn4qFP2i Rvqpc0Idz28qghr9EUymrJ3VBlRQLLc9gjsrAA7cw0s3j0Ntks2eE6NUWnfn9Fe90Pw6oU +CLlvMI+fm4j4K1G5F46+OipA/m2GX+hCj5es5s/gpoBpgaXSv5NTKII48E4lefy3g+YPh B878fZQWeXDu4IqBFe4090a8tCtuNRsxUN6B9/76864OG2iqLYo2h85P+2Sf0/RA== Original-Received: by recvd-6fc499775c-sfjtq with SMTP id recvd-6fc499775c-sfjtq-1-65FED922-F 2024-03-23 13:29:06.343065723 +0000 UTC m=+405002.857703640 Original-Received: from earth.catern.com (unknown) by geopod-ismtpd-21 (SG) with ESMTP id guRUgrWlT1WeDTeKT107dA Sat, 23 Mar 2024 13:29:06.162 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=127.0.0.1; helo=localhost; envelope-from=sbaugh@catern.com; receiver=gnu.org Original-Received: from localhost (localhost [127.0.0.1]) by earth.catern.com (Postfix) with ESMTPSA id B36BE6006B; Sat, 23 Mar 2024 09:29:05 -0400 (EDT) In-Reply-To: <864jcz3m57.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 21 Mar 2024 12:38:28 +0200") X-SG-EID: u001.v6RTqHFpv1T6krEot6UFAVAJmQ+4h1t8/TfqqE2B07P9mDCopJ1hNJown0Giu3fh1hTzAF6l60VLcZPW3/fWwFYzccUrjBAcFsjYvHUAObCQTReQ9YYoKhAtKsCf+fkQVFWIGdGnhK1FuAhmGkkTaHzKWmOlUhYoF5JLgsk/7Xhw9xkuBGxjQPSkP4u1zWCK8KMRfs6AbF0kKCiG3JabDQ== X-Entity-ID: u001.oW4JupFKOzCccZAQN2OOFQ== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:281979 Archived-At: --=-=-= Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Eli Zaretskii writes: >> From: sbaugh@catern.com >> Date: Sat, 16 Mar 2024 17:15:52 +0000 (UTC) >> Cc: Spencer Baugh , 69775@debbugs.gnu.org >> >> $ ./src/emacs -Q --batch -l ../emacs-29/bench-omit.elc >> files 1, ntimes 100: new 0.008839979999999999 old 0.018162129 new-uncached 0.031399762 >> files 10, ntimes 100: new 0.012037615 old 0.040232355000000004 new-uncached 0.037990543 >> files 100, ntimes 100: new 0.07368538100000001 old 0.314905271 new-uncached 0.10006527300000001 >> files 1000, ntimes 100: new 0.669103498 old 3.076339984 new-uncached 0.693134644 >> files 10000, ntimes 100: new 6.336211434 old 30.926320486 new-uncached 6.442762152999999 >> >> So the performance improvement is quite substantial for large >> directories. >> >> new-uncached is the performance if dired-omit-extensions changes on each >> call of dired-omit-regexp. For a directory of 1 file, the overhead of >> recomputing regexp-opt every time makes the performance perhaps 2x-3x >> worse, but around 10 files the performance improvement from regexp-opt >> exceeds the overhead, and above that the uncached version still >> outperforms the old version substantially. > > SGTM, thanks. > >> Certainly, updated patch attached. >> >> +@defvar dired-omit-size-limit >> +If non-@code{nil}, omitting will be skipped if the directory listing >> +exceeds this size in bytes. > > I'd rephrase > > If non-@code{nil}, @code{dired-omit-mode} will be effectively > disabled in directories whose listing has size (in bytes) larger > than the value of this option. > >> +*** The default value of 'dired-omit-size-limit' has increased. > ^^^ > "was" is better there. > >> +(defcustom dired-omit-size-limit 300000 >> + "Maximum buffer size for `dired-omit-mode'. >> + >> +Omitting will be skipped if the directory listing exceeds this size in > ^^^^^^^ > "disabled" Adjusted all these, here's the new patch. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Use-regexp-opt-in-dired-omit-regexp.patch >From 7ae71f994814f3eb7bad3387608e8c0cbe0d2d68 Mon Sep 17 00:00:00 2001 From: Spencer Baugh Date: Sat, 16 Mar 2024 17:11:24 +0000 Subject: [PATCH] Use regexp-opt in dired-omit-regexp In my benchmarking, for large dired buffers, using regexp-opt provides around a 3x speedup in omitting. regexp-opt takes around 5 milliseconds, so to avoid slowing down omitting in small dired buffers we cache the return value. Since omitting is now 3x faster, increase dired-omit-size-limit by 3x. Also, document dired-omit-size-limit better. * doc/misc/dired-x.texi (Omitting Variables): Document dired-omit-size-limit. * etc/NEWS: Announce increase of dired-omit-size-limit. * lisp/dired-x.el (dired-omit--extension-regexp-cache): Add. (dired-omit-regexp): Use regexp-opt. (bug#69775) (dired-omit-size-limit): Increase and improve docs. --- doc/misc/dired-x.texi | 9 +++++++++ etc/NEWS | 6 ++++++ lisp/dired-x.el | 26 ++++++++++++++++++++------ 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/doc/misc/dired-x.texi b/doc/misc/dired-x.texi index 4cad016a0f6..726b6653d0d 100644 --- a/doc/misc/dired-x.texi +++ b/doc/misc/dired-x.texi @@ -346,6 +346,15 @@ Omitting Variables match the file name relative to the buffer's top-level directory. @end defvar +@defvar dired-omit-size-limit +If non-@code{nil}, @code{dired-omit-mode} will be effectively disabled +in directories whose listing has size (in bytes) larger than the value +of this option. Since omitting can be slow for very large directories, +this avoids having to wait before seeing the directory. This variable +is ignored when @code{dired-omit-mode} is called interactively, such as +by @code{C-x M-o}, so you can still enable omitting in the directory +after the initial display. + @cindex omitting additional files @defvar dired-omit-marker-char Temporary marker used by Dired to implement omitting. Should never be used diff --git a/etc/NEWS b/etc/NEWS index 50f0ee4a1aa..9324d9f1dcf 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -692,6 +692,12 @@ marked or clicked on files according to the OS conventions. For example, on systems supporting XDG, this runs 'xdg-open' on the files. +*** The default value of 'dired-omit-size-limit' was increased. +After performance improvements to omitting in large directories, the new +default value is 300k, up from 100k. This means 'dired-omit-mode' will +omit files in directories whose directory listing is up to 300 kilobytes +in size. + +++ *** 'dired-listing-switches' handles connection-local values if exist. This allows to customize different switches for different remote machines. diff --git a/lisp/dired-x.el b/lisp/dired-x.el index 62fdd916e69..753d3054d2f 100644 --- a/lisp/dired-x.el +++ b/lisp/dired-x.el @@ -77,12 +77,17 @@ dired-vm-read-only-folders (other :tag "non-writable only" if-file-read-only)) :group 'dired-x) -(defcustom dired-omit-size-limit 100000 - "Maximum size for the \"omitting\" feature. +(defcustom dired-omit-size-limit 300000 + "Maximum buffer size for `dired-omit-mode'. + +Omitting will be disabled if the directory listing exceeds this size in +bytes. This variable is ignored when `dired-omit-mode' is called +interactively. + If nil, there is no maximum size." :type '(choice (const :tag "no maximum" nil) integer) :group 'dired-x - :version "29.1") + :version "30.1") (defcustom dired-omit-case-fold 'filesystem "Determine whether \"omitting\" patterns are case-sensitive. @@ -506,14 +511,23 @@ dired-omit-expunge (re-search-forward dired-re-mark nil t)))) count))) +(defvar dired-omit--extension-regexp-cache + nil + "A cache of `regexp-opt' applied to `dired-omit-extensions'. + +This is a cons whose car is a list of strings and whose cdr is a +regexp produced by `regexp-opt'.") + (defun dired-omit-regexp () + (unless (equal dired-omit-extensions (car dired-omit--extension-regexp-cache)) + (setq dired-omit--extension-regexp-cache + (cons dired-omit-extensions (regexp-opt dired-omit-extensions)))) (concat (if dired-omit-files (concat "\\(" dired-omit-files "\\)") "") (if (and dired-omit-files dired-omit-extensions) "\\|" "") (if dired-omit-extensions (concat ".";; a non-extension part should exist - "\\(" - (mapconcat 'regexp-quote dired-omit-extensions "\\|") - "\\)$") + (cdr dired-omit--extension-regexp-cache) + "$") ""))) ;; Returns t if any work was done, nil otherwise. -- 2.41.0 --=-=-=--