From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening large number of buffers Date: Fri, 29 Sep 2023 19:12:58 +0300 Message-ID: <83ttrdx8j9.fsf@gnu.org> References: <878r919qfh.fsf@localhost> <72c93fb0-bf3e-3dad-69c0-2147cfa40f57@gutov.dev> <875y42xyex.fsf@localhost> <87zg1ewfc2.fsf@localhost> <834jjm749q.fsf@gnu.org> <87cyyawd1a.fsf@localhost> <83pm2a5k85.fsf@gnu.org> <87wmwh2tae.fsf@localhost> <83zg1d468w.fsf@gnu.org> <87bkdr2651.fsf@localhost> <87pm2584oz.fsf@localhost> <83cyy11ln1.fsf@gnu.org> <87lecp84mf.fsf@localhost> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31064"; mail-complaints-to="usenet@ciao.gmane.io" Cc: dmitry@gutov.dev, 66117@debbugs.gnu.org To: Ihor Radchenko Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Sep 29 18:14:18 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qmG89-0007np-88 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 29 Sep 2023 18:14:17 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qmG7i-0000eJ-95; Fri, 29 Sep 2023 12:13:50 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qmG7g-0000cV-MR for bug-gnu-emacs@gnu.org; Fri, 29 Sep 2023 12:13:48 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qmG7g-0005bb-Dr for bug-gnu-emacs@gnu.org; Fri, 29 Sep 2023 12:13:48 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qmG7u-00060d-6j for bug-gnu-emacs@gnu.org; Fri, 29 Sep 2023 12:14:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 29 Sep 2023 16:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66117 X-GNU-PR-Package: emacs Original-Received: via spool by 66117-submit@debbugs.gnu.org id=B66117.169600402423062 (code B ref 66117); Fri, 29 Sep 2023 16:14:02 +0000 Original-Received: (at 66117) by debbugs.gnu.org; 29 Sep 2023 16:13:44 +0000 Original-Received: from localhost ([127.0.0.1]:57011 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qmG7c-0005zu-6A for submit@debbugs.gnu.org; Fri, 29 Sep 2023 12:13:44 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36058) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qmG7Y-0005zK-90 for 66117@debbugs.gnu.org; Fri, 29 Sep 2023 12:13:43 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qmG7B-0005Yf-U5; Fri, 29 Sep 2023 12:13:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=+j9OySZaBIS/DRcaWIYzmRH/8bZqLGtdUcdtTSBDuis=; b=B9gFxFAXaCG5 t+Hmft1sRzcTFuFaRp7LflgAwUoeWo/EaSybui0TBKk8zxzCgd8FTTz3ELKntzCT9MMmRyiaAWAxH JM3FgeGTXjqoe64NLfHPl/qcKGomASHwtGH06Dx1IvWnswFDuORhUHCCJ68CgCFkyMniF1V338kL5 VCMGMAUpWlkq3Qu89ZLVdr6sMlI3H8GsIuucd2WVUZtUl8Iz3/gujMnEH/xz3waGDbHiJdZOedm+G 9Pe6QaZl7EzlK5jvmEPQmTFuJEZtJ8YJHKz8JfnYdM81oUh3K65cAMEARkbXtM5zpoP+ttfABYD2L p8Sm1qJtvYklElUL1nNbKg==; In-Reply-To: <87lecp84mf.fsf@localhost> (message from Ihor Radchenko on Fri, 29 Sep 2023 13:56:40 +0000) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:271507 Archived-At: > From: Ihor Radchenko > Cc: dmitry@gutov.dev, 66117@debbugs.gnu.org > Date: Fri, 29 Sep 2023 13:56:40 +0000 > > Eli Zaretskii writes: > > >> 3. Open all the 1000 files one by one: > >> (dolist (file (directory-files "/tmp/test/" t "org")) > >> (unless (find-buffer-visiting file) (find-file-noselect file))) > >> > >> Step (3) takes 18.8 seconds on my machine. The CPU profile attached as > >> cpu-profile. > > > > Since find-file-noselect calls find-buffer-visiting internally, I'm > > not sure the above test case makes sense. A Lisp program should feel > > free to call find-file-noselect directly, and Emacs will find the > > visiting buffer, if it already exists, as part of the job of > > find-file-noselect. > > > > Let's please focus on test cases where the Lisp code being benchmarked > > doesn't do any unnecessary stuff, since what's at stake is a > > significant change in our internals. > > The reason I left an extra `find-buffer-visiting' call was because Org > mode does it (for a reason - we need information if a file was already > open or not). > > You may as well do > > (dolist (file (directory-files "/tmp/test/" t "org")) > (find-file-noselect file)) > > as step (3). > > The same conclusions will hold - `find-file-noselect' calls > `find-buffer-visiting' as well and it also takes most of the CPU time. > > I am attaching an updated set of the same profiles, but based on the > above `dolist' that only calls `find-file-noselect'. > > The run times are now: 12.0 seconds, 5.3 seconds, and 6.6 seconds. 12 sec is quite a far cry from 18.8, won't you agree? > >> If one uses `get-file-buffer' instead of `find-buffer-visiting', the > >> total runtime becomes 5.1 sec - almost 4x faster. > > > > This is also not very interesting, since find-file-noselect calls > > get-file-buffer as well. > > No. `find-file-noselect' calls `find-buffer-visiting'. Unless we use different Emacsen, find-file-noselect calls both get-file-buffer and find-buffer-visiting: (let* ((buf (get-file-buffer filename)) <<<<<<<<<<<<<<<<<<<<<<<<<<<< (truename (abbreviate-file-name (file-truename filename))) (attributes (file-attributes truename)) (number (file-attribute-file-identifier attributes)) ;; Find any buffer for a file that has same truename. (other (and (not buf) (find-buffer-visiting <<<<<<<<<<<<<<<<<<<<<<<<<<<< filename ;; We want to filter out buffers that we've ;; visited via symlinks and the like, where ;; the symlink no longer exists. (lambda (buffer) (let ((file (buffer-local-value 'buffer-file-name buffer))) (and file (file-exists-p file)))))))) > > If we come to the conclusion that those loops in find-buffer-visiting > > are the hot spot, the right thing is to implement them in C, where we > > don't need to use the equivalent of with-current-buffer to examine the > > truename and file-number of every buffer, we can just access them > > directly. > > I still think that my previous conclusions are true. And I agree that > rewriting these expensive loops in C makes sense. Maybe two new > subroutines to find buffer by `buffer-file-truename' and by > `buffer-file-number'? Yes, that's what I had in mind. > >> So, using `with-current-buffer' when looping over all the buffers is > >> certainly not optimal (maybe in other places as well). > > > > with-current-buffer is normally very expensive. Which is why any > > performance-critical loop should try to avoid it as much as possible. > > Aside: this reminds me about obsoletion of generalized buffer-local > variable. AFAIU, there is currently no way to set buffer-local value in > buffer without setting that buffer to current. It would be nice if such > setting were possible, especially in performance-critical code. Maybe, but is there any performance-critical code which needs that?