From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening large number of buffers Date: Fri, 29 Sep 2023 10:30:58 +0300 Message-ID: <83cyy11ln1.fsf@gnu.org> References: <878r919qfh.fsf@localhost> <72c93fb0-bf3e-3dad-69c0-2147cfa40f57@gutov.dev> <875y42xyex.fsf@localhost> <87zg1ewfc2.fsf@localhost> <834jjm749q.fsf@gnu.org> <87cyyawd1a.fsf@localhost> <83pm2a5k85.fsf@gnu.org> <87wmwh2tae.fsf@localhost> <83zg1d468w.fsf@gnu.org> <87bkdr2651.fsf@localhost> <87pm2584oz.fsf@localhost> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5392"; mail-complaints-to="usenet@ciao.gmane.io" Cc: dmitry@gutov.dev, 66117@debbugs.gnu.org To: Ihor Radchenko Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Sep 29 09:33:19 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qm7zy-000148-S2 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 29 Sep 2023 09:33:18 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qm7zZ-0004HQ-Cs; Fri, 29 Sep 2023 03:32:53 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qm7zV-0004EA-JW for bug-gnu-emacs@gnu.org; Fri, 29 Sep 2023 03:32:49 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qm7zT-0004Mi-8H for bug-gnu-emacs@gnu.org; Fri, 29 Sep 2023 03:32:48 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qm7zh-0001eg-Rz for bug-gnu-emacs@gnu.org; Fri, 29 Sep 2023 03:33:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 29 Sep 2023 07:33:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66117 X-GNU-PR-Package: emacs Original-Received: via spool by 66117-submit@debbugs.gnu.org id=B66117.16959727326296 (code B ref 66117); Fri, 29 Sep 2023 07:33:01 +0000 Original-Received: (at 66117) by debbugs.gnu.org; 29 Sep 2023 07:32:12 +0000 Original-Received: from localhost ([127.0.0.1]:55068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qm7ys-0001dT-AQ for submit@debbugs.gnu.org; Fri, 29 Sep 2023 03:32:11 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qm7yc-0001cJ-Bq for 66117@debbugs.gnu.org; Fri, 29 Sep 2023 03:32:08 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qm7yB-00045J-O4; Fri, 29 Sep 2023 03:31:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=4RGrRMDr2mb6xEJJ1NAiOdIMcQdSq4Icbl5WoLmnwlc=; b=IBwI6f/iY5vy HAFO34oOXIy1JGIsMc5d8bkCaQTEqdfnLA+NAqZFbryTVjBXoLJRpEPIQEUo1r+Mf/zFu424yD5nf x0eedO/D5QknVonS5VaLDPGOJHthkJ+fDS7AgJG9yHoRryiJDOZ8QbD4LX0Bt0uASRyKflp1RPHgF 8mw6OooIAeGKflfF2mLMpZOGz5L9bPjp9weo1hpDdA85D6ABZF4i6yc35Q8GVfDsSfocFl+UWbsRX GdyZ8cwmjYvSdT0J/pJDV6Mdq8fvK6oD3cg4Cnjlh9+wak2z9zH21cvIAXU8vsZspVu79VbO9/gjH ZQDCgv++XLkpjzfSTNDgvQ==; In-Reply-To: <87pm2584oz.fsf@localhost> (message from Ihor Radchenko on Tue, 26 Sep 2023 13:06:04 +0000) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:271472 Archived-At: > From: Ihor Radchenko > Cc: dmitry@gutov.dev, 66117@debbugs.gnu.org > Date: Tue, 26 Sep 2023 13:06:04 +0000 > > Here is a reproducer anyone can try locally: > > 1. Create a dummy set of 1000 files in /tmp/test/: > (dotimes (i 1000) (with-temp-file (format "/tmp/test/%d.org" i) (insert "* This is test"))) > > 2. emacs -Q > 3. Open all the 1000 files one by one: > (dolist (file (directory-files "/tmp/test/" t "org")) > (unless (find-buffer-visiting file) (find-file-noselect file))) > > Step (3) takes 18.8 seconds on my machine. The CPU profile attached as > cpu-profile. Since find-file-noselect calls find-buffer-visiting internally, I'm not sure the above test case makes sense. A Lisp program should feel free to call find-file-noselect directly, and Emacs will find the visiting buffer, if it already exists, as part of the job of find-file-noselect. Let's please focus on test cases where the Lisp code being benchmarked doesn't do any unnecessary stuff, since what's at stake is a significant change in our internals. > If one uses `get-file-buffer' instead of `find-buffer-visiting', the > total runtime becomes 5.1 sec - almost 4x faster. This is also not very interesting, since find-file-noselect calls get-file-buffer as well. > So, it looks like caching `get-file-buffer' is not really necessary. I don't think we are ready for conclusions yet, see above. > >From the profile, the slowest parts of `find-buffer-visiting' are the > two loops checking `buffer-file-truename' and `buffer-file-number' with > most of the time apparently spent executing `with-current-buffer'. I > tested whether `with-current-buffer' is the culprit by replacing it with > `buffer-local-value' calls: If we come to the conclusion that those loops in find-buffer-visiting are the hot spot, the right thing is to implement them in C, where we don't need to use the equivalent of with-current-buffer to examine the truename and file-number of every buffer, we can just access them directly. > The result is 7.8 sec execution time - much better compared to 18.8 > seconds in `with-current-buffer' version, but still worse compared to > 5.1 sec in `get-file-buffer' version. See the attached > cpu-profile-buffer-local-value. As explained above, both the 18.8 and the 5.1 figures are not good base lines upon which to make decisions. > So, using `with-current-buffer' when looping over all the buffers is > certainly not optimal (maybe in other places as well). with-current-buffer is normally very expensive. Which is why any performance-critical loop should try to avoid it as much as possible.