From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Date: Mon, 11 Sep 2023 14:57:10 +0300 Message-ID: <83tts0rkh5.fsf@gnu.org> References: <83sf9f6wm0.fsf@gnu.org> <83sf9eub9d.fsf@gnu.org> <2d844a34-857d-3d59-b897-73372baac480@gutov.dev> <83bkg2tsu6.fsf@gnu.org> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@gutov.dev> <834jlttv1p.fsf@gnu.org> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@gutov.dev> <83a5vlsanw.fsf@gnu.org> <69a98e2a-5816-d36b-9d04-8609291333cd@gutov.dev> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@gutov.dev> <878rb3m43b.fsf@localhost> <83v8e6lyi4.fsf@gnu.org> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@gutov.dev> <59c30342-a7e0-d83b-a128-0faae4cbd633@gutov.dev> <83pm4bi6qa.fsf@gnu.org> <83bkfs2tw5.fsf@gnu.org> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@gutov.dev> <83il8lxjcu.fsf@gnu.org> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@gutov.dev> <8334zmtwwi.fsf@gnu.org> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28876"; mail-complaints-to="usenet@ciao.gmane.io" Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, 64735@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Sep 11 13:58:16 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qffYV-0007F5-Ag for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 11 Sep 2023 13:58:15 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qffYH-0006Mu-Ca; Mon, 11 Sep 2023 07:58:01 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qffYF-0006ML-QL for bug-gnu-emacs@gnu.org; Mon, 11 Sep 2023 07:57:59 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qffYE-0004kM-Ho for bug-gnu-emacs@gnu.org; Mon, 11 Sep 2023 07:57:58 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qffYI-0002V2-4k for bug-gnu-emacs@gnu.org; Mon, 11 Sep 2023 07:58:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 11 Sep 2023 11:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64735 X-GNU-PR-Package: emacs Original-Received: via spool by 64735-submit@debbugs.gnu.org id=B64735.16944334629578 (code B ref 64735); Mon, 11 Sep 2023 11:58:02 +0000 Original-Received: (at 64735) by debbugs.gnu.org; 11 Sep 2023 11:57:42 +0000 Original-Received: from localhost ([127.0.0.1]:52273 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qffXx-0002UP-Ka for submit@debbugs.gnu.org; Mon, 11 Sep 2023 07:57:42 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43040) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qffXu-0002U9-Id for 64735@debbugs.gnu.org; Mon, 11 Sep 2023 07:57:40 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qffXk-0004cx-5q; Mon, 11 Sep 2023 07:57:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ApsTiErd2jgr2dnSg7gSl250EvRb98aN/jZKQ2h1of8=; b=UE7elJ/Jjkdl 5X2B2ZvL0opu+nsl9Zxwl0T9oEoO7fI8Ke9PraQkw2MWMruj8HSEAq/57pIT2bsGCww06y/r0s11U IK0BfXCG/nPX8A1QTg5et86IUg9NA10qFJlAXrYFKsMfi6/Vak92JQ5vjF1HAt0zk/rmBPdCNzUAK rYgFc5H/ElKrmO+9Hx90QxG9ZwnexQjMKkm/ip7jTupL3JTytlg36/hsRwBYhcgLWnfi878vgxr6L rf3vHs1S6CltmFkGQoPoRCP4HFKHXNDb24JIY+Y2uLU/362rQDxEfsyjYV23HLN4vV9vlN3tpX5v3 Xm+POh5xKWqI+onV57/DGg==; In-Reply-To: (message from Dmitry Gutov on Mon, 11 Sep 2023 03:02:55 +0300) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:270036 Archived-At: > Date: Mon, 11 Sep 2023 03:02:55 +0300 > Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, > 64735@debbugs.gnu.org > From: Dmitry Gutov > > > You could record its value in a local variable at the entry to > > garbage_collect, and the expose that value to Lisp. > > That also doesn't seem to give much, given that the condition for > entering 'maybe_garbage_collect' is (consing_until_gc < 0). I.e. we wait > until it's down to 0, then garbage-collect. No, we don't wait until it's zero, we perform GC on the first opportunity that we _notice_ that it crossed zero. So examining how negative is the value of consing_until_gc when GC is actually performed could tell us whether we checked the threshold with high enough frequency, and comparing these values between different runs could tell us whether the shorter time spend in GC means really less garbage or less frequent checks for the need to GC. > What's in there? First of all, for find-directory-files-recursively-3, > there are 0 garbage collections between the beginning of the function > and when we start parsing the output (no GCs while the process is > writing to the buffer synchronously). I guess inserting output in a > buffer doesn't increase consing, so there's nothing to GC? No, we just don't count increasing size of buffer text in the "consing since GC" counter. Basically, buffer text is never "garbage", except when a buffer is killed. > Next: for find-directory-files-recursively-2, the process only finishes > at the end, when all GC cycles are done for. I suppose that also means > we block the process's output while Lisp is running, and also that > whatever GC events occur might coincide with the chunks of output coming > from the process, and however many of them turn out to be in total. We don't block the process when GC runs. We do stop reading from the process, so if and when the pipe fills, the OS will block the process. > So there is also a second recording for > find-directory-files-recursively-2 with read-process-output-max=409600. > It does improve the performance significantly (and reduce the number of > GC pauses). I guess what I'm still not clear on, is whether the number > of GC pauses is fewer because of less consing (the only column that > looks significantly different is the 3rd: VECTOR-CELLS), or because the > process finishes faster due to larger buffers, which itself causes fewer > calls to maybe_gc. I think the latter.