From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Date: Wed, 13 Sep 2023 23:38:29 +0300 Message-ID: References: <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@gutov.dev> <878rb3m43b.fsf@localhost> <83v8e6lyi4.fsf@gnu.org> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@gutov.dev> <59c30342-a7e0-d83b-a128-0faae4cbd633@gutov.dev> <83pm4bi6qa.fsf@gnu.org> <83bkfs2tw5.fsf@gnu.org> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@gutov.dev> <83il8lxjcu.fsf@gnu.org> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@gutov.dev> <8334zmtwwi.fsf@gnu.org> <83tts0rkh5.fsf@gnu.org> <831qf3pd1y.fsf@gnu.org> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@gutov.dev> <83sf7jnq0m.fsf@gnu.org> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@gutov.dev> <83ledanvzw.fsf@gnu.org> <83r0n2m7qz.fsf@gnu.org> <26afa109-9ba3-78a3-0e68-7585ae8e3a19@gutov.dev> <83il8dna30.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24657"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, 64735@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Sep 13 22:39:25 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qgWdv-00062b-Cx for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 13 Sep 2023 22:39:24 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qgWdY-0004S8-E7; Wed, 13 Sep 2023 16:39:00 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgWdV-0004Rr-6F for bug-gnu-emacs@gnu.org; Wed, 13 Sep 2023 16:38:57 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qgWdU-0005sB-FD for bug-gnu-emacs@gnu.org; Wed, 13 Sep 2023 16:38:56 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qgWdZ-0007fG-S3 for bug-gnu-emacs@gnu.org; Wed, 13 Sep 2023 16:39:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 13 Sep 2023 20:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64735 X-GNU-PR-Package: emacs Original-Received: via spool by 64735-submit@debbugs.gnu.org id=B64735.169463753729451 (code B ref 64735); Wed, 13 Sep 2023 20:39:01 +0000 Original-Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 20:38:57 +0000 Original-Received: from localhost ([127.0.0.1]:36183 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgWdQ-0007et-Se for submit@debbugs.gnu.org; Wed, 13 Sep 2023 16:38:57 -0400 Original-Received: from wout3-smtp.messagingengine.com ([64.147.123.19]:57285) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgWdJ-0007ea-Ku for 64735@debbugs.gnu.org; Wed, 13 Sep 2023 16:38:51 -0400 Original-Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 3C2D832009A4; Wed, 13 Sep 2023 16:38:34 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 13 Sep 2023 16:38:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694637513; x=1694723913; bh=sELkUu/ROWCusXj+gqW6rAA6N72rSwsQEOe VjtxuYlQ=; b=hO5uJVW2YRocSDOQrqWSaNxW8PvO84YbuLiSBz5dDyyF24X/I48 aO/i4bg6WXC4GPCT821FwrqrrGawaYGG0741XCz1A139pOllBNvWOzYo/8QmPt/l FTFq+AvVnCkzD1ICQpL++7USCOTwQSdAz6gKcAutkoqsocdDWrDjhoYxLx6zTJEW rpH/uUuoccsQopp5iBMZAhyV6rx5dnWYjEQ5Kx196PWC8RqFGiCk6y3c2nfM8Ovi LSg/3gkBZRN1g6Q8wi/ubjnN60nutlnf2vsRgbLrAVgyDsau96vfGb8pHXcqCPYR dQnWUdfd1us2WAFCdNbSDe1kjPgOA6Rirnw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1694637513; x=1694723913; bh=sELkUu/ROWCusXj+gqW6rAA6N72rSwsQEOe VjtxuYlQ=; b=MWbwc3sdT2+wEXP1oj7L85dkyFDomklnwXR2zTqz4cLeCfyltZ0 MFU7AnFbYc4z5bcBDImEHx+OgD/zJUe7Ycxzzi8/Jf4yoNFdfRmipp7x35c6azhy daQFCktp39T61ep8ZZCT+EFpibPIh3mEaTDTFvzU+LVG0cBOy2gv6aipkm7/+PaL XqDLSG8p439wYyKUFYB24tl+BKEfs2ArGmQXJgE07y9zkqRAPrhSBdyxoEJQXmLz zTx9B6q7cIiDcBEjZUg+TquwlTYdZKayomDTAPOq8DuMn7RyL4sMtDscI6K7LX29 bekbJnBoz7/5730Gut30yUFsO3ymWowiUwA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeikedgudehvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthekredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpefhffehleejffegffeugefhkeektdffgfehjedvgeejtedtudehueffgffg feejheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 13 Sep 2023 16:38:31 -0400 (EDT) Content-Language: en-US In-Reply-To: <83il8dna30.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:270362 Archived-At: On 13/09/2023 22:32, Eli Zaretskii wrote: >>> We could change the ones we care about, though. >> >> Which ones do we care about? I've found a bunch of 'make-process' calls >> without :filter specified (flymake backends, ). Do we upgrade them all? >> >> The difference is likely not critical in most of them, but the change >> would likely result in small reduction of GC pressure in the >> corresponding Emacs sessions. >> >> We'll also need to version-guard the ones that are in ELPA. >> >> We don't touch the implementations of functions like start-file-process, >> right? >> >> What about the callers of functions like >> start-file-process-shell-command who want to take advantage of the >> improvement? Are we okay with them all having to call >> (set-process-filter proc 'buffer) on the returned process value? > > I think these questions are slightly premature. We should first have > the implementation of that filter, and then look for candidates that > could benefit from it. The implementation in that patch looks almost complete to me, unless you have any further comments. The main difference would be the change in the dispatch comparison from if (p->filter == Qinternal_default_process_filter) to if (p->filter == Qbuffer) , I think. Of course I can re-submit the amended patch, if you like. Regarding documentation, though. How will we describe that new value? The process filter is described like this in the manual: This function gives PROCESS the filter function FILTER. If FILTER is ‘nil’, it gives the process the default filter, which inserts the process output into the process buffer. If FILTER is ‘t’, Emacs stops accepting output from the process, unless it’s a network server process that listens for incoming connections. What can we add? If FILTER is ‘buffer’, it works like the default one, only a bit faster. ? > My tendency is to change only callers which > are in many cases expected to get a lot of stuff from a subprocess, so > shell buffers are probably out. But we could discuss that later. When I'm thinking of start-file-process-shell-command, I have in mind project--files-in-directory, which currently uses process-file-shell-command. Though I suppose most cases would be more easily converted to use make-process (like xref-matches-in-files uses process-file for launching a shell pipeline already). I was also thinking about Flymake backends because those work in the background. The outputs are usually small, but can easily grow in rare cases, without particular limit. Flymake also runs in the background, meaning whatever extra work it has to do (or especially GC pressure), affects the delays when editing. >>> We could also discuss changing >>> the default value, but that would require measurements in as many >>> cases as we can afford. >> >> If you have some particular scenarios in mind, and what to look out for, >> I could test them out at least on one platform. > > Didn't think about that enough to have scenarios. > >> I'm not sure what negatives to test for, though. Raising the limit 10x >> is unlikely to lead to an OOM, but I guess some processes could grow >> higher latency?.. > > With a large buffer and small subprocess output we will ask the OS for > a large memory increment for no good reason. Then the following GC > will want to compact the gap, which means it will be slower. I wonder what scenario that might become apparent in. Launching many small processes at once? Can't think of a realistic test case. Anyway, if you prefer to put off the discussion about changing the default, that's fine by me. Or split into a separate bug.