From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Date: Thu, 14 Sep 2023 08:41:11 +0300 Message-ID: <83bke5mhvs.fsf@gnu.org> References: <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@gutov.dev> <878rb3m43b.fsf@localhost> <83v8e6lyi4.fsf@gnu.org> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@gutov.dev> <59c30342-a7e0-d83b-a128-0faae4cbd633@gutov.dev> <83pm4bi6qa.fsf@gnu.org> <83bkfs2tw5.fsf@gnu.org> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@gutov.dev> <83il8lxjcu.fsf@gnu.org> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@gutov.dev> <8334zmtwwi.fsf@gnu.org> <83tts0rkh5.fsf@gnu.org> <831qf3pd1y.fsf@gnu.org> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@gutov.dev> <83sf7jnq0m.fsf@gnu.org> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@gutov.dev> <83ledanvzw.fsf@gnu.org> <83r0n2m7qz.fsf@gnu.org> <26afa109-9ba3-78a3-0e68-7585ae8e3a19@gutov.dev> <83il8dna30.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40365"; mail-complaints-to="usenet@ciao.gmane.io" Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, 64735@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Sep 14 07:42:24 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qgf7O-000AIG-U8 for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 14 Sep 2023 07:42:23 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qgf6z-0007uw-7o; Thu, 14 Sep 2023 01:41:57 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgf6y-0007uQ-E8 for bug-gnu-emacs@gnu.org; Thu, 14 Sep 2023 01:41:56 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qgf6y-00087I-4N for bug-gnu-emacs@gnu.org; Thu, 14 Sep 2023 01:41:56 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qgf73-0001kJ-Nc for bug-gnu-emacs@gnu.org; Thu, 14 Sep 2023 01:42:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 14 Sep 2023 05:42:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64735 X-GNU-PR-Package: emacs Original-Received: via spool by 64735-submit@debbugs.gnu.org id=B64735.16946701006684 (code B ref 64735); Thu, 14 Sep 2023 05:42:01 +0000 Original-Received: (at 64735) by debbugs.gnu.org; 14 Sep 2023 05:41:40 +0000 Original-Received: from localhost ([127.0.0.1]:36572 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgf6h-0001jj-UP for submit@debbugs.gnu.org; Thu, 14 Sep 2023 01:41:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:56714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgf6f-0001jU-7c for 64735@debbugs.gnu.org; Thu, 14 Sep 2023 01:41:39 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgf6S-00083R-UQ; Thu, 14 Sep 2023 01:41:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=CFuYYF71FoRm+7xntSM48ncTkuKifQE0ZqqGJ+bJEd8=; b=YwNZsL35mCFkOshehrsN Xxqua8iG1fvsCx4nGJe6ptj8d8g+Xvo0rjMrq9VY+PppgZywmsXL+UwUBfO7nB5t84oxg9l/sDd1R KrN7C3NsYX3CU4xObxVjyzDWgJpemf2aAUyXvQ5oeQq+l0ObX+r+/011+zjiNm87FCYDGzvo6BOwp tHc/T1J4CIHbuSl8cvtfCFnBG8fQkwf1RYWNHRVfpIq0BNmhWGx+XhDbRHBqjSRexzYyOsltbB7nk cM/YYNJqq1e6ud++5mbSo34hm16n3hgLHYDOJNOwBMQXvrvDHSoYmkgMkd/6h7guEnNmdefCYMMNA NZGSCEyNi/FaGw==; In-Reply-To: (message from Dmitry Gutov on Wed, 13 Sep 2023 23:38:29 +0300) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:270392 Archived-At: > Date: Wed, 13 Sep 2023 23:38:29 +0300 > Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, > 64735@debbugs.gnu.org > From: Dmitry Gutov > > > I think these questions are slightly premature. We should first have > > the implementation of that filter, and then look for candidates that > > could benefit from it. > > The implementation in that patch looks almost complete to me, unless you > have any further comments. Fine, then please post a complete patch with all the bells and whistles, and let's have it reviewed more widely. (I suggest a new bug report, as this one is already prohibitively long to follow, includes unrelated issues, and I fear some people will ignore patches posted to it). I think there are a few subtleties we still need to figure out. > The main difference would be the change in > the dispatch comparison from > > if (p->filter == Qinternal_default_process_filter) > > to > > if (p->filter == Qbuffer) Btw, both of the above are mistakes: you cannot compare Lisp objects as if they were simple values. You must use EQ. > This function gives PROCESS the filter function FILTER. If FILTER > is ‘nil’, it gives the process the default filter, which inserts > the process output into the process buffer. If FILTER is ‘t’, > Emacs stops accepting output from the process, unless it’s a > network server process that listens for incoming connections. > > What can we add? > > If FILTER is ‘buffer’, it works like the default one, only a bit faster. > > ? If FILTER is the symbol ‘buffer’, it works like the default filter, but makes some shortcuts to be faster: it doesn't adjust markers and the process mark (something else?). Of course, the real text will depend on what the final patch will look like: I'm not yet sure I understand which parts of internal-default-process-filter you want to keep in this alternative filter. (If you intend to keep all of them, it might be better to replace internal-default-process-filter completely, perhaps first with some variable exposed to Lisp which we could use to see if the new one causes issues.) > > My tendency is to change only callers which > > are in many cases expected to get a lot of stuff from a subprocess, so > > shell buffers are probably out. But we could discuss that later. > > When I'm thinking of start-file-process-shell-command, I have in mind > project--files-in-directory, which currently uses > process-file-shell-command. Though I suppose most cases would be more > easily converted to use make-process (like xref-matches-in-files uses > process-file for launching a shell pipeline already). > > I was also thinking about Flymake backends because those work in the > background. The outputs are usually small, but can easily grow in rare > cases, without particular limit. Flymake also runs in the background, > meaning whatever extra work it has to do (or especially GC pressure), > affects the delays when editing. I think we will have to address these on a case by case basis. The issues and aspects are not trivial and sometimes subtle. We might even introduce knobs to allow different pipe sizes if there's no one-fits-all value for a specific function using these primitives. > >> I'm not sure what negatives to test for, though. Raising the limit 10x > >> is unlikely to lead to an OOM, but I guess some processes could grow > >> higher latency?.. > > > > With a large buffer and small subprocess output we will ask the OS for > > a large memory increment for no good reason. Then the following GC > > will want to compact the gap, which means it will be slower. > > I wonder what scenario that might become apparent in. Launching many > small processes at once? Can't think of a realistic test case. One process suffices. The effect might not be significant, but slowdowns due to new features are generally considered regressions.