From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Date: Wed, 13 Sep 2023 20:27:09 +0300 Message-ID: <26afa109-9ba3-78a3-0e68-7585ae8e3a19@gutov.dev> References: <69a98e2a-5816-d36b-9d04-8609291333cd@gutov.dev> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@gutov.dev> <878rb3m43b.fsf@localhost> <83v8e6lyi4.fsf@gnu.org> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@gutov.dev> <59c30342-a7e0-d83b-a128-0faae4cbd633@gutov.dev> <83pm4bi6qa.fsf@gnu.org> <83bkfs2tw5.fsf@gnu.org> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@gutov.dev> <83il8lxjcu.fsf@gnu.org> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@gutov.dev> <8334zmtwwi.fsf@gnu.org> <83tts0rkh5.fsf@gnu.org> <831qf3pd1y.fsf@gnu.org> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@gutov.dev> <83sf7jnq0m.fsf@gnu.org> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@gutov.dev> <83ledanvzw.fsf@gnu.org> <83r0n2m7qz.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40181"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, 64735@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Sep 13 19:28:39 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qgTfK-000AEH-Iw for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 13 Sep 2023 19:28:39 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qgTei-0003SX-0d; Wed, 13 Sep 2023 13:28:00 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgTef-0003SF-GM for bug-gnu-emacs@gnu.org; Wed, 13 Sep 2023 13:27:57 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qgTef-0005Yz-86 for bug-gnu-emacs@gnu.org; Wed, 13 Sep 2023 13:27:57 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qgTek-0005oz-Ie for bug-gnu-emacs@gnu.org; Wed, 13 Sep 2023 13:28:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 13 Sep 2023 17:28:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64735 X-GNU-PR-Package: emacs Original-Received: via spool by 64735-submit@debbugs.gnu.org id=B64735.169462604922309 (code B ref 64735); Wed, 13 Sep 2023 17:28:02 +0000 Original-Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 17:27:29 +0000 Original-Received: from localhost ([127.0.0.1]:35891 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgTeC-0005nk-Qo for submit@debbugs.gnu.org; Wed, 13 Sep 2023 13:27:29 -0400 Original-Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:45405) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgTe9-0005nW-FP for 64735@debbugs.gnu.org; Wed, 13 Sep 2023 13:27:27 -0400 Original-Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id DB7DF3200989; Wed, 13 Sep 2023 13:27:13 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 13 Sep 2023 13:27:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694626033; x=1694712433; bh=VDCkBYEU3Ebxlnylz5EUpmpYkukUVkNSGuj 0ISVOiwk=; b=pg/+pvH8NcfvYCEDSx9sWJzy6L4LLntKqM41m8wyAHsrrKEsLYS vTOLIDFZEeGk7o6CdzsaJHj6pD1LNSElqEACMgjDZqG50VtGB2v3UDczT53R8vjg s7wB/kKDYOfWPfm94VIzL8HywPtGDgAzU0F2tHnmOz0QB/NxevrQRdVGq2u/OWnq 4zhJRWo+ACrzh12zrL+SXh+ShrshUBtmOHYGdTrJyH4SypTZHTwlkOTrUBEMbWWv UwK9qifyerpq6Ux0gYEO2dXJzpw/6wIl/oincYjFedD6bwB6htUu0g3JlP8zaS18 KLvXHJk6Llsqqny+/yFTllPwjdu1aMG7Zyw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1694626033; x=1694712433; bh=VDCkBYEU3Ebxlnylz5EUpmpYkukUVkNSGuj 0ISVOiwk=; b=b+Rrx2dFfS4hBpqLkjPfcE5RNnodJe2Y43NJAwfuo+R6hdEfXm3 wN8fxpj/tD84o+LiozB/Y0VnYfErBG6WyjALiy4a0J+DvgEkRxa9l+7rhtHxqZVM MMgcg4sPTEQDS+95sibneUjORFuXksJvMdNYBHUy3vesvWdkbH7DB36hKPy3Ha2w QOff9T5jF40dtMlLo51kHR7enwm3Ha5neESreKkWIWQvnSIHEWHQASnmxdZQO/ek XWTX0kKNYJHA4nYmOVx0Qaukk9K2giDA2q6UDRKFxU6qgXpsBHJY2yMcqUWuDPNp q2lqDSykXA0THwZtddBqCoe2At+iwrUvWeg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeikedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeev ledvveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 13 Sep 2023 13:27:11 -0400 (EDT) Content-Language: en-US In-Reply-To: <83r0n2m7qz.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:270341 Archived-At: On 13/09/2023 18:07, Eli Zaretskii wrote: >> Date: Wed, 13 Sep 2023 17:27:49 +0300 >> Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net, >> 64735@debbugs.gnu.org >> From: Dmitry Gutov >> >>>> Does the patch from my last patch-bearing email look similar enough to >>>> what you're describing? >>>> >>>> The one called read_and_insert_process_output.diff >>> >>> No, not entirely: it still produces a Lisp string when decoding is >>> needed, and then inserts that string into the buffer. >> >> Are you sure? IIUC the fact that is passes 'curbuf' as the last argument >> to 'decode_coding_c_string' means that decoding happens inside the >> buffer. This has been my explanation for the performance improvement anyway. > > Yes, you are right, sorry. So we're not going to try the gap-based approach? Okay. >>> Sure, but in this case we don't need any filtering. It's basically >>> the same idea as internal-default-process-filter: we just need to >>> insert the process output into a buffer, and optionally decode it. >> >> Pretty much. But that raises the question of what to do with the >> existing function internal-default-process-filter. > > Nothing. It will remain as the default filter. Okay, if you are sure. >>> We can provide a special filter identified by a symbol. Such a filter >>> will not be Lisp-callable, it will exist for the cases where we need >>> to insert the output into the process buffer. >> >> The would be the safest alternative. OTOH, this way we'd pass up on the >> opportunity to make all existing asynchronous processes without custom >> filters, a little bit faster in one fell swoop. > > We could change the ones we care about, though. Which ones do we care about? I've found a bunch of 'make-process' calls without :filter specified (flymake backends, ). Do we upgrade them all? The difference is likely not critical in most of them, but the change would likely result in small reduction of GC pressure in the corresponding Emacs sessions. We'll also need to version-guard the ones that are in ELPA. We don't touch the implementations of functions like start-file-process, right? What about the callers of functions like start-file-process-shell-command who want to take advantage of the improvement? Are we okay with them all having to call (set-process-filter proc 'buffer) on the returned process value? >> Should we also discuss increasing the default of >> read-process-output-max? Even increasing it 10x (not necessarily 100x) >> creates a noticeable difference, especially combined with the proposed >> change. > > That should be limited to specific cases where we expect to see a lot > of stuff coming from the subprocess. So it would be okay to bump it in particular functions? Okay. > We could also discuss changing > the default value, but that would require measurements in as many > cases as we can afford. If you have some particular scenarios in mind, and what to look out for, I could test them out at least on one platform. I'm not sure what negatives to test for, though. Raising the limit 10x is unlikely to lead to an OOM, but I guess some processes could grow higher latency?..