From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Spencer Baugh Newsgroups: gmane.emacs.devel Subject: Re: Excessive redisplay from lots of process output Date: Fri, 17 Feb 2023 11:41:27 -0500 Message-ID: References: <834jrk1dli.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5401"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org, azeng@janestreet.com To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Feb 17 18:05:48 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pT4BA-0001AJ-BV for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Feb 2023 18:05:48 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pT4AP-0002sP-AB; Fri, 17 Feb 2023 12:05:01 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pT3nq-0004Am-Su for emacs-devel@gnu.org; Fri, 17 Feb 2023 11:41:42 -0500 Original-Received: from mxout5.mail.janestreet.com ([64.215.233.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pT3np-0000Ob-8N for emacs-devel@gnu.org; Fri, 17 Feb 2023 11:41:42 -0500 Original-Received: from mail-yb1-f199.google.com ([209.85.219.199]) by mxgoog2.mail.janestreet.com with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.96) id 1pT3nm-00239S-0R for emacs-devel@gnu.org; Fri, 17 Feb 2023 11:41:38 -0500 Original-Received: by mail-yb1-f199.google.com with SMTP id a132-20020a25ca8a000000b00827819f87e5so1276153ybg.0 for ; Fri, 17 Feb 2023 08:41:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=OVZgxJjWxRHU8VCT6Pl+J8EGB04qTXjnLLxRVdkzDCM=; b=Nc6KWTDQeO5EGSS0W+H71O51MSL2JiNxzLZf76BoSGTa4lm914baY/fZinwWTfCmvJ /N7hrbqAxCBqkVSD/F5u+lMZqY2yaO4Yny4+1PGGdRRGM/YNb/yFJWyBwXe9UnlDVTv+ uieYUjLZ5AtkpOAAClFNahcR3P/PKtiWCQp8s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OVZgxJjWxRHU8VCT6Pl+J8EGB04qTXjnLLxRVdkzDCM=; b=vSr/8aJEJsGfjNCJLV+gbQGONTNLyAk+1ji6ijBFic/1FCSB5+baEqK2JzEX7MbdCm F2K9JvV3vl87/0aSI4CnIQM0SOhWvyV4KTNr6K6o834yyC0gAMisXAol5qTf4dJ+bUDD 9SM+PM/zpXAkO3mCZYbdLLlpfmHn0AacLL2vP4nvm28Nz6tjYSNWTj1qme+U/FZ/WzB8 HMOBz8c45wXA3uk7DW23T8aKpIgSy7AbLltamZddtyCc8p4TLSbelKi3KZXknn856dAp bjCpqb5N7MH7DUvchhn8o8nng5eHFXUEKpBcimrLnlRQqQ7fXLW68hFkjY8vHfOQCwD3 trHA== X-Gm-Message-State: AO0yUKVVPKGVPhKJqPZEjmgOJU06nvwweCCEJTNzsLTeq5wUt921WUC+ DT17T9F6mi73lEbyPGIqHzHyg7/jvz67OeeqpNzEnM/Yp6OFXuYe1jw+7Qsmb+TEe/qflfQBrxV TaaP0keJ8lojnO5hqmwdS2HO53Po= X-Received: by 2002:a81:5c04:0:b0:535:1880:d09e with SMTP id q4-20020a815c04000000b005351880d09emr392532ywb.492.1676652098572; Fri, 17 Feb 2023 08:41:38 -0800 (PST) X-Google-Smtp-Source: AK7set+2P2g8zAuHDX+MP5TX4BpjpZstN1kYTN6lA8TX+Y3l/C0b67wCJ/4EEPO5Mvy1lOtu1sN8G+I2jUKyG9BYD2k= X-Received: by 2002:a81:5c04:0:b0:535:1880:d09e with SMTP id q4-20020a815c04000000b005351880d09emr392529ywb.492.1676652098366; Fri, 17 Feb 2023 08:41:38 -0800 (PST) In-Reply-To: <834jrk1dli.fsf@gnu.org> Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@janestreet.com; helo=mxout5.mail.janestreet.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 17 Feb 2023 12:05:00 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:303505 Archived-At: On Fri, Feb 17, 2023 at 11:06 AM Eli Zaretskii wrote: > > > From: Spencer Baugh > > Cc: azeng@janestreet.com > > Date: Fri, 17 Feb 2023 10:16:11 -0500 > > > > Emacs calls redisplay every time it reads process output. This causes > > Emacs to become painfully slow when these two conditions are met: > > > > - When redisplay is expensive (e.g. when using ssh X forwarding) > > > > - And when there is frequent process output, say hundreds or thousands > > of times a second (e.g. ERC receiving network traffic, shells > > running commands which log heavily, other things which run in the > > background) > > This cannot be the whole story, because usually redisplay exits almost > immediately after being called, having determined that nothing needs > to be changed on display. > > So the important part of your situation that you didn't disclose is: > why does Emacs in your case perform the expensive redisplay each time > redisplay is entered? Can you figure that out by stepping through > redisplay_internal? (Let me know if you need guidance.) Oh, this was the key information for me to realize what was *really* going on. In Emacs 27, in-progress key sequences get replayed each time there is process output to read, see bug#32922. This causes redisplay to have actual work to do each time it reads process output. But this only happens mid key-sequence, so it's not really a general issue with Emacs performance, but it *does* cause the X server to get swamped and therefore slows down X in general, including the window manager and other X clients. But fortunately that key-sequence replaying behavior was deleted in Emacs 28, just as a general cleanup, see bug#5803. Which helpfully means that redisplay is once again always cheap, so the bad X performance of having lots of process output disappears. Sorry, a mix of upgrading to Emacs 28 and also applying this patch made it not clear what was helping performance. But now it is clear. > Not sure here is the right place for this kind of discussion, btw. It > might be better to submit a bug report with all the relevant details, > and discuss this on the bug tracker. > > > I realize that reading process output can trigger filters which can > > change window and frame configurations, which in turn means redisplay > > needs to happen. But isn't there some way we could improve this > > situation? Right now these redisplays are causing serious > > user-visible performance degradation. > > There are many variables and flags Emacs uses to decide when something > changed that needs the display to be updated. Those should generally > prevent Emacs from running the expensive parts of redisplay if they > are unnecessary. The question is why this doesn't work in your case. > > IOW, it is better to consider this as some specific failure in some > particular situation, and find why it happens, instead of considering > this as some general failure of the Emacs design. Design-wise, things > are fine; it's something in your specific case that could fail the > existing mechanisms, and the question is: what is that and why it > fails? Oh yes, that makes sense, now I see, thanks for the explanation.