From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Chris Hanson Newsgroups: gmane.emacs.bugs Subject: bug#66288: 29.1; Performance regression using pipe for subprocess Date: Mon, 2 Oct 2023 14:22:06 -0400 Message-ID: References: <83ttra91nu.fsf@gnu.org> <63df0312-f2e0-49cd-8536-f886d841e88e@chris-hanson.org> <83edid8vmx.fsf@gnu.org> <83bkdh8u2k.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="11900"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: 66288@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Oct 02 20:23:07 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qnNZS-0002iL-Fi for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 02 Oct 2023 20:23:06 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qnNZH-0002Hi-8r; Mon, 02 Oct 2023 14:22:55 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qnNZ8-0002H9-6a for bug-gnu-emacs@gnu.org; Mon, 02 Oct 2023 14:22:49 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qnNZ7-0000jO-5u for bug-gnu-emacs@gnu.org; Mon, 02 Oct 2023 14:22:45 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qnNZN-0002lI-Ri for bug-gnu-emacs@gnu.org; Mon, 02 Oct 2023 14:23:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Chris Hanson Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 02 Oct 2023 18:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66288 X-GNU-PR-Package: emacs Original-Received: via spool by 66288-submit@debbugs.gnu.org id=B66288.169627095510581 (code B ref 66288); Mon, 02 Oct 2023 18:23:01 +0000 Original-Received: (at 66288) by debbugs.gnu.org; 2 Oct 2023 18:22:35 +0000 Original-Received: from localhost ([127.0.0.1]:37913 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qnNYx-0002kb-5Q for submit@debbugs.gnu.org; Mon, 02 Oct 2023 14:22:35 -0400 Original-Received: from relay4-d.mail.gandi.net ([217.70.183.196]:44629) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qnNYv-0002kM-IJ for 66288@debbugs.gnu.org; Mon, 02 Oct 2023 14:22:34 -0400 Original-Received: by mail.gandi.net (Postfix) with ESMTPSA id DD143E0004; Mon, 2 Oct 2023 18:22:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chris-hanson.org; s=gm1; t=1696270930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ex1gk5GFUnrL12ZgSK3fEaY/KXujPbIRlMqPsdyjjyY=; b=oLKgJH2T5fIqy7WwpRB3132jrsqrl4K7dD6ODHt0VbW1g7n7cBbNU1GfUo+RYwcWUni6U+ 2W2V/FhCczlW0cNzOKuVMO8DQ9jMf3vGdf3bJtYm08HIpNGBwh1mC0KVcqSTeuT3G39GPN diEOVBP4VqPbwJAj2rjCGtWQ7IkZ7vSfWzyXxT2ed0zRDr9eKsBMewsm1NWaQF/LrujHdM 0Fjnj2dizlHNfr6WYfppMHS4fg58KW7UP1hYMJNXjf3jKPst2vW5KqMVQasoHUZQegU2D0 ggOOMqz7J2TE+vkJQ1adWIlW8WTHvjk5U1D34ai/3T66K8/deURoZceVma0ROQ== Content-Language: en-US In-Reply-To: <83bkdh8u2k.fsf@gnu.org> X-GND-Sasl: cph@chris-hanson.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:271668 Archived-At: On 10/2/23 01:36, Eli Zaretskii wrote: >> Cc: 66288@debbugs.gnu.org >> Date: Mon, 02 Oct 2023 08:02:14 +0300 >> From: Eli Zaretskii >> >>> I saw that there were no relevant differences in "xscheme.el" but I >>> never thought that was relevant. >>> >>> I believe this has something to do with how piped subprocesses are being >>> managed. I've not looked deeply into the C code for this, but I could >>> find no mention of anything to do with pipes in NEWS. >> >> Because AFAIK we didn't change anything in that department. > > I've now identified 3 changes in Emacs 29 which could potentially > affect your case. Not sure if they do, but it might be worth your > while to check them first. > > First, Emacs 29 uses posix_spawn by default on systems where it is > available and usable. You will see this fragment at the beginning of > callproc.c: > > /* In order to be able to use `posix_spawn', it needs to support some > variant of `chdir' as well as `setsid'. */ > #if defined HAVE_SPAWN_H && defined HAVE_POSIX_SPAWN \ > && defined HAVE_POSIX_SPAWNATTR_SETFLAGS \ > && (defined HAVE_POSIX_SPAWN_FILE_ACTIONS_ADDCHDIR \ > || defined HAVE_POSIX_SPAWN_FILE_ACTIONS_ADDCHDIR_NP) \ > && defined HAVE_DECL_POSIX_SPAWN_SETSID \ > && HAVE_DECL_POSIX_SPAWN_SETSID == 1 \ > /* posix_spawnattr_setflags rejects POSIX_SPAWN_SETSID on \ > Haiku */ \ > && !defined HAIKU > # include > # define USABLE_POSIX_SPAWN 1 > #else > # define USABLE_POSIX_SPAWN 0 > #endif > > If on your system USABLE_POSIX_SPAWN gets the value 1 here, edit > callproc.c to force it to zero, then rebuild Emacs, and see if this > affects the behavior. > > Next, we have the following two code fragments in > wait_reading_process_output, which are new in Emacs 29: > > Code fragment#1: > > if ((read_kbd > /* The following code doesn't make any sense for just the > wait_for_cell case, because detect_input_pending returns > whether or not the keyboard buffer isn't empty or there > is mouse movement. Any keyboard input that arrives > while waiting for a cell will cause the select call to > be skipped, and gobble_input to be called even when > there is no input available from the terminal itself. > Skipping the call to select also causes the timeout to > be ignored. (bug#46935) */ > /* || !NILP (wait_for_cell) */) > && detect_input_pending ()) > > Code fragment#2: > > #if !defined USABLE_SIGIO && !defined WINDOWSNT > /* If we're polling for input, don't get stuck in select for > more than 25 msec. */ > struct timespec short_timeout = make_timespec (0, 25000000); > if ((read_kbd || !NILP (wait_for_cell)) > && timespec_cmp (short_timeout, timeout) < 0) > timeout = short_timeout; > #endif > > (I think the second one should not affect you because your system > should have USABLE_SIGIO defined, but maybe I'm mistaken.) Compare > these with Emacs 28, and try reverting to 28.2 code to see if that > changes anything in your case. None of the three fragments made any difference. > Finally, if you describe in plain English how xscheme.el reads > subprocess output at the stage where you see the slowdown, it might > give further ideas. I'm not familiar with xscheme.el, and figuring > out which code gets executed when one runs "run-scheme" is not > trivial, so a detailed enough description might help. Specifically, > how does xscheme.el decide how much of the subprocess's output to read > and display? The process filter has one complexity: it looks for encoded commands from the subprocess, which are of the form "ESC " or "ESC ESC", depending on the . There is a small state machine to do that, which searches the output string for ESC using `string-search'. In this case there are no commands embedded, so that should not be relevant. The ordinary text is inserted into the process buffer using standard filter-output code, except it looks for BEL and translates that to (beep) if found. In this case there are no occurrences of BEL in the output, so that's not relevant. So, basically the output string is passed to `insert', making sure that process mark and point are updated appropriately. I contrived a small example test and ran it under both editors (see below). It does some printing and then shows the time taken in the subprocess. This should be valid since Scheme will block while waiting on Emacs to process the output. The reported times are in milliseconds, with 28.2 taking 1ms and 29.1 taking 880ms (increasing the test loop from 20 to 200, the times are 8ms and 9974ms respectively). As I said before, that's pretty dramatic: about 3 orders of magnitude. It feels like that in normal use too -- it's like being 30-40 years in the past, when that kind of performance was expected. 28.2: -------------------------------- (show-time (lambda () (for-each write-line (iota 20)))) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ;process time: 0 (0 RUN + 0 GC); real time: 1 -------------------------------- 29.1: -------------------------------- (show-time (lambda () (for-each write-line (iota 20)))) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ;process time: 0 (0 RUN + 0 GC); real time: 880 --------------------------------