From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jim Porter Newsgroups: gmane.emacs.bugs Subject: bug#71355: 30.0.50; [PATCH] Improve performance of buffered output in Eshell Date: Wed, 5 Jun 2024 09:42:43 -0700 Message-ID: <037ebce9-93af-f1ad-67d9-550fd1074294@gmail.com> References: <22b0dc8f-11dc-5fd2-c75d-88c17580d28d@gmail.com> <848772e9-5ef0-8a8a-decd-c0b79366ec27@gmail.com> <86ikynk30i.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33054"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 71355@debbugs.gnu.org, stefankangas@gmail.com To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Jun 05 18:53:12 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sEtsu-0008Pf-3h for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 05 Jun 2024 18:53:12 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sEtsX-0004Ep-FZ; Wed, 05 Jun 2024 12:52:49 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sEtsW-0004EM-40 for bug-gnu-emacs@gnu.org; Wed, 05 Jun 2024 12:52:48 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sEtsV-0004l1-Rc for bug-gnu-emacs@gnu.org; Wed, 05 Jun 2024 12:52:47 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sEtsj-0001ux-IP for bug-gnu-emacs@gnu.org; Wed, 05 Jun 2024 12:53:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Jim Porter Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 05 Jun 2024 16:53:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 71355 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 71355-submit@debbugs.gnu.org id=B71355.17176063257232 (code B ref 71355); Wed, 05 Jun 2024 16:53:01 +0000 Original-Received: (at 71355) by debbugs.gnu.org; 5 Jun 2024 16:52:05 +0000 Original-Received: from localhost ([127.0.0.1]:48363 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEtro-0001sT-GD for submit@debbugs.gnu.org; Wed, 05 Jun 2024 12:52:04 -0400 Original-Received: from mail-ot1-f42.google.com ([209.85.210.42]:43279) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEtrm-0001rq-Ps for 71355@debbugs.gnu.org; Wed, 05 Jun 2024 12:52:03 -0400 Original-Received: by mail-ot1-f42.google.com with SMTP id 46e09a7af769-6f944cae15cso413866a34.1 for <71355@debbugs.gnu.org>; Wed, 05 Jun 2024 09:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717606243; x=1718211043; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=v0pVI2ddW48XfhQ2R0x0+kMdWePxlRul2aKHlG8SWF8=; b=ETUW/Z9Lx4NW9W8twISmoHs62UFRDXNZeWkd1W1D8/Q/Xjth6j33cjsQBeilM2cUXC vaGQM+Cf/DzlyoZSG13zCRaCLku1Igeul4V7N/iY/DqxQPIDEHUqdD5VfvC/rJxnPDXi OwVZS9xFP9eSevMZnlq3wRN1apCAK8ZxOPExLZ1cLHsCImFfcOVNDzX6Jp0FGs5us0ZQ t0cKT+NgsGArdjO1UgihajnOYu1gXjQUPJOWd6GqpZsN8XmZ9IH82DqmVaLBNw9QmZso AS9t0VB1CI2LyxGHu+ckJqT4dSbV4lQPXqDsPOFpU1hHwxNAk90M6Ghy5q+7FIWgtYHJ oAsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717606243; x=1718211043; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=v0pVI2ddW48XfhQ2R0x0+kMdWePxlRul2aKHlG8SWF8=; b=oxfFvAY5vCAqEOzG0PtJrFSAGDKwbtzZMHZP1ogp4iudwnrH7HYhjtSZ4S+jnBoKpN 6QUtmcVwPLDnJPHe69C/0nyDDukXOEK6swesoyayKJ6xGyzEPBW67wF+jKXLDVss4nWa ZZ95JZ5p3pZk8bboD2JTxSFPHro739b6AsajLf9WOIQOaUYINq762VrDBIs4n6IAQjG0 VzibFMFbPLB+dDWe+szPuCeOsuEX8L802VsSl9+KPVhBlhZmQkhjYuIWaGS8C0d/+Pk7 SX/A8HuGN7ufaQ6kjvx7pe1ZtUnDd6iHkmGYViAzRezzkNIXuIIB6ULF7Xkj9l2ukMc0 N+zw== X-Gm-Message-State: AOJu0YxzAJNiRzJRNh5/IEEUNnN8DSFYxcZc8k/3eW8AS/uOaezrNSQm WoF4MBtj8XsndtcHsayeD94Edlg57jmxMLmgQteBGr9Rx6qQq4MbHaO0Dw== X-Google-Smtp-Source: AGHT+IGZ22UORSW3gZfsdpQUZGTMEIgx5tS8KeT8OFmcNksgCyoGAr/tKxgZlbaxXIq3BRnEyREQvA== X-Received: by 2002:a17:902:cf0f:b0:1f6:8351:293d with SMTP id d9443c01a7336-1f6b8f214f6mr2270145ad.34.1717605762108; Wed, 05 Jun 2024 09:42:42 -0700 (PDT) Original-Received: from [192.168.1.2] (syn-023-240-098-037.res.spectrum.com. [23.240.98.37]) by smtp.googlemail.com with ESMTPSA id d9443c01a7336-1f65d3466easm76968715ad.256.2024.06.05.09.42.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 05 Jun 2024 09:42:41 -0700 (PDT) Content-Language: en-US In-Reply-To: <86ikynk30i.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:286618 Archived-At: On 6/5/2024 5:06 AM, Eli Zaretskii wrote: > Is 2K indeed the optimal size? It is about 25 80-column lines, which > is quite a lot. "Normal" shells send output to the screen in smaller > chunks. How about 128 instead? or maybe some small multiple of the > line length, like 1 or 2? Yes, I believe 2k is the optimal size, or close to it. Trying a value of 128 results in basically no change in performance from baseline. That makes sense to me, since 128 is actually fairly close to the old value for this buffering (which was five *lines*[1]; the old code measured this differently). In starting on this work, I compared to the amount of data that I get each time a child process filter is called; that's normally 4095 bytes in my tests. Then I started a bit lower (512) and gradually doubled the buffer size until I was able to get most of the performance benefit. A lot of this improvement does come from reducing the number of times that we call 'eshell-output-filter-functions', so if we made those functions much faster, or throttled them somewhere else, then we could get a large perf improvement while using a small buffer size. However, even if I remove everything from 'eshell-output-filter-functions', a larger buffer is *still* faster (0.2s for 2048 vs 0.5s for 128). For comparison, the same command in an ordinary shell takes about 0.1s, so that's my ultimate target. Note that this buffered output code is really only used when Eshell will output a (relatively) large block of text all at once from a Lisp command. External processes and most ordinary Lisp code won't use this, since you have to use the special functions for buffering your writes. [1] Well, 5 calls to the print function, but most callers printed a line at a time.