From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#66020: (bug#64735 spin-off): regarding the default for read-process-output-max Date: Sun, 24 Sep 2023 08:29:37 +0300 Message-ID: <838r8w3zr2.fsf@gnu.org> References: <83il8lxjcu.fsf@gnu.org> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@gutov.dev> <8334zmtwwi.fsf@gnu.org> <83tts0rkh5.fsf@gnu.org> <831qf3pd1y.fsf@gnu.org> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@gutov.dev> <83sf7jnq0m.fsf@gnu.org> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@gutov.dev> <83ledanvzw.fsf@gnu.org> <83r0n2m7qz.fsf@gnu.org> <26afa109-9ba3-78a3-0e68-7585ae8e3a19@gutov.dev> <83il8dna30.fsf@gnu.org> <83bke5mhvs.fsf@gnu.org> <83a5tmk79p.fsf@gnu.org> <937d9927-506f-aa36-94e9-3cceb8f629dd@gutov.dev> <83zg1hay6q.fsf@gnu.org> <451d6012-e5ab-df6c-50e3-dac20b91781c@gutov.dev> <83led09dlk.fsf@gnu.org> <1d43341a-2e7c-9f1d-68e9-9bac6f4b5fba@gutov.dev> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37257"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Paul Eggert , stefankangas@gmail.com, 66020@debbugs.gnu.org, monnier@iro.umontreal.ca To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Sep 24 07:31:07 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qkHhz-0009U8-9O for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 24 Sep 2023 07:31:07 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qkHhk-0002o7-GF; Sun, 24 Sep 2023 01:30:52 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qkHhi-0002nj-JL for bug-gnu-emacs@gnu.org; Sun, 24 Sep 2023 01:30:50 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qkHhi-0006cg-BP for bug-gnu-emacs@gnu.org; Sun, 24 Sep 2023 01:30:50 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qkHht-0002Tv-U7 for bug-gnu-emacs@gnu.org; Sun, 24 Sep 2023 01:31:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 24 Sep 2023 05:31:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66020 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 66020-submit@debbugs.gnu.org id=B66020.16955334239188 (code B ref 66020); Sun, 24 Sep 2023 05:31:01 +0000 Original-Received: (at 66020) by debbugs.gnu.org; 24 Sep 2023 05:30:23 +0000 Original-Received: from localhost ([127.0.0.1]:40883 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qkHhG-0002O7-NG for submit@debbugs.gnu.org; Sun, 24 Sep 2023 01:30:23 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51246) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qkHhE-0002Nv-Gn for 66020@debbugs.gnu.org; Sun, 24 Sep 2023 01:30:21 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qkHgw-0006DQ-B3; Sun, 24 Sep 2023 01:30:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=hx69whFCd6nyy29p7bUM4wOGv5p/g0ZvrUmpFVz56Jc=; b=mwt1xBgAA83j YEZC/tlgMM5rYCq0TqeY9og5I+BUm5/LCvBHyfmAv+DYoWEekfHFdoHcfjsNAZDuLMeqgXqin2jaP nMuegghUbsyAgHmRzxBPUTWHElHUgKbGFUIwWhI0UT/drdcLUBP4zAXo4mPYYH/UrI1rzwbwDZkoC XiV//cemEVBS0aFoI2cNw/0wgnwJYrpQ37IL0jetQ0Yl22f5IDat8UZKiwgg8Efpd6+Yf9DdgQmBI 4gHVYr+FCmZcWBN51RyqHFFAFt3Gpdg9QAeoiNn3KBptXxxyVpLUCbRWW03az6/uvHHDdJrlhvKjr uxpeUsVh8tc1R6XRilVWUg==; In-Reply-To: (message from Dmitry Gutov on Sun, 24 Sep 2023 00:51:28 +0300) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:271213 Archived-At: > Date: Sun, 24 Sep 2023 00:51:28 +0300 > From: Dmitry Gutov > Cc: 66020@debbugs.gnu.org, monnier@iro.umontreal.ca, stefankangas@gmail.com > > But it seems I've found an answer to one previous question: "can we find > out how much is already buffered in advance?" > > The patch below asks that from the OS (how portable is this? not sure) > and allocates a larger buffer when more output has been buffered. If we > keep OS's default value of pipe buffer size (64K on Linux and 16K-ish on > macOS, according to > https://unix.stackexchange.com/questions/11946/how-big-is-the-pipe-buffer), > that means auto-scaling the buffer on Emacs's side depending on how much > the process outputs. The effect on performance is similar to the > previous (looping) patch (0.70 -> 0.65), and is comparable to bumping > read-process-output-max to 65536. > > So if we do decide to bump the default, I suppose the below should not > be necessary. And I don't know whether we should be concerned about > fragmentation: this way buffers do get allocates in different sizes > (almost always multiples of 4096, but with rare exceptions among larger > values). > > diff --git a/src/process.c b/src/process.c > index 2376d0f288d..13cf6d6c50d 100644 > --- a/src/process.c > +++ b/src/process.c > @@ -6137,7 +6145,18 @@ > specpdl_ref count = SPECPDL_INDEX (); > Lisp_Object odeactivate; > char *chars; > > +#ifdef USABLE_FIONREAD > +#ifdef DATAGRAM_SOCKETS > + if (!DATAGRAM_CHAN_P (channel)) > +#endif > + { > + int available_read; > + ioctl (p->infd, FIONREAD, &available_read); > + readmax = MAX (readmax, available_read); > + } > +#endif > + > USE_SAFE_ALLOCA; > chars = SAFE_ALLOCA (sizeof coding->carryover + readmax); > > What do people think? I think we should increase the default size, and the rest (querying the system about the pipe size) looks like an unnecessary complication to me. I've added Paul Eggert to this discussion, as I'd like to hear his opinions about this stuff.