From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#66020: (bug#64735 spin-off): regarding the default for read-process-output-max Date: Thu, 21 Sep 2023 20:40:35 +0300 Message-ID: <4dd10ea8-6124-a50d-2bc2-1fced50e34cc@gutov.dev> References: <83il8lxjcu.fsf@gnu.org> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@gutov.dev> <8334zmtwwi.fsf@gnu.org> <83tts0rkh5.fsf@gnu.org> <831qf3pd1y.fsf@gnu.org> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@gutov.dev> <83sf7jnq0m.fsf@gnu.org> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@gutov.dev> <83ledanvzw.fsf@gnu.org> <83r0n2m7qz.fsf@gnu.org> <26afa109-9ba3-78a3-0e68-7585ae8e3a19@gutov.dev> <83il8dna30.fsf@gnu.org> <83bke5mhvs.fsf@gnu.org> <83a5tmk79p.fsf@gnu.org> <937d9927-506f-aa36-94e9-3cceb8f629dd@gutov.dev> <83zg1hay6q.fsf@gnu.org> <451d6012-e5ab-df6c-50e3-dac20b91781c@gutov.dev> <83led09dlk.fsf@gnu.org> <83msxf8td3.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="19604"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: 66020@debbugs.gnu.org, stefankangas@gmail.com, monnier@iro.umontreal.ca To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Sep 21 19:41:35 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qjNgF-0004ym-9W for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 21 Sep 2023 19:41:35 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qjNfo-0001oV-0v; Thu, 21 Sep 2023 13:41:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qjNfY-0001VX-Tx for bug-gnu-emacs@gnu.org; Thu, 21 Sep 2023 13:40:53 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qjNfY-000085-LN for bug-gnu-emacs@gnu.org; Thu, 21 Sep 2023 13:40:52 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qjNfi-0003mj-8S for bug-gnu-emacs@gnu.org; Thu, 21 Sep 2023 13:41:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 21 Sep 2023 17:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66020 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 66020-submit@debbugs.gnu.org id=B66020.169531805714530 (code B ref 66020); Thu, 21 Sep 2023 17:41:02 +0000 Original-Received: (at 66020) by debbugs.gnu.org; 21 Sep 2023 17:40:57 +0000 Original-Received: from localhost ([127.0.0.1]:34771 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qjNfc-0003mI-I3 for submit@debbugs.gnu.org; Thu, 21 Sep 2023 13:40:57 -0400 Original-Received: from out4-smtp.messagingengine.com ([66.111.4.28]:48957) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qjNfZ-0003m4-Nb for 66020@debbugs.gnu.org; Thu, 21 Sep 2023 13:40:54 -0400 Original-Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 7295B5C0083; Thu, 21 Sep 2023 13:40:38 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 21 Sep 2023 13:40:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1695318038; x=1695404438; bh=BCbwPkLFW24nXas7752ku61z+VT8UPOlKLv ohBTMUgs=; b=IaFWWAFTN+AkYAvWylBs0GyE3LtOEQQz6XjJoEN0lQiJz4JoQEU qn21u96oODeSZmmxy+a9/vrfCXAxdEi4O4GAlhz6hOcyv8m9AGqYm4xueQ9hWrnm BbZwYt6YcUj0zhc4KWkA2nANb1ETLAY/mfaKjDfyST45gB4xNlmLIAvRp+KFJqzB 9juq2l6ogMLl05Pn9Zz4MPMbzU8saXZv8GGidk3/NCbTattpDOlcL0jjlBQ7krCx JDWElGwh9gdCcC1+vWKbqmjKlJ6P2krMPZS4LVxoon2NdNtrV3JTH4ImipNQqmin Vh/KE6auLA0UZdp3skmbsoqI8ZryHNU0mnw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1695318038; x=1695404438; bh=BCbwPkLFW24nXas7752ku61z+VT8UPOlKLv ohBTMUgs=; b=jSSWvIXkkxGhHM9LUFcKYsglhUaK9tTnzxEadwqbcqAKp7ZGz2o v2pDaKWNPgZVXAuksS089+oUJqZYHG0BUKIsRa5kCiI3Ks3Qe+NeErIQaal5Kexm PHDSuSPG051opUgvv8uSkrmj6eJsUiC1GWBSNQ2q2nVbrnDL5+YOFyAED6fM56hd +2KWYdyPpTa3MLPrqR+dA8lK98dFFqeY6AaldJL8DwFEHTImhrNYdnHVtcYvs3Rf 2YnIzF3QKcsvhXWT1rJRG76I08A5GNbyLxWfwViRvjjBRij24HZ6+lyGgYiM1dEX ZHGQsZ8pQvUTDAMIIV1p/y+7r588UC6OlEw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudekiedgudduhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeev ledvveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 21 Sep 2023 13:40:36 -0400 (EDT) Content-Language: en-US In-Reply-To: <83msxf8td3.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:271011 Archived-At: On 21/09/2023 17:59, Eli Zaretskii wrote: >> Date: Thu, 21 Sep 2023 17:37:23 +0300 >> Cc: stefankangas@gmail.com, monnier@iro.umontreal.ca, 66020@debbugs.gnu.org >> From: Dmitry Gutov >> >>> How does the throughput depend on this value? If the dependence curve >>> plateaus at some lower value, we could use that lower value as a >>> "good-enough" default. >> >> Depends on what we're prepared to call a plateau. Strictly speaking, not >> really. But we have a "sweet spot": for the process in my original >> benchmark ('find' with lots of output) it seems to be around 1009600. >> Here's a table (numbers are different from before because they're >> results of (benchmark 5 ...) divided by 5, meaning GC is amortized: >> >> | 4096 | 0.78 | >> | 16368 | 0.69 | >> | 40960 | 0.65 | >> | 409600 | 0.59 | >> | 1009600 | 0.56 | >> | 2009600 | 0.64 | >> | 4009600 | 0.65 | > > Not enough data points between 40960 and 409600, IMO. 40960 sounds > like a good spot for the default value. Or 32K, from the thread linked to previously (datagram size). And ifwe were to raise MAX_ALLOCA by 2x, we could still use 'alloca'. Neither would be optimal for my test scenario, though still an improvement. But see my other email with experimental patches, those bear improvement with the default 4096. >>>> And I think we should make the process "remember" the value at its >>>> creation either way (something touched on in bug#38561): in bug#55737 we >>>> added an fcntl call to make the larger values take effect. But this call >>>> is in create_process: so any subsequent increase to a large value of >>>> this var won't have effect. >>> >>> Why would the variable change after create_process? I'm afraid I >>> don't understand what issue you are trying to deal with here. >> >> Well, what could we lose by saving the value of read-process-output-max >> in create_process? > > It's already recorded in the size of the pipe, so why would we need to > record it once more? 'read_process_output' looks it up once more, to set the value of 'readmax' and allocate char*chars. Can we get the "recorded" value back from the pipe somehow?