From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Spencer Baugh Newsgroups: gmane.emacs.bugs Subject: bug#24531: process-send-string seems to truncate lines over 4096 characters Date: Fri, 21 Jul 2023 09:58:38 -0400 Message-ID: References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="30055"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, Stefan Monnier , jidanni@jidanni.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Jul 21 15:59:30 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qMqfJ-0007g4-U1 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 21 Jul 2023 15:59:29 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qMqf2-0006fT-KA; Fri, 21 Jul 2023 09:59:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMqev-0006dY-4s for bug-gnu-emacs@gnu.org; Fri, 21 Jul 2023 09:59:06 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qMqes-0006iq-QH for bug-gnu-emacs@gnu.org; Fri, 21 Jul 2023 09:59:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qMqes-0002kB-Ca for bug-gnu-emacs@gnu.org; Fri, 21 Jul 2023 09:59:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Spencer Baugh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Jul 2023 13:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 24531 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed Original-Received: via spool by 24531-submit@debbugs.gnu.org id=B24531.168994793110528 (code B ref 24531); Fri, 21 Jul 2023 13:59:02 +0000 Original-Received: (at 24531) by debbugs.gnu.org; 21 Jul 2023 13:58:51 +0000 Original-Received: from localhost ([127.0.0.1]:34321 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMqeg-0002jk-JI for submit@debbugs.gnu.org; Fri, 21 Jul 2023 09:58:51 -0400 Original-Received: from mxout5.mail.janestreet.com ([64.215.233.18]:59969) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMqea-0002j9-5b for 24531@debbugs.gnu.org; Fri, 21 Jul 2023 09:58:46 -0400 In-Reply-To: <83sf9h257b.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 21 Jul 2023 08:39:52 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:265713 Archived-At: Eli Zaretskii writes: >> Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, jidanni@jidanni.org >> Date: Thu, 20 Jul 2023 17:21:53 -0400 >> From: Stefan Monnier via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >> >> > I see that this bug is about 13 years old. I think there's a pretty >> > obvious solution: process-connection-type should default to nil. >> >> In practice, that can't fly because it'll break existing code. > > Indeed. I agree that we probably can't change the default. However... > A large portion (I think the majority, but I'm not sure) of > Lisp programs that use bidirectional communications with async > subprocesses actually _want_ the PTY interface, because they act as > GUI front-ends to other programs. Think "M-x term", "M-x gdb", > inferior-python-mode, etc. Even "M-x grep" and the likes need that > because they rely on default color output, which only happens if Grep > is connected to a terminal device. Some of the Emacs features based > on this don't work or don't work well on MS-Windows because Windows > only supports pipes. Do we really want such semi-broken behavior on > GNU and Unix systems? > > The number of applications that (a) don't need console-like behavior > and (b) need to send larger-than-4KB buffers to sub-processes is quite > small. Which is why this issue comes up only very rarely. So making > pipes the default will fix a very small fraction of applications, and > break the vast majority -- clearly a wrong balance. I see your point, but at the same time, the PTY interface on its own is not sufficient to make these applications work, not at all. Specialized modes are necessary to make M-x term (to implement a terminal) and M-x grep (to parse ANSI color codes) and other such programs work. Running things in a PTY without such specialized code doesn't give you anything, AFAIK, because a PTY alone is far from enough to make the Emacs end behave like a terminal. So such programs need to be aware and careful about such things anyway, and need additional infrastructure on top of make-process. So the default being "pty" gives such programs very little: it doesn't save them any complexity. Programs that just want to do some data processing with a subprocess, on the other hand, work fine with just make-process alone, they need no additional infrastructure, just process-send-string and reading directly from the process buffer. The default being "pipe" would take away a big footgun for such programs, since it's easy to forget that and then have a silently wrong program which will fail once you get large input. >> Also I don't think either value of `process-connection-type` is a good >> option. IOW, I think that the connection type should be a mandatory >> argument when creating an async process (except maybe for those >> processes with no input/output). > > If we go that way, we should start by specifying :connection-type for > all the uses of make-process and start-process we have in the core. > It's a large job, but before it is done we cannot in good faith make > such an incompatible transition. I can do that. However, what about my patch adding a warning about this to process-send-string? I think that is independently valuable. Right now we have no documentation of this problem... >> So maybe, the default value of `process-connection-type` should be >> `unspecified` and the process creation code should emit a warning when >> creating a process whose connection type is `unspecified` (just >> a warning, tho: it should then pursue execution as if that value was t, >> as usual, so as to preserve compatibility). > > Something like that, yes. > > But I'm actually wondering how come modern Linux kernels don't have a > way of lifting this restriction, or at least enlarging the limit so it > makes the problem even less frequent. Is there some inherent > limitation that this must be 4KB and nothing larger? Unfortunately from looking at Linux the limit of 4096 seems to be hardcoded.