bug#33050: 27.0.50; [macOS] Problem with process input with process-connection-type nil

[Thomas Fitzsimmons <fitzsim@fitzsim.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Thomas Fitzsimmons <fitzsim@fitzsim.org>
>> Cc: Filipp Gunbin <fgunbin@fastmail.fm>,  33050@debbugs.gnu.org,  alan@idiocy.org
>> Date: Thu, 25 Oct 2018 21:41:34 -0400
>> 
>> > I don't understand why: using nil process-connection-type for programs
>> > that prompt the user is a bug anyway.
>> 
>> I'm interested to know why you think this is a bug (or even not
>> preferable) in the specific case of ldap.el.  When ldap.el calls
>> ldapsearch, ldapsearch doesn't prompt the user for the password
>> directly.  ldap.el (having pre-read the password into Elisp by various
>> means) runs the command, and, in response to its prompt, sends it the
>> password.
>
> First, I must apologize in advance: I might be mis-interpreting the
> situation, because I don't use ldap.el or ldapsearch.  In that case,
> what I said before and what I say below might not make sense.
>
> That said, my rationale was simple: _any_ program that interacts with
> the user will generally behave better when connected through pty than
> when connected through a pipe, because the former looks and feels as a
> terminal, and thus will cause the program to use the appropriate
> buffering, will support terminal-only features like various escape
> sequences, cursor control, job control, etc.  Using pipes will
> probably work in most cases, but might fail in some of them, so good
> engineering tells us to prefer pty's, like the ELisp manual says.

OK, this makes sense, thanks for providing the rationale.  Given that
ldapsearch does issue a human-readable prompt for the password, it was
likely written and tested assuming a pty.

>> I re-read the Elisp "Asynchronous Processes" node; when I was deciding
>> how to set process-connection-type, I think I was following the
>> _underlined_ part in:
>> 
>>    [...] If available, ptys are usually preferable for processes visible
>>    to the user, as in Shell mode, because they allow for job control
>>    (‘C-c’, ‘C-z’, etc.) between the process and its children, and
>>    because interactive programs treat ptys as terminal devices, whereas
>>    pipes don’t support these features.  However, _for subprocesses used
>>    by Lisp programs for internal purposes, it is often better to use a
>>    pipe_, because pipes are more efficient, and because they are immune
>>    to stray character injections that ptys introduce for large (around
>>    500 byte) messages.  Also, the total number of ptys is limited on
>>    many systems and it is good not to waste them.
>> 
>> Given that ldapsearch can operate over a pipe (i.e., doesn't need
>> terminal features, seems to be unaffected by buffering), and is used for
>> internal purposes (i.e., is not visible to the user) isn't pipe mode
>> preferable for the reasons given there?
>
> Not in my opinion, no.  AFAIU, the efficiency considerations don't
> really apply here, since the amount of data exchanged is small
> (right?).

Right.

> All in all, I consider the "however" part above too vague to be useful
> (what exactly does "internal purposes" mean in this context?).  I
> think it's even slightly misleading.  I'll think about rewording it to
> make the point more clear and the decision easier.

OK, sounds good.  One worry I have about always leaving
process-connection-type t is that, depending on the external system
state -- specifically whether or not all ptys are busy --
process-connection-type might not have any effect, and the underlying
process will rarely (and silently AFAICT) operate in pipe mode.  By
forcing process-connection-type nil, one is always testing in the same
known mode.

That said, it is probably a bad idea to warn or signal an error if all
ptys are busy in the p-c-t t case.  Anyway, I thought I'd mention it;
maybe the documentation update can address this point as well.

How about I make the change conditional on Mac OS on the release branch,
and unconditional on the master branch (once the fix for 33154 is
committed)?  That way users on the master branch will be able to test
pty mode before the next release, and we fix the release branch without
potentially destabilizing non-Darwin users.

Thomas