bug#33050: 27.0.50; [macOS] Problem with process input with process-connection-type nil

[Thomas Fitzsimmons <fitzsim@fitzsim.org>]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> On 20/10/2018 13:09 +0300, Eli Zaretskii wrote:
>
>>> From: Filipp Gunbin <fgunbin@fastmail.fm>
>>> Date: Mon, 15 Oct 2018 22:03:19 +0300
>>>
>>> If I let-bind process-connection-type to t (use pty) in above code, then
>>> it works normally:
>>>
>>> *Messages* :
>>> proc status: run
>>> found prompt
>>> proc status: run
>>> exited
>>> "exited"
>>>
>>> Buffer "my-process-buf":
>>> enter something:
>>> Process my-process<2> finished
>>>
>>> But if I let-bind process-connection-type to nil (so does ldap.el), then
>>> it hangs, and after a few seconds wait and C-g, *Messages* has only
>>> this: "proc status: run [6 times]", and buffer my-process-buf is empty.
>>
>> The reason is almost certainly buffering: when the connection is via a
>> pipe, the subprocess writes in buffered mode, so it might take quite a
>> few characters of input before /usr/bin/read outputs something.  Try
>> using the -n or -N options, and see if that helps.
>
> Thanks for looking at this.
> I tried wrapping read in stdbuf, but it didn't change anything:
> (start-process "my-process" buf "stdbuf" "-o0" "/usr/bin/read" "-p" "enter something:")
>
> read -n won't help either, because it can affect the number of chars
> "read" reads - while in my example we are just waiting for prompt.
>
>> Why does ldap.el set process-connection-type to nil?
>
> I don't know, and it seems like it should not mingle with this setting
> at all (why should it?)
>
> I've CC'ed Thomas, who is the author of these lines (ldap.el line 649),
> according to git blame.

The overall change was to send the password to ldapsearch -W via a pipe
instead of in the clear on the command line (which made it visible to
other users e.g., in ps output).

As for that specific line, I may have copied it from another part of
Emacs that reads a password via a pipe; when I look now at other places
that bind this variable to nil, I see comments like this one in
lisp/gnus/nntp.el:

    ;; A non-nil connection type results in mightily odd behavior where
    ;; (process-send-string proc "\^M") ends up sending a "\n" to the
    ;; ssh process.  --Stef
    ;; Also a nil connection allow ssh-askpass to work under X11.
    (let ((process-connection-type nil))
      (apply 'start-process "nntpd" buffer command))

Today I tested my setup (x86_64 GNU/Linux, OpenLDAP ldapsearch 2.4.40)
without setting process-connection-type to nil, and it still works.  The
documentation for that variable says that the fallback is to use a pipe
if all ptys are busy in which case I guess this would still fail for
you.

Your test case behaves the same way for me on x86_64 GNU/Linux.

Maybe our ldapsearch commands are behaving differently.  What version of
ldapsearch are you using?

Thomas