bug#30762: 27.0.50; Apparent regression in process.c caused by checking pty_flag before calling setsid

[Thomas Fitzsimmons <fitzsim@fitzsim.org>]

Hi,

On x86_64, CentOS 7.3.1611, I noticed that on master tip (I tested
461e6818...), eudc-expand-inline was hanging.  It calls the ldapsearch
command line utility (by way of ldap-search-internal) using
start-process, and waits for a password prompt:

  (let* ((process-connection-type nil)
	 (proc-args (append arglist ldap-ldapsearch-args
			    filter))
	 (proc (apply #'start-process "ldapsearch" buf
		      ldap-ldapsearch-prog
		      proc-args)))
    (while (null (progn
		   (goto-char (point-min))
		   (re-search-forward
		    ldap-ldapsearch-password-prompt-regexp
		    (point-max) t)))
      (accept-process-output proc 1))
      [...])

master tip gets stuck in that while loop, whereas Emacs 25.1 is fine.

If I print the contents of the buffer within the loop, it shows:

Process ldapsearch stopped (tty output)

I isolated the working-to-non-working transition to this commit:

commit 8e5a769965313a7a1c42b5992ed24e8b0ea71ead
Author: Paul Eggert <eggert@cs.ucla.edu>
Date:   Tue Dec 6 23:41:45 2016 -0800

    Clean out some IRIX cruft
    
    * etc/MACHINES: Remove obsolete discussion of IRIX.
    * src/process.c (allocate_pty) [__sgi]: Remove SGI-specific code.
    (create_process) [HAVE_PTYS]: Don't worry about IRIX.
    * src/syntax.c (scan_sexps_forward): Remove obsolete comment.
    * src/unexelf.c [__sgi]: Don't include <syms.h>.
    (unexec) [__sgi]: Remove SGI-specific code.

and specifically the addition of this check in src/process.c:

-      /* We tried doing setsid only if pty_flag, but it caused
-	 process_set_signal to fail on SGI when using a pipe.  */
-      setsid ();
+      if (pty_flag)
+	setsid ();

If I remove the if (pty_flag) line, then eudc-expand-inline works again.

It may be that ldap-search-internal should change how it's invoking
ldapsearch, but I thought I would report the change in process.c
behavior anyway.

Thomas