bug#26397: [PATCH v2] Use vfork if possible on Darwin (bug#26397)

[Alan Third <alan@idiocy.org>]

* src/conf_post.h (HAVE_WORKING_VFORK): Don't undef.
(vfork): Don't define.
* src/process.c (create_process) [DARWIN_OS]: Use fork if pty_flag is
set, otherwise vfork.
* src/callproc.c (call_process) [DARWIN_OS]: Use TIOCNOTTY to detach
the controlling terminal instead of setsid.
---
 src/callproc.c  | 13 +++++++++++++
 src/conf_post.h |  6 ------
 src/process.c   |  9 +++++++++
 3 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/src/callproc.c b/src/callproc.c
index 05048576ce..792556c8e0 100644
--- a/src/callproc.c
+++ b/src/callproc.c
@@ -52,6 +52,8 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 #include "syswait.h"
 #include "blockinput.h"
 #include "frame.h"
+#include "systty.h"
+#include "keyboard.h"
 
 #ifdef MSDOS
 #include "msdos.h"
@@ -626,7 +628,18 @@ call_process (ptrdiff_t nargs, Lisp_Object *args, int filefd,
     {
       unblock_child_signal (&oldset);
 
+#ifdef DARWIN_OS
+      /* Darwin doesn't let us run setsid after a vfork, so use
+         TIOCNOTTY when necessary. */
+      int j = emacs_open (DEV_TTY, O_RDWR, 0);
+      if (j >= 0)
+        {
+          ioctl (j, TIOCNOTTY, 0);
+          emacs_close (j);
+        }
+#else
       setsid ();
+#endif
 
       /* Emacs ignores SIGPIPE, but the child should not.  */
       signal (SIGPIPE, SIG_DFL);
diff --git a/src/conf_post.h b/src/conf_post.h
index 30c948e39a..95ebd5511c 100644
--- a/src/conf_post.h
+++ b/src/conf_post.h
@@ -99,12 +99,6 @@ typedef bool bool_bf;
 #define realloc unexec_realloc
 #define free unexec_free
 #endif
-/* The following solves the problem that Emacs hangs when evaluating
-   (make-comint "test0" "/nodir/nofile" nil "") when /nodir/nofile
-   does not exist.  Also, setsid is not allowed in the vfork child's
-   context as of Darwin 9/Mac OS X 10.5.  */
-#undef HAVE_WORKING_VFORK
-#define vfork fork
 #endif  /* DARWIN_OS */
 
 /* If HYBRID_MALLOC is defined (e.g., on Cygwin), emacs will use
diff --git a/src/process.c b/src/process.c
index 2f2e5c1b25..47a85f1c95 100644
--- a/src/process.c
+++ b/src/process.c
@@ -2049,7 +2049,16 @@ create_process (Lisp_Object process, char **new_argv, Lisp_Object current_dir)
   int volatile forkerr_volatile = forkerr;
   struct Lisp_Process *p_volatile = p;
 
+#ifdef DARWIN_OS
+  /* Darwin doesn't let us run setsid after a vfork, so use fork when
+     necessary. */
+  if (pty_flag)
+    pid = fork ();
+  else
+    pid = vfork ();
+#else
   pid = vfork ();
+#endif
 
   current_dir = current_dir_volatile;
   lisp_pty_name = lisp_pty_name_volatile;
-- 

This patch includes everything so far.

I can’t say whether the TIOCNOTTY approach is correct either,
unfortunately.

Yamamoto‐san, I’m aware this is mostly your work, so if you’d prefer
I’m happy for you to commit it in your name.

-- 
Alan Third