Re: Opening a pty without a process

[Nick Roberts <nickrob@snap.net.nz>]

Previously I wrote:

 > I would like to associate a buffer with a pty without a process.  How easy
 > would it be able to adapt start-process to do this when given a nil value
 > for program name:
 > 
 > (start-process name buffer nil) or (start-process name buffer)
 > 
 > Perhaps a new function would be better because I would like it to return the
 > name of the pty, e.g,
 > 
 > (open-pty-buffer buffer)
 > ->"/dev/pts/1"
 > 
 > Emacs would need then to monitor the file descriptor using select in
 > wait_reading_process_output (process.c).
 > 
 > and I would  hook this up with a process using:
 > 
 > M-x gdb
 > Run gdb (like this): gdb -tty /dev/pts/1 -i=mi myprog
 > 
 > (In practice it's more likely that Emacs will set a Lisp variable to the
 > value of (open-pty-buffer buffer) and use it behind the users back.)

Here's a patch to sketch the ideas that I have.  I would like it to be
considered for eventual inclusion on the trunk.  With it if you evaluate

(start-process "mypty" "*mypty*" nil)
=> #<process mypty>


Then M-x list-processes gives something like:

Proc  Status   Buffer  Tty	  Command
----  ------   ------  ---	  -------
mypty run      *mypty* /dev/pts/1 
shell run      *shell* /dev/pts/3 /bin/bash --noediting -i


In buffer *mypty* do

  M-x comint-mode


Start GDB as follows ("-annotate=3" won't work)

  M-x gdb
  Run gdb (like this): gdb --fullname myprog

where myprog is a program that does IO.

  ...
  (gdb) tty /dev/pts/1
  (gdb) run

Then your program output should appear in *mypty* and GDB should accept
input from there too.


It's still rough: ptys aren't closed when the buffer is killed, input gets
echoed in *mypty*, assumes ptys etc., but I would like feedback before
polishing it any further.

For GDB it means that a dummy process like:

  (make-comint-in-buffer
   "gdb-inferior" (current-buffer) "sleep" nil "1000000000"))

isn't needed.  There must be other situations where this could be useful too.

-- 
Nick                                           http://www.inet.net.nz/~nickrob


2007-11-26  Nick Roberts  <nickrob@snap.net.nz>

	* process.c (create_pty): New function.  Set pid == -2.
	(list_processes_1): Check for no program.
	(Fstart_process): Create a pty if PROGRAM is nil.
	Condition for no program.  Update doc string.
	(start_process_unwind): Remove process if pid == -1.


*** process.c.~1.525~	2007-11-27 09:15:17.000000000 +1300
--- process.c	2007-11-27 09:18:35.000000000 +1300
*************** static int keyboard_bit_set P_ ((SELECT_
*** 309,314 ****
--- 309,315 ----
  static void deactivate_process P_ ((Lisp_Object));
  static void status_notify P_ ((struct Lisp_Process *));
  static int read_process_output P_ ((Lisp_Object, int));
+ static void create_pty P_ ((Lisp_Object));
  
  /* If we support a window system, turn on the code to poll periodically
     to detect C-g.  It isn't actually used when doing interrupt input.  */
*************** list_processes_1 (query_only)
*** 1522,1527 ****
--- 1523,1530 ----
  	  while (1)
  	    {
  	      tem1 = Fcar (tem);
+ 	      if (NILP (tem1))
+ 		break;
  	      Finsert (1, &tem1);
  	      tem = Fcdr (tem);
  	      if (NILP (tem))
*************** at end of BUFFER, unless you specify an 
*** 1571,1578 ****
  function to handle the output.  BUFFER may also be nil, meaning that
  this process is not associated with any buffer.
  
! PROGRAM is the program file name.  It is searched for in PATH.
! Remaining arguments are strings to give program as arguments.
  
  If you want to separate standard output from standard error, invoke
  the command through a shell and redirect one of them using the shell
--- 1574,1582 ----
  function to handle the output.  BUFFER may also be nil, meaning that
  this process is not associated with any buffer.
  
! PROGRAM is the program file name.  It is searched for in PATH. If nil,
! just associate a pty with the buffer.  Remaining arguments are strings
! to give program as arguments.
  
  If you want to separate standard output from standard error, invoke
  the command through a shell and redirect one of them using the shell
*************** usage: (start-process NAME BUFFER PROGRA
*** 1628,1634 ****
  
    program = args[2];
  
!   CHECK_STRING (program);
  
    proc = make_process (name);
    /* If an error occurs and we can't start the process, we want to
--- 1632,1639 ----
  
    program = args[2];
  
!   if (!NILP (program))
!     CHECK_STRING (program);
  
    proc = make_process (name);
    /* If an error occurs and we can't start the process, we want to
*************** usage: (start-process NAME BUFFER PROGRA
*** 1675,1681 ****
  	args2[0] = Qstart_process;
  	for (i = 0; i < nargs; i++) args2[i + 1] = args[i];
  	GCPRO2 (proc, current_dir);
! 	coding_systems = Ffind_operation_coding_system (nargs + 1, args2);
  	UNGCPRO;
  	if (CONSP (coding_systems))
  	  val = XCAR (coding_systems);
--- 1680,1687 ----
  	args2[0] = Qstart_process;
  	for (i = 0; i < nargs; i++) args2[i + 1] = args[i];
  	GCPRO2 (proc, current_dir);
! 	if (!NILP (program))
! 	  coding_systems = Ffind_operation_coding_system (nargs + 1, args2);
  	UNGCPRO;
  	if (CONSP (coding_systems))
  	  val = XCAR (coding_systems);
*************** usage: (start-process NAME BUFFER PROGRA
*** 1693,1699 ****
  	    args2[0] = Qstart_process;
  	    for (i = 0; i < nargs; i++) args2[i + 1] = args[i];
  	    GCPRO2 (proc, current_dir);
! 	    coding_systems = Ffind_operation_coding_system (nargs + 1, args2);
  	    UNGCPRO;
  	  }
  	if (CONSP (coding_systems))
--- 1699,1706 ----
  	    args2[0] = Qstart_process;
  	    for (i = 0; i < nargs; i++) args2[i + 1] = args[i];
  	    GCPRO2 (proc, current_dir);
! 	    if (!NILP (program))
! 	      coding_systems = Ffind_operation_coding_system (nargs + 1, args2);
  	    UNGCPRO;
  	  }
  	if (CONSP (coding_systems))
*************** usage: (start-process NAME BUFFER PROGRA
*** 1704,1784 ****
      XPROCESS (proc)->encode_coding_system = val;
    }
  
- #ifdef VMS
-   /* Make a one member argv with all args concatenated
-      together separated by a blank.  */
-   len = SBYTES (program) + 2;
-   for (i = 3; i < nargs; i++)
-     {
-       tem = args[i];
-       CHECK_STRING (tem);
-       len += SBYTES (tem) + 1;	/* count the blank */
-     }
-   new_argv = (unsigned char *) alloca (len);
-   strcpy (new_argv, SDATA (program));
-   for (i = 3; i < nargs; i++)
-     {
-       tem = args[i];
-       CHECK_STRING (tem);
-       strcat (new_argv, " ");
-       strcat (new_argv, SDATA (tem));
-     }
-   /* Need to add code here to check for program existence on VMS */
- 
- #else /* not VMS */
-   new_argv = (unsigned char **) alloca ((nargs - 1) * sizeof (char *));
- 
-   /* If program file name is not absolute, search our path for it.
-      Put the name we will really use in TEM.  */
-   if (!IS_DIRECTORY_SEP (SREF (program, 0))
-       && !(SCHARS (program) > 1
- 	   && IS_DEVICE_SEP (SREF (program, 1))))
-     {
-       struct gcpro gcpro1, gcpro2, gcpro3, gcpro4;
- 
-       tem = Qnil;
-       GCPRO4 (name, program, buffer, current_dir);
-       openp (Vexec_path, program, Vexec_suffixes, &tem, make_number (X_OK));
-       UNGCPRO;
-       if (NILP (tem))
- 	report_file_error ("Searching for program", Fcons (program, Qnil));
-       tem = Fexpand_file_name (tem, Qnil);
-     }
-   else
-     {
-       if (!NILP (Ffile_directory_p (program)))
- 	error ("Specified program for new process is a directory");
-       tem = program;
-     }
- 
-   /* If program file name starts with /: for quoting a magic name,
-      discard that.  */
-   if (SBYTES (tem) > 2 && SREF (tem, 0) == '/'
-       && SREF (tem, 1) == ':')
-     tem = Fsubstring (tem, make_number (2), Qnil);
- 
-   /* Encode the file name and put it in NEW_ARGV.
-      That's where the child will use it to execute the program.  */
-   tem = ENCODE_FILE (tem);
-   new_argv[0] = SDATA (tem);
- 
-   /* Here we encode arguments by the coding system used for sending
-      data to the process.  We don't support using different coding
-      systems for encoding arguments and for encoding data sent to the
-      process.  */
- 
-   for (i = 3; i < nargs; i++)
-     {
-       tem = args[i];
-       CHECK_STRING (tem);
-       if (STRING_MULTIBYTE (tem))
- 	tem = (code_convert_string_norecord
- 	       (tem, XPROCESS (proc)->encode_coding_system, 1));
-       new_argv[i - 2] = SDATA (tem);
-     }
-   new_argv[i - 2] = 0;
- #endif /* not VMS */
- 
    XPROCESS (proc)->decoding_buf = make_uninit_string (0);
    XPROCESS (proc)->decoding_carryover = 0;
    XPROCESS (proc)->encoding_buf = make_uninit_string (0);
--- 1711,1716 ----
*************** usage: (start-process NAME BUFFER PROGRA
*** 1786,1792 ****
    XPROCESS (proc)->inherit_coding_system_flag
      = !(NILP (buffer) || !inherit_process_coding_system);
  
!   create_process (proc, (char **) new_argv, current_dir);
  
    return unbind_to (count, proc);
  }
--- 1718,1804 ----
    XPROCESS (proc)->inherit_coding_system_flag
      = !(NILP (buffer) || !inherit_process_coding_system);
  
!   if (!NILP (program))
!     {
! #ifdef VMS
!       /* Make a one member argv with all args concatenated
! 	 together separated by a blank.  */
!       len = SBYTES (program) + 2;
!       for (i = 3; i < nargs; i++)
! 	{
! 	  tem = args[i];
! 	  CHECK_STRING (tem);
! 	  len += SBYTES (tem) + 1;	/* count the blank */
! 	}
!       new_argv = (unsigned char *) alloca (len);
!       strcpy (new_argv, SDATA (program));
!       for (i = 3; i < nargs; i++)
! 	{
! 	  tem = args[i];
! 	  CHECK_STRING (tem);
! 	  strcat (new_argv, " ");
! 	  strcat (new_argv, SDATA (tem));
! 	}
!       /* Need to add code here to check for program existence on VMS */
! 
! #else /* not VMS */
!       new_argv = (unsigned char **) alloca ((nargs - 1) * sizeof (char *));
! 
!       /* If program file name is not absolute, search our path for it.
! 	 Put the name we will really use in TEM.  */
!       if (!IS_DIRECTORY_SEP (SREF (program, 0))
! 	  && !(SCHARS (program) > 1
! 	       && IS_DEVICE_SEP (SREF (program, 1))))
! 	{
! 	  struct gcpro gcpro1, gcpro2, gcpro3, gcpro4;
! 
! 	  tem = Qnil;
! 	  GCPRO4 (name, program, buffer, current_dir);
! 	  openp (Vexec_path, program, Vexec_suffixes, &tem, make_number (X_OK));
! 	  UNGCPRO;
! 	  if (NILP (tem))
! 	    report_file_error ("Searching for program", Fcons (program, Qnil));
! 	  tem = Fexpand_file_name (tem, Qnil);
! 	}
!       else
! 	{
! 	  if (!NILP (Ffile_directory_p (program)))
! 	    error ("Specified program for new process is a directory");
! 	  tem = program;
! 	}
! 
!       /* If program file name starts with /: for quoting a magic name,
! 	 discard that.  */
!       if (SBYTES (tem) > 2 && SREF (tem, 0) == '/'
! 	  && SREF (tem, 1) == ':')
! 	tem = Fsubstring (tem, make_number (2), Qnil);
! 
!       /* Encode the file name and put it in NEW_ARGV.
! 	 That's where the child will use it to execute the program.  */
!       tem = ENCODE_FILE (tem);
!       new_argv[0] = SDATA (tem);
! 
!       /* Here we encode arguments by the coding system used for sending
! 	 data to the process.  We don't support using different coding
! 	 systems for encoding arguments and for encoding data sent to the
! 	 process.  */
! 
!       for (i = 3; i < nargs; i++)
! 	{
! 	  tem = args[i];
! 	  CHECK_STRING (tem);
! 	  if (STRING_MULTIBYTE (tem))
! 	    tem = (code_convert_string_norecord
! 		   (tem, XPROCESS (proc)->encode_coding_system, 1));
! 	  new_argv[i - 2] = SDATA (tem);
! 	}
!       new_argv[i - 2] = 0;
! #endif /* not VMS */
! 
!       create_process (proc, (char **) new_argv, current_dir);
!     }
!   else
!     create_pty (proc);
  
    return unbind_to (count, proc);
  }
*************** start_process_unwind (proc)
*** 1803,1809 ****
      abort ();
  
    /* Was PROC started successfully?  */
!   if (XPROCESS (proc)->pid <= 0)
      remove_process (proc);
  
    return Qnil;
--- 1815,1821 ----
      abort ();
  
    /* Was PROC started successfully?  */
!   if (XPROCESS (proc)->pid == -1)
      remove_process (proc);
  
    return Qnil;
*************** create_process_sigchld ()
*** 1836,1841 ****
--- 1848,1939 ----
  #endif
  #endif
  
+ void
+ create_pty (process)
+ {
+   int inchannel, outchannel;
+ 
+   /* Use volatile to protect variables from being clobbered by longjmp.  */
+   volatile int forkin, forkout;
+   volatile int pty_flag = 0;
+ 
+   inchannel = outchannel = -1;
+ 
+ #ifdef HAVE_PTYS
+   if (!NILP (Vprocess_connection_type))
+     outchannel = inchannel = allocate_pty ();
+ 
+   if (inchannel >= 0)
+     {
+ #if ! defined (USG) || defined (USG_SUBTTY_WORKS)
+       /* On most USG systems it does not work to open the pty's tty here,
+ 	 then close it and reopen it in the child.  */
+ #ifdef O_NOCTTY
+       /* Don't let this terminal become our controlling terminal
+ 	 (in case we don't have one).  */
+       forkout = forkin = emacs_open (pty_name, O_RDWR | O_NOCTTY, 0);
+ #else
+       forkout = forkin = emacs_open (pty_name, O_RDWR, 0);
+ #endif
+       if (forkin < 0)
+ 	report_file_error ("Opening pty", Qnil);
+ #if defined (RTU) || defined (UNIPLUS) || defined (DONT_REOPEN_PTY)
+       /* In the case that vfork is defined as fork, the parent process
+ 	 (Emacs) may send some data before the child process completes
+ 	 tty options setup.  So we setup tty before forking.  */
+       child_setup_tty (forkout);
+ #endif /* RTU or UNIPLUS or DONT_REOPEN_PTY */
+ #else
+       forkin = forkout = -1;
+ #endif /* not USG, or USG_SUBTTY_WORKS */
+       pty_flag = 1;
+     }
+   else
+ #endif /* HAVE_PTYS */
+ 
+ /* Stride people say it's a mystery why this is needed
+    as well as the O_NDELAY, but that it fails without this.  */
+ #if defined (STRIDE) || (defined (pfa) && defined (HAVE_PTYS))
+   {
+     int one = 1;
+     ioctl (inchannel, FIONBIO, &one);
+   }
+ #endif
+ 
+ #ifdef O_NONBLOCK
+   fcntl (inchannel, F_SETFL, O_NONBLOCK);
+   fcntl (outchannel, F_SETFL, O_NONBLOCK);
+ #else
+ #ifdef O_NDELAY
+   fcntl (inchannel, F_SETFL, O_NDELAY);
+   fcntl (outchannel, F_SETFL, O_NDELAY);
+ #endif
+ #endif
+ 
+   /* Record this as an active process, with its channels.
+      As a result, child_setup will close Emacs's side of the pipes.  */
+   chan_process[inchannel] = process;
+   XPROCESS (process)->infd = inchannel;
+   XPROCESS (process)->outfd = outchannel;
+ 
+   /* Previously we recorded the tty descriptor used in the subprocess.
+      It was only used for getting the foreground tty process, so now
+      we just reopen the device (see emacs_get_tty_pgrp) as this is
+      more portable (see USG_SUBTTY_WORKS above).  */
+ 
+   XPROCESS (process)->pty_flag = pty_flag;
+   XPROCESS (process)->status = Qrun;
+   setup_process_coding_systems (process);
+ 
+   FD_SET (inchannel, &input_wait_mask);
+   FD_SET (inchannel, &non_keyboard_wait_mask);
+   if (inchannel > max_process_desc)
+     max_process_desc = inchannel;
+ 
+   XPROCESS (process)->pid = -2;
+   XPROCESS (process)->tty_name = build_string (pty_name);
+ }
+ 
  #ifndef VMS /* VMS version of this function is in vmsproc.c.  */
  void
  create_process (process, new_argv, current_dir)