standard output/error/input streams

standard output/error/input streams

[Phillip Lord]

I've worked up the patch that I made to add input streams for writing to
the system standard out. There was some discussion about this in Jul.

https://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00910.html

The new version is on feature/stdout-stderr-stream.

Additions since last time. I've updated the names
(external-standard-output, external-standard-error) and I have also
added an input stream (external-standard-input).

The main motivation for this is the same as last time; it gives a
communication channel with Emacs which does not involve buffers at any
point. Although, if you search for "Emacs" and "standard output", other
people would like the same thing for other reasons.

Can I add this to master?

Phil

Re: standard output/error/input streams

[Noam Postavsky]

On Sat, Jan 14, 2017 at 7:22 AM, Phillip Lord
<phillip.lord@russet.org.uk> wrote:
>
>
> The new version is on feature/stdout-stderr-stream.

+DEFUN ("external-standard-error", Fexternal_standard_error,
Sexternal_standard_error, 1, 1, 0,
+       doc: /* Output character CHARACTER to system standard error. */)
+     (Lisp_Object character)
+{
+  return Fexternal_debugging_output (character);
+}

This could be just an alias instead, no?

Re: standard output/error/input streams

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Sat, 14 Jan 2017 12:22:50 +0000
> 
> I've worked up the patch that I made to add input streams for writing to
> the system standard out. There was some discussion about this in Jul.
> 
> https://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00910.html
> 
> The new version is on feature/stdout-stderr-stream.

Thanks.  Some of the concerns I expressed here:

  https://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00938.html

are still relevant, and at the very least should be reflected in the
docs.

> Can I add this to master?

I think this still "needs work", see my comments below:

  +@cindex @code{external-standard-input}
  +@defun external-standard-input &optional char

No need to explicitly index a function that is introduced in a @defun:
the latter automatically adds an index entry.

  +This function reads a single character from the system standard input
  +(as opposed to @var{standard-input}) and functions as a stream. Note,

Two spaces between sentences, please (here and elsewhere).

  +however, that if Emacs is running in a terminal its use can be
  +unpredictable.

I think we should explain more about the issue here, so that the
reader understands what she is up against.  (I think this function
will simply not work on a TTY frame.)

  +These functions are predominately useful for debugging, as they are a
  +mechanism for producing output that does not change any buffer. Note,
  +however, that if Emacs is running in a terminal their use can affect
  +the display unpredictably.

Likewise here.  This should also mention the problems with
stdout/stderr disposition in GUI sessions.

  +(defvar external-standard-input-pushback nil
  +  "Pushback character for `external-standard-input'.")

Isn't this an internal variable?  If so, it should use double hyphen
in its name.  And what is the purpose of this pushback?  I didn't see
any code that sets it in patch.

  +DEFUN ("external-standard-input-read-char",Fexternal_standard_input_read_char, Sexternal_standard_input_read_char, 0, 0, 0,
  +       doc: /* Read a single character from the system standard input.
  +
  +Returns -1 if standard input is at the end.*/)
  +     (void)
  +{
  +  int c;
  +  Lisp_Object val;
  +
  +  c = getchar();
  +  XSETINT(val,c);
  +
  +  return val;
  +}

This implementation has a number of issues:

  . getchar reads a _byte_, not a character, so unless input is
    plain-ASCII, what you return here is a raw byte, not a character
    in the Emacs sense of that word.  That is inconsistent with every
    other input facility we have, and could very well get the user in
    trouble.

  . getchar doesn't return until you type RET, at least on my system,
    which might come as a surprise to users.

  . (nitpicking) our coding style keeps one space between the
    function/macro name and the opening parenthesis.  (Same issue
    exists elsewhere in the patch.)

  +DEFUN ("external-standard-input-read-line", Fexternal_standard_input_read_line, Sexternal_standard_input_read_line, 0, 0, 0,
  +      doc: /* Read a line from the system standard input.*/)

This function doesn't seem to be documented in the ELisp manual.

  +  if (len || c == '\n' || c == '\r')
  +    {
  +      val = make_string (line, len);
  +      xfree (line);

What about EOL decoding, as in all the other Emacs input functions?

Also, make_string has its own ideas about when to produce unibyte
strings and when multibyte strings.  You should instead decode the
input text using coding-system-for-read (if non-nil) or
locale-coding-system (or maybe what terminal-coding-system returns).

  +DEFUN ("external-standard-output", Fexternal_standard_output, Sexternal_standard_output, 1, 1, 0,
  +       doc: /* Output character CHARACTER to system standard output. */)
  +     (Lisp_Object character)
  +{
  +  CHECK_NUMBER (character);
  +  printchar_to_stream (XINT(character), stdout);

printchar_to_stream converts the output text via
standard-display-table, if that is non-nil; do we really want that for
functionality that is supposed to be a debugging aid?  It also sends
its argument to the debugger on MS-Windows -- is that desirable?

Re: standard output/error/input streams

[Eli Zaretskii]

> Date: Sat, 14 Jan 2017 16:31:54 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
>   . getchar doesn't return until you type RET, at least on my system,
>     which might come as a surprise to users.

One other problem with this is that until getchar returns, Emacs is
stuck, and nothing runs.  That's not a good thing, IMO, so I think the
implementation should go via wait_reading_process_output and stuff
(which will have a nice side effect of letting some other thread run
while Emacs waits for input).

Re: standard output/error/input streams

[John Wiegley]

>>>>> "PL" == Phillip Lord <phillip.lord@russet.org.uk> writes:

PL> The main motivation for this is the same as last time; it gives a
PL> communication channel with Emacs which does not involve buffers at any
PL> point. Although, if you search for "Emacs" and "standard output", other
PL> people would like the same thing for other reasons.

Have you thought of allowing files to be "opened" for direct writing as well?
That is, I'd rather see file handles become a new output stream type, with
stdout and stderr instances of these, than hard-coded streams for only stdout
and stderr. Although, getting cleanup right makes me less certain.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: standard output/error/input streams

[Phillip Lord]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> "PL" == Phillip Lord <phillip.lord@russet.org.uk> writes:
>
> PL> The main motivation for this is the same as last time; it gives a
> PL> communication channel with Emacs which does not involve buffers at any
> PL> point. Although, if you search for "Emacs" and "standard output", other
> PL> people would like the same thing for other reasons.
>
> Have you thought of allowing files to be "opened" for direct writing as well?
> That is, I'd rather see file handles become a new output stream type, with
> stdout and stderr instances of these, than hard-coded streams for only stdout
> and stderr. Although, getting cleanup right makes me less certain.

Yes, that would be nice I agree, although standard out gives me my main
use case, and there is already support for clean up.

I'll take a look next maybe.

Phil

Re: standard output/error/input streams

[Stefan Monnier]

> Have you thought of allowing files to be "opened" for direct writing as well?

Last time this came up, I thought that mapping "opened files" to Emacs
subprocess objects was probably going to be more natural.


        Stefan

Re: standard output/error/input streams

[John Wiegley]

>>>>> "SM" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

SM> Last time this came up, I thought that mapping "opened files" to Emacs
SM> subprocess objects was probably going to be more natural.

That does make me realize that I could use `start-process` with an executable
that does what I want, and use `process-send-string` to bypass the involvement
of any buffers. Thanks for mentioning it.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: standard output/error/input streams

[Stefan Monnier]

> That does make me realize that I could use `start-process` with an executable
> that does what I want, and use `process-send-string` to bypass the involvement
> of any buffers. Thanks for mentioning it.

Right.  What I was thinking of was to add a `make-file-process`
primitive so you don't need an external executable.


        Stefan

Re: standard output/error/input streams

[John Wiegley]

>>>>> "SM" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

SM> Right. What I was thinking of was to add a `make-file-process` primitive
SM> so you don't need an external executable.

Sounds like a good workaround to me, and handily solves the cleanup problem.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: standard output/error/input streams

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> That does make me realize that I could use `start-process` with an executable
>> that does what I want, and use `process-send-string` to bypass the involvement
>> of any buffers. Thanks for mentioning it.
>
> Right.  What I was thinking of was to add a `make-file-process`
> primitive so you don't need an external executable.


If we had a function to go to generate a stream from a process then we
could have our cake and eat it.

Phil

Re: standard output/error/input streams

[Stefan Monnier]

> If we had a function to go to generate a stream from a process then we
> could have our cake and eat it.

(lambda (c) (process-send-string PROC (string c))) ?

BTW, w.r.t stdin/stdout/stderr: these are special because they may be
inaccessible to something like a subprocess (or at least it can be nigh
on impossible for the Emacs process itself to determine where
std(in|out|err) are going).

IOW, additionally to make-file-process, we'd need
a make-process-from-open-file-descriptor.


        Stefan