processes' filter functions and buffers -- data chunks

processes' filter functions and buffers -- data chunks

[Peter Tury]

Hi,

I read in (elisp)Filter Functions that "[A process'] output to the
[filter] function may come in chunks of any size.  A program that
produces the same output twice in a row may send it as one batch of
200 characters one time, and five batches of 40 characters the next."

What determines how and why this splitting happens? Is this an Emacs
feature? Or depends only on the process (e.g. the program what Emacs
connects to)? Or is this OS-dependent?

I see some similar thing happening in the process' buffer also:
sometimes I can't see the process' output until more output
arrives. It seems the output is buffered somewhere and it doesn't
displayed in the associated Emacs buffer until that (OS?) buffer is
filled up. (I use official GNU Emacs v22 on MS Windows XP.) However,
output is OK (=always showed up completely) in "Command/DOS windows",
thus it seems this is Emacs "feature"??

I found this "unpredictable" splitting quite annoying and I don't know
if I have chance to avoid this behaviour by modifying the peer program
or I absolutely have to live with this feature?

Thanks,
P

Re: processes' filter functions and buffers -- data chunks

[Eli Zaretskii]

> From: Peter Tury <tury.peter@gmail.com>
> Date: Sun, 30 Sep 2007 21:02:02 GMT
> 
> I read in (elisp)Filter Functions that "[A process'] output to the
> [filter] function may come in chunks of any size.  A program that
> produces the same output twice in a row may send it as one batch of
> 200 characters one time, and five batches of 40 characters the next."
> 
> What determines how and why this splitting happens? Is this an Emacs
> feature? Or depends only on the process (e.g. the program what Emacs
> connects to)? Or is this OS-dependent?

I think this is system-specific buffering of the pipe/pty used by
Emacs to communicate with subordinate processes.

> However, output is OK (=always showed up completely) in "Command/DOS
> windows"

AFAIK, that's because output to the console is normally line-buffered
or unbuffered, while pipes are usually fully buffered.

> I found this "unpredictable" splitting quite annoying and I don't know
> if I have chance to avoid this behaviour by modifying the peer program
> or I absolutely have to live with this feature?

If you can modify the subordinate program so that it uses less
aggressive buffering on its stdout/stderr, you _may_ see some
improvement (but not with certainty, since some of the buffering is in
the filesystem, not in the application).  But I think this solution is
impractical, since most programs cannot be practically changed in such
a way.