Re: [Emacs-diffs] /srv/bzr/emacs/trunk r100961: Enable ICANON (Bug#6771). Any long line problem must be solved differently.

Re: [Emacs-diffs] /srv/bzr/emacs/trunk r100961: Enable ICANON (Bug#6771). Any long line problem must be solved differently.

[Stefan Monnier]

I agree with the commit that reverts my ICANON change.  Yet:

> There is AFAIK no bug report or test case for the long line problem.

AFAIK, the missing bug-report is the one that shows the ills of sending
EOFs, while the bug-report for long-lines is bug#6149, which should be
re-opened.
 
> +  /* There is no more "send eof to flush" going on (which is wrong and
> +     unportable in itself), and disabling ICANON breaks a lot of stuff
> +     and shows literal ^D in many cases.  The correct way to handle too
> +     much output is to buffer what could not be written and then write it
> +     again when select returns ok for writing.

Could you spell out more precisely how it's different from what we do
now: we already check emacs_write's output to see if the buffer is full,
in which case we wait, don't we?

> +  s.main.c_lflag |= ICANON;	/* Enable line editing and eof processing */
> +  s.main.c_cc[VEOF] = 'D'&037;	/* Control-D */
>    s.main.c_cc[VMIN] = 1;
>    s.main.c_cc[VTIME] = 0;
 
IIUC VMIN and VTIME not only are specific to the non-ICANON mode, but
also they use the same slot as some of the other settings (specifically
the slots may be shared with VEOL and VEOF, according to my manpage).


        Stefan

Re: [Emacs-diffs] /srv/bzr/emacs/trunk r100961: Enable ICANON (Bug#6771). Any long line problem must be solved differently.

[Jan Djärv]

Stefan Monnier skrev 2010-08-04 17.56:
> I agree with the commit that reverts my ICANON change.  Yet:
>
>> There is AFAIK no bug report or test case for the long line problem.
>
> AFAIK, the missing bug-report is the one that shows the ills of sending
> EOFs, while the bug-report for long-lines is bug#6149, which should be
> re-opened.

As I said in relation to this, I could not reproduce the error in 6149.  But 
AFAIK, this all depends on the underlying tty code, so a longer input string 
might trigger the error.  But if a line is longer than MAX_CANON, it will be 
truncated by the tty code.

>
>> +  /* There is no more "send eof to flush" going on (which is wrong and
>> +     unportable in itself), and disabling ICANON breaks a lot of stuff
>> +     and shows literal ^D in many cases.  The correct way to handle too
>> +     much output is to buffer what could not be written and then write it
>> +     again when select returns ok for writing.
>
> Could you spell out more precisely how it's different from what we do
> now: we already check emacs_write's output to see if the buffer is full,
> in which case we wait, don't we?

Yes, but we wait a small time and then try to write again (if I read the code 
correctly).  If nobody is reading, that can turn into a (sort of) busy wait.
The point of handing it off to select is then we don't write until we know we 
can write something.

>
>> +  s.main.c_lflag |= ICANON;	/* Enable line editing and eof processing */
>> +  s.main.c_cc[VEOF] = 'D'&037;	/* Control-D */
>>     s.main.c_cc[VMIN] = 1;
>>     s.main.c_cc[VTIME] = 0;
>
> IIUC VMIN and VTIME not only are specific to the non-ICANON mode, but
> also they use the same slot as some of the other settings (specifically
> the slots may be shared with VEOL and VEOF, according to my manpage).

That is correct.

	Jan D.

Re: [Emacs-diffs] /srv/bzr/emacs/trunk r100961: Enable ICANON (Bug#6771). Any long line problem must be solved differently.

[Stefan Monnier]

>> I agree with the commit that reverts my ICANON change.  Yet:
>>> There is AFAIK no bug report or test case for the long line problem.
>> AFAIK, the missing bug-report is the one that shows the ills of sending
>> EOFs, while the bug-report for long-lines is bug#6149, which should be
>> re-opened.

> As I said in relation to this, I could not reproduce the error in 6149.

I could and still can:

  emacs -Q
  M-x shell
  echo
  SPC C-u 5000 a
  | wc
  RET

and instead of getting wc's output I just get 4090 "a"s
(the command appears to get cut at 4096 chars).  The shell is `zsh', but
last time I tried, the result was the same with bash.

>> Could you spell out more precisely how it's different from what we do
>> now: we already check emacs_write's output to see if the buffer is full,
>> in which case we wait, don't we?

> Yes, but we wait a small time and then try to write again (if I read the
> code correctly).  If nobody is reading, that can turn into a (sort of)
> busy wait.
> The point of handing it off to select is then we don't write until we know
> we can write something.

So this doesn't explain the loss of chars, it would only cause
a performance problem.

>> IIUC VMIN and VTIME not only are specific to the non-ICANON mode, but
>> also they use the same slot as some of the other settings (specifically
>> the slots may be shared with VEOL and VEOF, according to my manpage).
> That is correct.

Thanks for confirming, I've fixed it already,


        Stefan

Re: [Emacs-diffs] /srv/bzr/emacs/trunk r100961: Enable ICANON (Bug#6771). Any long line problem must be solved differently.

[Jan Djärv]

Stefan Monnier skrev 2010-08-06 15.03:
>>> I agree with the commit that reverts my ICANON change.  Yet:
>>>> There is AFAIK no bug report or test case for the long line problem.
>>> AFAIK, the missing bug-report is the one that shows the ills of sending
>>> EOFs, while the bug-report for long-lines is bug#6149, which should be
>>> re-opened.
>
>> As I said in relation to this, I could not reproduce the error in 6149.
>
> I could and still can:
>
>    emacs -Q
>    M-x shell
>    echo
>    SPC C-u 5000 a
>    | wc
>    RET
>
> and instead of getting wc's output I just get 4090 "a"s
> (the command appears to get cut at 4096 chars).  The shell is `zsh', but
> last time I tried, the result was the same with bash.

It is definitly a shell thing.  I tried with 100000 and bash: No problem.
It could be the tty driver that does things differently, but I tested on OSX 
and Fedora 12.

It really doesn't matter if Emacs sets icanon or not for bash at least and 
probably most shells that use command line editing.  Those shells will turn 
off icanon while reading input, and only turn it on when a command is executed.

	Jan D.



	Jan D.

Re: [Emacs-diffs] /srv/bzr/emacs/trunk r100961: Enable ICANON (Bug#6771). Any long line problem must be solved differently.

[Jan Djärv]

Jan Djärv skrev 2010-08-06 16.30:
>
>
> Stefan Monnier skrev 2010-08-06 15.03:
>>>> I agree with the commit that reverts my ICANON change. Yet:
>>>>> There is AFAIK no bug report or test case for the long line problem.
>>>> AFAIK, the missing bug-report is the one that shows the ills of sending
>>>> EOFs, while the bug-report for long-lines is bug#6149, which should be
>>>> re-opened.
>>
>>> As I said in relation to this, I could not reproduce the error in 6149.
>>
>> I could and still can:
>>
>> emacs -Q
>> M-x shell
>> echo
>> SPC C-u 5000 a
>> | wc
>> RET
>>
>> and instead of getting wc's output I just get 4090 "a"s
>> (the command appears to get cut at 4096 chars). The shell is `zsh', but
>> last time I tried, the result was the same with bash.
>
> It is definitly a shell thing. I tried with 100000 and bash: No problem.
> It could be the tty driver that does things differently, but I tested on OSX
> and Fedora 12.

I tried with zsh, then I get
zsh: command too long: ...

> It really doesn't matter if Emacs sets icanon or not for bash at least and
> probably most shells that use command line editing. Those shells will turn off
> icanon while reading input, and only turn it on when a command is executed.

I think this is a non-issue.  Emacs should not try to work around limitations 
set by the tty driver.  Pty:s are not a general communication mechanism, they 
are supposed to emulate a terminal.  I mean try this:

% read x ; echo "$x" > out

and type in a lot of input.  You simply can't get more than about 4096 
characters into out.  If you want more, use pipes.

Even if Emacs sets the tty to raw input, there is no guarantee that the 
subprocess doesn't set it back.  If a subprocess wants to read a lot of stuff 
from a tty input, it better put that tty into raw mode itself, like bash does.

There isn't a way for Emacs to detect this, the write from Emacs may indeed 
return success, but the tty driver silently discards the input.  I say just 
document it and close these bug reports.

	Jan D.