^D is broken in shell-mode

^D is broken in shell-mode

[Miles Bader]

It now results in a literal ^D being seen by the subproces.

I vaguely recall seeing recently that emacs stopped using ICANON mode
for ptys; I presume this is the cause?

It's been a long time since I did pty programming, but I seem to recall
that at least in some pty states, doing a zero-length write in the pty
master will send an end-of-file to the slave tty.  I don't recall the
details though...

Thanks,

-Miles

-- 
Future, n. That period of time in which our affairs prosper, our friends
are true and our happiness is assured.

Re: ^D is broken in shell-mode

[Andreas Schwab]

Miles Bader <miles.bader.fz@renesas.com> writes:

> I vaguely recall seeing recently that emacs stopped using ICANON mode
> for ptys; I presume this is the cause?

Yes, see bug#6350.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: ^D is broken in shell-mode

[Stefan Monnier]

> It now results in a literal ^D being seen by the subproces.
> I vaguely recall seeing recently that emacs stopped using ICANON mode
> for ptys; I presume this is the cause?

Yes.

> It's been a long time since I did pty programming, but I seem to recall
> that at least in some pty states, doing a zero-length write in the pty
> master will send an end-of-file to the slave tty.  I don't recall the
> details though...

I really wish someone could help.  At least if someone could point me to
some kind of documentation that would be very helpful.  The best "doc"
I could find so far is the xterm source code, which I do not really
understand and whose behavior I haven't been able to reproduce yet
(e.g. it seems not to disable ICANON mode, and it doesn't seem to ever
send EOF either, even when sending large chunks of data, yet that data
doesn't get truncated, whereas when I try to to make Emacs send large
amounts of data in ICANON mode without EOFs, it gets truncated).


        Stefan

Re: ^D is broken in shell-mode

[David De La Harpe Golden]

On 08/06/10 21:22, Stefan Monnier wrote:
>> It now results in a literal ^D being seen by the subproces.
>> I vaguely recall seeing recently that emacs stopped using ICANON mode
>> for ptys; I presume this is the cause?
>
> Yes.
>
>> It's been a long time since I did pty programming, but I seem to recall
>> that at least in some pty states, doing a zero-length write in the pty
>> master will send an end-of-file to the slave tty.  I don't recall the
>> details though...
>

FWIW, The write(3) manpage has this to say (about pipes not ptys):

 > Also,  some  existing  systems (for  example,  Eighth  Edition)
 > permit a write of zero bytes to mean that the reader should get an
 > end-of-file indication; for those systems, a return value of zero
 > from write() indicates a successful write of an end-of-file
 > indication.

That's possibly what Miles is recollecting?

I'm not sure it works for ptys, or that it's actually TRT here...

> I really wish someone could help.

Not necessarily helping:

If you turn off ICANON, it's little wonder EOF char is just
passed through?

This page here:
http://www.lafn.org/~dave/linux/terminalIO.html

makes the claim:

"""
Cooked mode is the default state of your terminal for almost all shells. 
At least when you execute another program with the shell, the terminal 
is put into cooked mode before it makes a call to an exec function.
"""

Stty in shell exec'd processes and shell children do indeed show a 
cooked terminal on my system, so likely emacs when in its capacity as a 
"normal" shell / superior process should be ensuring the pty slave it 
hands to children is ICANON out-of-box?

But n.b. that means the MAX_INPUT/MAX_CANON limits described at that 
link would apply.

Re: ^D is broken in shell-mode

[Miles Bader]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>> It's been a long time since I did pty programming, but I seem to recall
>> that at least in some pty states, doing a zero-length write in the pty
>> master will send an end-of-file to the slave tty.  I don't recall the
>> details though...
>
> I really wish someone could help.  At least if someone could point me to
> some kind of documentation that would be very helpful.  The best "doc"
> I could find so far is the xterm source code, which I do not really
> understand and whose behavior I haven't been able to reproduce yet
> (e.g. it seems not to disable ICANON mode, and it doesn't seem to ever
> send EOF either, even when sending large chunks of data, yet that data
> doesn't get truncated, whereas when I try to to make Emacs send large
> amounts of data in ICANON mode without EOFs, it gets truncated).

I may be thinking of TIOCREMOTE (from the solaris man page: "In normal
usage, a write of data is like the data typed as a line on the terminal;
a write of 0 bytes is like typing an EOF character"), but unfortunately
on linux the tty_ioctl man page says:  "The BSD ioctls TIOCSTOP,
TIOCSTART, TIOCUCNTL, TIOCREMOTE have not been implemented under Linux."

Still I'd think there would be some sort of equivalent functionality...

-Miles

-- 
Happiness, n. An agreeable sensation arising from contemplating the misery of
another.

Re: ^D is broken in shell-mode

[Jan Djärv]

2010-06-08 22:22, Stefan Monnier skrev:
>> It now results in a literal ^D being seen by the subproces.
>> I vaguely recall seeing recently that emacs stopped using ICANON mode
>> for ptys; I presume this is the cause?
>
> Yes.
>
>> It's been a long time since I did pty programming, but I seem to recall
>> that at least in some pty states, doing a zero-length write in the pty
>> master will send an end-of-file to the slave tty.  I don't recall the
>> details though...
>
> I really wish someone could help.  At least if someone could point me to
> some kind of documentation that would be very helpful.  The best "doc"
> I could find so far is the xterm source code, which I do not really
> understand and whose behavior I haven't been able to reproduce yet
> (e.g. it seems not to disable ICANON mode, and it doesn't seem to ever
> send EOF either, even when sending large chunks of data, yet that data
> doesn't get truncated, whereas when I try to to make Emacs send large
> amounts of data in ICANON mode without EOFs, it gets truncated).
>

I don't know what the original problem was (sending large amount of data may 
hang?) that prompted the removal of ICANON.  But as far as xterm goes (if it 
hasn't changed radically in the last year or so), it does something like this:

loop1:
try to write min(128 characters, what we want to write) to the pty (128 is 
half of what POSIX says is the minimum pty buffer size)

If write returned what we put in and there is more to write, goto loop1.

If write returned less or (-1 and errno is EAGAIN/EWOULDBLOCK), stop writing 
and select on the fd for write.

When select says it is OK to write again, start at loop1.

	Jan D.

Re: ^D is broken in shell-mode

[Stephen J. Turnbull]

Stefan Monnier writes:

 > I really wish someone could help.

Try Martin Buchholz <martin@xemacs.org>.  I think he did our pty
support.  If he didn't, he would know who.  Whoever did it had a real
good time -- not!, the profanity/LOC density is high there. ;-)

 > At least if someone could point me to some kind of documentation
 > that would be very helpful.

That's the reason for a lot of the profanity.  Apparently there isn't
much, outside of the kernels' sources.

Re: ^D is broken in shell-mode

[Miles Bader]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
> The best "doc" I could find so far is the xterm source code, which I
> do not really understand and whose behavior I haven't been able to
> reproduce yet (e.g. it seems not to disable ICANON mode, and it
> doesn't seem to ever send EOF either

Xterm seems like a somewhat different case though -- it acts like a
physical terminal, so just has to send raw keystrokes (including ^D, ^C,
etc), which are interpreted however the slave end of the pty wants, not
"high-level" events like EOF (or interrupt, etc).  So ICANON (etc) may
actually be set by the slave end, xterm itself doesn't have to care.

> even when sending large chunks of data, yet that data doesn't get
> truncated, whereas when I try to to make Emacs send large amounts of
> data in ICANON mode without EOFs, it gets truncated).

Can the data just be sent in chunks, as JanD mentioned?

I think support for higher-level pty control has always been kind of
poor in unix, and it sounds like linux is even worse... :(

-Miles

-- 
Liberty, n. One of imagination's most precious possessions.

Re: ^D is broken in shell-mode

[Jose A. Ortega Ruiz]

On Tue, Jun 08 2010, Miles Bader wrote:

> It now results in a literal ^D being seen by the subproces.

I've also noticed that this problem affects comint-mode, and is not
limited to xterm: i'm encountering it in rxvt too.

Cheers,
jao

Re: ^D is broken in shell-mode

[Miles Bader]

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>> It now results in a literal ^D being seen by the subproces.
>
> I've also noticed that this problem affects comint-mode, and is not
> limited to xterm: i'm encountering it in rxvt too.

Since shell-mode uses comint, this isn't surprising ... :]

Why do you mention xterm, BTW?  [I wasn't running Emacs in a TTY, rather
I was using X.]

-Miles

-- 
Alliance, n. In international politics, the union of two thieves who have
their hands so deeply inserted in each other's pockets that they cannot
separately plunder a third.

Re: ^D is broken in shell-mode

[Jose A. Ortega Ruiz]

On Thu, Jun 10 2010, Miles Bader wrote:

> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>>> It now results in a literal ^D being seen by the subproces.
>>
>> I've also noticed that this problem affects comint-mode, and is not
>> limited to xterm: i'm encountering it in rxvt too.
>
> Since shell-mode uses comint, this isn't surprising ... :]

Fair enough :) But it could have been a bug in shell-mode proper, i
guess.

> Why do you mention xterm, BTW?  [I wasn't running Emacs in a TTY, rather
> I was using X.]

I got the (apparently wrong) impression that xterm was being mentioned
in the ensuing discussion (which i just skimmed), and thought
that people were looking for problems related with that. Sorry for the
confusion.

Cheers,
jao

Re: ^D is broken in shell-mode

[Štěpán Němec]

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> On Tue, Jun 08 2010, Miles Bader wrote:
>
>> It now results in a literal ^D being seen by the subproces.
>
> I've also noticed that this problem affects comint-mode, and is not
> limited to xterm: i'm encountering it in rxvt too.

Could the problem with Magit being unusable with current Emacs be
related as well? (Perhaps through `process-send-eof' used in
`magit-run*' (?); interestingly, I experience no problems in shell or
comint modes, though):

http://github.com/philjackson/magit/issues/#issue/18


    Štěpán

Re: ^D is broken in shell-mode

[Jose A. Ortega Ruiz]

On Wed, Jun 23 2010, Štěpán Němec wrote:

> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>
>> On Tue, Jun 08 2010, Miles Bader wrote:
>>
>>> It now results in a literal ^D being seen by the subproces.
>>
>> I've also noticed that this problem affects comint-mode, and is not
>> limited to xterm: i'm encountering it in rxvt too.
>
> Could the problem with Magit being unusable with current Emacs be
> related as well? (Perhaps through `process-send-eof' used in
> `magit-run*' (?); interestingly, I experience no problems in shell or
> comint modes, though):

Certainly, the problems i'm observing in comint-derived modes are due to
`process-send-eof' not working (and magit doesn't work for me, either).

jao
-- 
I always pass on good advice. It's the only thing to do with it. It is
never any use to oneself.
 -Oscar Wilde

Re: ^D is broken in shell-mode

[Štěpán Němec]

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> On Wed, Jun 23 2010, Štěpán Němec wrote:
>
>> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>>
>>> On Tue, Jun 08 2010, Miles Bader wrote:
>>>
>>>> It now results in a literal ^D being seen by the subproces.
>>>
>>> I've also noticed that this problem affects comint-mode, and is not
>>> limited to xterm: i'm encountering it in rxvt too.
>>
>> Could the problem with Magit being unusable with current Emacs be
>> related as well? (Perhaps through `process-send-eof' used in
>> `magit-run*' (?); interestingly, I experience no problems in shell or
>> comint modes, though):
>
> Certainly, the problems i'm observing in comint-derived modes are due to
> `process-send-eof' not working (and magit doesn't work for me, either).

So IIUC the trunk is currently broken. Why is the change not reverted
until a solution to the problem is found?

    Štěpán

Re: ^D is broken in shell-mode

[Thierry Volpiatto]

Štěpán Němec <stepnem@gmail.com> writes:

> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>
>> On Wed, Jun 23 2010, Štěpán Němec wrote:
>>
>>> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>>>
>>>> On Tue, Jun 08 2010, Miles Bader wrote:
>>>>
>>>>> It now results in a literal ^D being seen by the subproces.
>>>>
>>>> I've also noticed that this problem affects comint-mode, and is not
>>>> limited to xterm: i'm encountering it in rxvt too.
>>>
>>> Could the problem with Magit being unusable with current Emacs be
>>> related as well? (Perhaps through `process-send-eof' used in
>>> `magit-run*' (?); interestingly, I experience no problems in shell or
>>> comint modes, though):
>>
>> Certainly, the problems i'm observing in comint-derived modes are due to
>> `process-send-eof' not working (and magit doesn't work for me, either).
>
> So IIUC the trunk is currently broken. Why is the change not reverted
> until a solution to the problem is found?
Agree, C-d work on 23 branch...

-- 
Thierry Volpiatto
Gpg key: http://pgp.mit.edu/