[GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[Nicholas Ochiel]

Steps:
0. Open gui emacs in X.
1. M-x term (char mode, /bin/zsh)
2. Enter a long input that causes text to wrap at fringe.
3. Notice that point jumps irregularly.
4. Move point and attempt to edit some portion of wrapped line in terminal.
5. Notice that prompt or previous terminal output is overwritten in an
unpredictable manner.
6. Further operations on text cause unpredictable editing behaviour.
7. This occurs regardless of truncate-lines, visual-line-mode.

Reproduced with:
- Emacs 24.5.1,
- emacs-snapshot 26.0.50.2

This odd behaviour was also reported here:
1. https://github.com/syl20bnr/spacemacs/issues/5520

Possibly related:
1. http://stackoverflow.com/questions/5541622/comint-mode-inserts-line-break-every-4096-characters?rq=1
2. http://stackoverflow.com/questions/5872686/linewrap-in-gui-emacs-in-linux

--
Sincerely,
Nicholas Ochiel

Re: [GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[Eli Zaretskii]

> From: Nicholas Ochiel <nochiel@gmail.com>
> Date: Sat, 26 Nov 2016 02:04:56 +0300
> Cc: emacs-devel@gnu.org
> 
> Steps:
> 0. Open gui emacs in X.
> 1. M-x term (char mode, /bin/zsh)

Are zsh and a GUI frame necessary for reproducing this issue?

> 2. Enter a long input that causes text to wrap at fringe.
> 3. Notice that point jumps irregularly.

Do you mean point jumps by itself, or do you mean it jumps when you
try moving it with cursor motion keys?  IOW, please describe what
triggers these jumps.

> 4. Move point and attempt to edit some portion of wrapped line in terminal.
> 5. Notice that prompt or previous terminal output is overwritten in an
> unpredictable manner.
> 6. Further operations on text cause unpredictable editing behaviour.
> 7. This occurs regardless of truncate-lines, visual-line-mode.

Did you try to disable the fringes?  Did it help?

> 
> Reproduced with:
> - Emacs 24.5.1,
> - emacs-snapshot 26.0.50.2
> 
> This odd behaviour was also reported here:
> 1. https://github.com/syl20bnr/spacemacs/issues/5520

The discussion there seems to say it has to do with something called
oh-my-zsh and/or multi-term?  Does the recipe you posted reproduce the
problem in "emacs -Q"?

Thanks.

Re: [GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[Nicholas Ochiel]

> The discussion there seems to say it has to do with something called
oh-my-zsh and/or multi-term?

Indeed, yes, oh-my-zsh (http://ohmyz.sh), which provides plugins for
zsh appears to be the source of the problem. (I apologise for not
investigating that a bit more before posting.)

Specifically, by disabling the plugin which provides *syntax
highlighting* in the shell,
(https://github.com/zsh-users/zsh-syntax-highlighting), ~ emacs -Q
--eval '(term "/bin/zsh")' ~ works without error.

(However, the syntax highlighting plugin works without any issues in xterm.)

I've provided further details below in case they are still relevant.

> Are zsh and a GUI frame necessary for reproducing this issue?

- zsh is necessary.
- The issue occurs both in gui and virtual console.

> Does the recipe you posted reproduce the
problem in "emacs -Q"?

Reproduced with ~ emacs -Q --eval '(term "/bin/zsh")' ~

> Do you mean point jumps by itself, or do you mean it jumps when you
try moving it with cursor motion keys?  IOW, please describe what
triggers these jumps.

- Type one char at a time until the edge of the frame is reached.
- Type one char; line wraps to the next line.
- Type one char; point moves to line above.

> Did you try to disable the fringes?  Did it help?

Issue still occurs when:
- ~(fringe-mode nil)~
- ~(setq overflow-newline-into-fringe nil)
--
Sincerely,
Nicholas Ochiel


On 26 November 2016 at 12:12, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nicholas Ochiel <nochiel@gmail.com>
>> Date: Sat, 26 Nov 2016 02:04:56 +0300
>> Cc: emacs-devel@gnu.org
>>
>> Steps:
>> 0. Open gui emacs in X.
>> 1. M-x term (char mode, /bin/zsh)
>
> Are zsh and a GUI frame necessary for reproducing this issue?
>
>> 2. Enter a long input that causes text to wrap at fringe.
>> 3. Notice that point jumps irregularly.
>
> Do you mean point jumps by itself, or do you mean it jumps when you
> try moving it with cursor motion keys?  IOW, please describe what
> triggers these jumps.
>
>> 4. Move point and attempt to edit some portion of wrapped line in terminal.
>> 5. Notice that prompt or previous terminal output is overwritten in an
>> unpredictable manner.
>> 6. Further operations on text cause unpredictable editing behaviour.
>> 7. This occurs regardless of truncate-lines, visual-line-mode.
>
> Did you try to disable the fringes?  Did it help?
>
>>
>> Reproduced with:
>> - Emacs 24.5.1,
>> - emacs-snapshot 26.0.50.2
>>
>> This odd behaviour was also reported here:
>> 1. https://github.com/syl20bnr/spacemacs/issues/5520
>
> The discussion there seems to say it has to do with something called
> oh-my-zsh and/or multi-term?  Does the recipe you posted reproduce the
> problem in "emacs -Q"?
>
> Thanks.

Re: [GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[Eli Zaretskii]

> From: Nicholas Ochiel <nochiel@gmail.com>
> Date: Sat, 26 Nov 2016 13:29:48 +0300
> Cc: Dan Nicolaescu <dann@ics.uci.edu>, emacs-devel@gnu.org
> 
> > The discussion there seems to say it has to do with something called
> oh-my-zsh and/or multi-term?
> 
> Indeed, yes, oh-my-zsh (http://ohmyz.sh), which provides plugins for
> zsh appears to be the source of the problem. (I apologise for not
> investigating that a bit more before posting.)
> 
> Specifically, by disabling the plugin which provides *syntax
> highlighting* in the shell,
> (https://github.com/zsh-users/zsh-syntax-highlighting), ~ emacs -Q
> --eval '(term "/bin/zsh")' ~ works without error.
> 
> (However, the syntax highlighting plugin works without any issues in xterm.)

I hope you are aware of the significant difference between any
terminal emulator and an Emacs terminal emulator.  Where the former
can rely on the terminal device driver to perform certain commands
sent by the shell, Emacs must reproduce all of those features in Lisp
code, and convert each command to the equivalent Emacs commands to
move the cursor, insert or delete text, change color of the text, etc.

> I've provided further details below in case they are still relevant.

Thanks.  I think the reason is that term.el's ANSI sequence handler is
simply unprepared to deal with the complex commands this plug-in
causes zsh to send.  As result, term.el becomes confused when text
deletion and reinsertion happen at the end of the screen line.
Patches to term.el to support that are welcome.

Re: [GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[Nicholas Ochiel]

[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]

>On 26 Nov 2016 15:53, "Eli Zaretskii" <eliz@gnu.org> wrote:
>
> > From: Nicholas Ochiel <nochiel@gmail.com>
> > Date: Sat, 26 Nov 2016 13:29:48 +0300
> > Cc: Dan Nicolaescu <dann@ics.uci.edu>, emacs-devel@gnu.org
> >

> I hope you are aware of the significant difference between any
> terminal emulator and an Emacs terminal emulator.  Where the former
> can rely on the terminal device driver to perform certain commands
> sent by the shell, Emacs must reproduce all of those features in Lisp
> code, and convert each command to the equivalent Emacs commands to
> move the cursor, insert or delete text, change color of the text, etc.

I'm aware of the difference though I admit that I'm not yet intimate with
the specifics of the different implementations.

> Thanks.  I think the reason is that term.el's ANSI sequence handler is
> simply unprepared to deal with the complex commands this plug-in
> causes zsh to send.  As result, term.el becomes confused when text
> deletion and reinsertion happen at the end of the screen line.
> Patches to term.el to support that are welcome.

I'll look into what the shell is doing in this case and how term.el is
handling it. I'll update this thread with any progress.

[-- Attachment #2: Type: text/html, Size: 1635 bytes --]

Re: [GUI emacs:term.el] Possible regression causes word wrapping to break term display in zsh

[Eli Zaretskii]

> From: Nicholas Ochiel <nochiel@gmail.com>
> Date: Sun, 27 Nov 2016 09:41:08 +0300
> Cc: emacs-devel@gnu.org, Dan Nicolaescu <dann@ics.uci.edu>
> 
> > Thanks. I think the reason is that term.el's ANSI sequence handler is
> > simply unprepared to deal with the complex commands this plug-in
> > causes zsh to send. As result, term.el becomes confused when text
> > deletion and reinsertion happen at the end of the screen line.
> > Patches to term.el to support that are welcome.
> 
> I'll look into what the shell is doing in this case and how term.el is handling it. I'll update this thread with any
> progress.

Thank you!