Setting bidi-paragraph-direction in comint modes

Setting bidi-paragraph-direction in comint modes

[Chong Yidong]

Any objections to setting bidi-paragraph-direction to `left-to-right' in
Comint and derived modes?  The bidi code appears to lead to a
performance regression.  You can observe this with

1. emacs -Q
2. M-x shell RET
3. ls /usr/share/doc    (Prints about 2000 lines on my computer)
4. Repeat step 3 many times.

As the buffer gets to about 8000 lines, redisplay slows to a crawl.
Setting bidi-paragraph-direction to `left-to-right' fixes this (mostly;
it's still a touch slower than in Emacs 23).  AFAICT, bidi paragraphs
don't make sense in Comint-like modes anyway.

Re: Setting bidi-paragraph-direction in comint modes

[Stefan Monnier]

> Any objections to setting bidi-paragraph-direction to `left-to-right' in
> Comint and derived modes?

None from me,


        Stefan

Re: Setting bidi-paragraph-direction in comint modes

[Eli Zaretskii]

> From: Chong Yidong <cyd@gnu.org>
> Date: Tue, 14 Feb 2012 14:35:52 +0800
> 
> 1. emacs -Q
> 2. M-x shell RET
> 3. ls /usr/share/doc    (Prints about 2000 lines on my computer)
> 4. Repeat step 3 many times.
> 
> As the buffer gets to about 8000 lines, redisplay slows to a crawl.

I cannot reproduce this here.  I tried the same in a directory with
2190 files, generated a 10K-line buffer with no empty lines, and I see
no "crawl", only some minor slowdown.  Maybe I didn't try the commands
you did?  Which Emacs commands caused such a slow redisplay on your
machine?

In any case, see this comment in bidi.c:

  /* On my 2005-vintage machine, searching back for paragraph start
     takes ~1 us per line.  And bidi_paragraph_init is called 4 times
     when user types C-p.  The number below limits each call to
     bidi_paragraph_init to about 10 ms.  */
  #define MAX_PARAGRAPH_SEARCH 7500

In my experience, 4 x 10 = 40 ms is not perceived as a terrible
slowdown, and most machines people use today are faster than mine.
This number was tuned to treat reasonably well hard cases such as the
one described in bug #9470.

> AFAICT, bidi paragraphs don't make sense in Comint-like modes
> anyway.

Unfortunately, it does: a console program that displays localized
messages through gettext will need dynamic determination of the
paragraph direction in bidirectional locales, to display the messages
as users of those locales expect.

In any case, I'd like first to reproduce the problem, and then look
into its root cause.  Somehow, I doubt that searching for the
paragraph start is it, with the above limit in effect.

Re: Setting bidi-paragraph-direction in comint modes

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Chong Yidong <cyd@gnu.org>
>> Date: Tue, 14 Feb 2012 14:35:52 +0800
>> 
>> 1. emacs -Q
>> 2. M-x shell RET
>> 3. ls /usr/share/doc    (Prints about 2000 lines on my computer)
>> 4. Repeat step 3 many times.
>> 
>> As the buffer gets to about 8000 lines, redisplay slows to a crawl.
>
> I cannot reproduce this here.  I tried the same in a directory with
> 2190 files, generated a 10K-line buffer with no empty lines, and I see
> no "crawl", only some minor slowdown.  Maybe I didn't try the commands
> you did?  Which Emacs commands caused such a slow redisplay on your
> machine?

Like I said, it's simply `ls /usr/share/doc'.

You say that there is "some minor slowdown": that's the same slowdown
I'm referring to, I think.  On my machine, the performance difference
with and without bidi-paragraph-direction is extremely noticeable.

Re: Setting bidi-paragraph-direction in comint modes

[Eli Zaretskii]

> From: Chong Yidong <cyd@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Wed, 15 Feb 2012 08:49:18 +0800
> 
> Like I said, it's simply `ls /usr/share/doc'.

You mean, the insertion of the next chunk of text that arrives from
`ls'?

> You say that there is "some minor slowdown": that's the same slowdown
> I'm referring to, I think.  On my machine, the performance difference
> with and without bidi-paragraph-direction is extremely noticeable.

Can you quantify it?  Like, how much time it takes to insert the first
2000 lines vs the last?

I didn't look at the insertion itself, I tried the various cursor
motion commands _after_ the output from `ls' was already in the
buffer.  Do you see the "crawl" there as well?

Re: Setting bidi-paragraph-direction in comint modes

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Chong Yidong <cyd@gnu.org>
>> Cc: emacs-devel@gnu.org
>> Date: Wed, 15 Feb 2012 08:49:18 +0800
>> 
>> Like I said, it's simply `ls /usr/share/doc'.
>
> You mean, the insertion of the next chunk of text that arrives from
> `ls'?

Yep.

> I didn't look at the insertion itself, I tried the various cursor
> motion commands _after_ the output from `ls' was already in the
> buffer.  Do you see the "crawl" there as well?

Yep.  After inserting about 7000 lines into the *shell* buffer, as
described, press and hold down M-v (or PageUp) for several seconds.
Redisplay freezes, not updating until about two seconds after the key is
released.  Then it updates the screen with the new cursor position.  But
if bidi-paragraph-direction is set to 'left-to-right, the display
updates in real time as I press and hold M-v.

> Can you quantify it?  Like, how much time it takes to insert the first
> 2000 lines vs the last?

Not easily, but it is definitely not my imagination.  You can't
reproduce the problem at all?

Re: Setting bidi-paragraph-direction in comint modes

[Eli Zaretskii]

> From: Chong Yidong <cyd@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Wed, 15 Feb 2012 14:24:11 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Chong Yidong <cyd@gnu.org>
> >> Cc: emacs-devel@gnu.org
> >> Date: Wed, 15 Feb 2012 08:49:18 +0800
> >> 
> >> Like I said, it's simply `ls /usr/share/doc'.
> >
> > You mean, the insertion of the next chunk of text that arrives from
> > `ls'?
> 
> Yep.

I will time this and see how frequently are we looking for the
paragraph beginning in this case.  I never tried to time repeated
insertions in small chunks, so I don't know what redisplay does in
that case.

> > I didn't look at the insertion itself, I tried the various cursor
> > motion commands _after_ the output from `ls' was already in the
> > buffer.  Do you see the "crawl" there as well?
> 
> Yep.  After inserting about 7000 lines into the *shell* buffer, as
> described, press and hold down M-v (or PageUp) for several seconds.
> Redisplay freezes, not updating until about two seconds after the key is
> released.  Then it updates the screen with the new cursor position.

I see nothing like that, the display never freezes.  Strange.  What
kind of CPU do you have there?  Is it a GUI session or a TTY session?
Does Emacs run on the local machine or remote one?

> > Can you quantify it?  Like, how much time it takes to insert the first
> > 2000 lines vs the last?
> 
> Not easily, but it is definitely not my imagination.

OK, I will do it myself.

> You can't reproduce the problem at all?

I see some slowdown, but I cannot in good faith characterize it as
anything like "crawl", and M-v doesn't get stuck.  This is on 7-year
old single-core CPU at 3GHz, in an Emacs compiled without
optimizations (-O0).  On the same machine, an optimized build (-O2)
runs significantly faster, so I see almost no slowdown at all.

If you say "ls /usr/share/doc >> foo" 4 times and then visit the
resulting file, do you see the same slow M-v in the buffer?

Anyway, please make a bug report about this, and I will see what I can
do about it.

Re: Setting bidi-paragraph-direction in comint modes

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> Anyway, please make a bug report about this, and I will see what I can
> do about it.

See Bug#10835.

Re: Setting bidi-paragraph-direction in comint modes

[Johan Bockgård]

Chong Yidong <cyd@gnu.org> writes:

> Not easily, but it is definitely not my imagination.  You can't
> reproduce the problem at all?

FWIW, I see the effect of bidi-paragraph-direction on scrolling clearly
after setting the keyboard repeat rate sufficiently high.

Re: Setting bidi-paragraph-direction in comint modes

[Eli Zaretskii]

> From: Johan Bockgård <bojohan@gnu.org>
> Date: Wed, 15 Feb 2012 18:46:48 +0100
> 
> Chong Yidong <cyd@gnu.org> writes:
> 
> > Not easily, but it is definitely not my imagination.  You can't
> > reproduce the problem at all?
> 
> FWIW, I see the effect of bidi-paragraph-direction on scrolling clearly
> after setting the keyboard repeat rate sufficiently high.

You can always outrun Emacs redisplay, even in Emacs 23, by setting
the keyboard auto-repeat rate sufficiently high.  That's not what this
thread is about, clearly.