bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[jamezmcclain]

This is pretty minor, but it surprised me. When a buffer is filled with
spaces going all the way to the last column without overwrapping. If one
tries to navigate by lines using C-n or C-p and holding them down, there
is noticable lag. My window-body-width is 115 and window-body-height is
60.
Recipe:
1. Open emacs         : Emacs -Q
2. Open a blank buffer: C-x b test-2
3. Fill the buffer up with spaces until it would force a linewrap, then
   use C-j to enter a newline. C-u 115 <space>
                               C-j
4. The buffer should appear blank unless you use whitespace mode, now try
   holding down C-n or C-p. Emacs will lag.

I also tested this in a tty emacs session on linux over ssh, and it
seemed the lag was also aparent over there.

The lag lasted at most around 5 seconds.

If the line has all spaces, one period, then a newline. (So if there is
one non-space character it seems) then this lag does not happen.

Frankly just curious,

James



In GNU Emacs 24.5.1 (x86_64-w64-mingw32)
 of 2015-05-16 on KAEL
Windowing system distributor `Microsoft Corp.', version 6.3.9600
Configured using:
 `configure --prefix=/z/emacs --host=x86_64-w64-mingw32
 --target=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --with-wide-int
 --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2
 --with-gnutls --with-sound=yes --with-file-notification=yes
 --without-dbus --without-imagemagick 'CFLAGS=-O3 -fomit-frame-pointer
 -g0 -pipe' 'LDFLAGS=-static-libgcc -static-libstdc++ -static -s
 -Wl,-s''

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Fundamental

Minor modes in effect:
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Checking 57 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/eshell...
Checking 70 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/erc...
Checking 34 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/emulation...
Checking 151 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/emacs-lisp...
Checking 24 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/cedet...
Checking 57 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/calendar...
Checking 87 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/calc...
Checking 111 files in c:/ProgramData/chocolatey/lib/emacs64/tools/emacs/share/emacs/24.5/lisp/obsolete...
Checking for load-path shadows...done
Mark set [2 times]
Quit

Load-path shadows:
None found.

Features:
(shadow sort mail-extr misearch multi-isearch noutline outline
easy-mmode view emacsbug message rfc822 mml mml-sec mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mail-utils eieio-opt speedbar sb-image
ezimage dframe find-func help-mode network-stream starttls tls erc-list
erc-menu easymenu erc-join erc-ring erc-networks erc-pcomplete pcomplete
comint ansi-color ring erc-track erc-match erc-button wid-edit erc-fill
erc-stamp erc-netsplit erc-goodies erc erc-backend erc-compat
format-spec auth-source eieio byte-opt bytecomp byte-compile cl-extra
cl-loaddefs cl-lib cconv eieio-core gnus-util mm-util help-fns
mail-prsvr password-cache thingatpt pp kmacro whitespace time-date
tooltip electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 ls-lisp w32-common-fns disp-table w32-win w32-vars tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)

Memory information:
((conses 16 133688 22081)
 (symbols 56 23279 0)
 (miscs 48 130 422)
 (strings 32 29961 2899)
 (string-bytes 1 788055)
 (vectors 16 17076)
 (vector-slots 8 461686 8433)
 (floats 8 87 296)
 (intervals 56 1762 121)
 (buffers 960 22))

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> From: jamezmcclain@gmail.com
> Date: Thu, 05 May 2016 02:01:54 -0700
> 
> This is pretty minor, but it surprised me. When a buffer is filled with
> spaces going all the way to the last column without overwrapping. If one
> tries to navigate by lines using C-n or C-p and holding them down, there
> is noticable lag.
> Recipe:
> 1. Open emacs         : Emacs -Q
> 2. Open a blank buffer: C-x b test-2
> 3. Fill the buffer up with spaces until it would force a linewrap, then
>    use C-j to enter a newline. C-u 115 <space>
>                                C-j
> 4. The buffer should appear blank unless you use whitespace mode, now try
>    holding down C-n or C-p. Emacs will lag.
> 
> I also tested this in a tty emacs session on linux over ssh, and it
> seemed the lag was also aparent over there.
> 
> The lag lasted at most around 5 seconds.

I cannot reproduce this.  I don't see any lags with this recipe.

>  My window-body-width is 115 and window-body-height is 60.

But that's not what you get with "emacs -Q", right?  So maybe the
problem only happens when you don't use -Q?

Also, is it possible that you have some optional display features
enabled, like ClearType?

Finally, maybe you could try a different build, preferably of the
latest pretest of Emacs 25.1, and see if you have the same problem
there.

Thanks.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

[Please keep the bug address on the CC line.]

> From: James McClain <jamezmcclain@gmail.com>
> Date: Thu, 5 May 2016 11:07:23 -0700
> 
> I expanded the window to full screen after using emacs -Q, then split
> it vertically to get that size. But I did start it with emacs -Q.
> I just tested it on emacs-25.0.93-x86_64-w64-mingw32, starting with
> .\emacs-25.0.93.exe -Q, and without changing the window size.
> window body width is 80, window body height is 34.
> Exactly what I did:
> .\emacs-25.0.93.exe -Q
> C-x b test-2 <ret> <ret> (to confirm)
> f3 to start a macro
> spaces until it would force a wrap (I do this by pressing space until
> I see it wrap, then backspacing by 1)
> C-j for a newline
> Then I repeat this until I have a bunch of lines of just spaces, in
> this case I did 67.
> Try holding C-p to move, and I experience a lot of lag.

Well, like I said: it doesn't happen here.  Not sure what else to try.

Did you look into the Emacs display features, like ClearType, in case
you have that turned on?

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> Date: Thu, 05 May 2016 22:26:13 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 23457@debbugs.gnu.org
> 
> Did you look into the Emacs display features, like ClearType, in case
> you have that turned on?

And 2 more questions:

 . what kind of CPU do you have on that machine?
 . does Task Manager show a significant CPU load when you press and
   hold C-p in that recipe?

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[James McClain]

I had cleartype on, but turning it off does not change the problem at all.
The task manager shows about a 50% load, which is quite high for emacs.
The processor is an intel core m 5y10c.

I also just tested it on Linux Mint 17, 64-bit. Compiling
emacs-25.0.93.tar.xz from source.
This machine has a Intel i7-4930K processor, and the problem still happens.
If you use my recipe and turn on whitespace-mode before you hold C-p,
do all the spaces show?
I would be happen to send you a video, this has worked on every
machine I have tested it on.

On Thu, May 5, 2016 at 12:59 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Thu, 05 May 2016 22:26:13 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: 23457@debbugs.gnu.org
>>
>> Did you look into the Emacs display features, like ClearType, in case
>> you have that turned on?
>
> And 2 more questions:
>
>  . what kind of CPU do you have on that machine?
>  . does Task Manager show a significant CPU load when you press and
>    hold C-p in that recipe?
>

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> From: James McClain <jamezmcclain@gmail.com>
> Date: Thu, 5 May 2016 17:34:36 -0700
> Cc: 23457@debbugs.gnu.org
> 
> I had cleartype on, but turning it off does not change the problem at all.
> The task manager shows about a 50% load, which is quite high for emacs.
> The processor is an intel core m 5y10c.
> 
> I also just tested it on Linux Mint 17, 64-bit. Compiling
> emacs-25.0.93.tar.xz from source.
> This machine has a Intel i7-4930K processor, and the problem still happens.

Does that GNU/Linux system has anything in common with your Windows
box?  The display, the keyboard, the filesystem, anything else?  Do
you have some common setup of any ind that makes those systems
similar?

> If you use my recipe and turn on whitespace-mode before you hold C-p,
> do all the spaces show?

Yes, the spaces show.

> I would be happen to send you a video, this has worked on every
> machine I have tested it on.

I believe you.  I just cannot reproduce this on any machine to which I
have access.

How many such empty lines do you have in the buffer where this lag
happens?  Your original recipe talked about a single line; is that all
that's needed, or are more such lines required?

Does the same happen in "emacs -Q -nw", i.e. a non-GUI (text-mode)
session?

Thanks.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 195 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> I believe you.  I just cannot reproduce this on any machine to which I
> have access.

I also can't reproduce it on my Linux box with Emacs 25.0.93.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> From: Nicolas Petton <nicolas@petton.fr>
> Cc: 23457@debbugs.gnu.org
> Date: Fri, 06 May 2016 09:59:20 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I believe you.  I just cannot reproduce this on any machine to which I
> > have access.
> 
> I also can't reproduce it on my Linux box with Emacs 25.0.93.

Thanks.  Would others please try reproducing the recipe?

Any ideas what could cause such a lag, in a way that affects both
Windows and GNU/Linux systems?  I'm quite stumped.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[James McClain]

The linux box has nothing in common with my windows box, there should
be no configurations shared that effect emacs, especially since I run
it with -Q.
For this to happen there needs to be many lines of just spaces and a
newline. I talked about in a previous message having 67 of those
lines, but I believe on my linux box it required more (less than 100
though). You also need to go to the very end of the file M->, then
scroll up through the lines up to the top. I did not make that clear.

Here is another take on the recipe, hopefully I can clear up any confusion.
This is without resizing the window, so window-body-width should be 80.
1. .\emacs-25.0.93.exe -Q
2. C-x b test-2 <ret>
3. <f3>
4. C-u 80 <space>
5. C-j
6. <f4>
7. C-u 120 <f4>
8. You should now be at the bottom of the buffer and at the first column,
    but it doesn't hurt to hit M-> and C-a, so do that.
9. Now try moving up to the top of the buffer with C-p

I experience lag doing this, to be clear about what I mean by that. I
can cancel by using C-g, but unless I do that, I cannot execute
commands until emacs finishes moving up lines. Which takes much longer
than if instead of all spaces, I had instead one period before the
newline.

I cannot remember if I tested with with emacs -Q -nw on linux. I will
report on this tomorrow as well as test the windows build on a newly
deployed computer and maybe test it in a virtual box VM.  I may also
try setting it up tonight if I have the energy.

On Fri, May 6, 2016 at 12:59 AM, Nicolas Petton <nicolas@petton.fr> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>> I believe you.  I just cannot reproduce this on any machine to which I
>> have access.
>
> I also can't reproduce it on my Linux box with Emacs 25.0.93.
>
> Nico

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Stephen Berman]

On Fri, 6 May 2016 01:43:55 -0700 James McClain <jamezmcclain@gmail.com> wrote:

> Here is another take on the recipe, hopefully I can clear up any confusion.
> This is without resizing the window, so window-body-width should be 80.
> 1. .\emacs-25.0.93.exe -Q
> 2. C-x b test-2 <ret>
> 3. <f3>
> 4. C-u 80 <space>
> 5. C-j
> 6. <f4>
> 7. C-u 120 <f4>
> 8. You should now be at the bottom of the buffer and at the first column,
>     but it doesn't hurt to hit M-> and C-a, so do that.
> 9. Now try moving up to the top of the buffer with C-p
>
> I experience lag doing this [...]

I can reproduce this with the above recipe.  At step 9 I held down C-p
for five seconds; after about 1 second point moved up one line (from 122
to 121), then Emacs froze for about 45 seconds, after which point jumped
to bob.

Here's my build:

In GNU Emacs 25.0.93.3 (x86_64-suse-linux-gnu, GTK+ Version 3.14.15)
 of 2016-05-04 built on rosalinde
Repository revision: adc80b7e238e09b1b8c392ecf902d2b978d9016d
Windowing system distributor 'The X.Org Foundation', version 11.0.11601000
System Description:	openSUSE 13.2 (Harlequin) (x86_64)

Configured using:
 'configure --with-xwidgets 'CFLAGS=-Og -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS NOTIFY
GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11 XWIDGETS

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix


And here's /proc/cpuinfo:

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 95
model name      : AMD Sempron(tm) Processor 3400+
stepping        : 2
cpu MHz         : 1000.000
cache size      : 256 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good nopl extd_apicid pni cx16 lahf_lm extapic cr8_legacy
bogomips        : 2010.12
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc

Steve Berman

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Fri, 06 May 2016 11:13:46 +0200
> Cc: Nicolas Petton <nicolas@petton.fr>, 23457@debbugs.gnu.org
> 
> I can reproduce this with the above recipe.  At step 9 I held down C-p
> for five seconds; after about 1 second point moved up one line (from 122
> to 121), then Emacs froze for about 45 seconds, after which point jumped
> to bob.

A large buffer with no text paragraph anywhere in sight caused Emacs
to search back for the end of the previous paragraph, in order to
decide what is the base direction for the current paragraph.  In
addition, every line is long, but empty, so regex search also takes
time.  The search is limited to a certain number of lines, so the
lag should level out after that, but otherwise just don't use such
empty lines in buffers whose bidi-paragraph-direction is nil.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> From: James McClain <jamezmcclain@gmail.com>
> Date: Fri, 6 May 2016 01:43:55 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 23457@debbugs.gnu.org
> 
> For this to happen there needs to be many lines of just spaces and a
> newline. I talked about in a previous message having 67 of those
> lines, but I believe on my linux box it required more (less than 100
> though). You also need to go to the very end of the file M->, then
> scroll up through the lines up to the top. I did not make that clear.

Ah!  Now I do see this.  And this doesn't happen in *scratch*, right?

To make the problem go away, do this:

  M-x set-variable RET bidi-paragraph-direction RET left-to-right RET

In any buffer whose major mode supports some programming language
(like *scratch*, which supports Emacs Lisp), the above variable is
already set to that value  by default, so the problem won't happen.

Anyway, now that I see the problem and understand its reasons, is
there any real-life use case where this problem happens?  If so,
please describe that use case.  Otherwise, this is expected behavior,
and this bug should be closed.

> I experience lag doing this, to be clear about what I mean by that. I
> can cancel by using C-g, but unless I do that, I cannot execute
> commands until emacs finishes moving up lines. Which takes much longer
> than if instead of all spaces, I had instead one period before the
> newline.

The contents of the buffer that you describe make redisplay work very
hard, so the time it takes to refresh the display after you move point
is long.  On my machine, a single C-p from the end of a 200-line
buffer with all-blank lines takes 2 sec in an unoptimized build, and
less than 1 sec in an optimized build.

> I cannot remember if I tested with with emacs -Q -nw on linux.

No need, now that I understand the problem.

Thanks.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[James McClain]

There is no real-life use case at all where I can see this problem
happening, I was just curious as to why.
Thank you for figuring that out,
Glad to see I'm not crazy!

Regards,

James

On Fri, May 6, 2016 at 2:15 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: James McClain <jamezmcclain@gmail.com>
>> Date: Fri, 6 May 2016 01:43:55 -0700
>> Cc: Eli Zaretskii <eliz@gnu.org>, 23457@debbugs.gnu.org
>>
>> For this to happen there needs to be many lines of just spaces and a
>> newline. I talked about in a previous message having 67 of those
>> lines, but I believe on my linux box it required more (less than 100
>> though). You also need to go to the very end of the file M->, then
>> scroll up through the lines up to the top. I did not make that clear.
>
> Ah!  Now I do see this.  And this doesn't happen in *scratch*, right?
>
> To make the problem go away, do this:
>
>   M-x set-variable RET bidi-paragraph-direction RET left-to-right RET
>
> In any buffer whose major mode supports some programming language
> (like *scratch*, which supports Emacs Lisp), the above variable is
> already set to that value  by default, so the problem won't happen.
>
> Anyway, now that I see the problem and understand its reasons, is
> there any real-life use case where this problem happens?  If so,
> please describe that use case.  Otherwise, this is expected behavior,
> and this bug should be closed.
>
>> I experience lag doing this, to be clear about what I mean by that. I
>> can cancel by using C-g, but unless I do that, I cannot execute
>> commands until emacs finishes moving up lines. Which takes much longer
>> than if instead of all spaces, I had instead one period before the
>> newline.
>
> The contents of the buffer that you describe make redisplay work very
> hard, so the time it takes to refresh the display after you move point
> is long.  On my machine, a single C-p from the end of a 200-line
> buffer with all-blank lines takes 2 sec in an unoptimized build, and
> less than 1 sec in an optimized build.
>
>> I cannot remember if I tested with with emacs -Q -nw on linux.
>
> No need, now that I understand the problem.
>
> Thanks.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> From: James McClain <jamezmcclain@gmail.com>
> Date: Fri, 6 May 2016 02:19:24 -0700
> Cc: Nicolas Petton <nicolas@petton.fr>, 23457@debbugs.gnu.org
> 
> There is no real-life use case at all where I can see this problem
> happening, I was just curious as to why.
> Thank you for figuring that out,
> Glad to see I'm not crazy!

Thanks, closing.

bug#23457: 24.5; interactive use of next-line and previous-line (holding down C-n or C-p) lag in a buffer with all spaces and newlines

[Eli Zaretskii]

> Date: Fri, 06 May 2016 12:24:10 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: nicolas@petton.fr, 23457-done@debbugs.gnu.org
> 
> > From: James McClain <jamezmcclain@gmail.com>
> > Date: Fri, 6 May 2016 02:19:24 -0700
> > Cc: Nicolas Petton <nicolas@petton.fr>, 23457@debbugs.gnu.org
> > 
> > There is no real-life use case at all where I can see this problem
> > happening, I was just curious as to why.
> > Thank you for figuring that out,
> > Glad to see I'm not crazy!
> 
> Thanks, closing.

Nevertheless, I tried to speed-up this use case on the master branch.