bug#44564: 27.1; C-n in macros causes long delays

bug#44564: 27.1; C-n in macros causes long delays

[Ted Lavarias]

[-- Attachment #1: Type: text/plain, Size: 5834 bytes --]

--text follows this line--

When editing a Python/Django models.py file for a very large database, I
was reformatting the database fields to be used elsewhere in the
project.  So I used a macro to edit 330 lines to change the format from:

    some_database_column_name = models.CharField(max_length=10, etc...)

to the following to be inserted into a Python dictionary:

    'some_database_column_name':

Doing a simple macro of:
F3
M-m ' C-s = C-b C-b ': C-k C-n
F4

and executing it for 329 more iterations takes 31.96 and 29.19 seconds!
When executing a similar macro, but instead of C-n, I was using "M-x
forward-line" to go to the next line, it only took 8.66 and 8.86
seconds.  Another similar macro, but without the C-n and then executing
the macro over the highlighted region with C-x C-k r (for
apply-macro-to-region-lines), it only takes 5.83 and 5.79 seconds!

I have run several tests in emacs over the last several days and have
gotten predictable results.  The above timed tests were always performed
in a "fresh" instance, started with "$ emacs -Q" and I had always
rebooted emacs in between test times to ensure nothing was cached.  I am
only displaying the results of 4 tests above, but I have tested this
SEVERAL times in many different configurations.  I have also tested this on
Emacs 26.1 from the Debian Stable repos and get the exact same results.  I
initially sought out help and tips on reddit, and other users have tried
running the macro on their machines and had similar results.  It was from
other users' input that we discovered that "M-x fo-lin" and "C-x C-k r"
give us macro execution times that are reasonable and comparable to vim
(which only takes 3.26 seconds for the equivalent macro).

In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23,
cairo version 1.16.0)
 of 2020-11-07, modified by Debian built on x86-ubc-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12009000
System Description: Debian GNU/Linux bullseye/sid

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-cairo
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars
 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-6jKC2B/emacs-27.1+1=.
-fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD
JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 45099 9140)
 (symbols 48 6002 1)
 (strings 32 15434 2143)
 (string-bytes 1 501833)
 (vectors 16 10078)
 (vector-slots 8 129851 8944)
 (floats 8 19 39)
 (intervals 56 244 0)
 (buffers 1000 11))

--
Very respectfully,

*Ted Lavarias*

*ted.lavarias@gmail.com <ted.lavarias@gmail.com>*

[-- Attachment #2: Type: text/html, Size: 6767 bytes --]

bug#44564: 27.1; C-n in macros causes long delays

[Lars Ingebrigtsen]

Ted Lavarias <ted.lavarias@gmail.com> writes:

> When executing a similar macro, but instead of C-n, I was using "M-x
> forward-line" to go to the next line, it only took 8.66 and 8.86
> seconds.

`next-line' is a more complex function -- it tries to land you
approximately at the same place horizontally after moving, while
`forward-line' doesn't.

So I'm wondering whether the macro part of this bug report is relevant
or not.  Try evaling the following two forms in one of these buffers and
report back the difference:

(benchmark-run 100 (next-line))

(benchmark-run 100 (forward-line))

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44564: 27.1; C-n in macros causes long delays

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So I'm wondering whether the macro part of this bug report is relevant
> or not.  Try evaling the following two forms in one of these buffers and
> report back the difference:
>
> (benchmark-run 100 (next-line))
>
> (benchmark-run 100 (forward-line))

More information was requested, but no response was given within a few
months, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44564: 27.1; C-n in macros causes long delays

[Lars Ingebrigtsen]

Ted Lavarias <ted.lavarias@gmail.com> writes:

> I apologize for such a late response! I never saw your initial email, where you
> requested me to eval those benchmark tests... I finally had a chance to do them
> and here are my results:
>
> (benchmark-run 100 (next-line)) ===> (0.28576279800000004 1
> 0.025205666000000013)
> (benchmark-run 100 (forward-line)) ===> (0.00010560799999999998 0 0.0)
>
> That's quite a significant difference between next-line and forward-line!

Yup; `next-line' is a surprisingly complex function.

bug#44564: 27.1; C-n in macros causes long delays

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 03 Feb 2021 18:58:47 +0100
> Cc: 44564@debbugs.gnu.org
> 
> > (benchmark-run 100 (next-line)) ===> (0.28576279800000004 1
> > 0.025205666000000013)
> > (benchmark-run 100 (forward-line)) ===> (0.00010560799999999998 0 0.0)
> >
> > That's quite a significant difference between next-line and forward-line!
> 
> Yup; `next-line' is a surprisingly complex function.

Only when line-move-visual is non-nil.  If the OP's application
doesn't need that, the function will run much faster if
line-move-visual is bound to nil.