bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Chen Bin]

Run below code in both Emacs 28.1 and Emacs 27.1

;; Run "base64 /dev/urandom | head -c 3000000000 > 3g.txt" to create 3g.txt
;; Then run below code in Emacs,
(setq content
      (with-temp-buffer
        (let ((coding-system-for-read 'utf-8-unix))
          (insert-file-contents "3g.txt"))
        (buffer-string)))

(message "content length=%s" (length content))
(let* ((gc-cons-threshold most-positive-fixnum))
  (message "%s vs %s"
           (benchmark-run-compiled 1
             (string-match "aaaaa" content))
           (benchmark-run-compiled 1
             (string-match "bbbbb" content))))

Using same machine, same emacs setup, 28 is alwasy much slower than 27.

Emacs 28 enables native compilation. But string-match is C API, so I'm not
sure native compilation matters.

In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu)
 of 2021-11-27, modified by Debian built on x86-csail-01
System Description: Debian GNU/Linux bookworm/sid

Recent messages:
Loading ivy...done
Loading /home/cb/.custom.el (source)...
*** Emacs loaded in 0.48 seconds with 0 garbage collections.
Loading /home/cb/.custom.el (source)...done
Loading /home/cb/.emacs.d/.session...done
Loading /home/cb/.emacs.d/company-statistics-cache.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Color theme [doom-material-dark] loaded.
[yas] Prepared just-in-time loading of snippets successfully.

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=no
 --without-gsettings 'CFLAGS=-g -O2
 -ffile-prefix-map=/build/emacs-fQHadv/emacs-27.1+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
SOUND GPM DBUS NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 ZLIB XIM
MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LC_COLLATE: zh_CN.UTF-8
  value of $LC_CTYPE: zh_CN.UTF-8
  value of $LC_MESSAGES: en_US.UTF-8
  value of $LC_MONETARY: zh_CN.UTF-8
  value of $LC_NUMERIC: zh_CN.UTF-8
  value of $LC_TIME: zh_CN.UTF-8
  value of $LANG: 
  value of $XMODIFIERS: @im=fcitx
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  global-auto-revert-mode: t
  midnight-mode: t
  evil-find-char-pinyin-mode: t
  winner-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  global-git-gutter-mode: t
  display-time-mode: t
  which-key-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  global-evil-visualstar-mode: t
  evil-visualstar-mode: t
  global-evil-matchit-mode: t
  evil-matchit-mode: t
  diredfl-global-mode: t
  winum-mode: t
  company-statistics-mode: t
  global-company-mode: t
  company-mode: t
  ivy-mode: t
  evil-escape-mode: t
  shell-dirtrack-mode: t
  evil-mode: t
  evil-local-mode: t
  save-place-mode: t
  keyfreq-autosave-mode: t
  keyfreq-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/cb/.emacs.d/site-lisp/keyfreq/keyfreq hides /home/cb/.emacs.d/elpa/keyfreq-20210630.1318/keyfreq
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-mua hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-mua
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-maildir-fcc hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-maildir-fcc
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-tag hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-tag
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-tree hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-tree
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-version hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-version
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-print hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-print
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/coolj hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/coolj
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-company hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-company
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-hello hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-hello
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-address hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-address
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-lib hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-lib
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-autoloads hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-autoloads
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-jump hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-jump
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-show hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-show
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-parser hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-parser
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-compat hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-compat
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-crypto hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-crypto
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-query hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-query
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-message hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-message
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-wash hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-wash
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-draft hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-draft
/usr/share/emacs/site-lisp/elpa/notmuch-0.36/notmuch-pkg hides /usr/share/emacs/site-lisp/elpa-src/notmuch-0.36/notmuch-pkg

Features:
(shadow sort bbdb-message bbdb bbdb-site timezone wucuo wucuo-sdk
find-lisp lazyflymake lazyflymake-sdk flymake-proc flymake mwheel
warnings mail-extr auto-save autorevert filenotify midnight
evil-exchange evil-find-char-pinyin pinyinlib winner
display-line-numbers git-gutter time image which-key evil-surround
evil-visualstar evil-nerd-commenter evil-nerd-commenter-operator
evil-nerd-commenter-sdk sgml-mode dom evil-matchit-evil-setup
yasnippet-snippets yasnippet emacsbug message rmc puny format-spec
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
doom-material-dark-theme doom-themes doom-themes-base cap-words
superword subword amx s counsel xdg xref project dired-aux dired-x
diredfl dired dired-loaddefs compile swiper winum dash term/xterm xterm
company-ispell company-oddmuse company-keywords company-gtags
company-dabbrev-code company-dabbrev company-files company-clang
company-capf company-semantic company-template company-bbdb
company-cmake company-c-headers company-ctags find-file
company-statistics company pcase session ivy delsel ivy-faces
ivy-overlay colir color cus-start cus-load init-no-byte-compile
init-evil general evil-escape evil evil-keybindings evil-integration
evil-maps evil-commands reveal flyspell ispell evil-jumps
evil-command-window evil-search evil-ex native-complete shell pcomplete
comint regexp-opt ansi-color evil-types evil-macros evil-repeat
evil-states evil-core advice evil-common windmove calc calc-loaddefs
calc-macs thingatpt rect evil-digraphs evil-vars init-hydra cl-extra
help-mode hydra ring lv init-writting init-dired init-shackle
init-emacs-w3m init-misc saveplace init-essential init-theme init-httpd
init-keyfreq keyfreq easy-mmode init-chinese init-company init-web-mode
init-term-mode init-lua-mode init-gnus init-bbdb init-ctags
init-clipboard init-gtags init-git init-linum-mode init-cc-mode
init-yasnippet init-lisp init-python init-org init-javascript
init-windows init-ivy init-ibuffer edmacro kmacro ibuf-macs
init-spelling init-elpa init-file-type init-utils init-modeline
init-autoload info tool-bar rx w3m-load package easymenu browse-url
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib early-init china-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 367525 448971)
 (symbols 48 26562 13)
 (strings 32 77860 38359)
 (string-bytes 1 2651900)
 (vectors 16 30280)
 (vector-slots 8 399594 211814)
 (floats 8 499 826)
 (intervals 56 1790 90)
 (buffers 1000 12))

-- 
Best Regards,
Chen Bin

--
Help me, help you

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Ihor Radchenko]

Chen Bin <chenbin.sh@gmail.com> writes:

> Run below code in both Emacs 28.1 and Emacs 27.1
>
> ;; Run "base64 /dev/urandom | head -c 3000000000 > 3g.txt" to create 3g.txt
> ;; Then run below code in Emacs,
>
> Using same machine, same emacs setup, 28 is alwasy much slower than 27.

On my system:

Emacs-27          (0.032783519 0 0.0) vs (0.093436952 0 0.0)
Emacs-28 (latest) (0.037079772999999996 0 0.0) vs (0.162106006 0 0.0)
Emacs-29 (latest) (0.03358834 0 0.0) vs (0.15387368299999998 0 0.0)

Best,
Ihor

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Lars Ingebrigtsen]

Chen Bin <chenbin.sh@gmail.com> writes:

> Using same machine, same emacs setup, 28 is alwasy much slower than 27.

How much slower?

I tried your test, and the results were within a few percentages of each
other in 27.1 and 28.1.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> How much slower?

D'oh -- you said so in the subject.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Eli Zaretskii]

> Resent-From: Lars Ingebrigtsen <larsi@gnus.org>
> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
> Resent-CC: bug-gnu-emacs@gnu.org
> Resent-Sender: help-debbugs@gnu.org
> Cc: 55729@debbugs.gnu.org
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 30 May 2022 17:09:58 +0200
> 
> Chen Bin <chenbin.sh@gmail.com> writes:
> 
> > Using same machine, same emacs setup, 28 is alwasy much slower than 27.
> 
> How much slower?
> 
> I tried your test, and the results were within a few percentages of each
> other in 27.1 and 28.1.

With a 3GB file being inserted non-literally, there could be factors
at work here that are entirely unrelated to processing speed.  The
implementation of string-match basically didn't change between Emacs
27 and Emacs 28, AFAICT.

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Eli Zaretskii]

> Cc: chenbin.sh@gmail.com, 55729@debbugs.gnu.org
> Date: Mon, 30 May 2022 18:58:17 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > I tried your test, and the results were within a few percentages of each
> > other in 27.1 and 28.1.
> 
> With a 3GB file being inserted non-literally, there could be factors
> at work here that are entirely unrelated to processing speed.  The
> implementation of string-match basically didn't change between Emacs
> 27 and Emacs 28, AFAICT.

Here's one random thought: were both Emacs 27 and Emacs 28 compiled
with the same compiler version, and do they both use the same version
of the Standard C Library?

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Chen Bin <chenbin.sh@gmail.com> writes:

> Run below code in both Emacs 28.1 and Emacs 27.1
>
> ;; Run "base64 /dev/urandom | head -c 3000000000 > 3g.txt" to create 3g.txt
> ;; Then run below code in Emacs,
> (setq content
>       (with-temp-buffer
>         (let ((coding-system-for-read 'utf-8-unix))
>           (insert-file-contents "3g.txt"))
>         (buffer-string)))
>
> (message "content length=%s" (length content))
> (let* ((gc-cons-threshold most-positive-fixnum))
>   (message "%s vs %s"
>            (benchmark-run-compiled 1
>              (string-match "aaaaa" content))
>            (benchmark-run-compiled 1
>              (string-match "bbbbb" content))))
>
> Using same machine, same emacs setup, 28 is alwasy much slower than 27.
>
> Emacs 28 enables native compilation. But string-match is C API, so I'm not
> sure native compilation matters.

Did you disable compiler optimizations when building Emacs?

bug#55729: 27.1; Emacs 28.1 `string-match' is five times slower than Emacs 27.1

[Eli Zaretskii]

> Cc: 55729@debbugs.gnu.org
> Date: Tue, 31 May 2022 09:04:13 +0800
> From:  Po Lu via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Chen Bin <chenbin.sh@gmail.com> writes:
> 
> > Run below code in both Emacs 28.1 and Emacs 27.1
> >
> > ;; Run "base64 /dev/urandom | head -c 3000000000 > 3g.txt" to create 3g.txt
> > ;; Then run below code in Emacs,
> > (setq content
> >       (with-temp-buffer
> >         (let ((coding-system-for-read 'utf-8-unix))
> >           (insert-file-contents "3g.txt"))
> >         (buffer-string)))
> >
> > (message "content length=%s" (length content))
> > (let* ((gc-cons-threshold most-positive-fixnum))
> >   (message "%s vs %s"
> >            (benchmark-run-compiled 1
> >              (string-match "aaaaa" content))
> >            (benchmark-run-compiled 1
> >              (string-match "bbbbb" content))))
> >
> > Using same machine, same emacs setup, 28 is alwasy much slower than 27.
> >
> > Emacs 28 enables native compilation. But string-match is C API, so I'm not
> > sure native compilation matters.
> 
> Did you disable compiler optimizations when building Emacs?

Yes.  Here's the response (sent only to myself):

> From: Chen Bin <chenbin.sh@gmail.com>
> Date: Tue, 31 May 2022 04:53:55 +1000
> 
> Thanks for the heads up.
> 
> I double checked my build script and found I forgot to setup `-O2'
> CFLAG. Sorry to bother you guys with such my mistake. 
> 
> It's all good now.

So I'm closing this bug.