bug#24048: 25.0.95; syntax-ppss can be slow

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 28106 bytes --]


In certain situations, the caching of syntax-ppss can seem to be
insufficient for good performance when syntax-ppss is invoked in a tight
loop. smartparens is a popular package that currently invokes
syntax-ppss in a tight loop (about 30-100 times depending on how many
match pairs are configured). syntax-ppss has caching built in, but it is
willing to use a cache from a further back position and do a
`parse-partial-sexp` from the old position to the current. Afterwards,
it does not set the cache with the current position. As such, in the
pathological case, it can parse ~2000 characters 30-100 times in a tight
loop. This is slow enough to show up in profiling and affect typing
latency.

To repro, install:

smartparens (you can use
https://github.com/aaronjensen/smartparens/tree/limit-binding-of-case-fold-search
to alleviate a different performance issue which could distract in the profiling)
elixir-mode

Load this file into a buffer, ensure elixir-mode is enabled: https://gist.github.com/aaronjensen/d4cd40347a916dd36c8239afecf64a7d

Go down to the chunk_by function and type inside of it. If you profile,
you'll see that syntax-ppss rises to the top. Here is an example
reversed profile:

+ redisplay 218 32%
- progn 120 18%
- if 120 18%
- condition-case 120 18%
- let 120 18%
+ syntax-ppss 120 18%
+ sit-for 101 15%
+ sp--looking-back 55 8%
+ font-lock-fontify-keywords-region 45 6%
+ alchemist-project-root 41 6%

For what it is worth, if the file is longer this tends to not happen as
syntax-ppss ends up executing for something (end of buffer it seems) outside of the "close"
heuristic which will ultimately cause the syntax-ppss-last to get set to
the current position for the tight loop. The issue is only when the
first branch of syntax-ppss is taken in a tight loop and pos - old-pos
is great enough.

Please see https://github.com/Fuco1/smartparens/pull/629 for additional
context and discussion.

Thanks!

Aaron

In GNU Emacs 25.0.95.1 (x86_64-apple-darwin15.5.0, NS appkit-1404.47 Version 10.11.5 (Build 15F34))
of 2016-06-10 built on aaronmbp.local
Windowing system distributor 'Apple', version 10.3.1404
Configured using:
'configure --disable-dependency-tracking --disable-silent-rules
--enable-locallisppath=/usr/local/share/emacs/site-lisp
--infodir=/usr/local/Cellar/emacs/25.0.95/share/info/emacs
--prefix=/usr/local/Cellar/emacs/25.0.95 --with-xml2 --without-dbus
--with-gnutls --with-rsvg --with-imagemagick --with-ns
--disable-ns-self-contained'

Configured features:
JPEG RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS

Important settings:
value of $LANG: en_US.UTF-8
locale-coding-system: utf-8-unix

Major mode: Elixir

Minor modes in effect:
alchemist-mode: t
goto-address-prog-mode: t
bug-reference-prog-mode: t
auto-highlight-symbol-mode: t
clean-aindent-mode: t
highlight-numbers-mode: t
highlight-parentheses-mode: t
rainbow-delimiters-mode: t
yas-global-mode: t
yas-minor-mode: t
projectile-global-mode: t
projectile-mode: t
recentf-mode: t
shackle-mode: t
global-hungry-delete-mode: t
hungry-delete-mode: t
evil-mc-mode: t
hl-todo-mode: t
global-spacemacs-whitespace-cleanup-mode: t
spacemacs-whitespace-cleanup-mode: t
ws-butler-global-mode: t
ws-butler-mode: t
winner-mode: t
window-numbering-mode: t
volatile-highlights-mode: t
global-vi-tilde-fringe-mode: t
vi-tilde-fringe-mode: t
spaceline-info-mode: t
spaceline-helm-mode: t
save-place-mode: t
savehist-mode: t
popwin-mode: t
persp-mode: t
Info-breadcrumbs-in-mode-line-mode: t
ido-vertical-mode: t
helm-mode: t
global-git-gutter+-mode: t
global-git-commit-mode: t
async-bytecomp-package-mode: t
shell-dirtrack-mode: t
flyspell-mode: t
global-flycheck-mode: t
flycheck-mode: t
flx-ido-mode: t
eyebrowse-mode: t
global-evil-surround-mode: t
evil-surround-mode: t
global-evil-search-highlight-persist: t
evil-search-highlight-persist: t
show-smartparens-global-mode: t
show-smartparens-mode: t
smartparens-mode: t
evil-escape-mode: t
global-anzu-mode: t
anzu-mode: t
eval-sexp-fu-flash-mode: t
diff-auto-refine-mode: t
company-mode: t
hybrid-mode: t
which-key-mode: t
override-global-mode: t
global-undo-tree-mode: t
undo-tree-mode: t
evil-mode: t
evil-local-mode: t
spacemacs-leader-override-mode: t
global-spacemacs-leader-override-mode: t
global-hl-line-mode: t
xterm-mouse-mode: t
global-auto-revert-mode: t
global-page-break-lines-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
column-number-mode: t
line-number-mode: t
transient-mark-mode: t
abbrev-mode: t
hs-minor-mode: t

Recent messages:
20394 19299 [100 times]
20390 19299 [2 times]
20394 19299
syntax-ppss
No profiler stopped
CPU profiler started
CPU profiler stopped
CPU profiler started
Quit
Mark saved where search started [2 times]

Load-path shadows:
/Users/aaronjensen/.emacs.d/elpa/helm-20160718.746/helm-multi-match hides /Users/aaronjensen/.emacs.d/elpa/helm-core-20160718.746/helm-multi-match
/Users/aaronjensen/.emacs.d/elpa/ox-gfm-20160520.2040/ox-gfm hides /Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-gfm
/Users/aaronjensen/.emacs.d/elpa/org-download-20160620.607/org-download hides /Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-download
/Users/aaronjensen/.emacs.d/elpa/org-bullets-20140918.1137/org-bullets hides /Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-bullets
/Users/aaronjensen/.emacs.d/elpa/ht-20150830.1115/ht hides ~/.emacs.d/core/libs/ht
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-texinfo hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-texinfo
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-publish hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-publish
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-org hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-org
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-odt hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-odt
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-md hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-md
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-man hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-man
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-latex hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-latex
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-icalendar hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-icalendar
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-html hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-html
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-beamer hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-beamer
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ox-ascii hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ox-ascii
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-w3m hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-w3m
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-version hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-version
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-timer hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-timer
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-table hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-table
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-src hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-src
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-rmail hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-rmail
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-protocol hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-protocol
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-plot hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-plot
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-pcomplete hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-pcomplete
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-mouse hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-mouse
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-mobile hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-mobile
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-mhe hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-mhe
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-macs hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-macs
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-macro hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-macro
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-loaddefs hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-loaddefs
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-list hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-list
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-irc hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-irc
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-install hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-install
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-inlinetask hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-inlinetask
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-info hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-info
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-indent hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-indent
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-id hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-id
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-habit hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-habit
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-gnus hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-gnus
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-footnote hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-footnote
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-feed hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-feed
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-faces hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-faces
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-eshell hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-eshell
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-entities hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-entities
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-element hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-element
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-docview hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-docview
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-datetree hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-datetree
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-ctags hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-ctags
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-crypt hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-crypt
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-compat hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-compat
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-colview hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-colview
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-clock hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-clock
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-capture hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-capture
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-bibtex hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-bibtex
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-bbdb hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-bbdb
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-attach hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-attach
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-archive hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-archive
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/org-agenda hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/org-agenda
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-tangle hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-tangle
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-table hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-table
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-sqlite hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-sqlite
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-sql hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-sql
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-shen hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-shen
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-screen hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-screen
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-scheme hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-scheme
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-scala hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-scala
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-sass hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-sass
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-ruby hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-ruby
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-ref hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-ref
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-R hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-R
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-python hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-python
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-plantuml hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-plantuml
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-picolisp hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-picolisp
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-perl hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-perl
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-org hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-org
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-octave hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-octave
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-ocaml hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-ocaml
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-mscgen hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-mscgen
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-maxima hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-maxima
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-matlab hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-matlab
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-makefile hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-makefile
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-lob hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-lob
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-lisp hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-lisp
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-lilypond hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-lilypond
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-ledger hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-ledger
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-latex hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-latex
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-keys hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-keys
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-js hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-js
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-java hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-java
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-io hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-io
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-haskell hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-haskell
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-gnuplot hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-gnuplot
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-fortran hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-fortran
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-exp hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-exp
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-eval hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-eval
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-emacs-lisp hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-emacs-lisp
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-dot hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-dot
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-ditaa hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-ditaa
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-css hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-css
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-core hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-core
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-comint hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-comint
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-clojure hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-clojure
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-calc hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-calc
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-C hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-C
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-awk hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-awk
/Users/aaronjensen/.emacs.d/elpa/org-plus-contrib-20160718/ob-asymptote hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/org/ob-asymptote
/Users/aaronjensen/.emacs.d/elpa/seq-2.16/seq hides /usr/local/Cellar/emacs/25.0.95/share/emacs/25.0.95/lisp/emacs-lisp/seq

Features:
(shadow sort mail-extr emacsbug evil-nerd-commenter
evil-nerd-commenter-operator tabify helm-command helm-elisp helm-eval
misearch multi-isearch network-stream nsm starttls dockerfile-mode
alchemist alchemist-macroexpand alchemist-company alchemist-help
alchemist-complete alchemist-refcard alchemist-phoenix alchemist-compile
alchemist-iex alchemist-message alchemist-hooks alchemist-hex
alchemist-mix alchemist-info alchemist-goto alchemist-scope
alchemist-eval alchemist-interact alchemist-server alchemist-execute
alchemist-report alchemist-test-mode alchemist-project alchemist-file
alchemist-key alchemist-utils elixir-mode pkg-info epl elixir-smie
company-tern dash-functional js-doc iswitchb js2-imenu-extras
js2-refactor js2r-paredit js2r-conveniences js2r-conditionals
js2r-wrapping js2r-functions js2r-vars js2r-iife js2r-formatting
js2r-helpers skewer-mode cache-table simple-httpd tern cursor-sensor
js2-mode js cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs imenu flycheck-elm company-robe
evil-matchit robe url-handlers inf-ruby rubocop ruby-tools sh-script
executable yaml-mode auto-compile packed elisp-slime-nav
evil-cleverparens evil-cleverparens-text-objects evil-cleverparens-util
paredit lispy swiper ivy delsel iedit iedit-lib multiple-cursors-core
lispy-inline avy semantic/db eieio-base semantic/util-modes
semantic/util semantic semantic/tag semantic/lex semantic/fw mode-local
cedet evil-ediff ediff-merg ediff-wind ediff-diff ediff-mult ediff-help
ediff-init ediff-util ediff edebug lispy-tags goto-addr bug-reference
auto-highlight-symbol clean-aindent-mode highlight-numbers parent-mode
highlight-parentheses hideshow rainbow-delimiters company-files
company-keywords company-etags company-gtags company-template
company-dabbrev-code company-dabbrev company-capf org-eldoc evil-org
ob-shell org-bullets org-download url-http tls gnutls url-auth url-gw
toc-org ert debug clojure-snippets yasnippet image-file org-rmail
org-mhe org-irc org-info org-gnus org-docview org-bibtex bibtex org-bbdb
org-w3m mu4e-maildirs-extension org-mu4e mu4e mu4e-speedbar speedbar
sb-image ezimage dframe mu4e-main mu4e-context mu4e-view epa
mu4e-headers mu4e-compose mu4e-draft mu4e-actions rfc2368 smtpmail
sendmail mu4e-mark mu4e-message html2text mu4e-proc mu4e-utils doc-view
jka-compr mu4e-lists mu4e-vars mu4e-meta ox-gfm ox-md ox-latex
ox-icalendar ox-html ox-ascii ox-publish ox org-element avl-tree org
org-macro org-footnote org-pcomplete org-list org-faces org-entities
org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp
org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs
org-loaddefs cal-menu calendar cal-loaddefs vc-git projectile grep
ibuf-ext ibuffer recentf tree-widget shackle hungry-delete
fill-or-unfill fix-autorevert-breaking-git eval-overlay
fix-bracketed-paste company-simple-complete init-terminal-cursor
evil-terminal-cursor-changer init-org init-magit init-elixir init-deft
evil-mc evil-mc-command-execute evil-mc-command-record
evil-mc-cursor-make evil-mc-region evil-mc-cursor-state evil-mc-undo
evil-mc-vars evil-mc-known-commands evil-mc-common hl-todo xterm-color
spacemacs-whitespace-cleanup ws-butler winner window-numbering etags
xref project volatile-highlights vi-tilde-fringe tmux spaceline-config
spaceline-segments spaceline powerline powerline-separators color
powerline-themes smartparens-config smartparens-ruby smartparens-html
saveplace savehist ruby-test-mode pcre2el rxt re-builder ruby-mode smie
popwin persp-mode osx-trash info+ image-mode ido-vertical-mode helm-mode
helm-files image-dired dired-x dired-aux ffap helm-buffers helm-elscreen
helm-tags helm-bookmark helm-adaptive helm-info bookmark pp helm-locate
helm-grep helm-regexp helm-external helm-net browse-url xml url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap helm-utils compile helm-help helm-types
git-gutter-fringe+ fringe-helper git-gutter+ git-commit with-editor
async-bytecomp async tramp-sh server log-edit message rfc822 mml mml-sec
epg mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log
docker-tramp tramp-cache tramp tramp-compat tramp-loaddefs trampver
shell pcomplete comint ansi-color flyspell-correct-helm helm helm-source
helm-multi-match helm-lib dired flyspell-correct flyspell ispell
flycheck find-func flx-ido flx ido eyebrowse format-spec
exec-path-from-shell evil-unimpaired f s evil-surround
evil-search-highlight-persist evil-numbers evil-lisp-state smartparens
evil-indent-plus evil-exchange evil-escape evil-args evil-anzu anzu
cider-eval-sexp-fu eval-sexp-fu rx highlight noutline outline diff-hl
vc-dir ewoc vc vc-dispatcher diff-mode company-web dash company-css
company web-completion-data adaptive-wrap hybrid-mode
evil-evilified-state which-key use-package diminish bind-key hydra lv
cus-edit cus-start cus-load evil evil-integration undo-tree diff
evil-maps evil-commands evil-jumps evil-command-window evil-types
evil-search evil-ex evil-macros evil-repeat evil-states evil-core
evil-common windmove thingatpt rect evil-digraphs evil-vars ring info
bind-map quelpa url-parse auth-source gnus-util password-cache url-vars
package-build mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 ietf-drums mm-util help-fns mail-prsvr json map lisp-mnt hl-line
xt-mouse autorevert filenotify disp-table wid-edit monokai-theme
finder-inf init-html init-javascript sgml-mode init-evil
core-configuration-layer eieio-compat cl-seq ht cl warnings package
epg-config seq eieio byte-opt bytecomp byte-compile cl-extra help-mode
cconv eieio-core core-spacemacs core-use-package-ext
core-transient-state core-micro-state core-toggle core-keybindings
core-fonts-support core-spacemacs-buffer derived edmacro kmacro
core-funcs cl-macs gv core-themes-support core-display-init
core-auto-completion core-release-management core-dotspacemacs
core-command-line core-debug advice profiler easymenu cl-loaddefs cl-lib
page-break-lines easy-mmode subr-x pcase time-date mule-util tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
ns-win ucs-normalize term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote kqueue cocoa ns multi-tty
make-network-process emacs)

Memory information:
((conses 16 1068122 891337)
(symbols 48 82031 35)
(miscs 40 6570 10733)
(strings 32 209973 395364)
(string-bytes 1 7368564)
(vectors 16 130382)
(vector-slots 8 2505920 346982)
(floats 8 1994 3740)
(intervals 56 13720 14404)
(buffers 976 61))

Aaron


[-- Attachment #2: Type: text/html, Size: 35254 bytes --]

bug#24048: 25.0.95; syntax-ppss can be slow

[Dmitry Gutov]

Hi Aaron,

On 07/21/2016 05:29 PM, Aaron Jensen wrote:
>
> In certain situations, the caching of syntax-ppss can seem to be
> insufficient for good performance when syntax-ppss is invoked in a tight
> loop. smartparens is a popular package that currently invokes
> syntax-ppss in a tight loop (about 30-100 times depending on how many
> match pairs are configured). syntax-ppss has caching built in, but it is
> willing to use a cache from a further back position and do a
> `parse-partial-sexp` from the old position to the current. Afterwards,
> it does not set the cache with the current position. As such, in the
> pathological case, it can parse ~2000 characters 30-100 times in a tight
> loop. This is slow enough to show up in profiling and affect typing
> latency.
>
> To repro, install:
>
> smartparens (you can use
> https://github.com/aaronjensen/smartparens/tree/limit-binding-of-case-fold-search
> to alleviate a different performance issue which could distract in the
> profiling)
> elixir-mode

Is there a way to reproduce it more easier? Maybe without smartparens, 
just with elixir-mode, if it's sufficient to illustrate the performance 
problem (I recall you stating parse-partial-sexp is also unusually slow 
in that major mode, for some reason).

Here, I'm seeing some similar behavior with elisp-mode.el. I can open 
this file, evaluate

(benchmark 1000 '(syntax-ppss))

at line 10, then move to line 70 and do it there, and see the latter 
result to be much faster. They're still below 100ms usually (even with 
1000 repetitions), so that's hardly a good example of a performance 
problem, by itself.

P.S. Please send a text email next time, we very much prefer them for 
various reasons, inability to sneak in a tracker like this among them:

<img 
src=3D=22h=ttp://pixels.canarymail.io/tracking/07C7B5E0-9441-40AD-9CC3-D2A176741827.=png=22 
id=3D=22141121424AC1=466A74CBB8A45B347C37B=22 style=3D=22width: 1p=x; 
height: 1px;=22>

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 2048 bytes --]

Hi Dmitry,

On Sun, Jul 24, 2016 at 6:16 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> Is there a way to reproduce it more easier? Maybe without smartparens,
> just with elixir-mode, if it's sufficient to illustrate the performance
> problem (I recall you stating parse-partial-sexp is also unusually slow
> in that major mode, for some reason).
>

Sure, just use elixir-mode and this file
https://gist.githubusercontent.com/aaronjensen/d4cd40347a916dd36c8239afecf64a7d/raw/b52440bb51b96193a365a7f4cc1c8733ba98476f/syntax-ppss-slow-repro.ex

Then do: (benchmark 100 '(syntax-ppss)) at the top of the file and compare
it to doing it within the `chunk_by` function at the bottom of the file.
For me it’s 0.2ms at the top and 5.6ms at the bottom. Smartparens can do
50-100 syntax-ppss calls for every self-insert-command, and 5.6ms is
significant latency to add on to every keypress.

I have a proposed pull to get smartparens to memoize syntax-ppss itself
https://github.com/Fuco1/smartparens/pull/629 but in theory it should be
redundant given syntax-ppss’s built in caching.

P.S. Please send a text email next time, we very much prefer them for
> various reasons, inability to sneak in a tracker like this among them:
>

Unfortunately, modern email clients don’t really offer this as an option
anymore. I started to set up mu4e but it just became too much work to set
up for the sole purpose of being able to send text emails to report emacs
bugs. This probably isn’t the place to debate the merits of easy for most
people to use bug reporting.

<img
> src=3D=22h=ttp://
> pixels.canarymail.io/tracking/07C7B5E0-9441-40AD-9CC3-D2A176741827.=png=22
> id=3D=22141121424AC1=466A74CBB8A45B347C37B=22 style=3D=22width: 1p=x;
> height: 1px;=22>
>

Yeah, canary isn’t even the client I’m using anymore, it’s just what emacs
happened to open when I told it to open my email client to send the bug
report. Unless Inbox is doing its own weirdness this mail should be
tracking pixel free.
-- 
Aaron

[-- Attachment #2: Type: text/html, Size: 3179 bytes --]

bug#24048: 25.0.95; syntax-ppss can be slow

[Dmitry Gutov]

On 07/25/2016 10:12 AM, Aaron Jensen wrote:

> Sure, just use elixir-mode and this
> file https://gist.githubusercontent.com/aaronjensen/d4cd40347a916dd36c8239afecf64a7d/raw/b52440bb51b96193a365a7f4cc1c8733ba98476f/syntax-ppss-slow-repro.ex
>
> Then do: (benchmark 100 '(syntax-ppss)) at the top of the file and
> compare it to doing it within the `chunk_by` function at the bottom of
> the file. For me it’s 0.2ms at the top and 5.6ms at the bottom.
> Smartparens can do 50-100 syntax-ppss calls for every
> self-insert-command, and 5.6ms is significant latency to add on to every
> keypress.

Is 5.6ms really significant? You should be typing like 200 characters 
per _second_ to really notice the difference.

Even 12ms that I'm getting at the bottom of your example file doesn't 
sound like it should have effect on the perceptible latency.

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

[-- Attachment #1.1: Type: text/plain, Size: 1267 bytes --]

On Mon, Jul 25, 2016 at 5:42 AM Dmitry Gutov <dgutov@yandex.ru> wrote:

> Is 5.6ms really significant? You should be typing like 200 characters
> per _second_ to really notice the difference.
>
> Even 12ms that I'm getting at the bottom of your example file doesn't
> sound like it should have effect on the perceptible latency.
>

That’s a totally fair question, but yes, I can feel the difference while
typing. Ever since switching to Emacs I could feel significant latency in
my setup so I’ve been on a quest to bring it down. That quest led me to
smartparens and first to that case-fold-search binding issue and now here.

Even if one can’t feel 5.6ms, it becomes another brick in the bag, i.e. if
another minor mode did something similar and added another 5.6ms. With
enough of them most people could probably feel it.

If you haven’t read it, I recommend looking over
https://pavelfatin.com/typing-with-pleasure/

He has a tool on there for measuring typing latency. Here is a histogram of
the difference between typing at the bottom of the sample file, the top,
and the bottom with memoizing syntax-ppss in smartparens:

[image: Bottom VS Top VS Memoize Bottom.png]
<https://pavelfatin.com/typing-with-pleasure/>
-- 
Aaron

[-- Attachment #1.2: Type: text/html, Size: 1876 bytes --]

[-- Attachment #2: Bottom VS Top VS Memoize Bottom.png --]
[-- Type: image/png, Size: 3177 bytes --]

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

On Mon, Jul 25, 2016 at 8:08 AM, Aaron Jensen <aaronjensen@gmail.com> wrote:
> On Mon, Jul 25, 2016 at 5:42 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>>
>> Is 5.6ms really significant? You should be typing like 200 characters
>> per _second_ to really notice the difference.
>>
>> Even 12ms that I'm getting at the bottom of your example file doesn't
>> sound like it should have effect on the perceptible latency.
>
> Even if one can’t feel 5.6ms, it becomes another brick in the bag, i.e. if another minor mode did something similar and added another 5.6ms. With enough of them most people could probably feel it.

Hey all, I'm trying to clean up bugs I've submitted. This one is still
an issue of sorts. I've fixed the downstream usage of syntax-ppss by
memoizing it, but that feels like it should be unnecessary given that
syntax-ppss is meant to do its own caching.

The problem here is that syntax-ppss will not update its cache if it
is "close enough" to the old position. In languages like Elixir (and
probably Ruby) the old position is often the beginning of the outer
module which spans most, if not all of the file. This means that the
further you get down the file, the further you get from the old
position and the more work syntax-ppss must do.

If a user of syntax-ppss is not careful (as was the case in
smartparens) then syntax-ppss can be called multiple times in
succession, which can add up to enough latency to be noticed.

Ultimately, it seems like it may be worth reconsidering this condition:

(and old-pos (< (- pos old-pos)
    ;; The time to use syntax-begin-function and
    ;; find PPSS is assumed to be about 2 * distance.
    (* 2 (/ (cdr (aref syntax-ppss-stats 5))
    (1+ (car (aref syntax-ppss-stats 5)))))))

But I do not know what other impacts that may have.

Does anybody have any new thoughts on this?

Thanks!

bug#24048: 25.0.95; syntax-ppss can be slow

[Lars Ingebrigtsen]

Aaron Jensen <aaronjensen@gmail.com> writes:

> Ultimately, it seems like it may be worth reconsidering this condition:
>
> (and old-pos (< (- pos old-pos)
>     ;; The time to use syntax-begin-function and
>     ;; find PPSS is assumed to be about 2 * distance.
>     (* 2 (/ (cdr (aref syntax-ppss-stats 5))
>     (1+ (car (aref syntax-ppss-stats 5)))))))

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

This bit is now:

	(if (and old-pos (< (- pos old-pos)
			    ;; The time to use syntax-begin-function and
			    ;; find PPSS is assumed to be about 2 * distance.
			    (let ((pair (aref syntax-ppss-stats 5)))
			      (/ (* 2 (cdr pair)) (car pair)))))

I.e., the last clause is gone.  I don't know whether this affects the
issue you were seeing, though.  Is syntax-ppss still mis-caching things
for you?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

On Wed, Dec 1, 2021 at 2:47 PM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> Aaron Jensen <aaronjensen@gmail.com> writes:
>
> > Ultimately, it seems like it may be worth reconsidering this condition:
> >
> > (and old-pos (< (- pos old-pos)
> >     ;; The time to use syntax-begin-function and
> >     ;; find PPSS is assumed to be about 2 * distance.
> >     (* 2 (/ (cdr (aref syntax-ppss-stats 5))
> >     (1+ (car (aref syntax-ppss-stats 5)))))))
>
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> This bit is now:
>
>         (if (and old-pos (< (- pos old-pos)
>                             ;; The time to use syntax-begin-function and
>                             ;; find PPSS is assumed to be about 2 * distance.
>                             (let ((pair (aref syntax-ppss-stats 5)))
>                               (/ (* 2 (cdr pair)) (car pair)))))
>
> I.e., the last clause is gone.  I don't know whether this affects the
> issue you were seeing, though.  Is syntax-ppss still mis-caching things
> for you?

I can still reproduce it with the specified repro:

Load https://gist.githubusercontent.com/aaronjensen/d4cd40347a916dd36c8239afecf64a7d/raw/b52440bb51b96193a365a7f4cc1c8733ba98476f/syntax-ppss-slow-repro.ex
Install and enable elixir-mode

At top of file:
M-: (benchmark 100 '(syntax-ppss))

On line 71:
M-: (benchmark 100 '(syntax-ppss))

For me it's 0.1ms vs 5ms

Aaron

bug#24048: 25.0.95; syntax-ppss can be slow

[Lars Ingebrigtsen]

Aaron Jensen <aaronjensen@gmail.com> writes:

> I can still reproduce it with the specified repro:
>
> Load
> https://gist.githubusercontent.com/aaronjensen/d4cd40347a916dd36c8239afecf64a7d/raw/b52440bb51b96193a365a7f4cc1c8733ba98476f/syntax-ppss-slow-repro.ex
> Install and enable elixir-mode
>
> At top of file:
> M-: (benchmark 100 '(syntax-ppss))
>
> On line 71:
> M-: (benchmark 100 '(syntax-ppss))
>
> For me it's 0.1ms vs 5ms

Thanks for checking.

Perhaps Stefan has some insights here; added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24048: 25.0.95; syntax-ppss can be slow

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> In certain situations, the caching of syntax-ppss can seem to be
> insufficient for good performance when syntax-ppss is invoked in
> a tight loop.

Yes, that's a known limitation.  Most uses of `syntax-ppss` that need to
call it in a tight loop move the call outside of the loop and then use
`parse-partial-sexp` inside the loop (and they're careful to move
forward so `parse-partial-sexp` is only called between the last
position and the current one so we avoid re-parsing the same text).

Currently `syntax-ppss` assumes that parsing <2500 chars is "fast
enough" and hence that it's better not to update the "last ppss data"
when this happens (the advantage is that we save time not-updating the
data and we also save time if the next call is somewhere within those
<2500 chars).


        Stefan

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

On Fri, Dec 3, 2021 at 2:04 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > In certain situations, the caching of syntax-ppss can seem to be
> > insufficient for good performance when syntax-ppss is invoked in
> > a tight loop.
>
> Yes, that's a known limitation.  Most uses of `syntax-ppss` that need to
> call it in a tight loop move the call outside of the loop and then use
> `parse-partial-sexp` inside the loop (and they're careful to move
> forward so `parse-partial-sexp` is only called between the last
> position and the current one so we avoid re-parsing the same text).
>
> Currently `syntax-ppss` assumes that parsing <2500 chars is "fast
> enough" and hence that it's better not to update the "last ppss data"
> when this happens (the advantage is that we save time not-updating the
> data and we also save time if the next call is somewhere within those
> <2500 chars).

Copy that, thanks for the explanation. I'm not presently affected by
this (no smartparens and no elixir) so I'd be happy to close this
given that it's working as expected and there are clear ways to avoid
it in many cases.

Thanks,

Aaron

bug#24048: 25.0.95; syntax-ppss can be slow

[Lars Ingebrigtsen]

Aaron Jensen <aaronjensen@gmail.com> writes:

> Copy that, thanks for the explanation. I'm not presently affected by
> this (no smartparens and no elixir) so I'd be happy to close this
> given that it's working as expected and there are clear ways to avoid
> it in many cases.

OK; I'm closing this report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24048: 25.0.95; syntax-ppss can be slow

[Dmitry Gutov]

On 04.12.2021 04:31, Aaron Jensen wrote:
> I'm not presently affected by
> this (no smartparens and no elixir)

No Ruby either?

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

On Sat, Dec 4, 2021 at 9:40 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 04.12.2021 04:31, Aaron Jensen wrote:
> > I'm not presently affected by
> > this (no smartparens and no elixir)
>
> No Ruby either?

I do use Ruby with enh-ruby-mode. I haven't noticed enough typing lag
for me to investigate. Why do you ask?

bug#24048: 25.0.95; syntax-ppss can be slow

[Dmitry Gutov]

On 04.12.2021 19:27, Aaron Jensen wrote:
> On Sat, Dec 4, 2021 at 9:40 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>>
>> On 04.12.2021 04:31, Aaron Jensen wrote:
>>> I'm not presently affected by
>>> this (no smartparens and no elixir)
>>
>> No Ruby either?
> 
> I do use Ruby with enh-ruby-mode. I haven't noticed enough typing lag
> for me to investigate. Why do you ask?

You mentioned Ruby previously. If you had identified particular 
performance problems with ruby-mode, we could try to fix it there (even 
if there's not much to be done in syntax-ppss).

enh-ruby-mode is a different matter, though.

bug#24048: 25.0.95; syntax-ppss can be slow

[Aaron Jensen]

On Sat, Dec 4, 2021 at 7:14 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 04.12.2021 19:27, Aaron Jensen wrote:
> > On Sat, Dec 4, 2021 at 9:40 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >>
> >> On 04.12.2021 04:31, Aaron Jensen wrote:
> >>> I'm not presently affected by
> >>> this (no smartparens and no elixir)
> >>
> >> No Ruby either?
> >
> > I do use Ruby with enh-ruby-mode. I haven't noticed enough typing lag
> > for me to investigate. Why do you ask?
>
> You mentioned Ruby previously. If you had identified particular
> performance problems with ruby-mode, we could try to fix it there (even
> if there's not much to be done in syntax-ppss).
>
> enh-ruby-mode is a different matter, though.

Ah. I don't have any cases right now, so I'm ok letting it go until
something comes up. I no longer understand enough about what's going
on in that function to even suggest an improvement, so if things are
working as expected and no one else is noticing a perf issue then I
think we are good.

Thanks,

Aaron