bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Tassilo Horn]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]


I use and like the breadcrumb ELPA package.

  https://elpa.gnu.org/packages/breadcrumb.html

It's installed/enabled using:

--8<---------------cut here---------------start------------->8---
(use-package breadcrumb
  :ensure t
  :init (breadcrumb-mode 1))
--8<---------------cut here---------------end--------------->8---

Now when I find the attached mirrorstatus.json using json-ts-mode (that
seems to be important), set point on the first { and insert some spaces,
Emacs will eventually freeze for several seconds or even minutes.

It can be reproduced with

  emacs -Q
  M-x package-initialize RET
  M-: (package-activate 'breadcrumb) RET
  M-x breadcrumb-mode RET

and then finding the file and doing as written above.

I've also attached a gdb backtrace which suggests that emacs is
collecting garbage (produced by the breadcrumb ELN code)?  I've also
tried profiler-start, reproduce the freeze, wait for it to go away,
profiler-report.  That said that 99% of cpu time where spent in
Automatic GC.

I currently have no emacs without native compilation handy so I cannot
tell if that's important, too.  Can I somehow tell emacs not to compile
breadcrumb natively after deleting the eln file?


[-- Attachment #2: mirrorstatus.json.gz --]
[-- Type: application/gzip, Size: 67854 bytes --]

[-- Attachment #3: backtrace.txt.gz --]
[-- Type: application/gzip, Size: 9207 bytes --]

[-- Attachment #4: Type: text/plain, Size: 12011 bytes --]



In GNU Emacs 30.0.50 (build 13, x86_64-pc-linux-gnu, GTK+ Version
 3.24.42, cairo version 1.18.0) of 2024-06-04 built on thinkpad-t440p
Repository revision: 799f78a92c6c31f4d181390523b83d036020ede1
Repository branch: master
System Description: Arch Linux

Configured using:
 'configure --with-tree-sitter --with-pgtk --with-modules'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBOTF LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LC_MONETARY: de_DE.utf8
  value of $LC_NUMERIC: de_DE.utf8
  value of $LC_TIME: de_DE.utf8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: mu4e:main

Minor modes in effect:
  breadcrumb-mode: t
  editorconfig-mode: t
  global-aggressive-indent-mode: t
  pdf-occur-global-minor-mode: t
  diredfl-global-mode: t
  mu4e-search-minor-mode: t
  mu4e-update-minor-mode: t
  mu4e-context-minor-mode: t
  mu4e-modeline-mode: t
  which-key-mode: t
  highlight-parentheses-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  corfu-popupinfo-mode: t
  corfu-history-mode: t
  global-corfu-mode: t
  corfu-mode: t
  vertico-mode: t
  marginalia-mode: t
  minibuffer-depth-indicate-mode: t
  switchy-window-minor-mode: t
  electric-pair-mode: t
  recentf-mode: t
  override-global-mode: t
  repeat-mode: t
  global-so-long-mode: t
  save-place-mode: t
  savehist-mode: t
  puni-global-mode: t
  puni-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  column-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  overwrite-mode: overwrite-mode-binary

Load-path shadows:
~/Repos/el/mu/mu4e/mu4e hides ~/Repos/el/mu/build/mu4e/mu4e
~/Repos/el/mu/mu4e/mu4e-modeline hides ~/Repos/el/mu/build/mu4e/mu4e-modeline
~/Repos/el/mu/mu4e/mu4e-context hides ~/Repos/el/mu/build/mu4e/mu4e-context
~/Repos/el/mu/mu4e/mu4e-main hides ~/Repos/el/mu/build/mu4e/mu4e-main
~/Repos/el/mu/mu4e/mu4e-vars hides ~/Repos/el/mu/build/mu4e/mu4e-vars
~/Repos/el/mu/mu4e/mu4e-window hides ~/Repos/el/mu/build/mu4e/mu4e-window
~/Repos/el/mu/mu4e/mu4e-speedbar hides ~/Repos/el/mu/build/mu4e/mu4e-speedbar
~/Repos/el/mu/mu4e/mu4e-view hides ~/Repos/el/mu/build/mu4e/mu4e-view
~/Repos/el/mu/mu4e/mu4e-thread hides ~/Repos/el/mu/build/mu4e/mu4e-thread
~/Repos/el/mu/mu4e/mu4e-bookmarks hides ~/Repos/el/mu/build/mu4e/mu4e-bookmarks
~/Repos/el/mu/mu4e/mu4e-org hides ~/Repos/el/mu/build/mu4e/mu4e-org
~/Repos/el/mu/mu4e/mu4e-lists hides ~/Repos/el/mu/build/mu4e/mu4e-lists
~/Repos/el/mu/mu4e/mu4e-actions hides ~/Repos/el/mu/build/mu4e/mu4e-actions
~/Repos/el/mu/mu4e/mu4e-helpers hides ~/Repos/el/mu/build/mu4e/mu4e-helpers
~/Repos/el/mu/mu4e/mu4e-search hides ~/Repos/el/mu/build/mu4e/mu4e-search
~/Repos/el/mu/mu4e/mu4e-server hides ~/Repos/el/mu/build/mu4e/mu4e-server
~/Repos/el/mu/mu4e/mu4e-obsolete hides ~/Repos/el/mu/build/mu4e/mu4e-obsolete
~/Repos/el/mu/mu4e/mu4e-update hides ~/Repos/el/mu/build/mu4e/mu4e-update
~/Repos/el/mu/mu4e/mu4e-draft hides ~/Repos/el/mu/build/mu4e/mu4e-draft
~/Repos/el/mu/mu4e/mu4e-message hides ~/Repos/el/mu/build/mu4e/mu4e-message
~/Repos/el/mu/mu4e/mu4e-compose hides ~/Repos/el/mu/build/mu4e/mu4e-compose
~/Repos/el/mu/mu4e/mu4e-headers hides ~/Repos/el/mu/build/mu4e/mu4e-headers
~/Repos/el/mu/mu4e/mu4e-query-items hides ~/Repos/el/mu/build/mu4e/mu4e-query-items
~/Repos/el/mu/mu4e/mu4e-notification hides ~/Repos/el/mu/build/mu4e/mu4e-notification
~/Repos/el/mu/mu4e/mu4e-contacts hides ~/Repos/el/mu/build/mu4e/mu4e-contacts
~/Repos/el/mu/mu4e/mu4e-icalendar hides ~/Repos/el/mu/build/mu4e/mu4e-icalendar
~/Repos/el/mu/mu4e/mu4e-mark hides ~/Repos/el/mu/build/mu4e/mu4e-mark
~/Repos/el/mu/mu4e/mu4e-contrib hides ~/Repos/el/mu/build/mu4e/mu4e-contrib
~/Repos/el/mu/mu4e/mu4e-folders hides ~/Repos/el/mu/build/mu4e/mu4e-folders
~/Repos/el/mu/mu4e/mu4e-mime-parts hides ~/Repos/el/mu/build/mu4e/mu4e-mime-parts
/home/horn/.emacs.d/elpa/ef-themes-1.7.0/theme-loaddefs hides /home/horn/Repos/el/emacs/lisp/theme-loaddefs
/home/horn/.emacs.d/elpa/transient-20240603.2200/transient hides /home/horn/Repos/el/emacs/lisp/transient

Features:
(shadow sort expreg cap-words superword subword face-remap mail-extr
cus-start shortdoc emacsbug view help-fns radix-tree tramp-cmds puni
display-fill-column-indicator display-line-numbers ef-dream-theme
generic yaml-mode fish-mode cargo xref cargo-process rust-utils
rust-mode-treesitter rust-ts-mode rust-mode rust-playpen rust-compile
rust-cargo rust-common rust-rustfmt web-mode disp-table auctex-autoloads
tex-site breadcrumb pulse project editorconfig editorconfig-core
editorconfig-core-handle editorconfig-fnmatch elfeed-show elfeed-search
vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view debbugs-browse
elfeed-csv elfeed elfeed-curl elfeed-log elfeed-db elfeed-lib avl-tree
url-queue xml-query hl-todo aggressive-indent rainbow-mode pdf-occur
tablist tablist-filter semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local cedet pdf-isearch
pdf-misc pdf-tools pdf-view jka-compr pdf-cache pdf-info tq pdf-util
pdf-macs image-mode exif vc-git vc-dir ewoc epa-file trashed diredfl
dired-x eshell esh-cmd generator esh-ext esh-opt esh-proc esh-io esh-arg
esh-module esh-module-loaddefs esh-util mu4e-icalendar gnus-icalendar
icalendar diary-lib diary-loaddefs mu4e mu4e-org mu4e-notification
notifications mu4e-main smtpmail mu4e-view mu4e-mime-parts mu4e-headers
mu4e-thread mu4e-actions org-capture org-refile org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-src ob-comint org-pcomplete
org-list org-footnote org-faces org-entities ob-emacs-lisp ob-core
ob-eval org-cycle org-table ol org-fold org-fold-core org-keys oc
org-loaddefs cal-menu calendar cal-loaddefs org-compat org-version
org-macs mu4e-compose mu4e-draft gnus-msg mu4e-search mu4e-lists
mu4e-bookmarks mu4e-mark mu4e-message flow-fill mule-util mu4e-contacts
mu4e-update mu4e-folders mu4e-context mu4e-query-items mu4e-server
mu4e-modeline mu4e-vars mu4e-helpers mu4e-config mu4e-window
magit-bookmark bookmark ido mu4e-obsolete hippie-exp auto-dictionary
flyspell ispell tramp-smb which-key highlight-parentheses restclient
forge-repos forge-topics forge-tablist hl-line forge-commands forge-semi
forge-bitbucket buck forge-gogs gogs forge-gitea gtea forge-gitlab glab
forge-github ghub-graphql treepy gsexp ghub let-alist forge-notify
forge-revnote forge-pullreq forge-issue forge-topic yaml forge-post
markdown-mode noutline outline forge-repo forge forge-core forge-db
closql emacsql-sqlite-common emacsql emacsql-compiler eieio-base
magit-submodule magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log which-func imenu magit-diff
smerge-mode diff diff-mode track-changes git-commit log-edit pcvs-util
add-log magit-core magit-autorevert autorevert filenotify magit-margin
magit-transient magit-process with-editor comp comp-cstr comp-run
comp-common server magit-mode magit-git magit-base magit-section
cursor-sensor crm dash visual-filename-abbrev rg vc vc-dispatcher
rg-info-hack advice rg-menu transient rg-ibuffer rg-result wgrep-rg
wgrep rg-history rg-header ibuf-ext ibuffer ibuffer-loaddefs grep
compile debbugs soap-client url-http url-auth url-gw nsm warnings
rng-xsd rng-dt rng-util xsd-regexp bug-reference thingatpt kind-icon
svg-lib color corfu-popupinfo corfu-history corfu vertico marginalia
icomplete mb-depth use-package-diminish switchy-window compat elec-pair
recentf tree-widget edmacro kmacro use-package-bind-key bind-key
diminish repeat toml-ts-mode json-ts-mode c++-ts-mode c-ts-mode
java-ts-mode c-ts-common find-func treesit so-long saveplace tramp-sh
tramp rx trampver tramp-integration files-x tramp-message tramp-compat
xdg shell pcomplete comint ansi-osc ring format-spec ansi-color
tramp-loaddefs savehist smiley gnus-art mm-uu mml2015 mm-view mml-smime
smime gnutls dig gnus-sum shr pixel-fill kinsoku url-file svg dom
gnus-group gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap
nnmail mail-source utf7 nnoo parse-time iso8601 gnus-spec gnus-int
gnus-range message sendmail yank-media puny dired dired-loaddefs rfc822
mml mml-sec epa derived epg rfc6068 epg-config mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev
gmm-utils mailheader gnus-win gnus nnheader gnus-util
text-property-search time-date mm-util mail-prsvr mail-utils range
ef-themes cl-extra help-mode use-package-ensure use-package-core
finder-inf cus-edit pp cus-load wid-edit aggressive-indent-autoloads
auto-dictionary-autoloads breadcrumb-autoloads cape-autoloads
cargo-autoloads clojure-mode-autoloads corfu-autoloads
csv-mode-autoloads debbugs-autoloads diminish-autoloads
diredfl-autoloads eat-autoloads editorconfig-autoloads
ef-themes-autoloads elfeed-autoloads ement-autoloads expreg-autoloads
fish-mode-autoloads forge-autoloads closql-autoloads emacsql-autoloads
ghub-autoloads highlight-parentheses-autoloads hl-todo-autoloads
kind-icon-autoloads magit-autoloads pcase git-commit-autoloads
marginalia-autoloads markdown-mode-autoloads mastodon-autoloads
pdf-tools-autoloads persist-autoloads plz-autoloads puni-autoloads
easy-mmode rainbow-mode-autoloads rcirc-color-autoloads
request-autoloads restclient-autoloads rg-autoloads rust-mode-autoloads
svg-lib-autoloads symbol-overlay-autoloads tablist-autoloads
taxy-magit-section-autoloads taxy-autoloads magit-section-autoloads
dash-autoloads totp-autoloads transient-autoloads trashed-autoloads
treepy-autoloads vertico-autoloads visual-filename-abbrev-autoloads
web-mode-autoloads wgrep-autoloads which-key-autoloads info
with-editor-autoloads yaml-autoloads yaml-mode-autoloads package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs icons
password-cache json subr-x map byte-opt gv bytecomp byte-compile
url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/pgtk-win pgtk-win term/common-win touch-screen pgtk-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty move-toolbar make-network-process native-compile emacs)

Memory information:
((conses 16 891519 133247) (symbols 48 53263 2) (strings 32 247420 6656)
 (string-bytes 1 7336948) (vectors 16 88961) (vector-slots 8 1039766 63137)
 (floats 8 784 130) (intervals 56 1927 97) (buffers 992 19))

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Andrea Corallo]

Tassilo Horn <tsdh@gnu.org> writes:

> I use and like the breadcrumb ELPA package.
>
>   https://elpa.gnu.org/packages/breadcrumb.html
>
> It's installed/enabled using:
>
> (use-package breadcrumb
>   :ensure t
>   :init (breadcrumb-mode 1))
>
> Now when I find the attached mirrorstatus.json using json-ts-mode (that
> seems to be important), set point on the first { and insert some spaces,
> Emacs will eventually freeze for several seconds or even minutes.
>
> It can be reproduced with
>
>   emacs -Q
>   M-x package-initialize RET
>   M-: (package-activate 'breadcrumb) RET
>   M-x breadcrumb-mode RET
>
> and then finding the file and doing as written above.
>
> I've also attached a gdb backtrace which suggests that emacs is
> collecting garbage (produced by the breadcrumb ELN code)?  I've also
> tried profiler-start, reproduce the freeze, wait for it to go away,
> profiler-report.  That said that 99% of cpu time where spent in
> Automatic GC.
>
> I currently have no emacs without native compilation handy so I cannot
> tell if that's important, too.  Can I somehow tell emacs not to compile
> breadcrumb natively after deleting the eln file?

Hi Tassilo,

yes, you can globally set 'native-comp-jit-compilation' to nil or be
more selective on the files you don't want to be compiled using
'native-comp-jit-compilation'.

Thanks

  Andrea

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Tassilo Horn]

Andrea Corallo <acorallo@gnu.org> writes:

Hi Andrea,

>> I currently have no emacs without native compilation handy so I
>> cannot tell if that's important, too.  Can I somehow tell emacs not
>> to compile breadcrumb natively after deleting the eln file?
>
> yes, you can globally set 'native-comp-jit-compilation' to nil or be
> more selective on the files you don't want to be compiled using
> 'native-comp-jit-compilation'.

Ok, now I tested with

  emacs -Q --load ~/recipe-71367@debbugs.gnu.org.el

where the recipe is:

--8<---------------cut here---------------start------------->8---
(progn
  ;; Native compile doesn't make a difference...
  (setq native-comp-jit-compilation-deny-list '(".*breadcrumb.*"))
  (package-initialize)
  (package-activate 'breadcrumb)
  (breadcrumb-mode)
  ;; Using json-ts-mode is essential!
  (add-to-list 'major-mode-remap-alist '(js-json-mode . json-ts-mode))
  (find-file "~/mirrorstatus.json"))
--8<---------------cut here---------------end--------------->8---

Before firing up Emacs, I've deleted the breadcrumb ELN file.

With the above recipe, Emacs instantly freezes when the file gets
displayed.  So native compilation of breadcrumb.el is not required in
order to observe this bug.  Using json-ts-mode instead of the default
js-json-mode is, though.

Bye,
Tassilo

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Andrea Corallo]

Tassilo Horn <tsdh@gnu.org> writes:

> Andrea Corallo <acorallo@gnu.org> writes:
>
> Hi Andrea,
>
>>> I currently have no emacs without native compilation handy so I
>>> cannot tell if that's important, too.  Can I somehow tell emacs not
>>> to compile breadcrumb natively after deleting the eln file?
>>
>> yes, you can globally set 'native-comp-jit-compilation' to nil or be
>> more selective on the files you don't want to be compiled using
>> 'native-comp-jit-compilation'.
>
> Ok, now I tested with
>
>   emacs -Q --load ~/recipe-71367@debbugs.gnu.org.el
>
> where the recipe is:
>
> (progn
>   ;; Native compile doesn't make a difference...
>   (setq native-comp-jit-compilation-deny-list '(".*breadcrumb.*"))
>   (package-initialize)
>   (package-activate 'breadcrumb)
>   (breadcrumb-mode)
>   ;; Using json-ts-mode is essential!
>   (add-to-list 'major-mode-remap-alist '(js-json-mode . json-ts-mode))
>   (find-file "~/mirrorstatus.json"))
>
> Before firing up Emacs, I've deleted the breadcrumb ELN file.
>
> With the above recipe, Emacs instantly freezes when the file gets
> displayed.  So native compilation of breadcrumb.el is not required in
> order to observe this bug.  Using json-ts-mode instead of the default
> js-json-mode is, though.

Thanks, I'm Cc'ing authors of breadcrumb and json-ts-mode in case they can
help.

Bests

  Andrea

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Eli Zaretskii]

> Cc: 71367@debbugs.gnu.org, Theodor Thornhill <theo@thornhill.no>,
>  João Távora <joaotavora@gmail.com>
> From: Andrea Corallo <acorallo@gnu.org>
> Date: Wed, 05 Jun 2024 05:36:23 -0400
> 
> Tassilo Horn <tsdh@gnu.org> writes:
> 
> > Ok, now I tested with
> >
> >   emacs -Q --load ~/recipe-71367@debbugs.gnu.org.el
> >
> > where the recipe is:
> >
> > (progn
> >   ;; Native compile doesn't make a difference...
> >   (setq native-comp-jit-compilation-deny-list '(".*breadcrumb.*"))
> >   (package-initialize)
> >   (package-activate 'breadcrumb)
> >   (breadcrumb-mode)
> >   ;; Using json-ts-mode is essential!
> >   (add-to-list 'major-mode-remap-alist '(js-json-mode . json-ts-mode))
> >   (find-file "~/mirrorstatus.json"))
> >
> > Before firing up Emacs, I've deleted the breadcrumb ELN file.
> >
> > With the above recipe, Emacs instantly freezes when the file gets
> > displayed.

It doesn't freeze, it just takes a very long time to perform the
initial redisplay after visiting the file.  With my unoptimized build
of Emacs 30, it takes something like a full minute until the "freeze"
ends.  Optimized builds should do that in, like, 15 to 20 sec, I'd
expect.  Thereafter the breadcrumbs display on the header line works
just fine.

The importance of using json-ts-mode is simply because it defines
support for Imenu, whereas the default js-json-mode does not.  With
Imenu support turned on, imenu--index-alist is a huge nested structure
for this large file: there are 17500 elements there.  The first time
breadcrumb-mode is invoked, it loops over all those elements doing its
thing in breadcrumb--summarize, and that takes both time and a lot of
consing (which triggers a lot of GC, which slows down this even more).
That's all.

Maybe João could suggest some trick or user option to cut down that
initial time to reasonable level.  Failing that, I suggest not to
enable breadcrumb-mode in such large JSON buffers (something that can
be done in a mode hook, I guess?)

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[João Távora]

> Maybe João could suggest some trick or user option to cut down that
> initial time to reasonable level.

No, but feel free to profile and suggest Elisp changes or algorithmic
changes.  Maybe the vconcat re-allocations in breadcrumb--ipath-plain
re-allocations are to blame?  It's O(N^2) allocations I think.  What's
the length of bc--ipath-plain?  17500ish?  If so, then the thing would
be to reserve some space in the vector upfront, but I don't know how
to do that easily in Elisp (in CL there's VECTOR-PUSH and
VECTOR-PUSH-EXTEND).  Or maybe the vector could be list instead, though
that would possibly slow down things elsewhere.

I'm of course just guessing and leaving these notes for the curious, I
won't be tackling this myself.

> Failing that, I suggest not to
> enable breadcrumb-mode in such large JSON buffers (something that can
> be done in a mode hook, I guess?)

Yes, there is `breadcrumb-local-mode`.

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 6 Jun 2024 11:07:54 +0100
> Cc: Andrea Corallo <acorallo@gnu.org>, tsdh@gnu.org, 71367@debbugs.gnu.org, 
> 	theo@thornhill.no
> 
> > Maybe João could suggest some trick or user option to cut down that
> > initial time to reasonable level.
> 
> No, but feel free to profile and suggest Elisp changes or algorithmic
> changes.  Maybe the vconcat re-allocations in breadcrumb--ipath-plain
> re-allocations are to blame?  It's O(N^2) allocations I think.

Could be.

> What's the length of bc--ipath-plain?  17500ish?  If so, then the
> thing would be to reserve some space in the vector upfront, but I
> don't know how to do that easily in Elisp (in CL there's VECTOR-PUSH
> and VECTOR-PUSH-EXTEND).

I'd say allocate some initial space with make-vector, and then enlarge
using vconcat or something?

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[João Távora]

On Thu, Jun 6, 2024 at 12:33 PM Eli Zaretskii <eliz@gnu.org> wrote:

> I'd say allocate some initial space with make-vector, and then enlarge
> using vconcat or something?

That amounts to implementing VECTOR-PUSH/VECTOR-PUSH-EXTEND
for a new "dynamic vector" data type, very common in other languages.
Doesn't belong in breadcrumb at all, in fact doesn't even belong in
Elisp IMO, but
I don't  mind if someone contributes it there (after confirming
this is the culprit, of course).  Could even be some other package already
has it.

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 6 Jun 2024 12:47:13 +0100
> Cc: acorallo@gnu.org, tsdh@gnu.org, 71367@debbugs.gnu.org, theo@thornhill.no
> 
> On Thu, Jun 6, 2024 at 12:33 PM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > I'd say allocate some initial space with make-vector, and then enlarge
> > using vconcat or something?
> 
> That amounts to implementing VECTOR-PUSH/VECTOR-PUSH-EXTEND
> for a new "dynamic vector" data type, very common in other languages.
> Doesn't belong in breadcrumb at all, in fact doesn't even belong in
> Elisp IMO, but
> I don't  mind if someone contributes it there (after confirming
> this is the culprit, of course).  Could even be some other package already
> has it.

I don't see why using make-vector and vconcat, two existing
primitives, would qualify as implementing something more general than
what is needed here.  But if we want this functionality as a
primitive, we could perhaps expose larger_vector to Lisp.

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[João Távora]

On Thu, Jun 6, 2024 at 1:08 PM Eli Zaretskii <eliz@gnu.org> wrote:

> I don't see why using make-vector and vconcat, two existing
> primitives, would qualify as implementing something more general than
> what is needed here.

If someone uses those two primitives in the context of that
particular  algorithm, they'll likely to keep track of the capacity of
the vector saying how many things fit and the size counting how many
things are already there. You'll increment size when adding things
pushing and enlarge/reallocate when  size = capacity. By then you'll
have "open coded" VECTOR-PUSH-EXTEND or C++'s std::vector::push_back
or whatever your preferred language calls it.

João

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 6 Jun 2024 14:00:20 +0100
> Cc: acorallo@gnu.org, tsdh@gnu.org, 71367@debbugs.gnu.org, theo@thornhill.no
> 
> On Thu, Jun 6, 2024 at 1:08 PM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > I don't see why using make-vector and vconcat, two existing
> > primitives, would qualify as implementing something more general than
> > what is needed here.
> 
> If someone uses those two primitives in the context of that
> particular  algorithm, they'll likely to keep track of the capacity of
> the vector saying how many things fit and the size counting how many
> things are already there. You'll increment size when adding things
> pushing and enlarge/reallocate when  size = capacity. By then you'll
> have "open coded" VECTOR-PUSH-EXTEND or C++'s std::vector::push_back
> or whatever your preferred language calls it.

I understand all that.  My point is that, given how little we use
vectors in our Lisp code, having to implement that in one or two
places is not a big deal, and doesn't necessarily mean we need a
general-purpose primitive.

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[João Távora]

On Thu, Jun 6, 2024 at 2:37 PM Eli Zaretskii <eliz@gnu.org> wrote:

> I understand all that.  My point is that, given how little we use
> vectors in our Lisp code, having to implement that in one or two
> places is not a big deal, and doesn't necessarily mean we need a
> general-purpose primitive.

I just said if someone comes with a breadcrumb.el solution, I'd like
to see it well encapsulated as an ADT, not mixed with the graph algorithm.
I don't care if Emacs keeps skipping this basic util for whatever reason.

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Tassilo Horn]

João Távora <joaotavora@gmail.com> writes:

>> I understand all that.  My point is that, given how little we use
>> vectors in our Lisp code, having to implement that in one or two
>> places is not a big deal, and doesn't necessarily mean we need a
>> general-purpose primitive.
>
> I just said if someone comes with a breadcrumb.el solution, I'd like
> to see it well encapsulated as an ADT, not mixed with the graph
> algorithm.  I don't care if Emacs keeps skipping this basic util for
> whatever reason.

Before turning to algorithm optimizations, I think there is some simpler
bug somewhere.  I wanted to understand how the graph imenu--in-alist (or
what (breadcrumb--ipath-alist) returns) looks like, so I've created a
simple test.json file:

--8<---------------cut here---------------start------------->8---
{
  "root": [
    {
      "foo": 1,
      "bar": "test1"
    },
    {
      "foo": 2,
      "bar": "test2"
    }
  ]
}
--8<---------------cut here---------------end--------------->8---

At the same time, I've had my sample mirrorstatus.json open and did M-:
stuff like M-: (pp (breadcrumb--ipath-alist)) here and there to get some
printed representations.  And somehow now my imenu--in-alist and
breadcrumb--ipath-plain-cache values in the test.json buffer contains
markers in both test.json AND mirrorstatus.json!

--8<---------------cut here---------------start------------->8---
imenu--index-alist is a variable defined in ‘imenu.el’.
...
Value:
(("root"
  (#(" " 0 1
     (breadcrumb-siblings #1=
                          ((#("active" 0 1 (breadcrumb-siblings #1#))
                            . #<marker at 137 in mirrorstatus.json>)
                           (#("completion_pct" 0 1 (breadcrumb-siblings #1#))
                            . #<marker at 159 in mirrorstatus.json>)
                           (#("country" 0 1 (breadcrumb-siblings #1#))
                            . #<marker at 188 in mirrorstatus.json>)
                           (#("country_code" 0 1 (breadcrumb-siblings #1#))
                            . #<marker at 218 in mirrorstatus.json>)
                           (#("delay" 0 1 (breadcrumb-siblings #1#)) . #<marker
                            at 246 in mirrorstatus.json>)
  ...
   . #<marker at 5 in test.json>)
  ("foo" . #<marker at 27 in test.json>) ("bar" . #<marker at 43 in test.json>)
  ("foo" . #<marker at 77 in test.json>) ("bar" . #<marker at 93 in test.json>)))
Local in buffer test.json; global value is nil
--8<---------------cut here---------------end--------------->8---

That's surely now how its supposed to be.  But I don't see how one can
get there.  I can only imagine a timer acting on the wrong buffer but
the breadcrumb--idle-timer is buffer-local and the timer function
ensures it runs on the buffer the timer was activated for...

Bye,
Tassilo

bug#71367: 30.0.50; ELPA package breadcrumb causes Emacs freeze when editing json file

[Jeremy Bryant via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Tassilo Horn <tsdh@gnu.org> writes:

> I use and like the breadcrumb ELPA package.
>
>   https://elpa.gnu.org/packages/breadcrumb.html
>
> It's installed/enabled using:
>
> --8<---------------cut here---------------start------------->8---
> (use-package breadcrumb
>   :ensure t
>   :init (breadcrumb-mode 1))
> --8<---------------cut here---------------end--------------->8---
>
> Now when I find the attached mirrorstatus.json using json-ts-mode (that
> seems to be important), set point on the first { and insert some spaces,
> Emacs will eventually freeze for several seconds or even minutes.
>
> It can be reproduced with
>
>   emacs -Q
>   M-x package-initialize RET
>   M-: (package-activate 'breadcrumb) RET
>   M-x breadcrumb-mode RET
>
> and then finding the file and doing as written above.
>
> I've also attached a gdb backtrace which suggests that emacs is
> collecting garbage (produced by the breadcrumb ELN code)?  I've also
> tried profiler-start, reproduce the freeze, wait for it to go away,
> profiler-report.  That said that 99% of cpu time where spent in
> Automatic GC.
>
> I currently have no emacs without native compilation handy so I cannot
> tell if that's important, too.  Can I somehow tell emacs not to compile
> breadcrumb natively after deleting the eln file?
>
> [2. application/gzip; mirrorstatus.json.gz]...
>
> [3. application/gzip; backtrace.txt.gz]...
>

João is the maintainer, CC'd