bug#70438: Emacs error 6 abort when starting rust-ts-mode

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Stefan Heitmann]

[-- Attachment #1: Type: text/plain, Size: 12841 bytes --]

From: sh@bytekomplex.de
To: bug-gnu-emacs@gnu.org
Subject: 29.3; After upgrading Arch - Emacs crashes when starting rust-ts-mode
Date: Wed, 17 Apr 2024 13:35:04 +0200
Message-ID: <87mspsi5nb.fsf@heistemaux.mail-host-address-is-not-set>
--text follows this line--

Opening a rust file (*.ts) and doing M-x rust-ts-mode <ENTER> crashes my
emacs. When I start emacs using the terminal I get the following when it
crashes:

*** stack smashing detected ***: terminated
Fatal error 6: Aborted
Backtrace:
emacs(+0x140096)[0x5cc8dc378096]
emacs(+0x20330)[0x5cc8dc258330]
emacs(+0x21153)[0x5cc8dc259153]
emacs(+0x29ad8d)[0x5cc8dc4d2d8d]
/usr/lib/libc.so.6(+0x3c770)[0x736916235770]
/usr/lib/libc.so.6(+0x8d32c)[0x73691628632c]
/usr/lib/libc.so.6(gsignal+0x18)[0x7369162356c8]
/usr/lib/libc.so.6(abort+0xd7)[0x73691621d4b8]
/usr/lib/libc.so.6(+0x25395)[0x73691621e395]
/usr/lib/libc.so.6(+0x11473b)[0x73691630d73b]
/usr/lib/libc.so.6(+0x115a56)[0x73691630ea56]
emacs(+0x24a063)[0x5cc8dc482063]
/usr/lib/emacs/29.3/native-lisp/29.3-4b59e32f/treesit-37439c61-97df641d.eln(F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0+0x1f2)[0x7369068e6832]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d666f6e746966792d73796e746163746963616c6c792d726567696f6e_font_lock_fontify_syntactically_region_0+0x62)[0x736911930f42]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0+0x4af)[0x73691192ebef]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0+0x93)[0x73691192d873]
emacs(+0x2078be)[0x5cc8dc43f8be]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
emacs(+0x1bac31)[0x5cc8dc3f2c31]
emacs(+0x1b567c)[0x5cc8dc3ed67c]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0+0xd8)[0x736911597c88]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0+0x80a)[0x73691159859a]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0+0x26f)[0x73691159798f]
emacs(+0x1b661e)[0x5cc8dc3ee61e]
emacs(+0x29e8d1)[0x5cc8dc4d68d1]
emacs(+0x4d90b)[0x5cc8dc28590b]
emacs(+0x53a10)[0x5cc8dc28ba10]
emacs(+0x57d96)[0x5cc8dc28fd96]
emacs(+0x5da76)[0x5cc8dc295a76]
emacs(+0x54976)[0x5cc8dc28c976]
emacs(+0x76af9)[0x5cc8dc2aeaf9]
emacs(+0x7ea02)[0x5cc8dc2b6a02]
emacs(+0x6f4c3)[0x5cc8dc2a74c3]
emacs(+0x1b3aec)[0x5cc8dc3ebaec]
emacs(+0x6f418)[0x5cc8dc2a7418]
emacs(+0x6f43d)[0x5cc8dc2a743d]
...
Aborted (core dumped)

My error seems to be similiar to this one:
https://github.com/tree-sitter/tree-sitter/issues/3296

However, I was encouraged on #emacs to file another bug report to the
emacs project.


In GNU Emacs 29.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41,
cairo version 1.18.0)
System Description: Arch Linux

Configured using:
 'configure --with-pgtk --with-native-compilation=aot --sysconfdir=/etc
 --prefix=/usr --libexecdir=/usr/lib --with-tree-sitter
 --localstatedir=/var --with-cairo --disable-build-details
 --with-harfbuzz --with-libsystemd --with-modules 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2
 -Wformat -Werror=format-security -fstack-clash-protection
 -fcf-protection -g
 -ffile-prefix-map=/build/emacs/src=/usr/src/debug/emacs -flto=auto'
 'LDFLAGS=-Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro
 -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto'
 'CXXFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -g
 -ffile-prefix-map=/build/emacs/src=/usr/src/debug/emacs -flto=auto''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  marginalia-mode: t
  vertico-mode: t
  global-corfu-mode: t
  corfu-mode: t
  global-git-commit-mode: t
  eshell-syntax-highlighting-global-mode: t
  shell-dirtrack-mode: t
  projectile-mode: t
  recentf-mode: t
  which-key-mode: t
  persp-mode: t
  global-evil-collection-unimpaired-mode: t
  evil-collection-unimpaired-mode: t
  evil-mode: t
  evil-local-mode: t
  minimap-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  doom-modeline-mode: t
  general-override-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  global-auto-revert-mode: t
  electric-pair-mode: t
  delete-selection-mode: t
  windmove-mode: t
  elpaca-use-package-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  global-visual-line-mode: t
  visual-line-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug mule-util vc-git vc-dispatcher
org-indent toc-org oc-basic ol-eww eww url-queue mm-url ol-rmail ol-mhe
ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015 mm-view mml-smime
smime gnutls dig gnus-sum shr pixel-fill kinsoku url-file svg dom
gnus-group gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap
nnmail mail-source utf7 nnoo gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range ol-docview doc-view jka-compr image-mode exif ol-bibtex
bibtex ol-bbdb ol-w3m ol-doi org-link-doi pulse org-agenda org-element
org-persist xdg org-id avl-tree org-refile time rainbow-delimiters
rainbow-mode hl-todo orderless marginalia vertico corfu ob-restclient
restclient magit-bookmark magit-submodule magit-blame magit-stash
magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit package browse-url url-handlers magit-repos
magit-apply magit-wip magit-log which-func imenu magit-diff smerge-mode
diff-mode git-commit log-edit message sendmail yank-media puny rfc822
mml mml-sec epa derived epg rfc6068 epg-config gnus-util mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader
pcvs-util add-log magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor server magit-mode transient
magit-git magit-base magit-section cursor-sensor crm org-bullets
sudo-edit dired-open dired-hacks-utils dired-x tldr vterm-toggle
tramp-sh vterm bookmark pp tramp tramp-loaddefs trampver
tramp-integration tramp-compat parse-time iso8601 face-remap color
vterm-module term/xterm xterm eshell-syntax-highlighting em-prompt
em-alias eshell-toggle term disp-table shell ehelp esh-mode eshell
esh-cmd generator esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util files-x neotree projectile project lisp-mnt grep
compile text-property-search ibuf-ext evil-collection-ibuffer ibuffer
ibuffer-loaddefs evil-collection-dashboard dashboard dashboard-widgets
recentf tree-widget wid-edit ffap diminish which-key perspective ido
evil-tutor evil-collection-unimpaired evil-collection-dired
evil-collection annalist evil evil-integration evil-maps evil-commands
reveal evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core advice evil-common
thingatpt rect evil-vars edmacro kmacro minimap undo-tree diff queue avy
all-the-icons-dired dired dired-loaddefs all-the-icons
all-the-icons-faces data-material data-weathericons data-octicons
data-fileicons data-faicons data-alltheicons doom-modeline
doom-modeline-segments doom-modeline-env doom-modeline-core comp
comp-cstr warnings shrink-path f s dash nerd-icons nerd-icons-faces
nerd-icons-data nerd-icons-data-mdicon nerd-icons-data-flicon
nerd-icons-data-codicon nerd-icons-data-devicon nerd-icons-data-sucicon
nerd-icons-data-wicon nerd-icons-data-faicon nerd-icons-data-powerline
nerd-icons-data-octicon nerd-icons-data-pomicon nerd-icons-data-ipsicon
compat doom-nord-theme doom-themes doom-themes-base general
orderless-autoloads marginalia-autoloads consult-autoloads
vertico-autoloads corfu-autoloads ob-restclient-autoloads
restclient-autoloads git-timemachine-autoloads magit-autoloads pcase
git-commit-autoloads magit-section-autoloads with-editor-autoloads
org-bullets-autoloads toc-org-autoloads sudo-edit-autoloads
peep-dired-autoloads dired-open-autoloads dired-hacks-utils-autoloads
tldr-autoloads vterm-toggle-autoloads vterm-autoloads
eshell-syntax-highlighting-autoloads eshell-toggle-autoloads
diminish-autoloads neotree-autoloads projectile-autoloads
dashboard-autoloads which-key-autoloads perspective-autoloads
evil-tutor-autoloads evil-collection-autoloads annalist-autoloads
evil-autoloads goto-chg-autoloads hl-todo-autoloads minimap-autoloads
undo-tree-autoloads queue-autoloads rainbow-mode-autoloads
rainbow-delimiters-autoloads avy-autoloads all-the-icons-dired-autoloads
all-the-icons-autoloads doom-modeline-autoloads compat-autoloads
nerd-icons-autoloads shrink-path-autoloads f-autoloads s-autoloads
dash-autoloads doom-themes-autoloads general-autoloads
app-launcher-autoloads org-tempo tempo display-line-numbers autorevert
filenotify elec-pair delsel cl-extra help-mode app-launchers buffer-move
windmove elpaca-setup elpaca-use-package use-package use-package-ensure
use-package-delight use-package-diminish use-package-bind-key bind-key
easy-mmode use-package-core elpaca-use-package-autoloads elpaca-log
elpaca-ui url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util url-parse auth-source
eieio eieio-core password-cache json map byte-opt bytecomp byte-compile
url-vars mailcap cl-seq cl-macs gv elpaca elpaca-process
elpaca-autoloads org ob ob-tangle ob-ref ob-lob ob-table ob-exp
org-macro org-src ob-comint org-pcomplete pcomplete comint ansi-osc
ansi-color ring org-list org-footnote org-faces org-entities time-date
subr-x noutline outline icons ob-emacs-lisp ob-core ob-eval org-cycle
org-table ol rx org-fold org-fold-core org-keys oc org-loaddefs
find-func cal-menu calendar cal-loaddefs org-version org-compat org-macs
format-spec cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 1019023 61146)
 (symbols 48 47444 3)
 (strings 32 317535 5547)
 (string-bytes 1 7916279)
 (vectors 16 95168)
 (vector-slots 8 2017207 65876)
 (floats 8 1478 287)
 (intervals 56 10218 397)
 (buffers 984 23))

[-- Attachment #2: Type: text/html, Size: 52314 bytes --]

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Stefan Heitmann <sh@bytekomplex.de>
> Date: Wed, 17 Apr 2024 11:41:16 +0000
> 
> From: sh@bytekomplex.de 
> To: bug-gnu-emacs@gnu.org
> Subject: 29.3; After upgrading Arch - Emacs crashes when starting rust-ts-mode
> Date: Wed, 17 Apr 2024 13:35:04 +0200
> Message-ID: <87mspsi5nb.fsf@heistemaux.mail-host-address-is-not-set>
> --text follows this line--
> 
> Opening a rust file (*.ts) and doing M-x rust-ts-mode <ENTER> crashes my
> emacs. When I start emacs using the terminal I get the following when it
> crashes:
> 
> *** stack smashing detected ***: terminated
> Fatal error 6: Aborted
> Backtrace:
> emacs(+0x140096)[0x5cc8dc378096]
> emacs(+0x20330)[0x5cc8dc258330]
> emacs(+0x21153)[0x5cc8dc259153]
> emacs(+0x29ad8d)[0x5cc8dc4d2d8d]
> /usr/lib/libc.so.6(+0x3c770)[0x736916235770]
> /usr/lib/libc.so.6(+0x8d32c)[0x73691628632c]
> /usr/lib/libc.so.6(gsignal+0x18)[0x7369162356c8]
> /usr/lib/libc.so.6(abort+0xd7)[0x73691621d4b8]
> /usr/lib/libc.so.6(+0x25395)[0x73691621e395]
> /usr/lib/libc.so.6(+0x11473b)[0x73691630d73b]
> /usr/lib/libc.so.6(+0x115a56)[0x73691630ea56]
> emacs(+0x24a063)[0x5cc8dc482063]
> /usr/lib/emacs/29.3/native-lisp/29.3-4b59e32f/treesit-37439c61-97df641d.eln
> (F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0+0x1f2)[0x7369068e6832]
> 

Yuan, can you please look into this?  It seems the latest versions of
tree-sitter made some incompatible ABI change (without changing the
shared-library version, I guess? not nice!), so can we have some
workaround for this?

Thanks.

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 17, 2024, at 9:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Stefan Heitmann <sh@bytekomplex.de>
>> Date: Wed, 17 Apr 2024 11:41:16 +0000
>> 
>> From: sh@bytekomplex.de 
>> To: bug-gnu-emacs@gnu.org
>> Subject: 29.3; After upgrading Arch - Emacs crashes when starting rust-ts-mode
>> Date: Wed, 17 Apr 2024 13:35:04 +0200
>> Message-ID: <87mspsi5nb.fsf@heistemaux.mail-host-address-is-not-set>
>> --text follows this line--
>> 
>> Opening a rust file (*.ts) and doing M-x rust-ts-mode <ENTER> crashes my
>> emacs. When I start emacs using the terminal I get the following when it
>> crashes:
>> 
>> *** stack smashing detected ***: terminated
>> Fatal error 6: Aborted
>> Backtrace:
>> emacs(+0x140096)[0x5cc8dc378096]
>> emacs(+0x20330)[0x5cc8dc258330]
>> emacs(+0x21153)[0x5cc8dc259153]
>> emacs(+0x29ad8d)[0x5cc8dc4d2d8d]
>> /usr/lib/libc.so.6(+0x3c770)[0x736916235770]
>> /usr/lib/libc.so.6(+0x8d32c)[0x73691628632c]
>> /usr/lib/libc.so.6(gsignal+0x18)[0x7369162356c8]
>> /usr/lib/libc.so.6(abort+0xd7)[0x73691621d4b8]
>> /usr/lib/libc.so.6(+0x25395)[0x73691621e395]
>> /usr/lib/libc.so.6(+0x11473b)[0x73691630d73b]
>> /usr/lib/libc.so.6(+0x115a56)[0x73691630ea56]
>> emacs(+0x24a063)[0x5cc8dc482063]
>> /usr/lib/emacs/29.3/native-lisp/29.3-4b59e32f/treesit-37439c61-97df641d.eln
>> (F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0+0x1f2)[0x7369068e6832]
>> 
> 
> Yuan, can you please look into this?  It seems the latest versions of
> tree-sitter made some incompatible ABI change (without changing the
> shared-library version, I guess? not nice!), so can we have some
> workaround for this?

I think tree-sitter people are working to fix this [1]. I don’t think we can do much here, AFAICT tree-site changed ABI without bumping ABI version, and Arch and Gentoo are serving old Emacs build this new tree-sitter so, which is incompatible and crashed Emacs. We can’t really do anything to the Emacs build on people’s machines (though, who knows, maybe you CAN cask spells and flip bits).

[1] https://github.com/tree-sitter/tree-sitter/issues/3296

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 20 Apr 2024 01:28:48 -0700
> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>  70438@debbugs.gnu.org
> 
> > Yuan, can you please look into this?  It seems the latest versions of
> > tree-sitter made some incompatible ABI change (without changing the
> > shared-library version, I guess? not nice!), so can we have some
> > workaround for this?
> 
> I think tree-sitter people are working to fix this [1]. I don’t think we can do much here, AFAICT tree-site changed ABI without bumping ABI version, and Arch and Gentoo are serving old Emacs build this new tree-sitter so, which is incompatible and crashed Emacs. We can’t really do anything to the Emacs build on people’s machines (though, who knows, maybe you CAN cask spells and flip bits).
> 
> [1] https://github.com/tree-sitter/tree-sitter/issues/3296

It isn't clear to me what would be their solution.  Couldn't we at
least reject these new versions at configure time, until they fix the
issue?

And when they do bump the ABI version, what would be our solution to
support both previous and new ABIs?

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 20, 2024, at 2:36 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 20 Apr 2024 01:28:48 -0700
>> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>> 70438@debbugs.gnu.org
>> 
>>> Yuan, can you please look into this?  It seems the latest versions of
>>> tree-sitter made some incompatible ABI change (without changing the
>>> shared-library version, I guess? not nice!), so can we have some
>>> workaround for this?
>> 
>> I think tree-sitter people are working to fix this [1]. I don’t think we can do much here, AFAICT tree-site changed ABI without bumping ABI version, and Arch and Gentoo are serving old Emacs build this new tree-sitter so, which is incompatible and crashed Emacs. We can’t really do anything to the Emacs build on people’s machines (though, who knows, maybe you CAN cask spells and flip bits).
>> 
>> [1] https://github.com/tree-sitter/tree-sitter/issues/3296
> 
> It isn't clear to me what would be their solution.  Couldn't we at
> least reject these new versions at configure time, until they fix the
> issue?

From what I understand, rebuilding Emacs with the new header file should fix the issue? Even if we push some change, users need to rebuild Emacs anyway.

> 
> And when they do bump the ABI version, what would be our solution to
> support both previous and new ABIs?

When then do bump ABI version, the old and new libtree-sitter.so will live alongside each other, Emacs built with the old libtree-sitter.so will use the old version, and Emacs built with the new libtree-sitter.so will use the new version.

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 20 Apr 2024 15:20:41 -0700
> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>  70438@debbugs.gnu.org
> 
> 
> 
> > On Apr 20, 2024, at 2:36 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri@gmail.com>
> >> Date: Sat, 20 Apr 2024 01:28:48 -0700
> >> Cc: Stefan Heitmann <sh@bytekomplex.de>,
> >> 70438@debbugs.gnu.org
> >> 
> >>> Yuan, can you please look into this?  It seems the latest versions of
> >>> tree-sitter made some incompatible ABI change (without changing the
> >>> shared-library version, I guess? not nice!), so can we have some
> >>> workaround for this?
> >> 
> >> I think tree-sitter people are working to fix this [1]. I don’t think we can do much here, AFAICT tree-site changed ABI without bumping ABI version, and Arch and Gentoo are serving old Emacs build this new tree-sitter so, which is incompatible and crashed Emacs. We can’t really do anything to the Emacs build on people’s machines (though, who knows, maybe you CAN cask spells and flip bits).
> >> 
> >> [1] https://github.com/tree-sitter/tree-sitter/issues/3296
> > 
> > It isn't clear to me what would be their solution.  Couldn't we at
> > least reject these new versions at configure time, until they fix the
> > issue?
> 
> From what I understand, rebuilding Emacs with the new header file should fix the issue? Even if we push some change, users need to rebuild Emacs anyway.

I didn't mean to avoid rebuilding Emacs -- that's impossible when
changes are made on the C level.  I meant to make our sources work
with both old and new ABIs.

But maybe I don't have a clear idea of what exactly was the
incompatible change.  Can you describe it here?  Is it the signature
of one or more tree-sitter functions, or is it something else?

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 20, 2024, at 9:47 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 20 Apr 2024 15:20:41 -0700
>> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>> 70438@debbugs.gnu.org
>> 
>> 
>> 
>>> On Apr 20, 2024, at 2:36 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri@gmail.com>
>>>> Date: Sat, 20 Apr 2024 01:28:48 -0700
>>>> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>>>> 70438@debbugs.gnu.org
>>>> 
>>>>> Yuan, can you please look into this?  It seems the latest versions of
>>>>> tree-sitter made some incompatible ABI change (without changing the
>>>>> shared-library version, I guess? not nice!), so can we have some
>>>>> workaround for this?
>>>> 
>>>> I think tree-sitter people are working to fix this [1]. I don’t think we can do much here, AFAICT tree-site changed ABI without bumping ABI version, and Arch and Gentoo are serving old Emacs build this new tree-sitter so, which is incompatible and crashed Emacs. We can’t really do anything to the Emacs build on people’s machines (though, who knows, maybe you CAN cask spells and flip bits).
>>>> 
>>>> [1] https://github.com/tree-sitter/tree-sitter/issues/3296
>>> 
>>> It isn't clear to me what would be their solution.  Couldn't we at
>>> least reject these new versions at configure time, until they fix the
>>> issue?
>> 
>> From what I understand, rebuilding Emacs with the new header file should fix the issue? Even if we push some change, users need to rebuild Emacs anyway.
> 
> I didn't mean to avoid rebuilding Emacs -- that's impossible when
> changes are made on the C level.  I meant to make our sources work
> with both old and new ABIs.
> 
> But maybe I don't have a clear idea of what exactly was the
> incompatible change.  Can you describe it here?  Is it the signature
> of one or more tree-sitter functions, or is it something else?

From what I understand, they changed a struct in the public interface [1]. So now the ABI changed. But they didn’t change the ABI version, and Arch and Gentoo swiftly replaced the old libtree-sitter.so with the new libtree-sitter.so. Then the old Emacs loaded the new libtree-sitter.so (which has a new ABI) and crashed.

Our code is still perfectly compatible with the new tree-sitter code; the changed field in that struct doesn’t affect us. The incompatibility is at the binary level.

[1] https://github.com/tree-sitter/tree-sitter/commit/09d2b23a640c60449b2b55ecae47d6483da82c95

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sun, 21 Apr 2024 16:57:46 -0700
> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>  70438@debbugs.gnu.org
> 
> > But maybe I don't have a clear idea of what exactly was the
> > incompatible change.  Can you describe it here?  Is it the signature
> > of one or more tree-sitter functions, or is it something else?
> 
> From what I understand, they changed a struct in the public interface [1]. So now the ABI changed. But they didn’t change the ABI version, and Arch and Gentoo swiftly replaced the old libtree-sitter.so with the new libtree-sitter.so. Then the old Emacs loaded the new libtree-sitter.so (which has a new ABI) and crashed.
> 
> Our code is still perfectly compatible with the new tree-sitter code; the changed field in that struct doesn’t affect us. The incompatibility is at the binary level.

And that struct is declared on the tree-sitter include file?  If so,
the only thing we can do is reject the bad version of tree-sitter at
configure time, I think.  This assumes that when they change the ABI
version, they will also bump the version of the library, so the test
for the bad version will only discover the ones that have an
incompatible ABI.

Can you add such a test to configure.ac?

(We could theoretically also fail the compilation, using the #error
directive, but AFAICT the tree-sitter header files don't declare any
version-related symbols, so we cannot check the library version at
compile time.)

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 21, 2024, at 10:54 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sun, 21 Apr 2024 16:57:46 -0700
>> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>> 70438@debbugs.gnu.org
>> 
>>> But maybe I don't have a clear idea of what exactly was the
>>> incompatible change.  Can you describe it here?  Is it the signature
>>> of one or more tree-sitter functions, or is it something else?
>> 
>> From what I understand, they changed a struct in the public interface [1]. So now the ABI changed. But they didn’t change the ABI version, and Arch and Gentoo swiftly replaced the old libtree-sitter.so with the new libtree-sitter.so. Then the old Emacs loaded the new libtree-sitter.so (which has a new ABI) and crashed.
>> 
>> Our code is still perfectly compatible with the new tree-sitter code; the changed field in that struct doesn’t affect us. The incompatibility is at the binary level.
> 
> And that struct is declared on the tree-sitter include file?  If so,
> the only thing we can do is reject the bad version of tree-sitter at
> configure time, I think.  This assumes that when they change the ABI
> version, they will also bump the version of the library, so the test
> for the bad version will only discover the ones that have an
> incompatible ABI.
> 
> Can you add such a test to configure.ac?
> 
> (We could theoretically also fail the compilation, using the #error
> directive, but AFAICT the tree-sitter header files don't declare any
> version-related symbols, so we cannot check the library version at
> compile time.)

There’s really no “bad” version—if someone downloads Emacs source and the new version of tree-sitter library (0.22.5), and compiles Emacs with it, Emacs wouldn’t crash. Emacs only crashes when it’s built with the old tree-sitter header, and loads a new tree-sitter .so file (which has incompatible ABI but has the same ABI version). We obviously can’t do anything to the already built Emacs to make it compatible with the new libtree-sitter.so; and any newly-built Emacs will not have issues. So I don’t really know what can we do here.

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

tags 70438 wontfix
close 70438
thanks

> From: Yuan Fu <casouri@gmail.com>
> Date: Sun, 21 Apr 2024 23:13:56 -0700
> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>  70438@debbugs.gnu.org
> 
> 
> There’s really no “bad” version—if someone downloads Emacs source and the new version of tree-sitter library (0.22.5), and compiles Emacs with it, Emacs wouldn’t crash. Emacs only crashes when it’s built with the old tree-sitter header, and loads a new tree-sitter .so file (which has incompatible ABI but has the same ABI version). We obviously can’t do anything to the already built Emacs to make it compatible with the new libtree-sitter.so; and any newly-built Emacs will not have issues. So I don’t really know what can we do here.

OK, so I'm closing this bug as wontfix.

bug#70438: AW: bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Stefan Heitmann]

Thank you very much for your help...

I think I understand roughly the issue and it makes sense that you can't do anything here... But I think, I could file another bug report for the repo maintainer of the arch package 😉

Thanks again

Stefan

-----Ursprüngliche Nachricht-----
Von: Yuan Fu <casouri@gmail.com> 
Gesendet: 22 April 2024 08:14
An: Eli Zaretskii <eliz@gnu.org>
Cc: Stefan Heitmann <sh@bytekomplex.de>; 70438@debbugs.gnu.org
Betreff: Re: bug#70438: Emacs error 6 abort when starting rust-ts-mode



> On Apr 21, 2024, at 10:54 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sun, 21 Apr 2024 16:57:46 -0700
>> Cc: Stefan Heitmann <sh@bytekomplex.de>, 70438@debbugs.gnu.org
>> 
>>> But maybe I don't have a clear idea of what exactly was the 
>>> incompatible change.  Can you describe it here?  Is it the signature 
>>> of one or more tree-sitter functions, or is it something else?
>> 
>> From what I understand, they changed a struct in the public interface [1]. So now the ABI changed. But they didn’t change the ABI version, and Arch and Gentoo swiftly replaced the old libtree-sitter.so with the new libtree-sitter.so. Then the old Emacs loaded the new libtree-sitter.so (which has a new ABI) and crashed.
>> 
>> Our code is still perfectly compatible with the new tree-sitter code; the changed field in that struct doesn’t affect us. The incompatibility is at the binary level.
> 
> And that struct is declared on the tree-sitter include file?  If so, 
> the only thing we can do is reject the bad version of tree-sitter at 
> configure time, I think.  This assumes that when they change the ABI 
> version, they will also bump the version of the library, so the test 
> for the bad version will only discover the ones that have an 
> incompatible ABI.
> 
> Can you add such a test to configure.ac?
> 
> (We could theoretically also fail the compilation, using the #error 
> directive, but AFAICT the tree-sitter header files don't declare any 
> version-related symbols, so we cannot check the library version at 
> compile time.)

There’s really no “bad” version—if someone downloads Emacs source and the new version of tree-sitter library (0.22.5), and compiles Emacs with it, Emacs wouldn’t crash. Emacs only crashes when it’s built with the old tree-sitter header, and loads a new tree-sitter .so file (which has incompatible ABI but has the same ABI version). We obviously can’t do anything to the already built Emacs to make it compatible with the new libtree-sitter.so; and any newly-built Emacs will not have issues. So I don’t really know what can we do here.

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Stefan Heitmann <sh@bytekomplex.de>
> CC: "70438@debbugs.gnu.org" <70438@debbugs.gnu.org>
> Date: Thu, 25 Apr 2024 11:24:13 +0000
> 
> Thank you very much for your help...
> 
> I think I understand roughly the issue and it makes sense that you can't do anything here... But I think, I could file another bug report for the repo maintainer of the arch package 😉

Yes, please.

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 25, 2024, at 6:19 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Stefan Heitmann <sh@bytekomplex.de>
>> CC: "70438@debbugs.gnu.org" <70438@debbugs.gnu.org>
>> Date: Thu, 25 Apr 2024 11:24:13 +0000
>> 
>> Thank you very much for your help...
>> 
>> I think I understand roughly the issue and it makes sense that you can't do anything here... But I think, I could file another bug report for the repo maintainer of the arch package 😉
> 
> Yes, please.

In the GitHub issue that tracks this incident, the author made clear that maintaining ABI versioning correctly is beyond their ability right now. So I think we should pin the tree-sitter version. But then IIUC we can only bump tree-sitter version with each Emacs release? This is a bit unfortunate but better than crashing Emacs.

This also brings me to the versioning of tree-sitter grammars—they really do change often, we should really consider pinning their version in some way. The current catch-up game we play can’t be scalable.

Yuan

Quote:

>> Just so that this is clear to me: The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?

> I really think that in order to provide a reliable end-user experience, and prevent issue like this, Emacs should just statically link a particular version of Tree-sitter.

> If some Linux distros mandates that all libraries, no matter how small, must be distributed as dynamic libraries (which is just... staggeringly impractical IMO), then the Emacs package should specify a particular version of the Tree-sitter package to use - not a version range.

>> I think we have to commit to not using the semver version in our library names, and to follow libtool-compatible rules for selecting a SONAME instead.
>> 
> I agree with this. Ideally, we should set up tooling to automatically bump the soname on breaking ABI changes. But if any Emacs package maintainer is seeing this issue - we do not yet have this tooling - this library is maintained by a small team, and may not have perfect ABI stability! Consider taking a more conservative approach to dependency versioning, so that end users don't have to deal with situations like this!

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 26 Apr 2024 09:49:01 -0700
> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>  70438@debbugs.gnu.org
> 
> In the GitHub issue that tracks this incident, the author made clear that maintaining ABI versioning correctly is beyond their ability right now. So I think we should pin the tree-sitter version. But then IIUC we can only bump tree-sitter version with each Emacs release? This is a bit unfortunate but better than crashing Emacs.

We cannot pin a tree-sitter version, because that makes sense only for
binary distributions.  We ship only source tarballs, and for those,
_any_ tree-sitter version will do -- provided that Emacs is built with
the same version of tree-sitter with which it will be used, or with
the version that uses the same ABI.

We could perhaps record the version with which Emacs was built, and
then reject incompatible versions we find at run time, but since
tree-sitter doesn't provide any version-related symbols in their
header files, we cannot do even that.

So the bottom line is still the same: we cannot do anything here, as
long as the tree-sitter developers think they can break the ABI at
will.

> This also brings me to the versioning of tree-sitter grammars—they really do change often, we should really consider pinning their version in some way. The current catch-up game we play can’t be scalable.

That's a separate issue: when there's incompatibility with grammar
libraries, Emacs doesn't crash.

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 26, 2024, at 10:21 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Fri, 26 Apr 2024 09:49:01 -0700
>> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>> 70438@debbugs.gnu.org
>> 
>> In the GitHub issue that tracks this incident, the author made clear that maintaining ABI versioning correctly is beyond their ability right now. So I think we should pin the tree-sitter version. But then IIUC we can only bump tree-sitter version with each Emacs release? This is a bit unfortunate but better than crashing Emacs.
> 
> We cannot pin a tree-sitter version, because that makes sense only for
> binary distributions.  We ship only source tarballs, and for those,
> _any_ tree-sitter version will do -- provided that Emacs is built with
> the same version of tree-sitter with which it will be used, or with
> the version that uses the same ABI.
> 
> We could perhaps record the version with which Emacs was built, and
> then reject incompatible versions we find at run time, but since
> tree-sitter doesn't provide any version-related symbols in their
> header files, we cannot do even that.
> 
> So the bottom line is still the same: we cannot do anything here, as
> long as the tree-sitter developers think they can break the ABI at
> will.

Can we statically link tree-sitter? From the look of it, tree-sitter devs don’t plan to not break ABI. We need to do something to prevent Emacs from crashing.

>> This also brings me to the versioning of tree-sitter grammars—they really do change often, we should really consider pinning their version in some way. The current catch-up game we play can’t be scalable.
> 
> That's a separate issue: when there's incompatibility with grammar
> libraries, Emacs doesn't crash.

I can open a separate thread for this.

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 26 Apr 2024 10:58:20 -0700
> Cc: sh@bytekomplex.de,
>  70438@debbugs.gnu.org
> 
> > We cannot pin a tree-sitter version, because that makes sense only for
> > binary distributions.  We ship only source tarballs, and for those,
> > _any_ tree-sitter version will do -- provided that Emacs is built with
> > the same version of tree-sitter with which it will be used, or with
> > the version that uses the same ABI.
> > 
> > We could perhaps record the version with which Emacs was built, and
> > then reject incompatible versions we find at run time, but since
> > tree-sitter doesn't provide any version-related symbols in their
> > header files, we cannot do even that.
> > 
> > So the bottom line is still the same: we cannot do anything here, as
> > long as the tree-sitter developers think they can break the ABI at
> > will.
> 
> Can we statically link tree-sitter? From the look of it, tree-sitter devs don’t plan to not break ABI. We need to do something to prevent Emacs from crashing.

Emacs can indeed be statically linked with tree-sitter.  But since we,
the Emacs project, don't distribute binaries, the decision how to link
Emacs with various libraries is made by the distros.  And they always
prefer shared libraries, because that allows to upgrade the libraries
without installing new binaries of dependent programs.

> >> This also brings me to the versioning of tree-sitter grammars—they really do change often, we should really consider pinning their version in some way. The current catch-up game we play can’t be scalable.
> > 
> > That's a separate issue: when there's incompatibility with grammar
> > libraries, Emacs doesn't crash.
> 
> I can open a separate thread for this.

Feel free (although we discussed that in the past already).

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Yuan Fu]

> On Apr 26, 2024, at 11:32 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Fri, 26 Apr 2024 10:58:20 -0700
>> Cc: sh@bytekomplex.de,
>> 70438@debbugs.gnu.org
>> 
>>> We cannot pin a tree-sitter version, because that makes sense only for
>>> binary distributions.  We ship only source tarballs, and for those,
>>> _any_ tree-sitter version will do -- provided that Emacs is built with
>>> the same version of tree-sitter with which it will be used, or with
>>> the version that uses the same ABI.
>>> 
>>> We could perhaps record the version with which Emacs was built, and
>>> then reject incompatible versions we find at run time, but since
>>> tree-sitter doesn't provide any version-related symbols in their
>>> header files, we cannot do even that.
>>> 
>>> So the bottom line is still the same: we cannot do anything here, as
>>> long as the tree-sitter developers think they can break the ABI at
>>> will.
>> 
>> Can we statically link tree-sitter? From the look of it, tree-sitter devs don’t plan to not break ABI. We need to do something to prevent Emacs from crashing.
> 
> Emacs can indeed be statically linked with tree-sitter.  But since we,
> the Emacs project, don't distribute binaries, the decision how to link
> Emacs with various libraries is made by the distros.  And they always
> prefer shared libraries, because that allows to upgrade the libraries
> without installing new binaries of dependent programs.

I wonder if we can make Emacs prefer static libtree-sitter in the makefile? Or it’s better done on the distro side?

Yuan

bug#70438: Emacs error 6 abort when starting rust-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 26 Apr 2024 20:06:26 -0700
> Cc: Stefan Heitmann <sh@bytekomplex.de>,
>  70438@debbugs.gnu.org
> 
> > Emacs can indeed be statically linked with tree-sitter.  But since we,
> > the Emacs project, don't distribute binaries, the decision how to link
> > Emacs with various libraries is made by the distros.  And they always
> > prefer shared libraries, because that allows to upgrade the libraries
> > without installing new binaries of dependent programs.
> 
> I wonder if we can make Emacs prefer static libtree-sitter in the makefile? Or it’s better done on the distro side?

The latter, I think.  Moreover, even if we do prefer static linking in
our configuration, distros can (and I think will) override it.