bug#61896: 30.0.50; Emacs crashes because of an invalid free

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 454 bytes --]


Emacs just crashes out of nowhere, e.g. after I open a my init file.

I have had this device for a while on a device of mine, that I couldn't
reproduce on my main workstation or using emacs -Q.  Apparently this
could be related to some faulty byte-code.

The best I could do to detect this issue was to build Emacs using
-fsanitize=address and I managed to reprodce the issue reliably by
invoking package-recompile-all.  I collected the following log:


[-- Attachment #2: log.1 --]
[-- Type: text/plain, Size: 5581 bytes --]

$ ./src/emacs
=================================================================
==74401==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x7ffe72b89e70 in thread T0
    #0 0x7fa972cb76a8 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
    #1 0x55dbfb6adcb7 in xfree /home/philip/Source/emacs/src/alloc.c:845
    #2 0x55dbfb7158cc in safe_free /home/philip/Source/emacs/src/lisp.h:5409
    #3 0x55dbfb72486b in apply_lambda /home/philip/Source/emacs/src/eval.c:3111
    #4 0x55dbfb7211b3 in eval_sub /home/philip/Source/emacs/src/eval.c:2547
    #5 0x55dbfb717052 in Fprogn /home/philip/Source/emacs/src/eval.c:436
    #6 0x55dbfb6f889e in Fsave_current_buffer /home/philip/Source/emacs/src/editfns.c:869
    #7 0x55dbfb7202af in eval_sub /home/philip/Source/emacs/src/eval.c:2451
    #8 0x55dbfb717052 in Fprogn /home/philip/Source/emacs/src/eval.c:436
    #9 0x55dbfb7202af in eval_sub /home/philip/Source/emacs/src/eval.c:2451
    #10 0x55dbfb716dd8 in Fif /home/philip/Source/emacs/src/eval.c:391
    #11 0x55dbfb7202af in eval_sub /home/philip/Source/emacs/src/eval.c:2451
    #12 0x55dbfb717052 in Fprogn /home/philip/Source/emacs/src/eval.c:436
    #13 0x55dbfb719c61 in Flet /home/philip/Source/emacs/src/eval.c:1026
    #14 0x55dbfb7202af in eval_sub /home/philip/Source/emacs/src/eval.c:2451
    #15 0x55dbfb717052 in Fprogn /home/philip/Source/emacs/src/eval.c:436
    #16 0x55dbfb7250fe in funcall_lambda /home/philip/Source/emacs/src/eval.c:3235
    #17 0x55dbfb7231d1 in funcall_general /home/philip/Source/emacs/src/eval.c:2959
    #18 0x55dbfb7c0ab9 in exec_byte_code /home/philip/Source/emacs/src/bytecode.c:811
    #19 0x55dbfb72446b in fetch_and_exec_byte_code /home/philip/Source/emacs/src/eval.c:3083
    #20 0x55dbfb724c00 in funcall_lambda /home/philip/Source/emacs/src/eval.c:3155
    #21 0x55dbfb7230ac in funcall_general /home/philip/Source/emacs/src/eval.c:2947
    #22 0x55dbfb723553 in Ffuncall /home/philip/Source/emacs/src/eval.c:2997
    #23 0x55dbfb72241c in run_hook_wrapped_funcall /home/philip/Source/emacs/src/eval.c:2775
    #24 0x55dbfb72286b in run_hook_with_args /home/philip/Source/emacs/src/eval.c:2856
    #25 0x55dbfb7224af in Frun_hook_wrapped /home/philip/Source/emacs/src/eval.c:2790
    #26 0x55dbfb7242e9 in funcall_subr /home/philip/Source/emacs/src/eval.c:3061
    #27 0x55dbfb7c0a90 in exec_byte_code /home/philip/Source/emacs/src/bytecode.c:809
    #28 0x55dbfb72446b in fetch_and_exec_byte_code /home/philip/Source/emacs/src/eval.c:3083
    #29 0x55dbfb724c00 in funcall_lambda /home/philip/Source/emacs/src/eval.c:3155
    #30 0x55dbfb7230ac in funcall_general /home/philip/Source/emacs/src/eval.c:2947
    #31 0x55dbfb723553 in Ffuncall /home/philip/Source/emacs/src/eval.c:2997
    #32 0x55dbfb57b1a5 in call1 /home/philip/Source/emacs/src/lisp.h:3247
    #33 0x55dbfb581f85 in Fkill_emacs /home/philip/Source/emacs/src/emacs.c:2884
    #34 0x55dbfb723a9e in funcall_subr /home/philip/Source/emacs/src/eval.c:3038
    #35 0x55dbfb7c0a90 in exec_byte_code /home/philip/Source/emacs/src/bytecode.c:809
    #36 0x55dbfb72446b in fetch_and_exec_byte_code /home/philip/Source/emacs/src/eval.c:3083
    #37 0x55dbfb724c00 in funcall_lambda /home/philip/Source/emacs/src/eval.c:3155
    #38 0x55dbfb7230ac in funcall_general /home/philip/Source/emacs/src/eval.c:2947
    #39 0x55dbfb723553 in Ffuncall /home/philip/Source/emacs/src/eval.c:2997
    #40 0x55dbfb70f126 in Ffuncall_interactively /home/philip/Source/emacs/src/callint.c:250
    #41 0x55dbfb7242e9 in funcall_subr /home/philip/Source/emacs/src/eval.c:3061
    #42 0x55dbfb723060 in funcall_general /home/philip/Source/emacs/src/eval.c:2943
    #43 0x55dbfb723553 in Ffuncall /home/philip/Source/emacs/src/eval.c:2997
    #44 0x55dbfb7130fa in Fcall_interactively /home/philip/Source/emacs/src/callint.c:787
    #45 0x55dbfb723b6b in funcall_subr /home/philip/Source/emacs/src/eval.c:3040
    #46 0x55dbfb7c0a90 in exec_byte_code /home/philip/Source/emacs/src/bytecode.c:809
    #47 0x55dbfb72446b in fetch_and_exec_byte_code /home/philip/Source/emacs/src/eval.c:3083
    #48 0x55dbfb724c00 in funcall_lambda /home/philip/Source/emacs/src/eval.c:3155
    #49 0x55dbfb7230ac in funcall_general /home/philip/Source/emacs/src/eval.c:2947
    #50 0x55dbfb723553 in Ffuncall /home/philip/Source/emacs/src/eval.c:2997
    #51 0x55dbfb58481a in call1 /home/philip/Source/emacs/src/lisp.h:3247
    #52 0x55dbfb58b2d4 in command_loop_1 /home/philip/Source/emacs/src/keyboard.c:1494
    #53 0x55dbfb71b9c0 in internal_condition_case /home/philip/Source/emacs/src/eval.c:1474
    #54 0x55dbfb58985d in command_loop_2 /home/philip/Source/emacs/src/keyboard.c:1124
    #55 0x55dbfb71a346 in internal_catch /home/philip/Source/emacs/src/eval.c:1197
    #56 0x55dbfb589785 in command_loop /home/philip/Source/emacs/src/keyboard.c:1102
    #57 0x55dbfb5880eb in recursive_edit_1 /home/philip/Source/emacs/src/keyboard.c:711
    #58 0x55dbfb5884af in Frecursive_edit /home/philip/Source/emacs/src/keyboard.c:794
    #59 0x55dbfb580b41 in main /home/philip/Source/emacs/src/emacs.c:2530
    #60 0x7fa970438189 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #61 0x7fa970438244 in __libc_start_main_impl ../csu/libc-start.c:381
    #62 0x55dbfb280830 in _start (/home/philip/Source/emacs/src/emacs+0x132830)

Address 0x7ffe72b89e70 is located in stack of thread T0
SUMMARY: AddressSanitizer: bad-free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 in __interceptor_free
==74401==ABORTING


[-- Attachment #3: Type: text/plain, Size: 3374 bytes --]


I ran the same command in batch mode, and now the issue appears to be
fixed.  This gives me no reassurance, as a few days ago the I had
temporary managed to acchive the same state and then Emacs crashed again
after rebuilding again.

In GNU Emacs 30.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
 3.24.36, cairo version 1.16.0) of 2023-03-01 built on quetzal
Repository revision: 4b99015e15a23bd5cbec021d53ef9fcca25b2441
Repository branch: master
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-pgtk 'CFLAGS=-O0 -ggdb3 -fsanitize=address''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort emacsbug mail-extr message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
cus-edit pp cus-start cus-load icons wid-edit misearch multi-isearch
vc-git diff-mode easy-mmode vc-dispatcher cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
term/common-win pgtk-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo gtk pgtk lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 65648 11383)
 (symbols 48 7380 0)
 (strings 32 19680 1617)
 (string-bytes 1 540967)
 (vectors 16 12795)
 (vector-slots 8 182734 13738)
 (floats 8 32 68)
 (intervals 56 625 8)
 (buffers 984 13))

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Date: Wed, 01 Mar 2023 20:25:11 +0000
> 
> Emacs just crashes out of nowhere, e.g. after I open a my init file.

It would help if you could run Emacs under GDB and show the backtrace
from one of those crashes, including the Lisp backtrace (the
"xbacktrace" command defined on src/.gdbinit).

> I have had this device for a while on a device of mine, that I couldn't
> reproduce on my main workstation or using emacs -Q.  Apparently this
> could be related to some faulty byte-code.
> 
> The best I could do to detect this issue was to build Emacs using
> -fsanitize=address and I managed to reprodce the issue reliably by
> invoking package-recompile-all.  I collected the following log:

Byte-code saw quite a bit of changes on master.  Adding Mattias in
case he has some ideas.

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Date: Wed, 01 Mar 2023 20:25:11 +0000
>> 
>> Emacs just crashes out of nowhere, e.g. after I open a my init file.
>
> It would help if you could run Emacs under GDB and show the backtrace
> from one of those crashes, including the Lisp backtrace (the
> "xbacktrace" command defined on src/.gdbinit).

I tried debugging it using GDB, but didn't know about xbacktrace.  Sadly
I cannot reproduce the issue any more (at least for now).

>> I have had this device for a while on a device of mine, that I couldn't
>> reproduce on my main workstation or using emacs -Q.  Apparently this
>> could be related to some faulty byte-code.
>> 
>> The best I could do to detect this issue was to build Emacs using
>> -fsanitize=address and I managed to reprodce the issue reliably by
>> invoking package-recompile-all.  I collected the following log:
>
> Byte-code saw quite a bit of changes on master.  Adding Mattias in
> case he has some ideas.

From what I recall, the address being freed was on the stack.  How does
the byte-code interpreter behave when the input is broken?  Is there
some way of validating if the byte-code is "coherent"?  If I manually
modify the byte code and replace random bytes, is the interpreter
written to expect this kind of issue?

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Mattias Engdegård <mattiase@acm.org>,
>   61896@debbugs.gnu.org
> Date: Thu, 02 Mar 2023 08:53:54 +0000
> 
> >From what I recall, the address being freed was on the stack.  How does
> the byte-code interpreter behave when the input is broken?  Is there
> some way of validating if the byte-code is "coherent"?  If I manually
> modify the byte code and replace random bytes, is the interpreter
> written to expect this kind of issue?

Sorry, I don't understand the questions.  Maybe Mattias will.

My interpretation of this problem is that some corruption happened to
the specpdl stuff, which causes SAFE_FREE decide that some data should
be 'free'd when it was actually allocated off the stack.  The question
is how could that happen.

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Rah Guzar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I encountered something very similar today after updating emacs.
In my case the crash was caused by trying to read an email from mu4e,
which I have installed as a systems package.

Like you I could everything worked fine with `emacs -Q`. After adding,
mu4e to the load path, I could load it and read messages successfully.
But with my own configuration it crashed even after removing all mu4e
related settings from my config. Emacs crashed and I could see the
following on the terminal I launched it from

free(): invalid pointer
Fatal error 6: Aborted

along with a backtrace.

After finding this thread, I copied the mu4e lisp files to a directory
writable by me and byte compiled those. Adding this directory to load-path
has fixed my problem.

I think the distro provided elc files were compiled by Emacs 28
and I am using a build of emacs 29 and some incompatible change
recently caused this problem.

For now, my fix works but is there a good way to deal with possibly
incompatible bytecode in site-lisp directory?

Rah Guzar

Philip Kaludercic <philipk@posteo.net> writes:

> Emacs just crashes out of nowhere, e.g. after I open a my init file.
>
> I have had this device for a while on a device of mine, that I couldn't
> reproduce on my main workstation or using emacs -Q.  Apparently this
> could be related to some faulty byte-code.
>
> The best I could do to detect this issue was to build Emacs using
> -fsanitize=address and I managed to reprodce the issue reliably by
> invoking package-recompile-all.  I collected the following log:
>
>
>
> I ran the same command in batch mode, and now the issue appears to be
> fixed.  This gives me no reassurance, as a few days ago the I had
> temporary managed to acchive the same state and then Emacs crashed again
> after rebuilding again.
>
> In GNU Emacs 30.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
>  3.24.36, cairo version 1.16.0) of 2023-03-01 built on quetzal
> Repository revision: 4b99015e15a23bd5cbec021d53ef9fcca25b2441
> Repository branch: master
> System Description: Debian GNU/Linux bookworm/sid
>
> Configured using:
>  'configure --with-pgtk 'CFLAGS=-O0 -ggdb3 -fsanitize=address''
>
> Configured features:
> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
> JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY
> PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
> TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB
>
> Important settings:
>   value of $LC_MONETARY: en_US.UTF-8
>   value of $LC_NUMERIC: en_US.UTF-8
>   value of $LC_TIME: en_US.UTF-8
>   value of $LANG: en_US.UTF-8
>   value of $XMODIFIERS: @im=ibus
>   locale-coding-system: utf-8-unix
>
> Major mode: ELisp/l
>
> Minor modes in effect:
>   tooltip-mode: t
>   global-eldoc-mode: t
>   eldoc-mode: t
>   show-paren-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   tool-bar-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>
> Load-path shadows:
> None found.
>
> Features:
> (shadow sort emacsbug mail-extr message mailcap yank-media puny dired
> dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
> epg-config gnus-util text-property-search time-date subr-x mm-decode
> mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
> sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
> cus-edit pp cus-start cus-load icons wid-edit misearch multi-isearch
> vc-git diff-mode easy-mmode vc-dispatcher cl-loaddefs cl-lib rmc
> iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
> vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
> term/common-win pgtk-dnd tool-bar dnd fontset image regexp-opt fringe
> tabulated-list replace newcomment text-mode lisp-mode prog-mode register
> page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
> scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
> frame minibuffer nadvice seq simple cl-generic indonesian philippine
> cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
> european ethiopic indian cyrillic chinese composite emoji-zwj charscript
> charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
> cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
> files window text-properties overlay sha1 md5 base64 format env
> code-pages mule custom widget keymap hashtable-print-readable backquote
> threads dbusbind inotify dynamic-setting system-font-setting
> font-render-setting cairo gtk pgtk lcms2 multi-tty make-network-process
> emacs)
>
> Memory information:
> ((conses 16 65648 11383)
>  (symbols 48 7380 0)
>  (strings 32 19680 1617)
>  (string-bytes 1 540967)
>  (vectors 16 12795)
>  (vector-slots 8 182734 13738)
>  (floats 8 32 68)
>  (intervals 56 625 8)
>  (buffers 984 13))

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Philip Kaludercic]

Rah Guzar <rahguzar@zohomail.eu> writes:

> I encountered something very similar today after updating emacs.
> In my case the crash was caused by trying to read an email from mu4e,
> which I have installed as a systems package.
>
> Like you I could everything worked fine with `emacs -Q`. After adding,
> mu4e to the load path, I could load it and read messages successfully.
> But with my own configuration it crashed even after removing all mu4e
> related settings from my config. Emacs crashed and I could see the
> following on the terminal I launched it from
>
> free(): invalid pointer
> Fatal error 6: Aborted
>
> along with a backtrace.

This was exactly the issue I had, though in my case the byte code was
not from a site directory.

Could you start Emacs using GDB print and run the xbacktrace command
that is defined in emacs.git's src/.gdbinit file (I believe this is best
done by starting GDB within the src directory)?

> After finding this thread, I copied the mu4e lisp files to a directory
> writable by me and byte compiled those. Adding this directory to load-path
> has fixed my problem.
>
> I think the distro provided elc files were compiled by Emacs 28
> and I am using a build of emacs 29 and some incompatible change
> recently caused this problem.
>
> For now, my fix works but is there a good way to deal with possibly
> incompatible bytecode in site-lisp directory?
>
> Rah Guzar
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Emacs just crashes out of nowhere, e.g. after I open a my init file.
>>
>> I have had this device for a while on a device of mine, that I couldn't
>> reproduce on my main workstation or using emacs -Q.  Apparently this
>> could be related to some faulty byte-code.
>>
>> The best I could do to detect this issue was to build Emacs using
>> -fsanitize=address and I managed to reprodce the issue reliably by
>> invoking package-recompile-all.  I collected the following log:
>>
>>
>>
>> I ran the same command in batch mode, and now the issue appears to be
>> fixed.  This gives me no reassurance, as a few days ago the I had
>> temporary managed to acchive the same state and then Emacs crashed again
>> after rebuilding again.
>>
>> In GNU Emacs 30.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
>>  3.24.36, cairo version 1.16.0) of 2023-03-01 built on quetzal
>> Repository revision: 4b99015e15a23bd5cbec021d53ef9fcca25b2441
>> Repository branch: master
>> System Description: Debian GNU/Linux bookworm/sid
>>
>> Configured using:
>>  'configure --with-pgtk 'CFLAGS=-O0 -ggdb3 -fsanitize=address''
>>
>> Configured features:
>> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
>> JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY
>> PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
>> TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB
>>
>> Important settings:
>>   value of $LC_MONETARY: en_US.UTF-8
>>   value of $LC_NUMERIC: en_US.UTF-8
>>   value of $LC_TIME: en_US.UTF-8
>>   value of $LANG: en_US.UTF-8
>>   value of $XMODIFIERS: @im=ibus
>>   locale-coding-system: utf-8-unix
>>
>> Major mode: ELisp/l
>>
>> Minor modes in effect:
>>   tooltip-mode: t
>>   global-eldoc-mode: t
>>   eldoc-mode: t
>>   show-paren-mode: t
>>   electric-indent-mode: t
>>   mouse-wheel-mode: t
>>   tool-bar-mode: t
>>   menu-bar-mode: t
>>   file-name-shadow-mode: t
>>   global-font-lock-mode: t
>>   font-lock-mode: t
>>   blink-cursor-mode: t
>>   line-number-mode: t
>>   transient-mark-mode: t
>>   auto-composition-mode: t
>>   auto-encryption-mode: t
>>   auto-compression-mode: t
>>
>> Load-path shadows:
>> None found.
>>
>> Features:
>> (shadow sort emacsbug mail-extr message mailcap yank-media puny dired
>> dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
>> epg-config gnus-util text-property-search time-date subr-x mm-decode
>> mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
>> sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
>> cus-edit pp cus-start cus-load icons wid-edit misearch multi-isearch
>> vc-git diff-mode easy-mmode vc-dispatcher cl-loaddefs cl-lib rmc
>> iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
>> vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
>> term/common-win pgtk-dnd tool-bar dnd fontset image regexp-opt fringe
>> tabulated-list replace newcomment text-mode lisp-mode prog-mode register
>> page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
>> scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
>> frame minibuffer nadvice seq simple cl-generic indonesian philippine
>> cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
>> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
>> european ethiopic indian cyrillic chinese composite emoji-zwj charscript
>> charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
>> cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
>> files window text-properties overlay sha1 md5 base64 format env
>> code-pages mule custom widget keymap hashtable-print-readable backquote
>> threads dbusbind inotify dynamic-setting system-font-setting
>> font-render-setting cairo gtk pgtk lcms2 multi-tty make-network-process
>> emacs)
>>
>> Memory information:
>> ((conses 16 65648 11383)
>>  (symbols 48 7380 0)
>>  (strings 32 19680 1617)
>>  (string-bytes 1 540967)
>>  (vectors 16 12795)
>>  (vector-slots 8 182734 13738)
>>  (floats 8 32 68)
>>  (intervals 56 625 8)
>>  (buffers 984 13))

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Mattias Engdegård]

2 mars 2023 kl. 09.53 skrev Philip Kaludercic <philipk@posteo.net>:

>> Byte-code saw quite a bit of changes on master.  Adding Mattias in
>> case he has some ideas.
> 
> From what I recall, the address being freed was on the stack.  How does
> the byte-code interpreter behave when the input is broken?  Is there
> some way of validating if the byte-code is "coherent"?  If I manually
> modify the byte code and replace random bytes, is the interpreter
> written to expect this kind of issue?

The very first thing is to make sure you don't have any lingering *.elc files generated during the period of incompatibility regarding `save-restriction`. That issue should have been resolved by now; let's not chase ghosts. The indication of a specpdl imbalance does point to this being a possible cause.

The byte-code interpreter normally assumes the code to be correct and performs few checks since every cycle counts here. There are some additional checks to be enabled: the general --enable-checking=all, and/or compiling with -DBYTE_CODE_SAFE=1 (or just adding

#define BYTE_CODE_SAFE 1

early in bytecode.c, which is what I tend to do).

These checks do not audit the specpdl balance directly but that would be something to add if you don't make further progress.

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 200 bytes --]

> These checks do not audit the specpdl balance directly but that would be something to add if you don't make further progress.

You could try this patch if you build with --enable-checking=all:


[-- Attachment #2: bytecode-specpdl-depth-check.diff --]
[-- Type: application/octet-stream, Size: 1659 bytes --]

diff --git a/src/bytecode.c b/src/bytecode.c
index 74a94859aba..8f30fa55829 100644
--- a/src/bytecode.c
+++ b/src/bytecode.c
@@ -382,6 +382,9 @@ #define BC_STACK_SIZE (512 * 1024 * sizeof (Lisp_Object))
   const unsigned char *saved_pc;    /* previous program counter */
 
   Lisp_Object fun;                  /* current function object */
+#ifdef ENABLE_CHECKING
+  specpdl_ref entry_specpdl_depth;  /* specpdl depth at function entry */
+#endif
 
   Lisp_Object next_stack[];	    /* data stack of next frame */
 };
@@ -484,10 +487,6 @@ exec_byte_code (Lisp_Object fun, ptrdiff_t args_template,
  setup_frame: ;
   eassert (!STRING_MULTIBYTE (bytestr));
   eassert (string_immovable_p (bytestr));
-  /* FIXME: in debug mode (!NDEBUG, BYTE_CODE_SAFE or enabled checking),
-     save the specpdl index on function entry and check that it is the same
-     when returning, to detect unwind imbalances.  This would require adding
-     a field to the frame header.  */
 
   Lisp_Object vector = AREF (fun, COMPILED_CONSTANTS);
   Lisp_Object maxdepth = AREF (fun, COMPILED_STACK_DEPTH);
@@ -511,6 +510,9 @@ exec_byte_code (Lisp_Object fun, ptrdiff_t args_template,
   fp->saved_pc = pc;
   fp->saved_fp = bc->fp;
   bc->fp = fp;
+#ifdef ENABLE_CHECKING
+  fp->entry_specpdl_depth = SPECPDL_INDEX ();
+#endif
 
   top = frame_base - 1;
   unsigned char const *bytestr_data = SDATA (bytestr);
@@ -878,6 +880,8 @@ #define DEFINE(name, value) [name] = &&insn_ ## name,
 
 	CASE (Breturn):
 	  {
+	    eassert (specpdl_ref_eq (bc->fp->entry_specpdl_depth,
+				     SPECPDL_INDEX ()));
 	    Lisp_Object *saved_top = bc->fp->saved_top;
 	    if (saved_top)
 	      {

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Philip Kaludercic]

Mattias Engdegård <mattiase@acm.org> writes:

>> These checks do not audit the specpdl balance directly but that would be something to add if you don't make further progress.
>
> You could try this patch if you build with --enable-checking=all:

As I mentioned in my other response, I cannot reproduce the issue for
now.  Rah mentioned that he still has the files that caused the issue,
so perhaps he can help?

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Rah Guzar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I have never used gdb before so I will need to figure that out. I am traveling
today so this will not happen before Monday or Tuesday. But I will try it
sometime next week.


Philip Kaludercic <philipk@posteo.net> writes:

> Rah Guzar <rahguzar@zohomail.eu> writes:
>
>> I encountered something very similar today after updating emacs.
>> In my case the crash was caused by trying to read an email from mu4e,
>> which I have installed as a systems package.
>>
>> Like you I could everything worked fine with `emacs -Q`. After adding,
>> mu4e to the load path, I could load it and read messages successfully.
>> But with my own configuration it crashed even after removing all mu4e
>> related settings from my config. Emacs crashed and I could see the
>> following on the terminal I launched it from
>>
>> free(): invalid pointer
>> Fatal error 6: Aborted
>>
>> along with a backtrace.
>
> This was exactly the issue I had, though in my case the byte code was
> not from a site directory.
>
> Could you start Emacs using GDB print and run the xbacktrace command
> that is defined in emacs.git's src/.gdbinit file (I believe this is best
> done by starting GDB within the src directory)?
>
>> After finding this thread, I copied the mu4e lisp files to a directory
>> writable by me and byte compiled those. Adding this directory to load-path
>> has fixed my problem.
>>
>> I think the distro provided elc files were compiled by Emacs 28
>> and I am using a build of emacs 29 and some incompatible change
>> recently caused this problem.
>>
>> For now, my fix works but is there a good way to deal with possibly
>> incompatible bytecode in site-lisp directory?
>>
>> Rah Guzar
>>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Emacs just crashes out of nowhere, e.g. after I open a my init file.
>>>
>>> I have had this device for a while on a device of mine, that I couldn't
>>> reproduce on my main workstation or using emacs -Q.  Apparently this
>>> could be related to some faulty byte-code.
>>>
>>> The best I could do to detect this issue was to build Emacs using
>>> -fsanitize=address and I managed to reprodce the issue reliably by
>>> invoking package-recompile-all.  I collected the following log:
>>>
>>>
>>>
>>> I ran the same command in batch mode, and now the issue appears to be
>>> fixed.  This gives me no reassurance, as a few days ago the I had
>>> temporary managed to acchive the same state and then Emacs crashed again
>>> after rebuilding again.
>>>
>>> In GNU Emacs 30.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
>>>  3.24.36, cairo version 1.16.0) of 2023-03-01 built on quetzal
>>> Repository revision: 4b99015e15a23bd5cbec021d53ef9fcca25b2441
>>> Repository branch: master
>>> System Description: Debian GNU/Linux bookworm/sid
>>>
>>> Configured using:
>>>  'configure --with-pgtk 'CFLAGS=-O0 -ggdb3 -fsanitize=address''
>>>
>>> Configured features:
>>> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
>>> JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY
>>> PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
>>> TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB
>>>
>>> Important settings:
>>>   value of $LC_MONETARY: en_US.UTF-8
>>>   value of $LC_NUMERIC: en_US.UTF-8
>>>   value of $LC_TIME: en_US.UTF-8
>>>   value of $LANG: en_US.UTF-8
>>>   value of $XMODIFIERS: @im=ibus
>>>   locale-coding-system: utf-8-unix
>>>
>>> Major mode: ELisp/l
>>>
>>> Minor modes in effect:
>>>   tooltip-mode: t
>>>   global-eldoc-mode: t
>>>   eldoc-mode: t
>>>   show-paren-mode: t
>>>   electric-indent-mode: t
>>>   mouse-wheel-mode: t
>>>   tool-bar-mode: t
>>>   menu-bar-mode: t
>>>   file-name-shadow-mode: t
>>>   global-font-lock-mode: t
>>>   font-lock-mode: t
>>>   blink-cursor-mode: t
>>>   line-number-mode: t
>>>   transient-mark-mode: t
>>>   auto-composition-mode: t
>>>   auto-encryption-mode: t
>>>   auto-compression-mode: t
>>>
>>> Load-path shadows:
>>> None found.
>>>
>>> Features:
>>> (shadow sort emacsbug mail-extr message mailcap yank-media puny dired
>>> dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
>>> epg-config gnus-util text-property-search time-date subr-x mm-decode
>>> mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
>>> sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
>>> cus-edit pp cus-start cus-load icons wid-edit misearch multi-isearch
>>> vc-git diff-mode easy-mmode vc-dispatcher cl-loaddefs cl-lib rmc
>>> iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
>>> vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
>>> term/common-win pgtk-dnd tool-bar dnd fontset image regexp-opt fringe
>>> tabulated-list replace newcomment text-mode lisp-mode prog-mode register
>>> page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
>>> scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
>>> frame minibuffer nadvice seq simple cl-generic indonesian philippine
>>> cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
>>> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
>>> european ethiopic indian cyrillic chinese composite emoji-zwj charscript
>>> charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
>>> cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
>>> files window text-properties overlay sha1 md5 base64 format env
>>> code-pages mule custom widget keymap hashtable-print-readable backquote
>>> threads dbusbind inotify dynamic-setting system-font-setting
>>> font-render-setting cairo gtk pgtk lcms2 multi-tty make-network-process
>>> emacs)
>>>
>>> Memory information:
>>> ((conses 16 65648 11383)
>>>  (symbols 48 7380 0)
>>>  (strings 32 19680 1617)
>>>  (string-bytes 1 540967)
>>>  (vectors 16 12795)
>>>  (vector-slots 8 182734 13738)
>>>  (floats 8 32 68)
>>>  (intervals 56 625 8)
>>>  (buffers 984 13))

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Philip Kaludercic]

Rah Guzar <rahguzar@zohomail.eu> writes:

> I have never used gdb before so I will need to figure that out. I am traveling
> today so this will not happen before Monday or Tuesday. But I will try it
> sometime next week.

It shouldn't be that difficult, I usually first reconfigure Emacs with
debugging information:

   $ pwd
   /home/user/src/emacs/
   $ ./configure CFLAGS="-ggdb3"

Then all you need to do is to start Emacs in the src sub-directory using
GDB and then try to provoke the bug:

   $ pwd
   /home/user/src/emacs/src
   $ gdb emacs |& tee error.log
   ... # Copyright and stuff here.
   (gdb) run -Q
   ... # Emacs is running now and you can load the broken bytecode.
       # The next GDB prompt will appear when Emacs aborts.
   (gdb) xbacktrace
   ... # The Lisp backtrace should appear here.
   (gdb) quit
   
If you do this, then the entire session log should be store in the
"error.log" file.

-- 
Philip Kaludercic

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Rah Guzar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi!
  I tried again today and I can no longer reproduce the bug with the bytecode
  in the site-lisp directory. The byte code hasn't been updated but my emacs
  was updated yesterday. I have also tried building emacs-29 from source and
  that also works fine. So it seems like the problem has been fixed.

Philip Kaludercic <philipk@posteo.net> writes:

> Rah Guzar <rahguzar@zohomail.eu> writes:
>
>> I have never used gdb before so I will need to figure that out. I am traveling
>> today so this will not happen before Monday or Tuesday. But I will try it
>> sometime next week.
>
> It shouldn't be that difficult, I usually first reconfigure Emacs with
> debugging information:
>
>    $ pwd
>    /home/user/src/emacs/
>    $ ./configure CFLAGS="-ggdb3"
>
> Then all you need to do is to start Emacs in the src sub-directory using
> GDB and then try to provoke the bug:
>
>    $ pwd
>    /home/user/src/emacs/src
>    $ gdb emacs |& tee error.log
>    ... # Copyright and stuff here.
>    (gdb) run -Q
>    ... # Emacs is running now and you can load the broken bytecode.
>        # The next GDB prompt will appear when Emacs aborts.
>    (gdb) xbacktrace
>    ... # The Lisp backtrace should appear here.
>    (gdb) quit
>
> If you do this, then the entire session log should be store in the
> "error.log" file.

bug#61896: 30.0.50; Emacs crashes because of an invalid free

[Stefan Kangas]

Rah Guzar <rahguzar@zohomail.eu> writes:

> Hi!
>   I tried again today and I can no longer reproduce the bug with the bytecode
>   in the site-lisp directory. The byte code hasn't been updated but my emacs
>   was updated yesterday. I have also tried building emacs-29 from source and
>   that also works fine. So it seems like the problem has been fixed.

Thanks, I'm therefore closing this bug report.