bug#55654: 28.1.50; HEAP Error with ImageMagick on Windows MSYS2

bug#55654: 28.1.50; HEAP Error with ImageMagick on Windows MSYS2

[Fabio Leimgruber]

Compiled Emacs with these options:

--host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --with-wide-int --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2 --with-gnutls --with-xft --with-imagemagick --with-modules --with-json --with-native-compilation LDFLAGS=-fstack-protector

The error is triggered when I do find-file, which uses helm and treemacs in turn loading images via ImageMagick.

Running Emacs under GDB shows the backtrace below.

Any hints on debugging this further?

#+begin_example
warning: HEAP[emacs.exe]:
warning: Invalid address specified to RtlValidateHeap( 000001EE1E190000, 000001EE25BC1B80 )

Thread 1 received signal SIGTRAP, Trace/breakpoint trap.
0x00007ff99eaca313 in ntdll!RtlRegisterSecureMemoryCacheCallback () from C:\WINDOWS\SYSTEM32\ntdll.dll
(gdb) bt
#0  0x00007ff99eaca313 in ntdll!RtlRegisterSecureMemoryCacheCallback () from C:\WINDOWS\SYSTEM32\ntdll.dll
#1  0x00007ff99ea8cd24 in ntdll!memset () from C:\WINDOWS\SYSTEM32\ntdll.dll
#2  0x00007ff99ea2e115 in ntdll!RtlValidateHeap () from C:\WINDOWS\SYSTEM32\ntdll.dll
#3  0x00007ff92596750f in NotifyShims () from C:\WINDOWS\SYSTEM32\AcLayers.dll
#4  0x00007ff99e269c9c in msvcrt!free () from C:\WINDOWS\System32\msvcrt.dll
#5  0x00007ff92575b662 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#6  0x00007ff92575ce15 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#7  0x00007ff9258e6269 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#8  0x00007ff9258e7a91 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#9  0x00007ff9258e7cbb in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#10 0x00007ff9258e7e97 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#11 0x00007ff90f674ee7 in ?? ()
   from C:\Users\LeimgruberF\opt\msys64\mingw64\lib\ImageMagick-7.0.10\modules-Q16HDRI\coders\png.dll
#12 0x00007ff90f677230 in ?? ()
   from C:\Users\LeimgruberF\opt\msys64\mingw64\lib\ImageMagick-7.0.10\modules-Q16HDRI\coders\png.dll
#13 0x00007ff925783ca1 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll
#14 0x00007ff9256823b6 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickWand-7.Q16HDRI-7.dll
#15 0x00007ff6095bac81 in imagemagick_load_image (f=f@entry=0x1ee2139ad80, img=img@entry=0x1ee3f390a10,
    contents=contents@entry=0x0, size=size@entry=0,
    filename=0x1ee291a35f8 "c:/Users/LeimgruberF/.emacs.d/elpa/28.1/develop/treemacs-20220519.1908/icons/default/vsc/dir-closed.png") at image.c:9139
#16 0x00007ff6095bbc7a in imagemagick_load (f=0x1ee2139ad80, img=0x1ee3f390a10) at image.c:9497
#17 0x00007ff6095bfd62 in lookup_image (f=0x1ee2139ad80, spec=spec@entry=0x1ee304453d3, face_id=<optimized out>)
    at image.c:2469
#18 0x00007ff6093e927b in handle_single_display_spec (it=it@entry=0xab88ff68b0, spec=<optimized out>,
    spec@entry=0x1ee304453d3, object=<optimized out>, object@entry=0x1ee27e79825, overlay=overlay@entry=0x0,
    position=position@entry=0xab88ff69e8, bufpos=bufpos@entry=12, display_replaced=display_replaced@entry=0,
    frame_window_p=frame_window_p@entry=true, enable_eval_p=true) at xdisp.c:5820
#19 0x00007ff6093e9978 in handle_display_spec (it=it@entry=0xab88ff68b0, spec=<optimized out>,
--Type <RET> for more, q to quit, c to continue without paging--c
    object=object@entry=0x1ee27e79825, overlay=0x0, position=position@entry=0xab88ff69e8, bufpos=bufpos@entry=12, frame_window_p=true) at xdisp.c:5301
#20 0x00007ff6093e9b6e in handle_display_prop (it=0xab88ff68b0) at xdisp.c:5210
#21 0x00007ff6093ecb65 in handle_stop (it=it@entry=0xab88ff68b0) at xdisp.c:3870
#22 0x00007ff6093f47d8 in next_element_from_buffer (it=0xab88ff68b0) at xdisp.c:8943
#23 0x00007ff6093f2775 in get_next_display_element (it=it@entry=0xab88ff68b0) at xdisp.c:7530
#24 0x00007ff6093fc58b in display_line (it=it@entry=0xab88ff68b0, cursor_vpos=cursor_vpos@entry=0) at xdisp.c:23693
#25 0x00007ff609401e5b in try_window (window=window@entry=0x1ee350b7aa5, pos=..., flags=<optimized out>, flags@entry=1) at xdisp.c:19597
#26 0x00007ff60941761a in redisplay_window (window=0x1ee350b7aa5, just_this_one_p=just_this_one_p@entry=false) at xdisp.c:19004
#27 0x00007ff60941a1f7 in redisplay_window_0 (window=<optimized out>) at xdisp.c:16714
#28 0x00007ff6094f60a5 in internal_condition_case_1 (bfun=bfun@entry=0x7ff60941a1c0 <redisplay_window_0>, arg=arg@entry=0x1ee350b7aa5, handlers=<optimized out>, hfun=hfun@entry=0x7ff6093d7850 <redisplay_window_error>) at eval.c:1474
#29 0x00007ff6093dafbd in redisplay_windows (window=0x1ee350b7aa5) at xdisp.c:16694
#30 0x00007ff6093daf7a in redisplay_windows (window=0x1ee34e8ced5) at xdisp.c:16688
#31 0x00007ff609404c33 in redisplay_internal () at xdisp.c:16162
#32 0x00007ff609406c35 in redisplay () at xdisp.c:15369
#33 0x00007ff609485912 in read_char (commandflag=<optimized out>, map=<optimized out>, map@entry=0x1ee26c3b723, prev_event=<optimized out>, used_mouse_menu=<optimized out>, used_mouse_menu@entry=0xab88ffd26b, end_time=<optimized out>, end_time@entry=0x0) at keyboard.c:2555
#34 0x00007ff6094884db in read_key_sequence (keybuf=keybuf@entry=0xab88ffd3c0, prompt=prompt@entry=0x0, dont_downcase_last=dont_downcase_last@entry=false, can_return_switch_frame=can_return_switch_frame@entry=true, fix_current_buffer=fix_current_buffer@entry=true, prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9635
#35 0x00007ff60948a045 in command_loop_1 () at keyboard.c:1392
#36 0x00007ff6094f600d in internal_condition_case (bfun=bfun@entry=0x7ff609489e40 <command_loop_1>, handlers=handlers@entry=0x90, hfun=hfun@entry=0x7ff609480f80 <cmd_error>) at eval.c:1450
#37 0x00007ff60947a7ce in command_loop_2 (handlers=0x90) at keyboard.c:1133
#38 0x00007ff6094f5f7b in internal_catch (tag=tag@entry=0x64e0, func=func@entry=0x7ff60947a7a0 <command_loop_2>, arg=arg@entry=0x90) at eval.c:1181
#39 0x00007ff60947a6af in command_loop () at keyboard.c:1103
#40 0x00007ff609480b34 in recursive_edit_1 () at keyboard.c:720
#41 0x00007ff6094a7e0c in read_minibuf (inherit_input_method=<optimized out>, allow_props=<optimized out>, defalt=<optimized out>, histpos=<optimized out>, histvar=0xae90, expflag=<optimized out>, prompt=<optimized out>, initial=<optimized out>, map=<optimized out>) at minibuf.c:899
#42 Fread_from_minibuffer (prompt=<optimized out>, initial_contents=<optimized out>, keymap=<optimized out>, sys_read=<optimized out>, hist=<optimized out>, default_value=<optimized out>, inherit_input_method=<optimized out>) at minibuf.c:1351
#43 0x00007ff91b2cdf07 in F68656c6d2d726561642d66726f6d2d6d696e69627566666572_helm_read_from_minibuffer_0 (par_0=<optimized out>, par_1=0x1ee2cb2d494, par_2=0x0, par_3=0x0, par_4=0x0, par_5=<optimized out>, par_6=0x0) from c:\Users\LeimgruberF\.emacs.d\eln-cache\28.1.50-93a37783\helm-core-6b42f8de-26353aa9.eln
#44 0x00007ff6094f8a20 in funcall_subr (subr=0x1ee2e46b3e0, numargs=numargs@entry=7, args=args@entry=0xab88ffdcc0) at eval.c:3118
#45 0x00007ff6094f70bf in Ffuncall (nargs=<optimized out>, args=<optimized out>) at eval.c:3023
#46 0x00007ff91b2c7b0d in F68656c6d2d696e7465726e616c_helm_internal_0 (nargs=<optimized out>, args=<optimized out>) from c:\Users\LeimgruberF\.emacs.d\eln-cache\28.1.50-93a37783\helm-core-6b42f8de-26353aa9.eln
#47 0x00007ff6094f70bf in Ffuncall (nargs=nargs@entry=10, args=args@entry=0xab88ffdde0) at eval.c:3023
#48 0x00007ff6094f92a0 in Fapply (nargs=2, args=0xab88ffdef0) at eval.c:2653
#49 0x00007ff91b2c6a9a in F68656c6d_helm_0 (nargs=<optimized out>, args=<optimized out>) from c:\Users\LeimgruberF\.emacs.d\eln-cache\28.1.50-93a37783\helm-core-6b42f8de-26353aa9.eln
#50 0x00007ff6094f70bf in Ffuncall (nargs=nargs@entry=10, args=args@entry=0xab88ffdfc0) at eval.c:3023
#51 0x00007ff6094f92a0 in Fapply (nargs=2, args=0xab88ffe0d0) at eval.c:2653
#52 0x00007ff91b2c6a4d in F68656c6d_helm_0 (nargs=<optimized out>, args=<optimized out>) from c:\Users\LeimgruberF\.emacs.d\eln-cache\28.1.50-93a37783\helm-core-6b42f8de-26353aa9.eln
#53 0x00007ff6094f9a6d in eval_sub (form=<optimized out>) at eval.c:2470
#54 0x00007ff6094fb16c in Funwind_protect (args=0x1ee2fb9d1f3) at lisp.h:1420
#55 0x00007ff6094f9c08 in eval_sub (form=<optimized out>) at eval.c:2451
#56 0x00007ff6094fb09d in Fprogn (body=0x0) at eval.c:465
#57 FletX (args=0x1ee2fb9cb93) at eval.c:983
#58 0x00007ff6094f9c08 in eval_sub (form=<optimized out>) at eval.c:2451
#59 0x00007ff6094fa6dd in Fprogn (body=0x0) at eval.c:465
#60 funcall_lambda (fun=0x1ee2fb9c443, fun@entry=0x2f8, nargs=nargs@entry=1, arg_vector=arg_vector@entry=0xab88ffe600) at eval.c:3305
#61 0x00007ff6094fa8b9 in apply_lambda (fun=0x2f8, fun@entry=0x1ee2fb9c453, args=<optimized out>, count=2122375504051, count@entry=-138572329258512) at eval.c:3172
#62 0x00007ff6094f96e6 in eval_sub (form=<optimized out>) at eval.c:2575
#63 0x00007ff6094fb09d in Fprogn (body=0x0) at eval.c:465
#64 FletX (args=0x1ee27702013) at eval.c:983
#65 0x00007ff6094f9c08 in eval_sub (form=<optimized out>) at eval.c:2451
#66 0x00007ff6094fa6dd in Fprogn (body=0x0) at eval.c:465
#67 funcall_lambda (fun=0x1ee27702193, nargs=nargs@entry=1, arg_vector=arg_vector@entry=0xab88ffeaf0) at eval.c:3305
#68 0x00007ff6094f7079 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0xab88ffeae8) at eval.c:3039
#69 0x00007ff6094f36a6 in Ffuncall_interactively (nargs=2, args=0xab88ffeae8) at callint.c:260
#70 0x00007ff6094f70bf in Ffuncall (nargs=nargs@entry=2122256158061, args=args@entry=0x0) at eval.c:3023
#71 0x00007ff6094f49df in Fcall_interactively (function=<optimized out>, record_flag=<optimized out>, keys=<optimized out>) at callint.c:798
#72 0x00007ff6094f9bc3 in eval_sub (form=<optimized out>) at eval.c:2504
#73 0x00007ff6094fa6dd in Fprogn (body=0x0) at eval.c:465
#74 funcall_lambda (fun=0x1ee2ddf3253, nargs=nargs@entry=0, arg_vector=arg_vector@entry=0xab88fff0c0) at eval.c:3305
#75 0x00007ff6094f7079 in Ffuncall (nargs=nargs@entry=1, args=args@entry=0xab88fff0b8) at eval.c:3039
#76 0x00007ff6094f36a6 in Ffuncall_interactively (nargs=1, args=0xab88fff0b8) at callint.c:260
#77 0x00007ff6094f70bf in Ffuncall (nargs=nargs@entry=2, args=args@entry=0xab88fff0b0) at eval.c:3023
#78 0x00007ff6094f9360 in Fapply (nargs=nargs@entry=3, args=0xab88fff0b0, args@entry=0xab88fff1a0) at eval.c:2606
#79 0x00007ff6094f4cc7 in Fcall_interactively (function=0x1ee26ac8e90, record_flag=<optimized out>, keys=0x7ff6099f7940 <pending_malloc_warning>) at callint.c:353
#80 0x00007ff92472bd2f in F636f6d6d616e642d65786563757465_command_execute_0 (par_0=0xffff81f8243fb0b0, par_1=0x0, par_2=0x0, par_3=<optimized out>) from c:\Users\LeimgruberF\opt\emacs-28\lib\emacs\28.1.50\native-lisp\28.1.50-93a37783\preloaded\simple-fab5b0cf-0f48f2a4.eln
#81 0x00007ff6094f70bf in Ffuncall (nargs=nargs@entry=2, args=args@entry=0xab88fff290) at eval.c:3023
#82 0x00007ff6094f721d in call1 (fn=fn@entry=0x45f0, arg1=<optimized out>) at eval.c:2883
#83 0x00007ff60948a228 in command_loop_1 () at keyboard.c:1505
#84 0x00007ff6094f600d in internal_condition_case (bfun=bfun@entry=0x7ff609489e40 <command_loop_1>, handlers=handlers@entry=0x90, hfun=hfun@entry=0x7ff609480f80 <cmd_error>) at eval.c:1450
#85 0x00007ff60947a7ce in command_loop_2 (handlers=0x90) at keyboard.c:1133
#86 0x00007ff6094f5f7b in internal_catch (tag=tag@entry=0xf750, func=func@entry=0x7ff60947a7a0 <command_loop_2>, arg=arg@entry=0x90) at eval.c:1181
#87 0x00007ff60947a75c in command_loop () at keyboard.c:1111
#88 0x00007ff609480b34 in recursive_edit_1 () at keyboard.c:720
#89 0x00007ff609480ea9 in Frecursive_edit () at keyboard.c:803
#90 0x00007ff6095e3e6e in main (argc=<optimized out>, argv=0x1ee1f964170) at emacs.c:2354
#+end_example

bug#55654: 28.1.50; HEAP Error with ImageMagick on Windows MSYS2

[Eli Zaretskii]

> From: Fabio Leimgruber <fabio.leimgruber@web.de>
> Date: Thu, 26 May 2022 12:06:46 +0200
> 
> Compiled Emacs with these options:
> 
> --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --with-wide-int --with-jpeg --with-xpm --with-png --with-tiff --with-rsvg --with-xml2 --with-gnutls --with-xft --with-imagemagick --with-modules --with-json --with-native-compilation LDFLAGS=-fstack-protector
> 
> The error is triggered when I do find-file, which uses helm and treemacs in turn loading images via ImageMagick.
> 
> Running Emacs under GDB shows the backtrace below.
> 
> Any hints on debugging this further?
> 
> #+begin_example
> warning: HEAP[emacs.exe]:
> warning: Invalid address specified to RtlValidateHeap( 000001EE1E190000, 000001EE25BC1B80 )
> 
> Thread 1 received signal SIGTRAP, Trace/breakpoint trap.
> 0x00007ff99eaca313 in ntdll!RtlRegisterSecureMemoryCacheCallback () from C:\WINDOWS\SYSTEM32\ntdll.dll
> (gdb) bt
> #0  0x00007ff99eaca313 in ntdll!RtlRegisterSecureMemoryCacheCallback () from C:\WINDOWS\SYSTEM32\ntdll.dll
> #1  0x00007ff99ea8cd24 in ntdll!memset () from C:\WINDOWS\SYSTEM32\ntdll.dll
> #2  0x00007ff99ea2e115 in ntdll!RtlValidateHeap () from C:\WINDOWS\SYSTEM32\ntdll.dll
> #3  0x00007ff92596750f in NotifyShims () from C:\WINDOWS\SYSTEM32\AcLayers.dll
> #4  0x00007ff99e269c9c in msvcrt!free () from C:\WINDOWS\System32\msvcrt.dll
> #5  0x00007ff92575b662 in ?? () from C:\Users\LeimgruberF\opt\msys64\mingw64\bin\libMagickCore-7.Q16HDRI-7.dll

I think this happens because ImageMagick tries to free memory that
Emacs allocated.  Emacs on MS-Windows uses its own implementation of
malloc and free, so a scenario where Emacs allocates and some library
frees, or vice versa, will never work.

Does ImageMagick has a facility whereby an application can tell it
what functions to use for allocation and deallocation of memory?  If
so, we can use those facilities to force ImageMagick to use our
allocation functions.

If the above doesn't help, I think the way to debug this is to install
ImageMagick sources and debug info, or build it with debug information
on your machine, and see what memory is being free'd here, and where
was it allocated.

bug#55654: 28.1.50; HEAP Error with ImageMagick on Windows MSYS2

[Fabio Leimgruber]

Eli Zaretskii <eliz@gnu.org> writes:

> I think this happens because ImageMagick tries to free memory that
> Emacs allocated.  Emacs on MS-Windows uses its own implementation of
> malloc and free, so a scenario where Emacs allocates and some library
> frees, or vice versa, will never work.
>
> Does ImageMagick has a facility whereby an application can tell it
> what functions to use for allocation and deallocation of memory?  If
> so, we can use those facilities to force ImageMagick to use our
> allocation functions.
>
> If the above doesn't help, I think the way to debug this is to install
> ImageMagick sources and debug info, or build it with debug information
> on your machine, and see what memory is being free'd here, and where
> was it allocated.

Thanks for your analysis and the suggestions.

I just recompiled after upgrading MSYS2 ImageMagick to 7.1.0.35 and the error is gone.

Your comment about malloc and free on MS-Windows got me thinking on how ImageMagick has been working on this machine with Emacs 27.2.50 in the first place.  I was then searching for supported ImageMagick versions, but it seems that starting with Emacs 27 ImageMagick is disabled by default.  Maybe a note on ImageMagick compat for Emacs 28 on MS-Windows MSYS2 could be added to the docs, as long as the option --with-imagemagick is still available?

In any case, please consider this solved from my side.

bug#55654: 28.1.50; HEAP Error with ImageMagick on Windows MSYS2

[Eli Zaretskii]

> From: Fabio Leimgruber <fabio.leimgruber@web.de>
> Cc: Fabio Leimgruber <fabio.leimgruber@web.de>,  55654@debbugs.gnu.org
> Date: Sun, 29 May 2022 11:12:20 +0200
> 
> I just recompiled after upgrading MSYS2 ImageMagick to 7.1.0.35 and the error is gone.

So this sounds like it's a bug in ImageMagick?  Any chance to verify
this with ImageMagick developers or the MSYS2 folks?

> Your comment about malloc and free on MS-Windows got me thinking on how ImageMagick has been working on this machine with Emacs 27.2.50 in the first place.  I was then searching for supported ImageMagick versions, but it seems that starting with Emacs 27 ImageMagick is disabled by default.  Maybe a note on ImageMagick compat for Emacs 28 on MS-Windows MSYS2 could be added to the docs, as long as the option --with-imagemagick is still available?

What kind of note would you suggest adding, and how is it relevant to
MS-Windows and MSYS2?

> In any case, please consider this solved from my side.

OK, so I'm closing the bug.  (We can continue discussing the issue
even after the bug is closed.)

Thanks.

bug#55654: 28.1.50; HEAP Error with ImageMagick on Windows MSYS2

[Fabio Leimgruber]

Eli Zaretskii <eliz@gnu.org> writes:

> So this sounds like it's a bug in ImageMagick?

Yes, I guess so.

> Any chance to verify this with ImageMagick developers or the MSYS2
> folks?

I do not have the capacity at the moment to get involved for a
meaningful bug report (including debug symbols) and I personally do not
need Emacs 28.1 working with an older ImageMagick.

> What kind of note would you suggest adding, and how is it relevant to
> MS-Windows and MSYS2?

The note would be about version compatibility of Emacs 28.1 with
ImageMagick on MS-Windows and MSYS2 if the --with-imagemagick option is
used, i.e. minimal ImageMagick version known to work is 7.1.0.35 in that
environment.  Just an idea, and also depending on whether the
--with-imagemagick option will be dropped altogether at some point.