bug#31771: 26.1; Writing image file with emacs hangs entire system

bug#31771: 26.1; Writing image file with emacs hangs entire system

[Adam]

Ensure everything is saved and your system can tolerate a hard reboot;
reproducing this bug may cause a full-system lock-up.

In your shell,
$ cat > black-pixel.png.b64 << EOF
iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVQI12NgYGAAAAAEAAEnNCcK
AAAAAElFTkSuQmCC
EOF
$ emacs -Q black-pixel.png.b64

From emacs,
C-x h
M-x base64-decode-region RET
C-x C-w RET black-pixel.png RET

The decoded buffer (before pressing C-x C-w) should contain:

\211PNG^M
^Z
^@^@^@^MIHDR^@^@^@^A^@^@^@^A^H^B^@^@^@\220wS\336^@^@^@^LIDAT^H\327c```^@^@^@^D^@^A'4'
^@^@^@^@IEND\256B`\202

After following these commands, your emacs should freeze, and then your
entire system should hang a few seconds later.

My hunch: this is caused by emacs trying to render the image after it's
saved. Unfortunately, this causes emacs and my system to hang, so I
can't provide any core dumps. I'm using linux 4.16.13, mesa 18.1.1,
libpng 1.6.34, and imagemagick 7.0.7.38

In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-05-29 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12000000
System Description:	Arch Linux

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11 MODULES THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LC_COLLATE: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 95362 9471)
 (symbols 48 20349 1)
 (miscs 40 44 168)
 (strings 32 28460 1052)
 (string-bytes 1 743428)
 (vectors 16 14643)
 (vector-slots 8 496724 7908)
 (floats 8 49 68)
 (intervals 56 218 0)
 (buffers 992 11))

bug#31771: 26.1; Writing image file with emacs hangs entire system

[Noam Postavsky]

tags 31771 + unreproducible
quit

Adam <adam.niederer@gmail.com> writes:

> After following these commands, your emacs should freeze, and then your
> entire system should hang a few seconds later.

I couldn't reproduce this.  No hang, and if I look closely, I can see
the one black pixel rendered correctly in the top left corner.

Is your whole system completely frozen, or is it perhaps just using so
much RAM that it goes into swap and starts thrashing and becomes very
slow?  If the former, I guess it's some kind of graphics driver thing,
though it seems a bit funny that an image as simple as a single black
pixel could wreck it.

> My hunch: this is caused by emacs trying to render the image after it's
> saved. Unfortunately, this causes emacs and my system to hang, so I
> can't provide any core dumps. I'm using linux 4.16.13, mesa 18.1.1,
> libpng 1.6.34, and imagemagick 7.0.7.38

> Configured features:
> XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
> ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
> GTK3 X11 MODULES THREADS LIBSYSTEMD LCMS2

Do you really have imagemagick 7?  Or do maybe also have imagemagic 6?
As far as I know, the patch for Emacs to use v7 is not merged yet:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25967#41

bug#31771: 26.1; Writing image file with emacs hangs entire system

[Adam]

On 06/09/2018 03:47 PM, Noam Postavsky wrote:
> tags 31771 + unreproducible
> quit
> 
> Adam <adam.niederer@gmail.com> writes:
> 
>> After following these commands, your emacs should freeze, and then your
>> entire system should hang a few seconds later.
> 
> I couldn't reproduce this.  No hang, and if I look closely, I can see
> the one black pixel rendered correctly in the top left corner.
> 
> Is your whole system completely frozen, or is it perhaps just using so
> much RAM that it goes into swap and starts thrashing and becomes very
> slow?  If the former, I guess it's some kind of graphics driver thing,
> though it seems a bit funny that an image as simple as a single black
> pixel could wreck it.

I don't think it's RAM-related; I'm seeing ~500MB used of 16GB whenever
the system freezes.

It's actually quite similar to a graphics driver hang, now that you
mention it. The system is unresponsive to input and my cursor freezes,
but audio keeps playing. Emacs also doesn't successfully write the
decoded file to my disk.

I can reproduce the issue in a debug build of 26.1, but the freeze also
truncates my gdb log so I'm having a hard time nailing down exactly
where it occurs (too many lisp interpreter frames :( ). I've also found
that the issue only occurs iff the imagemagick feature is enabled, and
can simply be reproduced with C-x C-f (the decoded black pixel image).

If it's any help, the system freezes before the minibuffer is cleared
and the new image is rendered (The find/write file prompt is still up
and the current buffer isn't changed)

>> My hunch: this is caused by emacs trying to render the image after it's
>> saved. Unfortunately, this causes emacs and my system to hang, so I
>> can't provide any core dumps. I'm using linux 4.16.13, mesa 18.1.1,
>> libpng 1.6.34, and imagemagick 7.0.7.38
> 
>> Configured features:
>> XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
>> ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
>> GTK3 X11 MODULES THREADS LIBSYSTEMD LCMS2
> 
> Do you really have imagemagick 7?  Or do maybe also have imagemagic 6?
> As far as I know, the patch for Emacs to use v7 is not merged yet:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25967#41

Ah, my apologies. Here's libmagick6:

$ pacman -Qs libmagick
libmagick6 6.9.9.50-1
libmagick 7.0.7.38-1

bug#31771: 26.1; Writing image file with emacs hangs entire system

[Noam Postavsky]

Adam <adam.niederer@gmail.com> writes:

> It's actually quite similar to a graphics driver hang, now that you
> mention it. The system is unresponsive to input and my cursor freezes,
> but audio keeps playing. Emacs also doesn't successfully write the
> decoded file to my disk.
>
> I can reproduce the issue in a debug build of 26.1, but the freeze also
> truncates my gdb log so I'm having a hard time nailing down exactly
> where it occurs (too many lisp interpreter frames :( ). I've also found
> that the issue only occurs iff the imagemagick feature is enabled, and
> can simply be reproduced with C-x C-f (the decoded black pixel image).

So it definitely sounds like a graphics driver problem.  Do you get
anything in the Xorg logs?  Are there any graphics driver settings you
can fiddle with?  I suppose it doesn't happen if you run imagemagick's
'display' command on the image?

bug#31771: 26.1; Writing image file with emacs hangs entire system

[Adam]

On 06/09/2018 10:53 PM, Noam Postavsky wrote:
> So it definitely sounds like a graphics driver problem.  Do you get
> anything in the Xorg logs?  Are there any graphics driver settings you
> can fiddle with?  I suppose it doesn't happen if you run imagemagick's
> 'display' command on the image?

`display` doesn't hang my system, and I see nothing out of the ordinary
in my Xorg logs. I was able to figure out which line causes the hang,
though: A call to MagickMergeImageLayers at image.c:8728 (on the emacs
26.1 tag)

 8724   {
 8725     MagickWand *new_wand;
 8726     MagickSetImageBackgroundColor (image_wand, bg_wand);
 8727 #ifdef HAVE_MAGICKMERGEIMAGELAYERS
 8728     new_wand = MagickMergeImageLayers (image_wand, MergeLayer);
 8729 #else
 8730     new_wand = MagickFlattenImages (image_wand);
 8731 #endif
 8732     DestroyMagickWand (image_wand);
 8733     image_wand = new_wand;
 8734   }

Using the deprecated MagickFlattenImages call also causes the hang. I'll
compile a libmagick6 later and see if I can find the underlying problem,
but this definitely isn't emacs' fault.

Thanks for all the help!

bug#31771: 26.1; [imagemagick] Writing image file with emacs hangs entire system

[Joerg Kulbartz]

A workaround for the bug, at least for me, seems to disable OpenCL for
ImageMagick by setting the environment variable

MAGICK_OCL_DEVICE=OFF


(h/t https://bugs.archlinux.org/task/47864 )