bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[Simon Leinen]

I regularly build Emacs from Git master on my work system, an Apple
laptop running Mac OS 10.14.5 "Mojave".  My current binaries have been
built from commit fd5410217ff23810edc16e97c10934ad622f8e4b.

As I want to use open standards as far as possible, I've always been
building with X11 libraries (--with-x --with-x-toolkit=lucid
--with-ns=no).

Lately I have experienced several crashes, mostly while using GNUS
(which fontifies articles in interesting ways).  From running under the
debugger, it seems they are related to libharfbuzz in some way.
Unfortunately I have no easy way to reproduce this from emacs -Q.


I haven't yet been able to work around Mac OS's security restrictions
enough to run Emacs under GDB.  But I've been running it under Apple's
"lldb" debugger and was able to create a backtrace:

Last login: Sat Jul 20 13:23:56 on ttys044
## No news is good news.
: leinen@macsl[leinen]; lldb
: 1leinen@macsl[leinen]; cd
: 1leinen@macsl[leinen]; lldb /var/tmp/emacs/gbuild/src/emacs
(lldb) target create "/var/tmp/emacs/gbuild/src/emacs"
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy.py", line 52, in <module>
    import weakref
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/weakref.py", line 14, in <module>
    from _weakref import (
ImportError: cannot import name _remove_dead_weakref
Current executable set to '/var/tmp/emacs/gbuild/src/emacs' (x86_64).
(lldb) r
Process 9545 launched: '/var/tmp/emacs/gbuild/src/emacs' (x86_64)
2019-07-21 11:07:57.544779+0200 emacs[25606:421986] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:18:42.815391+0200 emacs[25752:425967] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:22:16.379676+0200 emacs[25802:427348] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:27:09.345678+0200 emacs[25863:429541] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:27:24.670984+0200 emacs[25865:429681] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:37:24.323339+0200 emacs[26051:433355] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:44:54.299349+0200 emacs[26136:436109] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:45:16.788719+0200 emacs[26151:436419] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:47:17.212474+0200 emacs[26176:437212] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 11:47:34.105429+0200 emacs[26181:437444] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 12:22:46.221838+0200 emacs[26623:452631] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 12:26:16.117381+0200 emacs[26679:454282] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 12:26:58.124286+0200 emacs[26691:454761] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 12:27:00.180428+0200 emacs[26694:454846] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 12:27:02.224276+0200 emacs[26708:454913] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 13:50:46.001067+0200 emacs[27746:488845] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 13:50:54.757005+0200 emacs[27748:488967] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 13:51:14.189815+0200 emacs[27764:489204] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 14:46:07.639186+0200 emacs[28712:516697] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 14:47:23.180579+0200 emacs[28804:518329] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 14:48:48.027322+0200 emacs[28834:519841] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]

Process 9545 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
    frame #0: 0x00007fff6fb795c2 libsystem_kernel.dylib`__pselect + 10
libsystem_kernel.dylib`__pselect:
->  0x7fff6fb795c2 <+10>: jae    0x7fff6fb795cc            ; <+20>
    0x7fff6fb795c4 <+12>: movq   %rax, %rdi
    0x7fff6fb795c7 <+15>: jmp    0x7fff6fb73421            ; cerror
    0x7fff6fb795cc <+20>: retq
Target 0: (emacs) stopped.
(lldb)
error: No auto repeat.
(lldb) c
Process 9545 resuming
2019-07-21 15:30:13.339659+0200 emacs[29660:557040] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 15:55:56.078789+0200 emacs[29998:574005] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 15:56:09.184720+0200 emacs[30011:574122] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 15:56:26.626043+0200 emacs[30013:574226] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 15:56:44.489415+0200 emacs[30016:574337] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 15:57:10.106427+0200 emacs[30029:574522] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 15:58:05.589844+0200 emacs[30045:574937] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 16:21:39.400825+0200 emacs[30614:587945] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 16:40:30.080012+0200 emacs[30849:597560] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 16:42:52.885387+0200 emacs[30884:598627] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 16:42:56.622956+0200 emacs[30886:598656] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:46:40.041993+0200 emacs[32751:653893] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:49:41.491249+0200 emacs[32830:656133] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:50:18.395142+0200 emacs[32844:656537] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:50:46.436173+0200 emacs[32852:656919] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:51:10.653719+0200 emacs[32866:657140] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:51:36.764702+0200 emacs[32870:657330] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:53:22.954997+0200 emacs[32896:658067] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 18:53:32.343062+0200 emacs[32899:658140] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:11:16.060831+0200 emacs[33149:667448] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:11:21.477075+0200 emacs[33151:667483] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:15:28.201008+0200 emacs[33219:669175] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:16:07.899816+0200 emacs[33235:669559] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:16:13.187729+0200 emacs[33237:669586] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:16:20.108804+0200 emacs[33239:669664] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:16:36.219786+0200 emacs[33243:669807] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:16:56.210873+0200 emacs[33245:669957] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:18:35.343660+0200 emacs[33277:670645] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:19:04.349868+0200 emacs[33290:670858] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:20:32.108903+0200 emacs[33307:671522] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:22:03.469323+0200 emacs[33340:672302] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:22:38.048582+0200 emacs[33343:672521] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:23:44.267080+0200 emacs[33360:673008] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:24:56.869375+0200 emacs[33373:673517] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:30:21.086736+0200 emacs[33459:676300] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:31:33.563610+0200 emacs[33473:676766] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 19:32:56.002587+0200 emacs[33488:677221] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-21 22:48:46.824487+0200 emacs[35959:751002] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
2019-07-22 08:55:18.190641+0200 emacs[43351:983764] nw_path_close_fd Failed to close guarded necp fd 8 [9: Bad file descriptor]
Process 9545 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x11e4c6000)
    frame #0: 0x0000000100d361f6 libharfbuzz.0.dylib`OT::OpenTypeFontFile::get_face(unsigned int, unsigned int*) const + 18
libharfbuzz.0.dylib`OT::OpenTypeFontFile::get_face:
->  0x100d361f6 <+18>: movl   (%rcx), %edi
    0x100d361f8 <+20>: bswapl %edi
    0x100d361fa <+22>: leaq   0x6304f(%rip), %rax       ; _hb_NullPool
    0x100d36201 <+29>: cmpl   $0x74727564, %edi         ; imm = 0x74727564
Target 0: (emacs) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x11e4c6000)
  * frame #0: 0x0000000100d361f6 libharfbuzz.0.dylib`OT::OpenTypeFontFile::get_face(unsigned int, unsigned int*) const + 18
    frame #1: 0x0000000100d35d8f libharfbuzz.0.dylib`_hb_face_for_data_reference_table(hb_face_t*, unsigned int, void*) + 67
    frame #2: 0x0000000100d360f5 libharfbuzz.0.dylib`hb_face_t::reference_table(unsigned int) const + 19
    frame #3: 0x0000000100d88f89 libharfbuzz.0.dylib`hb_blob_t* hb_sanitize_context_t::reference_table<OT::maxp>(hb_face_t const*, unsigned int) + 51
    frame #4: 0x0000000100d88f1e libharfbuzz.0.dylib`hb_face_t::load_num_glyphs() const + 60
    frame #5: 0x0000000100d5fe81 libharfbuzz.0.dylib`hb_blob_t* hb_sanitize_context_t::reference_table<OT::GSUB>(hb_face_t const*, unsigned int) + 33
    frame #6: 0x0000000100d5fd60 libharfbuzz.0.dylib`OT::GSUBGPOS::accelerator_t<OT::GSUB>::init(hb_face_t*) + 70
    frame #7: 0x0000000100d5fd12 libharfbuzz.0.dylib`hb_lazy_loader_t<OT::GSUB_accelerator_t, hb_face_lazy_loader_t<OT::GSUB_accelerator_t, 18u>, hb_face_t, 18u, OT::GSUB_accelerator_t>::create(hb_face_t*) + 44
    frame #8: 0x0000000100d5fcab libharfbuzz.0.dylib`hb_lazy_loader_t<OT::GSUB_accelerator_t, hb_face_lazy_loader_t<OT::GSUB_accelerator_t, 18u>, hb_face_t, 18u, OT::GSUB_accelerator_t>::get_stored() const + 59
    frame #9: 0x0000000100d51f8e libharfbuzz.0.dylib`get_gsubgpos_table(hb_face_t*, unsigned int) + 32
    frame #10: 0x0000000100d51f33 libharfbuzz.0.dylib`hb_ot_layout_table_get_script_tags + 23
    frame #11: 0x00000001001f2fc8 emacs`hbotf_check_features(otf=0x000000010e61c3a0, gsubp=<unavailable>, script=1634885986, language=0, features=0x00000001105af7b0, n_features=4) at ftfont.c:2915:7 [opt]
    frame #12: 0x00000001001f0e48 emacs`ftfont_list(f=<unavailable>, spec=0x00000001033cbb6d) at ftfont.c:942:14 [opt]
    frame #13: 0x00000001001f0a2e emacs`ftfont_list2(f=<unavailable>, spec=<unavailable>, type=0x000000000000eb50) at ftfont.c:1007:22 [opt]
    frame #14: 0x0000000100197504 emacs`font_list_entities(f=0x0000000102805600, spec=0x0000000118185375) at font.c:2786:12 [opt]
    frame #15: 0x00000001001999c0 emacs`font_find_for_lface(f=0x0000000102805600, attrs=0x00000001173d0c40, spec=<unavailable>, c=-1) at font.c:3277:16 [opt]
    frame #16: 0x00000001001fa147 emacs`fontset_find_font(fontset=0x000000011bad0865, c=1641, face=0x00000001173d0c40, charset_id=<unavailable>, fallback=<unavailable>) at fontset.c:653:18 [opt]
    frame #17: 0x00000001001f6de8 emacs`fontset_font(fontset=<unavailable>, c=1641, face=0x00000001173d0c40, id=-1) at fontset.c:775:4 [opt]
    frame #18: 0x00000001001f7044 emacs`font_for_char(face=0x00000001173d0c40, c=<unavailable>, pos=<unavailable>, object=<unavailable>) at fontset.c:1056:15 [opt]
    frame #19: 0x000000010019b6f6 emacs`font_range(pos=13500, pos_byte=13502, limit=0x00007ffeefbfd628, w=<unavailable>, face=0x00000001173d0c40, string=0x0000000000000000) at font.c:3860:18 [opt]
    frame #20: 0x00000001001e6049 emacs`autocmp_chars(rule=<unavailable>, charpos=<unavailable>, bytepos=13502, limit=13501, win=0x0000000118841d80, face=0x0000000000000000, string=0x0000000000000000, direction=0x0000000000002580) at composite.c:899:21 [opt]
    frame #21: 0x00000001001e5c54 emacs`composition_reseat_it(cmp_it=0x00007ffeefbfd740, charpos=13500, bytepos=13502, endpos=<unavailable>, w=0x0000000118841d80, bidi_level='\0', face=0x0000000000000000, string=0x0000000000000000) at composite.c:1222:19 [opt]
    frame #22: 0x000000010013a49c emacs`scan_for_column(endpos=0x00007ffeefbfd838, goalcol=0x00007ffeefbfd848, prevcol=0x0000000000000000) at indent.c:600:11 [opt]
    frame #23: 0x0000000100138bf8 emacs`current_column [inlined] current_column_1 at indent.c:725:3 [opt]
    frame #24: 0x0000000100138be1 emacs`current_column at indent.c:351 [opt]
    frame #25: 0x00000001001c2055 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x000000010f14a6c5, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:1109:4 [opt]
    frame #26: 0x0000000100182b98 emacs`funcall_lambda(fun=0x0000000110b0fae5, nargs=1, arg_vector=0x00007ffeefbfdac0) at eval.c:3072:13 [opt]
    frame #27: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #28: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x0000000110a6d705, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #29: 0x0000000100182b98 emacs`funcall_lambda(fun=0x0000000110b0faa5, nargs=2, arg_vector=0x00007ffeefbfdd50) at eval.c:3072:13 [opt]
    frame #30: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #31: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x000000010f132635, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #32: 0x0000000100182b98 emacs`funcall_lambda(fun=0x0000000110b0fa75, nargs=0, arg_vector=0x00007ffeefbfdf10) at eval.c:3072:13 [opt]
    frame #33: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #34: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x00000001068d0a75, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #35: 0x0000000100182b98 emacs`funcall_lambda(fun=0x00000001068d0ba5, nargs=4, arg_vector=0x00007ffeefbfe1d0) at eval.c:3072:13 [opt]
    frame #36: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #37: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x0000000101d6e035, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #38: 0x0000000100182b98 emacs`funcall_lambda(fun=0x0000000101ca1e35, nargs=0, arg_vector=0x00007ffeefbfe4b0) at eval.c:3072:13 [opt]
    frame #39: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #40: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x0000000101d6b105, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #41: 0x0000000100182b98 emacs`funcall_lambda(fun=0x00000001068a6aa5, nargs=0, arg_vector=0x00007ffeefbfe6a0) at eval.c:3072:13 [opt]
    frame #42: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #43: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x00000001068a6835, maxdepth=<unavailable>, args_template=0x0000000000000000, nargs=0, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #44: 0x0000000100182b98 emacs`funcall_lambda(fun=0x00000001068a6a75, nargs=2, arg_vector=0x00007ffeefbfea10) at eval.c:3072:13 [opt]
    frame #45: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #46: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x0000000106a08d95, maxdepth=<unavailable>, args_template=0x0000000000000806, nargs=1, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #47: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #48: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x0000000101e404b5, maxdepth=<unavailable>, args_template=0x0000000000000c02, nargs=1, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #49: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #50: 0x000000010017ae09 emacs`Ffuncall_interactively(nargs=<unavailable>, args=<unavailable>) at callint.c:255:32 [opt]
    frame #51: 0x0000000100181d3b emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:2799:11 [opt]
    frame #52: 0x000000010017c28f emacs`Fcall_interactively(function=<unavailable>, record_flag=0x0000000000000000, keys=0x0000000105edfa65) at callint.c:783:21 [opt]
    frame #53: 0x000000010018279b emacs`funcall_subr(subr=0x0000000100258af0, numargs=3, args=<unavailable>) at eval.c:2877:19 [opt]
    frame #54: 0x0000000100181d3b emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:2799:11 [opt]
    frame #55: 0x00000001001c1046 emacs`exec_byte_code(bytestr=<unavailable>, vector=0x0000000103248595, maxdepth=<unavailable>, args_template=0x0000000000001006, nargs=1, args=<unavailable>) at bytecode.c:633:12 [opt]
    frame #56: 0x0000000100181cd9 emacs`Ffuncall(nargs=<unavailable>, args=<unavailable>) at eval.c:0:4 [opt]
    frame #57: 0x00000001001823cc emacs`call1(fn=<unavailable>, arg1=<unavailable>) at eval.c:2652:10 [opt]
    frame #58: 0x00000001000f8b98 emacs`command_loop_1 at keyboard.c:1461:13 [opt]
    frame #59: 0x0000000100180289 emacs`internal_condition_case(bfun=(emacs`command_loop_1 at keyboard.c:1239), handlers=0x0000000000000090, hfun=<unavailable>) at eval.c:1351:25 [opt]
    frame #60: 0x00000001001082f0 emacs`command_loop_2(ignore=<unavailable>) at keyboard.c:1091:11 [opt]
    frame #61: 0x000000010017f957 emacs`internal_catch(tag=0x000000000000ced0, func=(emacs`command_loop_2 at keyboard.c:1087), arg=0x0000000000000000) at eval.c:1112:25 [opt]
    frame #62: 0x00000001000f790e emacs`command_loop at keyboard.c:1070:2 [opt]
    frame #63: 0x00000001000f7823 emacs`recursive_edit_1 at keyboard.c:714:9 [opt]
    frame #64: 0x00000001000f7acb emacs`Frecursive_edit at keyboard.c:786:3 [opt]
    frame #65: 0x00000001000f6287 emacs`main(argc=<unavailable>, argv=0x00007ffeefbff760) at emacs.c:2086:3 [opt]
    frame #66: 0x00007fff6fa3e3d5 libdyld.dylib`start + 1
(lldb)

In GNU Emacs 27.0.50 (build 4, x86_64-apple-darwin18.6.0, X toolkit, Xaw3d scroll bars)
 of 2019-06-18 built on macsl
Repository revision: e8751c0ce374e1fde4fdabf5dc3504996013fd79
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description:  Mac OS X 10.14.5

Recent messages:
BBDB could not be loaded.
Loading experimental modules...
Done loading experimental modules.
emacs/lisp/solarized-definitions.el: ‘flet’ is an obsolete macro (as of 24.3); use either ‘cl-flet’ or ‘cl-letf’.
Specified image bit depth is not supported by XRender [10 times]
Loading autoinsert...done
Loading gnus...done
Loading avoid...done
Waiting for git... [2 times]
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --verbose --with-x --with-x-toolkit=lucid --with-ns=no
 --without-makeinfo
 LIBXML2_CFLAGS=-I/usr/local/opt/libxml2/include/libxml2
 'LIBXML2_LIBS=-L/usr/local/opt/libxml2/lib -lxml2' --with-jpeg=no
 --with-gif=no --with-tiff=no --x-libraries=/usr/X11/lib
 --x-includes=/usr/X11/include --with-xpm=no
 PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/X11/lib/pkgconfig:/usr/X11/share/pkgconfig'

Configured features:
XAW3D PNG DBUS NOTIFY KQUEUE ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM THREADS JSON PDUMPER LCMS2
GMP

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  auto-insert-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/Users/leinen/.emacs.d/elpa/lsp-go-20180914.515/lsp-go hides /Users/leinen/.emacs.d/elpa/lsp-mode-20190709.442/lsp-go
/Users/leinen/.emacs.d/elpa/lsp-rust-20180305.1308/lsp-rust hides /Users/leinen/.emacs.d/elpa/lsp-mode-20190709.442/lsp-rust

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader imenu elec-pair
warnings advice finder-inf tex-site confluence-edit-autoloads
docker-tramp tramp-cache epoch-view-autoloads idle-highlight-autoloads
rx cl-extra help-mode gh-common marshal eieio-compat logito-autoloads
pod-mode-autoloads rnc-mode-autoloads slime-autoloads
windresize-autoloads info package easymenu epg-config url-handlers
url-parse url-vars tramp tramp-loaddefs trampver tramp-integration
files-x tramp-compat shell pcomplete comint ansi-color ring parse-time
format-spec avoid gnus nnheader gnus-util rmail rmail-loaddefs
text-property-search time-date wid-edit autoinsert cus-start cus-load
solarized-light-theme solarized-definitions cl smtpmail auth-source
cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json
subr-x map seq byte-opt gv bytecomp byte-compile cconv sendmail rfc2047
rfc2045 ietf-drums mail-utils timeclock cl-loaddefs cl-lib vm-startup
vm-search vm-edit vm-reply vm-mark vm-smime vm-delete vm-digest vm-undo
vm-page vm-virtual vm-summary-faces vm-pop utf7 mm-util mail-prsvr
vm-imap vm-thread vm-mime vm-motion vm-mouse vm-toolbar vm-menu
vm-window vm-crypto vm-summary vm-folder vm-minibuf vm-misc vm-sort
vm-autoloads vm-vars vm-version vm mule-util early-init tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind kqueue lcms2 dynamic-setting
font-render-setting x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 283671 16225)
 (symbols 48 23464 1)
 (strings 32 96293 4705)
 (string-bytes 1 2948897)
 (vectors 16 27044)
 (vector-slots 8 381510 12246)
 (floats 8 69 13)
 (intervals 56 306 0)
 (buffers 992 14))
-- 
Simon.

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[YAMAMOTO Mitsuharu]

On Mon, 22 Jul 2019 16:09:21 +0900,
Simon Leinen wrote:
> 
> I regularly build Emacs from Git master on my work system, an Apple
> laptop running Mac OS 10.14.5 "Mojave".  My current binaries have been
> built from commit fd5410217ff23810edc16e97c10934ad622f8e4b.
> 
> As I want to use open standards as far as possible, I've always been
> building with X11 libraries (--with-x --with-x-toolkit=lucid
> --with-ns=no).
> 
> Lately I have experienced several crashes, mostly while using GNUS
> (which fontifies articles in interesting ways).  From running under the
> debugger, it seems they are related to libharfbuzz in some way.
> Unfortunately I have no easy way to reproduce this from emacs -Q.

Could you try the following instructions?

  1. recompile Emacs with CFLAGS="-O0 -g3"
  2. run it under lldb.
  3. run GNUS and wait for crash happens.
  4. repeat lldb "up" command until it reaches emacs`ftfont_list
     (frame #12 in the original stack trace)
  5. run "p file" under lldb.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[Simon Leinen]

Dear Mitshuaru,

thank you for your prompt reply and clear instructions - this is very
helpful for me as I don't have any experience with lldb!

> Could you try the following instructions?

>   1. recompile Emacs with CFLAGS="-O0 -g3"
>   2. run it under lldb.
>   3. run GNUS and wait for crash happens.
>   4. repeat lldb "up" command until it reaches emacs`ftfont_list
>      (frame #12 in the original stack trace)
>   5. run "p file" under lldb.

(lldb) p file
(FcChar8 *) $0 = 0x00000001071522e0 "/System/Library/Fonts/ArabicUIDisplay.ttc"

The file exists

: leinen@macsl[leinen]; ls -l /System/Library/Fonts/ArabicUIDisplay.ttc
-rw-r--r--  1 root  wheel  983632 Aug 18  2018 /System/Library/Fonts/ArabicUIDisplay.ttc

Maybe the code has issue with right-to-left scripts?
-- 
Simon.

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[YAMAMOTO Mitsuharu]

On Mon, 22 Jul 2019 17:42:02 +0900,
Simon Leinen wrote:
> 
> Dear Mitshuaru,
> 
> thank you for your prompt reply and clear instructions - this is very
> helpful for me as I don't have any experience with lldb!
> 
> > Could you try the following instructions?
> 
> >   1. recompile Emacs with CFLAGS="-O0 -g3"
> >   2. run it under lldb.
> >   3. run GNUS and wait for crash happens.
> >   4. repeat lldb "up" command until it reaches emacs`ftfont_list
> >      (frame #12 in the original stack trace)
> >   5. run "p file" under lldb.
> 
> (lldb) p file
> (FcChar8 *) $0 = 0x00000001071522e0 "/System/Library/Fonts/ArabicUIDisplay.ttc"
> 
> The file exists
> 
> : leinen@macsl[leinen]; ls -l /System/Library/Fonts/ArabicUIDisplay.ttc
> -rw-r--r--  1 root  wheel  983632 Aug 18  2018 /System/Library/Fonts/ArabicUIDisplay.ttc
> 
> Maybe the code has issue with right-to-left scripts?

Thanks.  I rather suspect it has something to do with the versions of
Harbuzz.  Could you try to see if the following test program crash on
your side?

/* cc -g hb-ot-test.c `pkg-config freetype2 harfbuzz --cflags --libs` */
#include <stdio.h>
#include <ft2build.h>
#include <freetype/freetype.h>
#include <hb.h>
#include <hb-ft.h>
#include <hb-ot.h>

int
main ()
{
  printf ("HarfBuzz Version: %s\n", hb_version_string ());

  static FT_Library ft_library;
  FT_Init_FreeType (&ft_library);

  FT_Face ft_face;
  FT_New_Face (ft_library, "/System/Library/Fonts/ArabicUIDisplay.ttc", 0,
	       &ft_face);

  hb_face_t *face = hb_ft_face_create_referenced (ft_face);
  FT_Done_Face (ft_face);

  unsigned int script_count
    = hb_ot_layout_table_get_script_tags (face, HB_OT_TAG_GSUB, 0, NULL, NULL);
  printf ("GSUB script_count = %d\n", script_count);
  script_count
    = hb_ot_layout_table_get_script_tags (face, HB_OT_TAG_GPOS, 0, NULL, NULL);
  printf ("GPOS script_count = %d\n", script_count);
}


On macOS 10.14.6 Beta, I could successfully run it with the following
output:

HarfBuzz Version: 2.5.3
GSUB script_count = 1
GPOS script_count = 1

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[Simon Leinen]

YAMAMOTO Mitsuharu writes:
> Thanks.  I rather suspect it has something to do with the versions of
> Harbuzz.  Could you try to see if the following test program crash on
> your side?

> /* cc -g hb-ot-test.c `pkg-config freetype2 harfbuzz --cflags --libs` */

No, that runs fine (return code 0) and outputs

  HarfBuzz Version: 2.5.3
  GSUB script_count = 1
  GPOS script_count = 1

: leinen@macsl[leinen]; objdump -macho -dylibs-used /var/tmp/a.out
/var/tmp/a.out:
	/usr/local/opt/freetype/lib/libfreetype.6.dylib (compatibility version 24.0.0, current version 24.1.0)
	/usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 20504.0.0, current version 20504.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

This is the same version of libharfbuzz that my Emacs binary uses:

: leinen@macsl[leinen]; objdump -macho -dylibs-used /var/tmp/emacs/gbuild/src/emacs
/var/tmp/emacs/gbuild/src/emacs:
	/opt/X11/lib/libpng16.16.dylib (compatibility version 43.0.0, current version 43.0.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/opt/X11/lib/libXaw3d.8.dylib (compatibility version 9.0.0, current version 9.0.0)
	/opt/X11/lib/libXmu.6.dylib (compatibility version 9.0.0, current version 9.0.0)
	/opt/X11/lib/libXt.6.dylib (compatibility version 7.0.0, current version 7.0.0)
	/opt/X11/lib/libSM.6.dylib (compatibility version 7.0.0, current version 7.1.0)
	/opt/X11/lib/libICE.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current version 11.0.0)
	/opt/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current version 10.0.0)
	/opt/X11/lib/libX11-xcb.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/X11/lib/libxcb.1.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/X11/lib/libXft.2.dylib (compatibility version 6.0.0, current version 6.2.0)
	/opt/X11/lib/libXrender.1.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/local/opt/dbus/lib/libdbus-1.3.dylib (compatibility version 23.0.0, current version 23.11.0)
	/opt/X11/lib/libXrandr.2.dylib (compatibility version 5.0.0, current version 5.0.0)
	/opt/X11/lib/libXinerama.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/X11/lib/libXfixes.3.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/local/opt/libxml2/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.9.0)
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
	/opt/X11/lib/libfreetype.6.dylib (compatibility version 19.0.0, current version 19.6.0)
	/opt/X11/lib/libfontconfig.1.dylib (compatibility version 11.0.0, current version 11.2.0)
	/usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 20504.0.0, current version 20504.0.0)
	/usr/local/opt/gnutls/lib/libgnutls.30.dylib (compatibility version 55.0.0, current version 55.0.0)
	/usr/local/opt/little-cms2/lib/liblcms2.2.dylib (compatibility version 3.0.0, current version 3.8.0)
	/usr/local/opt/jansson/lib/libjansson.4.dylib (compatibility version 16.0.0, current version 16.1.0)
	/usr/local/opt/gmp/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.2.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

Thanks a lot for your help! Let me know if I can run more tests for
you.  I'm optimistic that we'll find the issue soon.

Best regards,
-- 
Simon.

> #include <stdio.h>
> #include <ft2build.h>
> #include <freetype/freetype.h>
> #include <hb.h>
> #include <hb-ft.h>
> #include <hb-ot.h>

> int
> main ()
> {
>   printf ("HarfBuzz Version: %s\n", hb_version_string ());

>   static FT_Library ft_library;
>   FT_Init_FreeType (&ft_library);

>   FT_Face ft_face;
>   FT_New_Face (ft_library, "/System/Library/Fonts/ArabicUIDisplay.ttc", 0,
> 	       &ft_face);

>   hb_face_t *face = hb_ft_face_create_referenced (ft_face);
>   FT_Done_Face (ft_face);

>   unsigned int script_count
>     = hb_ot_layout_table_get_script_tags (face, HB_OT_TAG_GSUB, 0, NULL, NULL);
>   printf ("GSUB script_count = %d\n", script_count);
>   script_count
>     = hb_ot_layout_table_get_script_tags (face, HB_OT_TAG_GPOS, 0, NULL, NULL);
>   printf ("GPOS script_count = %d\n", script_count);
> }


> On macOS 10.14.6 Beta, I could successfully run it with the following
> output:

> HarfBuzz Version: 2.5.3
> GSUB script_count = 1
> GPOS script_count = 1

> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[YAMAMOTO Mitsuharu]

On Mon, 22 Jul 2019 19:52:07 +0900,
Simon Leinen wrote:
> 
> YAMAMOTO Mitsuharu writes:
> > Thanks.  I rather suspect it has something to do with the versions of
> > Harbuzz.  Could you try to see if the following test program crash on
> > your side?
> 
> > /* cc -g hb-ot-test.c `pkg-config freetype2 harfbuzz --cflags --libs` */
> 
> No, that runs fine (return code 0) and outputs
> 
>   HarfBuzz Version: 2.5.3
>   GSUB script_count = 1
>   GPOS script_count = 1
> 
> : leinen@macsl[leinen]; objdump -macho -dylibs-used /var/tmp/a.out
> /var/tmp/a.out:
> 	/usr/local/opt/freetype/lib/libfreetype.6.dylib (compatibility version 24.0.0, current version 24.1.0)
> 	/usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 20504.0.0, current version 20504.0.0)
> 	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
> 
> This is the same version of libharfbuzz that my Emacs binary uses:
> 
> : leinen@macsl[leinen]; objdump -macho -dylibs-used /var/tmp/emacs/gbuild/src/emacs
(snip)
> 	/opt/X11/lib/libfreetype.6.dylib (compatibility version 19.0.0, current version 19.6.0)

I noticed the versions of libfreetype are different between the test
program and Emacs.  What happens if you link the test program with the
latter one using the environment variable DYLD_LIBRARY_PATH ?

  $ DYLD_LIBRARY_PATH=/opt/X11/lib ./a.out

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[Simon Leinen]

YAMAMOTO Mitsuharu writes:
>> This is the same version of libharfbuzz that my Emacs binary uses:
>> 
>> : leinen@macsl[leinen]; objdump -macho -dylibs-used /var/tmp/emacs/gbuild/src/emacs
> (snip)
>> /opt/X11/lib/libfreetype.6.dylib (compatibility version 19.0.0, current version 19.6.0)

> I noticed the versions of libfreetype are different between the test
> program and Emacs.

Ah, good point - I hadn't noticed that!

> What happens if you link the test program with the latter one using
> the environment variable DYLD_LIBRARY_PATH ?

>   $ DYLD_LIBRARY_PATH=/opt/X11/lib ./a.out

Unfortunately it is not that easy:

  : leinen@macsl[tmp]; DYLD_LIBRARY_PATH=/opt/X11/lib ./a.out
  dyld: Library not loaded: /usr/local/opt/freetype/lib/libfreetype.6.dylib
    Referenced from: /private/var/tmp/./a.out
    Reason: Incompatible library version: a.out requires version 24.0.0 or later, but libfreetype.6.dylib provides version 19.0.0
  Abort trap: 6

...but if I recompile the test program with -L/opt/X11/lib, then it
crashes:

  : leinen@macsl[tmp]; cc -g hb-ot-test.c -L/opt/X11/lib `pkg-config freetype2 harfbuzz --cflags --libs`
  : leinen@macsl[tmp]; objdump -macho -dylibs-used /var/tmp/a.out
  /var/tmp/a.out:
  	/opt/X11/lib/libfreetype.6.dylib (compatibility version 19.0.0, current version 19.6.0)
  	/usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 20504.0.0, current version 20504.0.0)
  	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
  : leinen@macsl[tmp]; ./a.out
  HarfBuzz Version: 2.5.3
  Segmentation fault: 11

So I guess all I have to do is make sure Emacs doesn't use the old
libfreetype in /opt/X11/lib, but rather the newer one in /usr/local/lib
(and hope that that one works in an X11 environment...)

I'll be in meetings for the next few hours, but will let you know what
happens once I find time to do the above.

Thanks a lot so far!!!

Cheers,
-- 
Simon.

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[Simon Leinen]

Simon Leinen writes:
> So I guess all I have to do is make sure Emacs doesn't use the old
> libfreetype in /opt/X11/lib, but rather the newer one in
> /usr/local/lib (and hope that that one works in an X11 environment...)

I finally managed to get the Emacs Makefiles put
-L/usr/local/opt/freetype/lib in front of -L/opt/X11/lib, and rebuilt my
Emacs binaries.  They now no longer crash when I open the message with
the Arabic font in GNUS.  Hooray!

So the problem was a typical "DLL hell" issue that I brought upon myself
by selecting an unusual combination of libraries.  I don't think there's
much for the Emacs developers to improve in this situation.  Therefore I
think this bug can safely be closed.

I wrote up my (fixed) build procedure on my blog:

https://blog.simon.leinen.ch/2019/07/compiling-emacs-on-mac-os-against-x11.html

This is mainly for my own reference - it's so convoluted that I rely on
config.log to reconstruct all the options... but the Internet is big,
and maybe it's even useful for someone else.

Thanks again for your kind help with this!

Best regards,
-- 
Simon.

bug#36758: 27.0.50; Mac OS/Lucid/X11: crash with libharfbuzz

[YAMAMOTO Mitsuharu]

On Tue, 23 Jul 2019 02:20:23 +0900,
Simon Leinen wrote:
> 
> Simon Leinen writes:
> > So I guess all I have to do is make sure Emacs doesn't use the old
> > libfreetype in /opt/X11/lib, but rather the newer one in
> > /usr/local/lib (and hope that that one works in an X11 environment...)
> 
> I finally managed to get the Emacs Makefiles put
> -L/usr/local/opt/freetype/lib in front of -L/opt/X11/lib, and rebuilt my
> Emacs binaries.  They now no longer crash when I open the message with
> the Arabic font in GNUS.  Hooray!
> 
> So the problem was a typical "DLL hell" issue that I brought upon myself
> by selecting an unusual combination of libraries.  I don't think there's
> much for the Emacs developers to improve in this situation.  Therefore I
> think this bug can safely be closed.

Thanks for confirming it.  Closing the bug.

For future reference, FreeType 2.7.0 bundled with XQuartz 2.7.11
crashes if used with HarfBuzz 2.5.3, but FreeType 2.10.1 doesn't.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp