changed dlopen flags in dynlib.c, gccemacs crash

changed dlopen flags in dynlib.c, gccemacs crash

[hx]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

Hi,

I changed the flags of dlopen in dynlib.c:

dynlib_handle_ptr
dynlib_open (const char *path)
{
  // return dlopen (path, RTLD_LAZY);



  return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
}

then my dynamic module can load other .so in emacs-27.

but it make gccemacs-28 crash:

$ /usr/src/emacs/src/emacs -nw -q
Fatal error 11: Segmentation fault
Backtrace:
/usr/src/emacs/src/emacs(+0x1c4c13)[0x558a8d05fc13]
/usr/src/emacs/src/emacs(+0x199a0e)[0x558a8d034a0e]
/usr/src/emacs/src/emacs(+0x1c44d8)[0x558a8d05f4d8]
/usr/src/emacs/src/emacs(+0x1c44ad)[0x558a8d05f4ad]
/usr/src/emacs/src/emacs(+0x1c4513)[0x558a8d05f513]
/usr/src/emacs/src/emacs(+0x1c46ab)[0x558a8d05f6ab]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x7f3bba1a4140]
/usr/src/emacs/src/emacs(+0x2b60b6)[0x558a8d1510b6]
/home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/cconv-3b1f1f98-cca24f72.eln(top_level_run+0x87)[0x7f3bb37045a7]
/usr/src/emacs/src/emacs(+0x2b5d18)[0x558a8d150d18]
/usr/src/emacs/src/emacs(+0x2b6830)[0x558a8d151830]

[-- Attachment #2: Type: text/html, Size: 1530 bytes --]

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: hx <silent2600@gmail.com>
> Date: Wed, 8 Dec 2021 14:01:29 +0800
> 
> I changed the flags of dlopen in dynlib.c:
> 
> dynlib_handle_ptr
> dynlib_open (const char *path)
> {
>   // return dlopen (path, RTLD_LAZY);                                                                                                                    
>                                                                                                                       
>   return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
> }
> 
> then my dynamic module can load other .so in emacs-27.
> 
> but it make gccemacs-28 crash:
> 
> $ /usr/src/emacs/src/emacs -nw -q
> Fatal error 11: Segmentation fault
> Backtrace:
> /usr/src/emacs/src/emacs(+0x1c4c13)[0x558a8d05fc13]
> /usr/src/emacs/src/emacs(+0x199a0e)[0x558a8d034a0e]
> /usr/src/emacs/src/emacs(+0x1c44d8)[0x558a8d05f4d8]

Please run this under GDB and show the backtrace from the crash in
human-readable form.

Also, please describe what you did immediately before the crash.

Thanks.

Re: changed dlopen flags in dynlib.c, gccemacs crash

[hx]

[-- Attachment #1: Type: text/plain, Size: 11231 bytes --]

remove the directory (28.0.90-xxx) from ~/.emacs.d/eln-cache/,
start emacs with -nw -q, it will crash after a few seconds without any
operation.
after that, it will crash immediately when I start it with the same
parameters.

here is the backtrace:

Thread 1 "emacs" received signal SIGSEGV, Segmentation fault.

 Fcomp__register_lambda (reloc_idx=0xa, c_name=0x555555e5c6f3, minarg=0x16,
maxarg=0x16, type=0x0, rest=0x555555e76854, comp_u=0x555555e65ab5) at
comp.c:5195
5195  Lisp_Object intspec = SECOND (rest);
(gdb) bt
#0  Fcomp__register_lambda (reloc_idx=0xa, c_name=0x555555e5c6f3,
minarg=0x16, maxarg=0x16, type=0x0, rest=0x555555e76854,
comp_u=0x555555e65ab5) at comp.c:5195
#1  0x00007fffef1605a7 in top_level_run () at
/home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/cconv-3b1f1f98-cca24f72.eln
#2  0x0000555555809d18 in load_comp_unit (comp_u=0x555555e65ab0,
loading_dump=false, late_load=false) at comp.c:5093
#3  0x000055555580a830 in Fnative_elisp_load (filename=0x555555e51534,
late_load=0x0) at comp.c:5316
#4  0x00005555557e1d36 in Fload (file=0x555555e081c4, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1565
#5  0x00005555557e207e in save_match_data_load (file=0x555555e081c4,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#6  0x00005555557bdf33 in Frequire (feature=0x19b100, filename=0x0,
noerror=0x0) at fns.c:3184
#7  0x00005555557af134 in funcall_subr (subr=0x555555c67ce0 <Srequire>,
numargs=1, args=0x7fffffff85f0) at eval.c:3103
#8  0x00005555557aecde in Ffuncall (nargs=2, args=0x7fffffff85e8) at
eval.c:3023
#9  0x00005555557fb3e2 in exec_byte_code (bytestr=0x555555e081e4,
vector=0x555555dc9e05, maxdepth=0x22, args_template=0x0, nargs=0, args=0x0)
at bytecode.c:632
#10 0x00005555557fa8de in Fbyte_code (bytestr=0x555555e081e4,
vector=0x555555dc9e05, maxdepth=0x22) at bytecode.c:334
#11 0x00005555557ad73c in eval_sub (form=0x555555dcb8f3) at eval.c:2504
#12 0x00005555557e40d4 in readevalloop (readcharfun=0x7830,
infile0=0x7fffffff8d30, sourcename=0x555555e07844, printflag=false,
unibyte=0x0, readfun=0x0, start=0x0, end=0x0) at lread.c:2326
#13 0x00005555557e1df8 in Fload (file=0x7ffff01f0ae4, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1579
#14 0x00005555557e207e in save_match_data_load (file=0x7ffff01f0ae4,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#15 0x00005555557acc82 in Fautoload_do_load (fundef=0x7ffff01f0ab3,
funname=0x2aaa9a5158c0, macro_only=0x0) at eval.c:2295
#16 0x00005555557aee50 in Ffuncall (nargs=2, args=0x7fffffff8f98) at
eval.c:3042
#17 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff05cf9ac,
vector=0x7ffff01ebf3d, maxdepth=0x76, args_template=0x406, nargs=1,
args=0x7fffffff95b8) at bytecode.c:632
#18 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff01ebf0d,
syms_left=0x406, nargs=1, args=0x7fffffff95b0) at eval.c:3147
#19 0x00005555557af71a in funcall_lambda (fun=0x7ffff01ebf0d, nargs=1,
arg_vector=0x7fffffff95b0) at eval.c:3228
#20 0x00005555557aed32 in Ffuncall (nargs=2, args=0x7fffffff95a8) at
eval.c:3027
#21 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff05d1754,
vector=0x7ffff01ebebd, maxdepth=0x26, args_template=0xc0e, nargs=3,
args=0x7fffffff9ad0) at bytecode.c:632
#22 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff01ebe8d,
syms_left=0xc0e, nargs=3, args=0x7fffffff9ab8) at eval.c:3147
#23 0x00005555557af71a in funcall_lambda (fun=0x7ffff01ebe8d, nargs=3,
arg_vector=0x7fffffff9ab8) at eval.c:3228
#24 0x00005555557aed32 in Ffuncall (nargs=4, args=0x7fffffff9ab0) at
eval.c:3027
#25 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff053bb64,
vector=0x7ffff053bb1d, maxdepth=0x22, args_template=0x406, nargs=1,
args=0x7fffffffa030) at bytecode.c:632
#26 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff053baed,
syms_left=0x406, nargs=1, args=0x7fffffffa028) at eval.c:3147
#27 0x00005555557af71a in funcall_lambda (fun=0x7ffff053baed, nargs=1,
arg_vector=0x7fffffffa028) at eval.c:3228
#28 0x00005555557aed32 in Ffuncall (nargs=2, args=0x7fffffffa020) at
eval.c:3027
#29 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff05cef34,
vector=0x7ffff01f47ed, maxdepth=0x66, args_template=0x1416, nargs=5,
args=0x7fffffffa698) at bytecode.c:632
#30 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff01f47bd,
syms_left=0x1416, nargs=5, args=0x7fffffffa670) at eval.c:3147
#31 0x00005555557af71a in funcall_lambda (fun=0x7ffff01f47bd, nargs=5,
arg_vector=0x7fffffffa670) at eval.c:3228
#32 0x00005555557aed32 in Ffuncall (nargs=6, args=0x7fffffffa668) at
eval.c:3027
#33 0x00005555557fb3e2 in exec_byte_code (bytestr=0x555555e76514,
vector=0x555555e75485, maxdepth=0x1a, args_template=0x0, nargs=0, args=0x0)
at bytecode.c:632
#34 0x00005555557fa8de in Fbyte_code (bytestr=0x555555e76514,
vector=0x555555e75485, maxdepth=0x1a) at bytecode.c:334
#35 0x00005555557ad73c in eval_sub (form=0x555555e5af83) at eval.c:2504
#36 0x00005555557ace05 in Feval (form=0x555555e5af83, lexical=0x30) at
eval.c:2327
#37 0x00007fffef176107 in top_level_run () at
/home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/xterm-6ea901d1-05e64e57.eln
#38 0x0000555555809d18 in load_comp_unit (comp_u=0x555555e43630,
loading_dump=false, late_load=false) at comp.c:5093
#39 0x000055555580a830 in Fnative_elisp_load (filename=0x555555e55af4,
late_load=0x0) at comp.c:5316
#40 0x00005555557e1d36 in Fload (file=0x555555e52124, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x0) at lread.c:1565
#41 0x00005555557e207e in save_match_data_load (file=0x555555e52124,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#42 0x00005555557bdf33 in Frequire (feature=0x117670, filename=0x0,
noerror=0x0) at fns.c:3184
#43 0x00005555557af134 in funcall_subr (subr=0x555555c67ce0 <Srequire>,
numargs=1, args=0x7fffffffb1d0) at eval.c:3103
#44 0x00005555557aecde in Ffuncall (nargs=2, args=0x7fffffffb1c8) at
eval.c:3023
#45 0x00005555557fb3e2 in exec_byte_code (bytestr=0x555555e52104,
vector=0x555555e4359d, maxdepth=0x22, args_template=0x0, nargs=0, args=0x0)
at bytecode.c:632
#46 0x00005555557fa8de in Fbyte_code (bytestr=0x555555e52104,
vector=0x555555e4359d, maxdepth=0x22) at bytecode.c:334
#47 0x00005555557ad73c in eval_sub (form=0x555555e34933) at eval.c:2504
#48 0x00005555557e40d4 in readevalloop (readcharfun=0x7830,
infile0=0x7fffffffb900, sourcename=0x555555e52044, printflag=false,
unibyte=0x0, readfun=0x0, start=0x0, end=0x0) at lread.c:2326
#49 0x00005555557e1df8 in Fload (file=0x555555e51e14, noerror=0x30,
nomessage=0x30, nosuffix=0x0, must_suffix=0x0) at lread.c:1579
#50 0x00005555557af1b8 in funcall_subr (subr=0x555555c69900 <Sload>,
numargs=3, args=0x7fffffffbb00) at eval.c:3110
#51 0x00005555557aecde in Ffuncall (nargs=4, args=0x7fffffffbaf8) at
eval.c:3023
#52 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff025a724,
vector=0x7ffff025a2ed, maxdepth=0x1e, args_template=0x406, nargs=1,
args=0x7fffffffc010) at bytecode.c:632
#53 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff025a2bd,
syms_left=0x406, nargs=1, args=0x7fffffffc008) at eval.c:3147
#54 0x00005555557af71a in funcall_lambda (fun=0x7ffff025a2bd, nargs=1,
arg_vector=0x7fffffffc008) at eval.c:3228
#55 0x00005555557aed32 in Ffuncall (nargs=2, args=0x7fffffffc000) at
eval.c:3027
#56 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff025b344,
vector=0x7ffff025b2ed, maxdepth=0x2a, args_template=0x80a, nargs=2,
args=0x7fffffffc550) at bytecode.c:632
#57 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff025b2bd,
syms_left=0x80a, nargs=2, args=0x7fffffffc540) at eval.c:3147
#58 0x00005555557af71a in funcall_lambda (fun=0x7ffff025b2bd, nargs=2,
arg_vector=0x7fffffffc540) at eval.c:3228
#59 0x00005555557aed32 in Ffuncall (nargs=3, args=0x7fffffffc538) at
eval.c:3027
#60 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff025b384,
vector=0x7ffff0259f45, maxdepth=0x2a, args_template=0xc06, nargs=3,
args=0x7fffffffca98) at bytecode.c:632
#61 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff0259f15,
syms_left=0xc06, nargs=3, args=0x7fffffffca80) at eval.c:3147
#62 0x00005555557af71a in funcall_lambda (fun=0x7ffff0259f15, nargs=3,
arg_vector=0x7fffffffca80) at eval.c:3228
#63 0x00005555557aed32 in Ffuncall (nargs=4, args=0x7fffffffca78) at
eval.c:3027
#64 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff04d5c4c,
vector=0x7ffff03295c5, maxdepth=0x3a, args_template=0x2, nargs=0,
args=0x7fffffffd538) at bytecode.c:632
--Type <RET> for more, q to quit, c to continue without paging--
#65 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff0329595,
syms_left=0x2, nargs=0, args=0x7fffffffd538) at eval.c:3147
#66 0x00005555557af71a in funcall_lambda (fun=0x7ffff0329595, nargs=0,
arg_vector=0x7fffffffd538) at eval.c:3228
#67 0x00005555557aed32 in Ffuncall (nargs=1, args=0x7fffffffd530) at
eval.c:3027
#68 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff04d7f34,
vector=0x7ffff0328c0d, maxdepth=0x26, args_template=0x2, nargs=0,
args=0x7fffffffdc20) at bytecode.c:632
#69 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff0328bdd,
syms_left=0x2, nargs=0, args=0x7fffffffdc20) at eval.c:3147
#70 0x00005555557af71a in funcall_lambda (fun=0x7ffff0328bdd, nargs=0,
arg_vector=0x7fffffffdc20) at eval.c:3228
#71 0x00005555557af537 in apply_lambda (fun=0x7ffff0328bdd, args=0x0,
count=4) at eval.c:3172
#72 0x00005555557ad91c in eval_sub (form=0x7ffff08cae13) at eval.c:2545
#73 0x00005555557ace05 in Feval (form=0x7ffff08cae13, lexical=0x0) at
eval.c:2327
#74 0x00005555556f462a in top_level_2 () at keyboard.c:1142
#75 0x00005555557ab0f0 in internal_condition_case (bfun=0x5555556f4607
<top_level_2>, handlers=0x90, hfun=0x5555556f3f73 <cmd_error>) at
eval.c:1450
#76 0x00005555556f4672 in top_level_1 (ignore=0x0) at keyboard.c:1150
#77 0x00005555557aa822 in internal_catch (tag=0xe7f0, func=0x5555556f462c
<top_level_1>, arg=0x0) at eval.c:1181
#78 0x00005555556f455b in command_loop () at keyboard.c:1110
#79 0x00005555556f3b28 in recursive_edit_1 () at keyboard.c:720
#80 0x00005555556f3cc7 in Frecursive_edit () at keyboard.c:803
#81 0x00005555556f0717 in main (argc=3, argv=0x7fffffffe198) at emacs.c:2354
(gdb) q


$ ls 28.0.90-8dcada16/
cconv-3b1f1f98-cca24f72.eln  xterm-6ea901d1-05e64e57.eln
$

On Wed, Dec 8, 2021 at 9:15 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: hx <silent2600@gmail.com>
> > Date: Wed, 8 Dec 2021 14:01:29 +0800
> >
> > I changed the flags of dlopen in dynlib.c:
> >
> > dynlib_handle_ptr
> > dynlib_open (const char *path)
> > {
> >   // return dlopen (path, RTLD_LAZY);
>
>
> >
>
> >   return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
> > }
> >
> > then my dynamic module can load other .so in emacs-27.
> >
> > but it make gccemacs-28 crash:
> >
> > $ /usr/src/emacs/src/emacs -nw -q
> > Fatal error 11: Segmentation fault
> > Backtrace:
> > /usr/src/emacs/src/emacs(+0x1c4c13)[0x558a8d05fc13]
> > /usr/src/emacs/src/emacs(+0x199a0e)[0x558a8d034a0e]
> > /usr/src/emacs/src/emacs(+0x1c44d8)[0x558a8d05f4d8]
>
> Please run this under GDB and show the backtrace from the crash in
> human-readable form.
>
> Also, please describe what you did immediately before the crash.
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 12650 bytes --]

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: hx <silent2600@gmail.com>
> Date: Thu, 9 Dec 2021 08:51:03 +0800
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> remove the directory (28.0.90-xxx) from ~/.emacs.d/eln-cache/,
> start emacs with -nw -q, it will crash after a few seconds without any operation.
> after that, it will crash immediately when I start it with the same parameters.

It doesn't crash when I try reproducing it, so something is missing
from the recipe.  Is this with an unmodified Emacs 28.0.90, or is it
with your change in dynlib.c?

Anyway, does the patch below help?

Andrea, any better ideas?  I don't understand the reason for the
crash, as we seem to have defenses against this in Fnative_elisp_load,
so I'm probably missing something.

diff --git a/src/comp.c b/src/comp.c
index 43feac6..42eb0af 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -5191,6 +5191,8 @@ DEFUN ("comp--register-lambda", Fcomp__register_lambda, Scomp__register_lambda,
    Lisp_Object maxarg, Lisp_Object type, Lisp_Object rest,
    Lisp_Object comp_u)
 {
+  CHECK_CONS (rest);
+
   Lisp_Object doc_idx = FIRST (rest);
   Lisp_Object intspec = SECOND (rest);
   struct Lisp_Native_Comp_Unit *cu = XNATIVE_COMP_UNIT (comp_u);

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: hx <silent2600@gmail.com>
>> Date: Thu, 9 Dec 2021 08:51:03 +0800
>> Cc: emacs-devel <emacs-devel@gnu.org>
>> 
>> remove the directory (28.0.90-xxx) from ~/.emacs.d/eln-cache/,
>> start emacs with -nw -q, it will crash after a few seconds without any operation.
>> after that, it will crash immediately when I start it with the same parameters.
>
> It doesn't crash when I try reproducing it, so something is missing
> from the recipe.  Is this with an unmodified Emacs 28.0.90, or is it
> with your change in dynlib.c?
>
> Anyway, does the patch below help?
>
> Andrea, any better ideas?  I don't understand the reason for the
> crash, as we seem to have defenses against this in Fnative_elisp_load,
> so I'm probably missing something.

Hello all,

I'm no expert on 'dlopen' so I'm not sure about the impact of using
RTLD_GLOBAL.  But should be noted that .eln files do share a number of
symbol names, I'm wondering if this could be somehow problematic when
using RTLD_GLOBAL, that's my suspect here.

  Andrea

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: hx <silent2600@gmail.com>, emacs-devel@gnu.org
> Date: Thu, 09 Dec 2021 10:53:57 +0000
> 
> > Andrea, any better ideas?  I don't understand the reason for the
> > crash, as we seem to have defenses against this in Fnative_elisp_load,
> > so I'm probably missing something.
> 
> Hello all,
> 
> I'm no expert on 'dlopen' so I'm not sure about the impact of using
> RTLD_GLOBAL.  But should be noted that .eln files do share a number of
> symbol names, I'm wondering if this could be somehow problematic when
> using RTLD_GLOBAL, that's my suspect here.

Assuming the RTLD_GLOBAL flag was added.

But still, the crash is because we assume a Lisp object is a cons cell
and take its CDR.  So evidently it isn't a cons cell (probably nil),
and I don't understand how that could be caused, not even if
RTLD_GLOBAL was used.

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: hx <silent2600@gmail.com>, emacs-devel@gnu.org
>> Date: Thu, 09 Dec 2021 10:53:57 +0000
>> 
>> > Andrea, any better ideas?  I don't understand the reason for the
>> > crash, as we seem to have defenses against this in Fnative_elisp_load,
>> > so I'm probably missing something.
>> 
>> Hello all,
>> 
>> I'm no expert on 'dlopen' so I'm not sure about the impact of using
>> RTLD_GLOBAL.  But should be noted that .eln files do share a number of
>> symbol names, I'm wondering if this could be somehow problematic when
>> using RTLD_GLOBAL, that's my suspect here.
>
> Assuming the RTLD_GLOBAL flag was added.

Yep

> But still, the crash is because we assume a Lisp object is a cons cell
> and take its CDR.  So evidently it isn't a cons cell (probably nil),
> and I don't understand how that could be caused, not even if
> RTLD_GLOBAL was used.

I'm just speculating *but*: `comp--register-lambda' it's called directly
from the eln synthetized code that is run when the .eln is loaded.

The arguments of `comp--register-lambda' are immediates that are set
into the eln not long before, and this process (in 'load_comp_unit')
indeed requires symbol resolution (through our `dynlib_sym').

My guess is that something goes wrong in that area (but as mentioned
I've no proof of that).

  Andrea

Re: changed dlopen flags in dynlib.c, gccemacs crash

[hx]

[-- Attachment #1: Type: text/plain, Size: 15356 bytes --]

git diff
diff --git a/src/comp.c b/src/comp.c
index 43feac6..2941f84 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -5191,6 +5191,8 @@ DEFUN ("comp--register-lambda",
Fcomp__register_lambda, Scomp__register_lambda,
    Lisp_Object maxarg, Lisp_Object type, Lisp_Object rest,
    Lisp_Object comp_u)
 {
+  CHECK_CONS(rest);
+
   Lisp_Object doc_idx = FIRST (rest);
   Lisp_Object intspec = SECOND (rest);
   struct Lisp_Native_Comp_Unit *cu = XNATIVE_COMP_UNIT (comp_u);
diff --git a/src/dynlib.c b/src/dynlib.c
index a8c8843..362530b 100644
--- a/src/dynlib.c
+++ b/src/dynlib.c
@@ -270,7 +270,8 @@ dynlib_close (dynlib_handle_ptr h)
 dynlib_handle_ptr
 dynlib_open (const char *path)
 {
-  return dlopen (path, RTLD_LAZY);
+  // return dlopen (path, RTLD_LAZY);
+  return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
 }


delete the eln-cache, execute with -nw -q in gdb, wait a few seconds:
(gdb) bt
#0  _dl_lookup_symbol_x (undef_name=0x555555e3814300 <error: Cannot access
memory at address 0x555555e3814300>, undef_map=0x555556059380,
ref=0x7fffffffc1b8, symbol_scope=0x555556059718, version=0x0, type_class=0,
flags=2, skip_map=0x0) at dl-lookup.c:842
#1  0x00007ffff4082b74 in do_sym (handle=<optimized out>,
name=0x555555e3814300 <error: Cannot access memory at address
0x555555e3814300>, who=0x55555580b073 <dynlib_sym+35>, vers=vers@entry=0x0,
flags=flags@entry=2) at dl-sym.c:165
#2  0x00007ffff408305d in _dl_sym (handle=<optimized out>, name=<optimized
out>, who=<optimized out>) at dl-sym.c:274
#3  0x00007ffff5bdf3b4 in dlsym_doit (a=a@entry=0x7fffffffc3f0) at
dlsym.c:50
#4  0x00007ffff4083260 in __GI__dl_catch_exception
(exception=exception@entry=0x7fffffffc390, operate=0x7ffff5bdf3a0
<dlsym_doit>, args=0x7fffffffc3f0) at dl-error-skeleton.c:208
#5  0x00007ffff408331f in __GI__dl_catch_error (objname=0x555555dab830,
errstring=0x555555dab838, mallocedp=0x555555dab828, operate=<optimized
out>, args=<optimized out>) at dl-error-skeleton.c:227
#6  0x00007ffff5bdfa65 in _dlerror_run (operate=operate@entry=0x7ffff5bdf3a0
<dlsym_doit>, args=args@entry=0x7fffffffc3f0) at dlerror.c:170
#7  0x00007ffff5bdf41c in __dlsym (handle=<optimized out>, name=<optimized
out>) at dlsym.c:70
#8  0x000055555580b073 in dynlib_sym (h=0x555556059380,
sym=0x555555e3814300 <error: Cannot access memory at address
0x555555e3814300>) at dynlib.c:280
#9  0x0000555555809f63 in make_subr (symbol_name=0x555555e38153,
minarg=0x16, maxarg=0x16, c_name=0x555555e38153, type=0xfc30, doc_idx=0xba,
intspec=0x0, comp_u=0x555555df6d1d) at comp.c:5147
#10 0x000055555580a156 in Fcomp__register_lambda (reloc_idx=0xa,
c_name=0x555555e38153, minarg=0x16, maxarg=0x16, type=0xfc30,
rest=0x555555e17c03, comp_u=0x555555df6d1d) at comp.c:5203
#11 0x00007fffef0eed34 in late_top_level_run () at
/home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/cconv-3b1f1f98-cca24f72.eln
#12 0x0000555555809d51 in load_comp_unit (comp_u=0x555555df6d18,
loading_dump=false, late_load=true) at comp.c:5093
#13 0x000055555580a875 in Fnative_elisp_load (filename=0x555556058174,
late_load=0x30) at comp.c:5318
#14 0x00005555557af101 in funcall_subr (subr=0x555555c6aaa0
<Snative_elisp_load>, numargs=2, args=0x7fffffffc848) at eval.c:3100
#15 0x00005555557aecde in Ffuncall (nargs=3, args=0x7fffffffc840) at
eval.c:3023
#16 0x00005555557fb3e2 in exec_byte_code (bytestr=0x55555600ac44,
vector=0x555555f472bd, maxdepth=0x1e, args_template=0x80a, nargs=2,
args=0x7fffffffcd38) at bytecode.c:632
#17 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x555555ee0ce5,
syms_left=0x80a, nargs=2, args=0x7fffffffcd28) at eval.c:3147
#18 0x00005555557af71a in funcall_lambda (fun=0x555555ee0ce5, nargs=2,
arg_vector=0x7fffffffcd28) at eval.c:3228
#19 0x00005555557aed32 in Ffuncall (nargs=3, args=0x7fffffffcd20) at
eval.c:3027
#20 0x00005555557adfe8 in Fapply (nargs=2, args=0x7fffffffcde0) at
eval.c:2653
#21 0x00005555557ae552 in apply1 (fn=0x555555ee0ce5, arg=0x555555f0eb43) at
eval.c:2869
#22 0x0000555555818f75 in read_process_output_call
(fun_and_args=0x555555f0eaf3) at process.c:5971
#23 0x00005555557ab197 in internal_condition_case_1 (bfun=0x555555818f47
<read_process_output_call>, arg=0x555555f0eaf3, handlers=0x90,
hfun=0x55555581bee7 <exec_sentinel_error_handler>) at eval.c:1474
#24 0x000055555581c103 in exec_sentinel (proc=0x555555e1699d,
reason=0x55555605a154) at process.c:7457
#25 0x000055555581c338 in status_notify (deleting_process=0x0,
wait_proc=0x0) at process.c:7549
#26 0x0000555555817885 in wait_reading_process_output (time_limit=0,
nsecs=0, read_kbd=-1, do_display=true, wait_for_cell=0x0, wait_proc=0x0,
just_wait_proc=0) at process.c:5374
#27 0x00005555556facff in kbd_buffer_get_event (kbp=0x7fffffffd488,
used_mouse_menu=0x7fffffffdacf, end_time=0x0) at keyboard.c:3926
#28 0x00005555556f6fbd in read_event_from_main_queue (end_time=0x0,
local_getcjmp=0x7fffffffd860, used_mouse_menu=0x7fffffffdacf) at
keyboard.c:2198
#29 0x00005555556f7254 in read_decoded_event_from_main_queue (end_time=0x0,
local_getcjmp=0x7fffffffd860, prev_event=0x0,
used_mouse_menu=0x7fffffffdacf) at keyboard.c:2262
#30 0x00005555556f88cf in read_char (commandflag=1, map=0x5555560ae943,
prev_event=0x0, used_mouse_menu=0x7fffffffdacf, end_time=0x0) at
keyboard.c:2892
#31 0x0000555555706f8f in read_key_sequence (keybuf=0x7fffffffdcd0,
prompt=0x0, dont_downcase_last=false, can_return_switch_frame=true,
fix_current_buffer=true, prevent_redisplay=false) at keyboard.c:9635
#32 0x00005555556f4d5d in command_loop_1 () at keyboard.c:1392
#33 0x00005555557ab0f0 in internal_condition_case (bfun=0x5555556f4917
<command_loop_1>, handlers=0x90, hfun=0x5555556f3f73 <cmd_error>) at
eval.c:1450
#34 0x00005555556f45df in command_loop_2 (handlers=0x90) at keyboard.c:1133
#35 0x00005555557aa822 in internal_catch (tag=0xe7f0, func=0x5555556f45b8
<command_loop_2>, arg=0x90) at eval.c:1181
#36 0x00005555556f4584 in command_loop () at keyboard.c:1111
#37 0x00005555556f3b28 in recursive_edit_1 () at keyboard.c:720
#38 0x00005555556f3cc7 in Frecursive_edit () at keyboard.c:803
#39 0x00005555556f0717 in main (argc=3, argv=0x7fffffffe1b8) at emacs.c:2354
(gdb)

~/.emacs.d/eln-cache$ ls 28.0.90-8dcada16/
bytecomp-12882072-29d9ad62.eln  cconv-3b1f1f98-cca24f72.eln
 xterm-6ea901d1-05e64e57.eln

 execute it again, emacs display message in minibuffer: Wrong type
argument: consp,
"F787465726d2d7472616e736c6174652d666f6375732d696e_xterm_translate_focus_in_0"
*Messages* buffer: cl--generic-get-dispatcher: Wrong type argument: consp,
"F787465726d2d7472616e736c6174652d666f6375732d696e_xterm_translate_focus_in_0"

then I open my init.el, execute (package-initialize)  c-x c-e, segfault:
(gdb) bt
#0  Fcomp__register_lambda (reloc_idx=0x1a, c_name=0x555555e79234,
minarg=0x2, maxarg=0x2, type=0x0, rest=0x555555e5bb73,
comp_u=0x555555eade25) at comp.c:5197
#1  0x00007fffef10c886 in top_level_run () at
/home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/bytecomp-12882072-29d9ad62.eln
#2  0x0000555555809d51 in load_comp_unit (comp_u=0x555555eade20,
loading_dump=false, late_load=false) at comp.c:5093
#3  0x000055555580a875 in Fnative_elisp_load (filename=0x555555e3b0a4,
late_load=0x0) at comp.c:5318
#4  0x00005555557e1d36 in Fload (file=0x7ffff03a682c, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1565
#5  0x00005555557e207e in save_match_data_load (file=0x7ffff03a682c,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#6  0x00005555557bdf33 in Frequire (feature=0x2aaa9a6cb638, filename=0x0,
noerror=0x0) at fns.c:3184
#7  0x00005555557af134 in funcall_subr (subr=0x555555c67ce0 <Srequire>,
numargs=1, args=0x7fffffffa9b0) at eval.c:3103
#8  0x00005555557aecde in Ffuncall (nargs=2, args=0x7fffffffa9a8) at
eval.c:3023
#9  0x00005555557fb3e2 in exec_byte_code (bytestr=0x555555e4ea34,
vector=0x555555dd07a5, maxdepth=0x2a, args_template=0x0, nargs=0, args=0x0)
at bytecode.c:632
#10 0x00005555557fa8de in Fbyte_code (bytestr=0x555555e4ea34,
vector=0x555555dd07a5, maxdepth=0x2a) at bytecode.c:334
#11 0x00005555557ad73c in eval_sub (form=0x555555eb2773) at eval.c:2504
#12 0x00005555557e40d4 in readevalloop (readcharfun=0x7830,
infile0=0x7fffffffb240, sourcename=0x555555e4e074, printflag=false,
unibyte=0x0, readfun=0x0, start=0x0, end=0x0) at lread.c:2326
#13 0x00005555557e1df8 in Fload (file=0x7ffff04df174, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1579
#14 0x00005555557e207e in save_match_data_load (file=0x7ffff04df174,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#15 0x00005555557bdf33 in Frequire (feature=0x4680, filename=0x0,
noerror=0x0) at fns.c:3184
#16 0x000055555580936f in maybe_defer_native_compilation
(function_name=0x17dbf0, definition=0x555555de50fd) at comp.c:4870
#17 0x000055555578d8d1 in Fdefalias (symbol=0x555555e5bb73,
definition=0x555555de50fd, docstring=0x0) at data.c:830
#18 0x00005555557ad73c in eval_sub (form=0x555555eb2923) at eval.c:2504
#19 0x00005555557e40d4 in readevalloop (readcharfun=0x7830,
infile0=0x7fffffffb850, sourcename=0x555555ddcac4, printflag=false,
unibyte=0x0, readfun=0x0, start=0x0, end=0x0) at lread.c:2326
#20 0x00005555557e1df8 in Fload (file=0x7ffff02b12fc, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1579
#21 0x00005555557e207e in save_match_data_load (file=0x7ffff02b12fc,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#22 0x00005555557bdf33 in Frequire (feature=0x2aaa9a5d6108, filename=0x0,
noerror=0x0) at fns.c:3184
#23 0x00005555557af134 in funcall_subr (subr=0x555555c67ce0 <Srequire>,
numargs=1, args=0x7fffffffbb50) at eval.c:3103
#24 0x00005555557aecde in Ffuncall (nargs=2, args=0x7fffffffbb48) at
eval.c:3023
#25 0x00005555557fb3e2 in exec_byte_code (bytestr=0x555555e07694,
vector=0x555555e2c4c5, maxdepth=0x32, args_template=0x0, nargs=0, args=0x0)
at bytecode.c:632
#26 0x00005555557fa8de in Fbyte_code (bytestr=0x555555e07694,
vector=0x555555e2c4c5, maxdepth=0x32) at bytecode.c:334
#27 0x00005555557ad73c in eval_sub (form=0x555555eb3db3) at eval.c:2504
#28 0x00005555557e40d4 in readevalloop (readcharfun=0x7830,
infile0=0x7fffffffc370, sourcename=0x555555e07c94, printflag=false,
unibyte=0x0, readfun=0x0, start=0x0, end=0x0) at lread.c:2326
#29 0x00005555557e1df8 in Fload (file=0x7ffff01f3474, noerror=0x0,
nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1579
#30 0x00005555557e207e in save_match_data_load (file=0x7ffff01f3474,
noerror=0x0, nomessage=0x30, nosuffix=0x0, must_suffix=0x30) at lread.c:1629
#31 0x00005555557acc82 in Fautoload_do_load (fundef=0x7ffff07c8973,
funname=0x2aaa9aaed780, macro_only=0x0) at eval.c:2295
#32 0x00005555557ad9f2 in eval_sub (form=0x555555eb3e83) at eval.c:2557
#33 0x00005555557ace05 in Feval (form=0x555555eb3e83, lexical=0x0) at
eval.c:2327
#34 0x00005555557af101 in funcall_subr (subr=0x555555c654c0 <Seval>,
numargs=2, args=0x7fffffffc7d8) at eval.c:3100
#35 0x00005555557aecde in Ffuncall (nargs=3, args=0x7fffffffc7d0) at
eval.c:3023
#36 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff039cf54,
vector=0x7ffff039c5f5, maxdepth=0x3e, args_template=0x406, nargs=1,
args=0x7fffffffcd00) at bytecode.c:632
#37 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff039c5c5,
syms_left=0x406, nargs=1, args=0x7fffffffccf8) at eval.c:3147
#38 0x00005555557af71a in funcall_lambda (fun=0x7ffff039c5c5, nargs=1,
arg_vector=0x7fffffffccf8) at eval.c:3228
#39 0x00005555557aed32 in Ffuncall (nargs=2, args=0x7fffffffccf0) at
eval.c:3027
#40 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff039d154,
vector=0x7ffff039c565, maxdepth=0x12, args_template=0x406, nargs=1,
args=0x7fffffffd2f8) at bytecode.c:632
#41 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff039c52d,
syms_left=0x406, nargs=1, args=0x7fffffffd2f0) at eval.c:3147
#42 0x00005555557af71a in funcall_lambda (fun=0x7ffff039c52d, nargs=1,
arg_vector=0x7fffffffd2f0) at eval.c:3228
#43 0x00005555557aed32 in Ffuncall (nargs=2, args=0x7fffffffd2e8) at
eval.c:3027
#44 0x00005555557a5579 in Ffuncall_interactively (nargs=2,
args=0x7fffffffd2e8) at callint.c:260
#45 0x00005555557af000 in funcall_subr (subr=0x555555c64920
<Sfuncall_interactively>, numargs=2, args=0x7fffffffd2e8) at eval.c:3078
#46 0x00005555557aecde in Ffuncall (nargs=3, args=0x7fffffffd2e0) at
eval.c:3023
#47 0x00005555557a792c in Fcall_interactively (function=0x2aaa9a6c1338,
record_flag=0x0, keys=0x555555dd112d) at callint.c:798
#48 0x00005555557af134 in funcall_subr (subr=0x555555c64980
<Scall_interactively>, numargs=3, args=0x7fffffffd720) at eval.c:3103
#49 0x00005555557aecde in Ffuncall (nargs=4, args=0x7fffffffd718) at
eval.c:3023
#50 0x00005555557fb3e2 in exec_byte_code (bytestr=0x7ffff02aa224,
vector=0x7ffff02a9e8d, maxdepth=0x36, args_template=0x1006, nargs=1,
args=0x7fffffffdca0) at bytecode.c:632
#51 0x00005555557af38d in fetch_and_exec_byte_code (fun=0x7ffff02a9e5d,
syms_left=0x1006, nargs=1, args=0x7fffffffdc98) at eval.c:3147
#52 0x00005555557af71a in funcall_lambda (fun=0x7ffff02a9e5d, nargs=1,
arg_vector=0x7fffffffdc98) at eval.c:3228
#53 0x00005555557aed32 in Ffuncall (nargs=2, args=0x7fffffffdc90) at
eval.c:3027
#54 0x00005555557ae5a4 in call1 (fn=0x4590, arg1=0x2aaa9a6c1338) at
eval.c:2883
#55 0x00005555556f5100 in command_loop_1 () at keyboard.c:1505
#56 0x00005555557ab0f0 in internal_condition_case (bfun=0x5555556f4917
<command_loop_1>, handlers=0x90, hfun=0x5555556f3f73 <cmd_error>) at
eval.c:1450
#57 0x00005555556f45df in command_loop_2 (handlers=0x90) at keyboard.c:1133
#58 0x00005555557aa822 in internal_catch (tag=0xe7f0, func=0x5555556f45b8
<command_loop_2>, arg=0x90) at eval.c:1181
#59 0x00005555556f4584 in command_loop () at keyboard.c:1111
#60 0x00005555556f3b28 in recursive_edit_1 () at keyboard.c:720
#61 0x00005555556f3cc7 in Frecursive_edit () at keyboard.c:803
#62 0x00005555556f0717 in main (argc=3, argv=0x7fffffffe1b8) at emacs.c:2354

I realized I should not change dynlib.c
but src/emacs-module.c: emacs-module.c:  handle = dynlib_open (SSDATA
(file));



On Thu, Dec 9, 2021 at 4:38 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: hx <silent2600@gmail.com>
> > Date: Thu, 9 Dec 2021 08:51:03 +0800
> > Cc: emacs-devel <emacs-devel@gnu.org>
> >
> > remove the directory (28.0.90-xxx) from ~/.emacs.d/eln-cache/,
> > start emacs with -nw -q, it will crash after a few seconds without any
> operation.
> > after that, it will crash immediately when I start it with the same
> parameters.
>
> It doesn't crash when I try reproducing it, so something is missing
> from the recipe.  Is this with an unmodified Emacs 28.0.90, or is it
> with your change in dynlib.c?
>
> Anyway, does the patch below help?
>
> Andrea, any better ideas?  I don't understand the reason for the
> crash, as we seem to have defenses against this in Fnative_elisp_load,
> so I'm probably missing something.
>
> diff --git a/src/comp.c b/src/comp.c
> index 43feac6..42eb0af 100644
> --- a/src/comp.c
> +++ b/src/comp.c
> @@ -5191,6 +5191,8 @@ DEFUN ("comp--register-lambda",
> Fcomp__register_lambda, Scomp__register_lambda,
>     Lisp_Object maxarg, Lisp_Object type, Lisp_Object rest,
>     Lisp_Object comp_u)
>  {
> +  CHECK_CONS (rest);
> +
>    Lisp_Object doc_idx = FIRST (rest);
>    Lisp_Object intspec = SECOND (rest);
>    struct Lisp_Native_Comp_Unit *cu = XNATIVE_COMP_UNIT (comp_u);
>

[-- Attachment #2: Type: text/html, Size: 16720 bytes --]

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: hx <silent2600@gmail.com>
> Date: Sat, 11 Dec 2021 10:29:33 +0800
> Cc: Andrea Corallo <akrl@sdf.org>, emacs-devel <emacs-devel@gnu.org>
> 
> git diff
> diff --git a/src/comp.c b/src/comp.c
> index 43feac6..2941f84 100644
> --- a/src/comp.c
> +++ b/src/comp.c
> @@ -5191,6 +5191,8 @@ DEFUN ("comp--register-lambda", Fcomp__register_lambda,
> Scomp__register_lambda,
>     Lisp_Object maxarg, Lisp_Object type, Lisp_Object rest,
>     Lisp_Object comp_u)
>  {
> +  CHECK_CONS(rest);
> +
>    Lisp_Object doc_idx = FIRST (rest);
>    Lisp_Object intspec = SECOND (rest);
>    struct Lisp_Native_Comp_Unit *cu = XNATIVE_COMP_UNIT (comp_u);
> diff --git a/src/dynlib.c b/src/dynlib.c
> index a8c8843..362530b 100644
> --- a/src/dynlib.c
> +++ b/src/dynlib.c
> @@ -270,7 +270,8 @@ dynlib_close (dynlib_handle_ptr h)
>  dynlib_handle_ptr
>  dynlib_open (const char *path)
>  {
> -  return dlopen (path, RTLD_LAZY);
> +  // return dlopen (path, RTLD_LAZY);
> +  return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
>  }
>  
> 
> delete the eln-cache, execute with -nw -q in gdb, wait a few seconds:
> (gdb) bt
> #0  _dl_lookup_symbol_x (undef_name=0x555555e3814300 <error: Cannot access memory at address
> 0x555555e3814300>, undef_map=0x555556059380, ref=0x7fffffffc1b8, symbol_scope=0x555556059718,
> version=0x0, type_class=0, flags=2, skip_map=0x0) at dl-lookup.c:842
> #1  0x00007ffff4082b74 in do_sym (handle=<optimized out>, name=0x555555e3814300 <error: Cannot
> access memory at address 0x555555e3814300>, who=0x55555580b073 <dynlib_sym+35>,
> vers=vers@entry=0x0, flags=flags@entry=2) at dl-sym.c:165
> #2  0x00007ffff408305d in _dl_sym (handle=<optimized out>, name=<optimized out>, who=<optimized out>)
> at dl-sym.c:274
> #3  0x00007ffff5bdf3b4 in dlsym_doit (a=a@entry=0x7fffffffc3f0) at dlsym.c:50
> #4  0x00007ffff4083260 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffc390,
> operate=0x7ffff5bdf3a0 <dlsym_doit>, args=0x7fffffffc3f0) at dl-error-skeleton.c:208
> #5  0x00007ffff408331f in __GI__dl_catch_error (objname=0x555555dab830, errstring=0x555555dab838,
> mallocedp=0x555555dab828, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:227
> #6  0x00007ffff5bdfa65 in _dlerror_run (operate=operate@entry=0x7ffff5bdf3a0 <dlsym_doit>,
> args=args@entry=0x7fffffffc3f0) at dlerror.c:170
> #7  0x00007ffff5bdf41c in __dlsym (handle=<optimized out>, name=<optimized out>) at dlsym.c:70
> #8  0x000055555580b073 in dynlib_sym (h=0x555556059380, sym=0x555555e3814300 <error: Cannot
> access memory at address 0x555555e3814300>) at dynlib.c:280
> #9  0x0000555555809f63 in make_subr (symbol_name=0x555555e38153, minarg=0x16, maxarg=0x16,
> c_name=0x555555e38153, type=0xfc30, doc_idx=0xba, intspec=0x0, comp_u=0x555555df6d1d) at
> comp.c:5147
> #10 0x000055555580a156 in Fcomp__register_lambda (reloc_idx=0xa, c_name=0x555555e38153,
> minarg=0x16, maxarg=0x16, type=0xfc30, rest=0x555555e17c03, comp_u=0x555555df6d1d) at comp.c:5203
> #11 0x00007fffef0eed34 in late_top_level_run () at
> /home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/cconv-3b1f1f98-cca24f72.eln

So I think this is a manifestation of the issue that Andrea was
talking about: the way we produce native code is incompatible with
RTLD_GLOBAL.

So at this point, I think this bug report should be treated as
"wishlist", i.e. a feature request: to support RTLD_GLOBAL in dynlib
with native-comp.  Andrea, is that feasible, and if so, what would it
entail?

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: hx <silent2600@gmail.com>
>> Date: Sat, 11 Dec 2021 10:29:33 +0800
>> Cc: Andrea Corallo <akrl@sdf.org>, emacs-devel <emacs-devel@gnu.org>
>> 
>> git diff
>> diff --git a/src/comp.c b/src/comp.c
>> index 43feac6..2941f84 100644
>> --- a/src/comp.c
>> +++ b/src/comp.c
>> @@ -5191,6 +5191,8 @@ DEFUN ("comp--register-lambda", Fcomp__register_lambda,
>> Scomp__register_lambda,
>>     Lisp_Object maxarg, Lisp_Object type, Lisp_Object rest,
>>     Lisp_Object comp_u)
>>  {
>> +  CHECK_CONS(rest);
>> +
>>    Lisp_Object doc_idx = FIRST (rest);
>>    Lisp_Object intspec = SECOND (rest);
>>    struct Lisp_Native_Comp_Unit *cu = XNATIVE_COMP_UNIT (comp_u);
>> diff --git a/src/dynlib.c b/src/dynlib.c
>> index a8c8843..362530b 100644
>> --- a/src/dynlib.c
>> +++ b/src/dynlib.c
>> @@ -270,7 +270,8 @@ dynlib_close (dynlib_handle_ptr h)
>>  dynlib_handle_ptr
>>  dynlib_open (const char *path)
>>  {
>> -  return dlopen (path, RTLD_LAZY);
>> +  // return dlopen (path, RTLD_LAZY);
>> +  return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
>>  }
>>  
>> 
>> delete the eln-cache, execute with -nw -q in gdb, wait a few seconds:
>> (gdb) bt
>> #0  _dl_lookup_symbol_x (undef_name=0x555555e3814300 <error: Cannot access memory at address
>> 0x555555e3814300>, undef_map=0x555556059380, ref=0x7fffffffc1b8, symbol_scope=0x555556059718,
>> version=0x0, type_class=0, flags=2, skip_map=0x0) at dl-lookup.c:842
>> #1  0x00007ffff4082b74 in do_sym (handle=<optimized out>, name=0x555555e3814300 <error: Cannot
>> access memory at address 0x555555e3814300>, who=0x55555580b073 <dynlib_sym+35>,
>> vers=vers@entry=0x0, flags=flags@entry=2) at dl-sym.c:165
>> #2  0x00007ffff408305d in _dl_sym (handle=<optimized out>, name=<optimized out>, who=<optimized out>)
>> at dl-sym.c:274
>> #3  0x00007ffff5bdf3b4 in dlsym_doit (a=a@entry=0x7fffffffc3f0) at dlsym.c:50
>> #4  0x00007ffff4083260 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffc390,
>> operate=0x7ffff5bdf3a0 <dlsym_doit>, args=0x7fffffffc3f0) at dl-error-skeleton.c:208
>> #5  0x00007ffff408331f in __GI__dl_catch_error (objname=0x555555dab830, errstring=0x555555dab838,
>> mallocedp=0x555555dab828, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:227
>> #6  0x00007ffff5bdfa65 in _dlerror_run (operate=operate@entry=0x7ffff5bdf3a0 <dlsym_doit>,
>> args=args@entry=0x7fffffffc3f0) at dlerror.c:170
>> #7  0x00007ffff5bdf41c in __dlsym (handle=<optimized out>, name=<optimized out>) at dlsym.c:70
>> #8  0x000055555580b073 in dynlib_sym (h=0x555556059380, sym=0x555555e3814300 <error: Cannot
>> access memory at address 0x555555e3814300>) at dynlib.c:280
>> #9  0x0000555555809f63 in make_subr (symbol_name=0x555555e38153, minarg=0x16, maxarg=0x16,
>> c_name=0x555555e38153, type=0xfc30, doc_idx=0xba, intspec=0x0, comp_u=0x555555df6d1d) at
>> comp.c:5147
>> #10 0x000055555580a156 in Fcomp__register_lambda (reloc_idx=0xa, c_name=0x555555e38153,
>> minarg=0x16, maxarg=0x16, type=0xfc30, rest=0x555555e17c03, comp_u=0x555555df6d1d) at comp.c:5203
>> #11 0x00007fffef0eed34 in late_top_level_run () at
>> /home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/cconv-3b1f1f98-cca24f72.eln
>
> So I think this is a manifestation of the issue that Andrea was
> talking about: the way we produce native code is incompatible with
> RTLD_GLOBAL.
>
> So at this point, I think this bug report should be treated as
> "wishlist", i.e. a feature request: to support RTLD_GLOBAL in dynlib
> with native-comp.  Andrea, is that feasible, and if so, what would it
> entail?

Hi Eli,

yes, I think once we have verified the issue is what we suspect we could
dlopen eln files as before and have RTLD_GLOBAL added for the other
cases.

I'll have a look into next week and report on the feature request.

Thanks

  Andrea

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: hx <silent2600@gmail.com>, emacs-devel@gnu.org
> Date: Sat, 11 Dec 2021 18:09:56 +0000
> 
> > So at this point, I think this bug report should be treated as
> > "wishlist", i.e. a feature request: to support RTLD_GLOBAL in dynlib
> > with native-comp.  Andrea, is that feasible, and if so, what would it
> > entail?
> 
> Hi Eli,
> 
> yes, I think once we have verified the issue is what we suspect we could
> dlopen eln files as before and have RTLD_GLOBAL added for the other
> cases.
> 
> I'll have a look into next week and report on the feature request.

Thanks.  One (unchecked) idea is to use 2 different versions of
dynlib_open: one for loading *.eln files, the other for every other
kind of dynamic loading.

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: hx <silent2600@gmail.com>, emacs-devel@gnu.org
>> Date: Sat, 11 Dec 2021 18:09:56 +0000
>> 
>> > So at this point, I think this bug report should be treated as
>> > "wishlist", i.e. a feature request: to support RTLD_GLOBAL in dynlib
>> > with native-comp.  Andrea, is that feasible, and if so, what would it
>> > entail?
>> 
>> Hi Eli,
>> 
>> yes, I think once we have verified the issue is what we suspect we could
>> dlopen eln files as before and have RTLD_GLOBAL added for the other
>> cases.
>> 
>> I'll have a look into next week and report on the feature request.
>
> Thanks.  One (unchecked) idea is to use 2 different versions of
> dynlib_open: one for loading *.eln files, the other for every other
> kind of dynamic loading.

Agree, 2 different versions of dynlib_open was my idea as well.  I'll
just investigate a bit to confirm the reason of the issue.

  Andrea

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Andrea Corallo]

Andrea Corallo <akrl@sdf.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: hx <silent2600@gmail.com>, emacs-devel@gnu.org
>>> Date: Sat, 11 Dec 2021 18:09:56 +0000
>>>
>>> > So at this point, I think this bug report should be treated as
>>> > "wishlist", i.e. a feature request: to support RTLD_GLOBAL in dynlib
>>> > with native-comp.  Andrea, is that feasible, and if so, what would it
>>> > entail?
>>>
>>> Hi Eli,
>>>
>>> yes, I think once we have verified the issue is what we suspect we could
>>> dlopen eln files as before and have RTLD_GLOBAL added for the other
>>> cases.
>>>
>>> I'll have a look into next week and report on the feature request.
>>
>> Thanks.  One (unchecked) idea is to use 2 different versions of
>> dynlib_open: one for loading *.eln files, the other for every other
>> kind of dynamic loading.
>
> Agree, 2 different versions of dynlib_open was my idea as well.

Hi all,

I've implemented the mentioned idea of having two different
'dynlib_open' and pushed that as e6b93245fe.  Contextually I've added
the RTLD_GLOBAL flag to 'dlopen'.

Note the Windows version of the new 'dynlib_open' is just a wrapper to
the old one as I don't know if something has to be done there or not
(nor I know how to test it).

BR

  Andrea

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: silent2600@gmail.com, emacs-devel@gnu.org
> Date: Wed, 15 Dec 2021 15:26:20 +0000
> 
> I've implemented the mentioned idea of having two different
> 'dynlib_open' and pushed that as e6b93245fe.  Contextually I've added
> the RTLD_GLOBAL flag to 'dlopen'.

Thanks.

> Note the Windows version of the new 'dynlib_open' is just a wrapper to
> the old one as I don't know if something has to be done there or not
> (nor I know how to test it).

The equivalent RTLD_GLOBAL flag cannot be easily supported on
MS-Windows.  So we currently don't, as it isn't required yet.

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Philipp Stephani]

Am Sa., 11. Dez. 2021 um 19:10 Uhr schrieb Andrea Corallo <akrl@sdf.org>:
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> From: hx <silent2600@gmail.com>
> >> Date: Sat, 11 Dec 2021 10:29:33 +0800
> >> Cc: Andrea Corallo <akrl@sdf.org>, emacs-devel <emacs-devel@gnu.org>
> >>
> >> git diff
> >> diff --git a/src/comp.c b/src/comp.c
> >> index 43feac6..2941f84 100644
> >> --- a/src/comp.c
> >> +++ b/src/comp.c
> >> @@ -5191,6 +5191,8 @@ DEFUN ("comp--register-lambda", Fcomp__register_lambda,
> >> Scomp__register_lambda,
> >>     Lisp_Object maxarg, Lisp_Object type, Lisp_Object rest,
> >>     Lisp_Object comp_u)
> >>  {
> >> +  CHECK_CONS(rest);
> >> +
> >>    Lisp_Object doc_idx = FIRST (rest);
> >>    Lisp_Object intspec = SECOND (rest);
> >>    struct Lisp_Native_Comp_Unit *cu = XNATIVE_COMP_UNIT (comp_u);
> >> diff --git a/src/dynlib.c b/src/dynlib.c
> >> index a8c8843..362530b 100644
> >> --- a/src/dynlib.c
> >> +++ b/src/dynlib.c
> >> @@ -270,7 +270,8 @@ dynlib_close (dynlib_handle_ptr h)
> >>  dynlib_handle_ptr
> >>  dynlib_open (const char *path)
> >>  {
> >> -  return dlopen (path, RTLD_LAZY);
> >> +  // return dlopen (path, RTLD_LAZY);
> >> +  return dlopen (path, RTLD_LAZY|RTLD_GLOBAL);
> >>  }
> >>
> >>
> >> delete the eln-cache, execute with -nw -q in gdb, wait a few seconds:
> >> (gdb) bt
> >> #0  _dl_lookup_symbol_x (undef_name=0x555555e3814300 <error: Cannot access memory at address
> >> 0x555555e3814300>, undef_map=0x555556059380, ref=0x7fffffffc1b8, symbol_scope=0x555556059718,
> >> version=0x0, type_class=0, flags=2, skip_map=0x0) at dl-lookup.c:842
> >> #1  0x00007ffff4082b74 in do_sym (handle=<optimized out>, name=0x555555e3814300 <error: Cannot
> >> access memory at address 0x555555e3814300>, who=0x55555580b073 <dynlib_sym+35>,
> >> vers=vers@entry=0x0, flags=flags@entry=2) at dl-sym.c:165
> >> #2  0x00007ffff408305d in _dl_sym (handle=<optimized out>, name=<optimized out>, who=<optimized out>)
> >> at dl-sym.c:274
> >> #3  0x00007ffff5bdf3b4 in dlsym_doit (a=a@entry=0x7fffffffc3f0) at dlsym.c:50
> >> #4  0x00007ffff4083260 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffc390,
> >> operate=0x7ffff5bdf3a0 <dlsym_doit>, args=0x7fffffffc3f0) at dl-error-skeleton.c:208
> >> #5  0x00007ffff408331f in __GI__dl_catch_error (objname=0x555555dab830, errstring=0x555555dab838,
> >> mallocedp=0x555555dab828, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:227
> >> #6  0x00007ffff5bdfa65 in _dlerror_run (operate=operate@entry=0x7ffff5bdf3a0 <dlsym_doit>,
> >> args=args@entry=0x7fffffffc3f0) at dlerror.c:170
> >> #7  0x00007ffff5bdf41c in __dlsym (handle=<optimized out>, name=<optimized out>) at dlsym.c:70
> >> #8  0x000055555580b073 in dynlib_sym (h=0x555556059380, sym=0x555555e3814300 <error: Cannot
> >> access memory at address 0x555555e3814300>) at dynlib.c:280
> >> #9  0x0000555555809f63 in make_subr (symbol_name=0x555555e38153, minarg=0x16, maxarg=0x16,
> >> c_name=0x555555e38153, type=0xfc30, doc_idx=0xba, intspec=0x0, comp_u=0x555555df6d1d) at
> >> comp.c:5147
> >> #10 0x000055555580a156 in Fcomp__register_lambda (reloc_idx=0xa, c_name=0x555555e38153,
> >> minarg=0x16, maxarg=0x16, type=0xfc30, rest=0x555555e17c03, comp_u=0x555555df6d1d) at comp.c:5203
> >> #11 0x00007fffef0eed34 in late_top_level_run () at
> >> /home/silent/.emacs.d/eln-cache/28.0.90-8dcada16/cconv-3b1f1f98-cca24f72.eln
> >
> > So I think this is a manifestation of the issue that Andrea was
> > talking about: the way we produce native code is incompatible with
> > RTLD_GLOBAL.
> >
> > So at this point, I think this bug report should be treated as
> > "wishlist", i.e. a feature request: to support RTLD_GLOBAL in dynlib
> > with native-comp.  Andrea, is that feasible, and if so, what would it
> > entail?
>
> Hi Eli,
>
> yes, I think once we have verified the issue is what we suspect we could
> dlopen eln files as before and have RTLD_GLOBAL added for the other
> cases.

This thread is already quite old, but let me point out that
https://www.akkadia.org/drepper/dsohowto.pdf warns very strongly
against using RTLD_GLOBAL (section 1.5.4), and I don't see a strong
reason to not heed that warning. So I'd recommend removing RTLD_GLOBAL
again.

Re: changed dlopen flags in dynlib.c, gccemacs crash

[Eli Zaretskii]

> From: Philipp Stephani <p.stephani2@gmail.com>
> Date: Sun, 6 Mar 2022 15:32:14 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, hx <silent2600@gmail.com>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> > yes, I think once we have verified the issue is what we suspect we could
> > dlopen eln files as before and have RTLD_GLOBAL added for the other
> > cases.
> 
> This thread is already quite old, but let me point out that
> https://www.akkadia.org/drepper/dsohowto.pdf warns very strongly
> against using RTLD_GLOBAL (section 1.5.4), and I don't see a strong
> reason to not heed that warning. So I'd recommend removing RTLD_GLOBAL
> again.

We added it for a reason, and I don't see anything in Ulrich Drepper's
paper that could affect Emacs modules badly enough to back up on this.

That said, I don't see myself an expert on this stuff, so if enough
people here think it will have adverse effect on Emacs, I'm okay with
removing that flag, provided that we see some detailed explanations of
how this could be detrimental to use of Emacs modules.