From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#46256: [feature/native-comp] AOT eln files ignored if run from build tree Date: Mon, 08 Mar 2021 10:14:35 +0000 Message-ID: References: <83h7m84h9g.fsf@gnu.org> <86v9ao5czu.fsf@gmail.com> <86wnutogrh.fsf@gmail.com> <86wnut8fb9.fsf@gmail.com> <861rd1tbpa.fsf@gmail.com> <83pn0km6y3.fsf@gnu.org> <86ft1f8ara.fsf@gmail.com> <83sg5cjdn8.fsf@gnu.org> <83r1kwjcy2.fsf@gnu.org> <8335x6u9o4.fsf@gnu.org> <83zgzesrku.fsf@gnu.org> <83tupms4mp.fsf@gnu.org> Reply-To: Andrea Corallo Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8147"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Eli Zaretskii , andrewjmoreton@gmail.com, 46256@debbugs.gnu.org To: Pip Cet Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Mar 08 11:15:22 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lJCv3-0001vI-I9 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 08 Mar 2021 11:15:21 +0100 Original-Received: from localhost ([::1]:34136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJCv2-0002O9-HY for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 08 Mar 2021 05:15:20 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36178) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJCuk-0002O3-Ty for bug-gnu-emacs@gnu.org; Mon, 08 Mar 2021 05:15:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:58881) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJCuk-0007cT-MZ for bug-gnu-emacs@gnu.org; Mon, 08 Mar 2021 05:15:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lJCuk-00031L-Fs for bug-gnu-emacs@gnu.org; Mon, 08 Mar 2021 05:15:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Andrea Corallo Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 08 Mar 2021 10:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46256 X-GNU-PR-Package: emacs Original-Received: via spool by 46256-submit@debbugs.gnu.org id=B46256.161519848111568 (code B ref 46256); Mon, 08 Mar 2021 10:15:02 +0000 Original-Received: (at 46256) by debbugs.gnu.org; 8 Mar 2021 10:14:41 +0000 Original-Received: from localhost ([127.0.0.1]:42194 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lJCuP-00030V-6O for submit@debbugs.gnu.org; Mon, 08 Mar 2021 05:14:41 -0500 Original-Received: from mx.sdf.org ([205.166.94.24]:59424) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lJCuL-00030I-V0 for 46256@debbugs.gnu.org; Mon, 08 Mar 2021 05:14:40 -0500 Original-Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id 128AEZ6O019698 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO); Mon, 8 Mar 2021 10:14:35 GMT In-Reply-To: (Pip Cet's message of "Mon, 8 Mar 2021 06:48:40 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:201818 Archived-At: --=-=-= Content-Type: text/plain Pip Cet writes: > On Mon, Mar 8, 2021 at 5:54 AM Pip Cet wrote: >> Note that this might not always work because of conservative GC. > > If it doesn't work, can you simply retry a few times? Eventually there > shouldn't be references to the stale native_comp_unit on the stack. > > However, I think I've worked out why dynlib_close doesn't do its job: > > Fnative_elisp_load creates a comp unit, but, if the shared library has > already been initialized, it doesn't set that comp unit's comp_unit > variable to point to the new comp unit; instead, it will continue > pointing to the first comp unit which still has it open. > > Then, the original comp unit is unloaded but not the new one created > by Fnative_elisp_load. We call dynlib_close() once, but we called it > twice before, leaving the shared library open and initialized. > > Then, we try to load the comp unit again, and follow the stale > comp_unit variable pointing to the original comp unit. > > Fix should be as attached. Note the fix is, at worst, harmless (unless > I messed up), so we should apply it anyway just because it's good not > to leave stale pointers lying around even if we hope that the OS > unmaps them at some point. > > Pip Hi Pip, thanks for the analysis, I'm not sure I followed 100% so I'll repeat to make sure we are on the same page, please correct me in case. IIUC (and make sense to me) the issue is that we are leaving two pointer pointing to the same handle: One is in the CU_2 allocated by 'Fnative_elisp_load' and later discarded by 'load_comp_unit' when reloading the same filename. The other is the original CU_1 created the first time this filename was loaded. When CU_2 will be GC'ed because discarded we'll get the problem because we'll dlclose the handle. Is this correct? In case isn't the attached curing the issue as well? Thanks Andrea PS I couldn't reproduce using the lisp reproducer both on my 64bit both on my 32bit system (I left it looping for a while), is that reproducer working for you? --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=dlclose.patch diff --git a/src/alloc.c b/src/alloc.c index af08336177..0bea10610f 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -3157,8 +3157,8 @@ cleanup_vector (struct Lisp_Vector *vector) { struct Lisp_Native_Comp_Unit *cu = PSEUDOVEC_STRUCT (vector, Lisp_Native_Comp_Unit); - eassert (cu->handle); - dynlib_close (cu->handle); + if (cu->handle) + dynlib_close (cu->handle); } else if (NATIVE_COMP_FLAG && PSEUDOVECTOR_TYPEP (&vector->header, PVEC_SUBR)) diff --git a/src/comp.c b/src/comp.c index e6f672de25..abc3535dc6 100644 --- a/src/comp.c +++ b/src/comp.c @@ -4832,6 +4832,10 @@ load_comp_unit (struct Lisp_Native_Comp_Unit *comp_u, bool loading_dump, We must *never* mess with static pointers in an already loaded eln. */ { + /* Invalidate the handle for the CU we are leaving for garbage + collection. */ + comp_u->handle = NULL; + /* Swap CU for the old one. */ comp_u_lisp_obj = *saved_cu; comp_u = XNATIVE_COMP_UNIT (comp_u_lisp_obj); comp_u->loaded_once = true; --=-=-=--