From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Paul Eggert Newsgroups: gmane.emacs.bugs Subject: bug#49261: Segfault during loadup Date: Sun, 11 Jul 2021 01:36:21 -0700 Organization: UCLA Computer Science Department Message-ID: References: <87o8bn7bie.fsf@gnus.org> <87zgv6vuon.fsf@gmx.de> <837di9lwbm.fsf@gnu.org> <87a6n5vuu4.fsf@gnus.org> <8735sqnmei.fsf@gnus.org> <87zguyf4ht.fsf@gmx.de> <87pmvum54p.fsf@gnus.org> <87v95mf2lj.fsf@gmx.de> <87pmvt3ob1.fsf@gnus.org> <83r1g9evoy.fsf@gnu.org> <83pmvtevgg.fsf@gnu.org> <87czrt3mj4.fsf@gnus.org> <878s2h3m1d.fsf@gnus.org> <874kd53lnl.fsf@gnus.org> <87zgux26ab.fsf@gnus.org> <87pmvt25uq.fsf@gnus.org> <87lf6h2294.fsf@gnus.org> <87h7h51x9y.fsf_-_@gnus.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------F1DFBA2D6341F22CE1972BAC" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27809"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 Cc: 49261@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Jul 11 10:37:23 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m2Uxm-00078A-US for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 11 Jul 2021 10:37:23 +0200 Original-Received: from localhost ([::1]:42414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m2Uxm-0007bz-1T for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 11 Jul 2021 04:37:22 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51514) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m2UxS-0007bl-E6 for bug-gnu-emacs@gnu.org; Sun, 11 Jul 2021 04:37:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:50532) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m2UxS-00060a-0B for bug-gnu-emacs@gnu.org; Sun, 11 Jul 2021 04:37:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1m2UxR-0008HP-MX for bug-gnu-emacs@gnu.org; Sun, 11 Jul 2021 04:37:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 11 Jul 2021 08:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49261 X-GNU-PR-Package: emacs Original-Received: via spool by 49261-submit@debbugs.gnu.org id=B49261.162599259131791 (code B ref 49261); Sun, 11 Jul 2021 08:37:01 +0000 Original-Received: (at 49261) by debbugs.gnu.org; 11 Jul 2021 08:36:31 +0000 Original-Received: from localhost ([127.0.0.1]:33845 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m2Uwx-0008Gh-Al for submit@debbugs.gnu.org; Sun, 11 Jul 2021 04:36:31 -0400 Original-Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57904) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m2Uwv-0008GQ-N7 for 49261@debbugs.gnu.org; Sun, 11 Jul 2021 04:36:30 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 99B6416006F; Sun, 11 Jul 2021 01:36:23 -0700 (PDT) Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id IQGioXllqwx2; Sun, 11 Jul 2021 01:36:22 -0700 (PDT) Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5657A16008D; Sun, 11 Jul 2021 01:36:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 1n0jqanoLdOm; Sun, 11 Jul 2021 01:36:22 -0700 (PDT) Original-Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 18A3716006F; Sun, 11 Jul 2021 01:36:22 -0700 (PDT) In-Reply-To: <87h7h51x9y.fsf_-_@gnus.org> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:209788 Archived-At: This is a multi-part message in MIME format. --------------F1DFBA2D6341F22CE1972BAC Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 7/7/21 5:09 PM, Lars Ingebrigtsen wrote: > There seems to be a corruption in the hash tables somewhere. It's > totally reproducible with the recipe Thanks for the recipe; it let me reproduce the problem on Fedora 34 x86-6= 4. The problem comes from the fact that mark_maybe_pointer works=20 differently on pdumper objects than it works on ordinary objects. On=20 ordinary objects, roots can point anywhere into an object (because this=20 sort of thing has happened on real machines), whereas on pdumper=20 objects, roots had to point to the start of the object. I worked around this particular problem changing mark_maybe_pointer so=20 that pdumper roots can also be tagged (see first attached patch).=20 However, I suspect this is not a complete fix, as it doesn't cover the=20 case where a root points to some part of a pdumper object that is not at=20 the object's start. I added a FIXME about this. Perhaps Daniel can take=20 a look at it sometime. I think the remaining bug will be hit only rarely=20 (if ever). The second attached patch is in the same area, but is not part of the=20 fix. It causes the GC to be a bit more accurate (i.e., less=20 conservative) for roots, which can help avoid some leaks. --------------F1DFBA2D6341F22CE1972BAC Content-Type: text/x-patch; charset=UTF-8; name="0001-Fix-pdumper-related-GC-bug.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0001-Fix-pdumper-related-GC-bug.patch" =46rom 2f7afef5ffe023a7a12520201ab70643f826abfd Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Sun, 11 Jul 2021 00:27:43 -0700 Subject: [PATCH 1/2] Fix pdumper-related GC bug MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit * src/alloc.c (mark_maybe_pointer): Also mark pointers to pdumper objects, even when the pointers are tagged. Add a FIXME saying why this isn=E2=80=99t enough. --- src/alloc.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/alloc.c b/src/alloc.c index 76d8c7ddd1..752eaec135 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -4755,6 +4755,17 @@ mark_maybe_pointer (void *p) definitely _don't_ have an object. */ if (pdumper_object_p (p)) { + /* FIXME: This code assumes that every reachable pdumper object + is addressed either by a pointer to the object start, or by + the same pointer with an LSB-style tag. This assumption + fails if a pdumper object is reachable only via machine + addresses of non-initial object components. Although such + addressing is rare in machine code generated by C compilers + from Emacs source code, it can occur in some cases. To fix + this problem, the pdumper code should grok non-initial + addresses, as the non-pdumper code does. */ + uintptr_t mask =3D VALMASK; + p =3D (void *) ((uintptr_t) p & mask); /* Don't use pdumper_object_p_precise here! It doesn't check the tag bits. OBJ here might be complete garbage, so we need to verify both the pointer and the tag. */ --=20 2.31.1 --------------F1DFBA2D6341F22CE1972BAC Content-Type: text/x-patch; charset=UTF-8; name="0002-Make-pdumper-marking-pickier.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0002-Make-pdumper-marking-pickier.patch" =46rom f6472cc8e2fdcfd7365240783f34e101fe44142b Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Sun, 11 Jul 2021 00:54:32 -0700 Subject: [PATCH 2/2] Make pdumper-marking pickier MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit Prevent some false-positives in conservative GC marking. This doesn=E2=80=99t fix any correctness bugs; it=E2=80=99s merely to reclaim some memory instead of keeping it unnecessarily. * src/alloc.c (mark_maybe_pointer): New arg SYMBOL_ONLY. All callers changed. Check that the pointer=E2=80=99s tag, if any, matches the pdumper-reported type. --- src/alloc.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/src/alloc.c b/src/alloc.c index 752eaec135..b3668d2131 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -4740,7 +4740,7 @@ live_small_vector_p (struct mem_node *m, void *p) marked. */ =20 static void -mark_maybe_pointer (void *p) +mark_maybe_pointer (void *p, bool symbol_only) { struct mem_node *m; =20 @@ -4765,15 +4765,21 @@ mark_maybe_pointer (void *p) this problem, the pdumper code should grok non-initial addresses, as the non-pdumper code does. */ uintptr_t mask =3D VALMASK; - p =3D (void *) ((uintptr_t) p & mask); + void *po =3D (void *) ((uintptr_t) p & mask); + char *cp =3D p; + char *cpo =3D po; /* Don't use pdumper_object_p_precise here! It doesn't check the tag bits. OBJ here might be complete garbage, so we need to verify both the pointer and the tag. */ - int type =3D pdumper_find_object_type (p); - if (pdumper_valid_object_type_p (type)) - mark_object (type =3D=3D Lisp_Symbol - ? make_lisp_symbol (p) - : make_lisp_ptr (p, type)); + int type =3D pdumper_find_object_type (po); + if (pdumper_valid_object_type_p (type) + && (!USE_LSB_TAG || p =3D=3D po || cp - cpo =3D=3D type)) + { + if (type =3D=3D Lisp_Symbol) + mark_object (make_lisp_symbol (po)); + else if (!symbol_only) + mark_object (make_lisp_ptr (po, type)); + } return; } =20 @@ -4791,6 +4797,8 @@ mark_maybe_pointer (void *p) =20 case MEM_TYPE_CONS: { + if (symbol_only) + return; struct Lisp_Cons *h =3D live_cons_holding (m, p); if (!h) return; @@ -4800,6 +4808,8 @@ mark_maybe_pointer (void *p) =20 case MEM_TYPE_STRING: { + if (symbol_only) + return; struct Lisp_String *h =3D live_string_holding (m, p); if (!h) return; @@ -4818,6 +4828,8 @@ mark_maybe_pointer (void *p) =20 case MEM_TYPE_FLOAT: { + if (symbol_only) + return; struct Lisp_Float *h =3D live_float_holding (m, p); if (!h) return; @@ -4827,6 +4839,8 @@ mark_maybe_pointer (void *p) =20 case MEM_TYPE_VECTORLIKE: { + if (symbol_only) + return; struct Lisp_Vector *h =3D live_large_vector_holding (m, p); if (!h) return; @@ -4836,6 +4850,8 @@ mark_maybe_pointer (void *p) =20 case MEM_TYPE_VECTOR_BLOCK: { + if (symbol_only) + return; struct Lisp_Vector *h =3D live_small_vector_holding (m, p); if (!h) return; @@ -4897,7 +4913,7 @@ mark_memory (void const *start, void const *end) for (pp =3D start; (void const *) pp < end; pp +=3D GC_POINTER_ALIGNME= NT) { void *p =3D *(void *const *) pp; - mark_maybe_pointer (p); + mark_maybe_pointer (p, false); =20 /* Unmask any struct Lisp_Symbol pointer that make_lisp_symbol previously disguised by adding the address of 'lispsym'. @@ -4906,7 +4922,7 @@ mark_memory (void const *start, void const *end) non-adjacent words and P might be the low-order word's value. */ intptr_t ip; INT_ADD_WRAPV ((intptr_t) p, (intptr_t) lispsym, &ip); - mark_maybe_pointer ((void *) ip); + mark_maybe_pointer ((void *) ip, true); } } =20 --=20 2.31.1 --------------F1DFBA2D6341F22CE1972BAC--