From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Paul Eggert Newsgroups: gmane.emacs.bugs Subject: bug#41321: 27.0.91; Emacs aborts due to invalid pseudovector objects Date: Thu, 28 May 2020 00:47:05 -0700 Organization: UCLA Computer Science Department Message-ID: <21d399ef-5d63-dcf5-d124-64d2820885ea@cs.ucla.edu> References: <83zha8cgpi.fsf@gnu.org> <83y2phwb9x.fsf@gnu.org> <83r1v9w9vi.fsf@gnu.org> <83mu5xw50d.fsf@gnu.org> <83k110wxte.fsf@gnu.org> <4bab5f55-95fe-cf34-e490-1d4319728395@cs.ucla.edu> <837dwyvi74.fsf@gnu.org> <1484f569-c260-9fb0-bfe1-67897de289d3@cs.ucla.edu> <83blm9tn4j.fsf@gnu.org> <4aeb8963-4fd1-fcd4-e6e1-be409ab54775@cs.ucla.edu> <753eed58-287b-1c47-deef-0e343bca8e69@cs.ucla.edu> <1ce49934-ee4c-78b5-20ff-83f281aed4ee@cs.ucla.edu> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------0687281901E7D082DDD4D59C" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="50686"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 Cc: 41321@debbugs.gnu.org, Stefan Monnier To: Pip Cet Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu May 28 09:48:11 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jeDGt-000D8n-7i for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 28 May 2020 09:48:11 +0200 Original-Received: from localhost ([::1]:42042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeDGr-0005b9-V7 for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 28 May 2020 03:48:09 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeDGk-0005ap-5n for bug-gnu-emacs@gnu.org; Thu, 28 May 2020 03:48:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jeDGj-0007FF-T3 for bug-gnu-emacs@gnu.org; Thu, 28 May 2020 03:48:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jeDGj-0004XD-Re for bug-gnu-emacs@gnu.org; Thu, 28 May 2020 03:48:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 28 May 2020 07:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41321 X-GNU-PR-Package: emacs Original-Received: via spool by 41321-submit@debbugs.gnu.org id=B41321.159065203517375 (code B ref 41321); Thu, 28 May 2020 07:48:01 +0000 Original-Received: (at 41321) by debbugs.gnu.org; 28 May 2020 07:47:15 +0000 Original-Received: from localhost ([127.0.0.1]:50875 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jeDFy-0004W8-NV for submit@debbugs.gnu.org; Thu, 28 May 2020 03:47:15 -0400 Original-Received: from zimbra.cs.ucla.edu ([131.179.128.68]:56868) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jeDFw-0004Vv-5G for 41321@debbugs.gnu.org; Thu, 28 May 2020 03:47:12 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 9EF341600C3; Thu, 28 May 2020 00:47:06 -0700 (PDT) Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id WSkExrpTw6nH; Thu, 28 May 2020 00:47:05 -0700 (PDT) Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 7483B1600CC; Thu, 28 May 2020 00:47:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id X-aDHD4nwwy8; Thu, 28 May 2020 00:47:05 -0700 (PDT) Original-Received: from [192.168.1.9] (cpe-23-242-74-103.socal.res.rr.com [23.242.74.103]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 436341600C3; Thu, 28 May 2020 00:47:05 -0700 (PDT) Autocrypt: addr=eggert@cs.ucla.edu; prefer-encrypt=mutual; keydata= LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tCgptUUlOQkV5QWNtUUJFQURB QXlIMnhvVHU3cHBHNUQzYThGTVpFb243NGRDdmM0K3ExWEEySjJ0QnkycHdhVHFmCmhweHhk R0E5Smo1MFVKM1BENGJTVUVnTjh0TFowc2FuNDdsNVhUQUZMaTI0NTZjaVNsNW04c0thSGxH ZHQ5WG0KQUF0bVhxZVpWSVlYL1VGUzk2ZkR6ZjR4aEVtbS95N0xiWUVQUWRVZHh1NDd4QTVL aFRZcDVibHRGM1dZRHoxWQpnZDdneDA3QXV3cDdpdzdlTnZub0RUQWxLQWw4S1lEWnpiRE5D UUdFYnBZM2VmWkl2UGRlSStGV1FONFcra2doCnkrUDZhdTZQcklJaFlyYWV1YTdYRGRiMkxT MWVuM1NzbUUzUWpxZlJxSS9BMnVlOEpNd3N2WGUvV0szOEV6czYKeDc0aVRhcUkzQUZINmls QWhEcXBNbmQvbXNTRVNORnQ3NkRpTzFaS1FNcjlhbVZQa25qZlBtSklTcWRoZ0IxRApsRWR3 MzRzUk9mNlY4bVp3MHhmcVQ2UEtFNDZMY0ZlZnpzMGtiZzRHT1JmOHZqRzJTZjF0azVlVThN Qml5Ti9iClowM2JLTmpOWU1wT0REUVF3dVA4NGtZTGtYMndCeHhNQWhCeHdiRFZadWR6eERa SjFDMlZYdWpDT0pWeHEya2wKakJNOUVUWXVVR3FkNzVBVzJMWHJMdzYrTXVJc0hGQVlBZ1Jy NytLY3dEZ0JBZndoU In-Reply-To: Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:181131 Archived-At: This is a multi-part message in MIME format. --------------0687281901E7D082DDD4D59C Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 5/27/20 11:31 PM, Pip Cet wrote: > I hope you're right, in that compilers will support GC better before > they move on to clever optimizations that break it :-) After looking into it, I decided it wasn't worth the hassle of treating pointers just past the end of a Lisp object as pointing into the object. Although such pointers can exist, I can't think of a realistic-with-today's-compilers scenario at the machine level where (1) a pointer like that will exist, (2) no pointers into the middle or start of the object will exist, and (3) the object might be accessed later. In contrast we have seen scenarios with pointers into the middle of Lisp objects. With that in mind, attached is a proposed patch to master that I hope deals with some of the more-serious problems mentioned so far in this thread, in particular the problem with Lisp_Object representations of symbols being split into two registers in a --with-wide-int build. I haven't tested this as much as I'd like, but I need to turn my attention to sleep and work and so this is a good place to broadcast a checkpoint. This patch doesn't address the LISP_ALIGNMENT issues you mentioned, both in lisp.h and in the pdumper; I can work on that soon, I think. PS. Thanks for helping bring this problem to our attention; it's been fun to look into it. --------------0687281901E7D082DDD4D59C Content-Type: text/x-patch; charset=UTF-8; name="0001-Fix-crashes-due-to-misidentified-pointers.patch" Content-Disposition: attachment; filename="0001-Fix-crashes-due-to-misidentified-pointers.patch" Content-Transfer-Encoding: quoted-printable >From 023344217e05d2a23b5e8157da2f9aea16a5df78 Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Thu, 28 May 2020 00:11:08 -0700 Subject: [PATCH] Fix crashes due to misidentified pointers MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit Problem reported by Pip Cet (Bug#41321#74, Bug#41321#80) A compiler can create temporaries that point somewhere into a Lisp object but are not GCALIGNED, and these temporaries may be the only thing that addresses the object. So, if any value points within an object, treat the object as being addressed. However, do not worry about pointers that point just past the end of an object, as these do not seem to be a problem in practice and attempting to worry about them would complicate and slow the code. * src/alloc.c (live_float_p): Don=E2=80=99t insist that the offset be aligned properly for a float, since it might be tagged or offset. (GC_OBJECT_ALIGNMENT_MINIMUM, maybe_lisp_pointer): Remove. All uses removed. (mark_maybe_pointer): New arg SYMBOL_ONLY. All callers changed. Don=E2=80=99t insist on pointers being aligned. Align pointers before doing pdumper checks on them, and before giving them to make_lisp_ptr. (mark_memory): Do not use mark_maybe_object here. Instead, use mark_maybe_pointer alone; that suffices. Also look for offsets from lispsym, to mark symbols more reliably. --- src/alloc.c | 65 +++++++++++++++-------------------------------------- 1 file changed, 18 insertions(+), 47 deletions(-) diff --git a/src/alloc.c b/src/alloc.c index e241b9933a..b1d45dbb33 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -4560,7 +4560,6 @@ live_float_p (struct mem_node *m, void *p) /* P must point to the start of a Lisp_Float and not be one of the unused cells in the current float block. */ return (offset >=3D 0 - && offset % sizeof b->floats[0] =3D=3D 0 && offset < (FLOAT_BLOCK_SIZE * sizeof b->floats[0]) && (b !=3D float_block || offset / sizeof b->floats[0] < float_block_index)); @@ -4687,54 +4686,25 @@ mark_maybe_objects (Lisp_Object const *array, ptr= diff_t nelts) mark_maybe_object (*array); } =20 -/* A lower bound on the alignment of Lisp objects that need marking. - Although 1 is safe, higher values speed up mark_maybe_pointer. - If USE_LSB_TAG, this value is typically GCALIGNMENT; otherwise, - it's determined by the natural alignment of Lisp structs. - All vectorlike objects have alignment at least that of union - vectorlike_header and it's unlikely they all have alignment greater, - so use the union as a safe and likely-accurate standin for - vectorlike objects. */ - -enum { GC_OBJECT_ALIGNMENT_MINIMUM - =3D max (GCALIGNMENT, - min (alignof (union vectorlike_header), - min (min (alignof (struct Lisp_Cons), - alignof (struct Lisp_Float)), - min (alignof (struct Lisp_String), - alignof (struct Lisp_Symbol))))) }; - -/* Return true if P might point to Lisp data that can be garbage - collected, and false otherwise (i.e., false if it is easy to see - that P cannot point to Lisp data that can be garbage collected). - Symbols are implemented via offsets not pointers, but the offsets - are also multiples of GC_OBJECT_ALIGNMENT_MINIMUM. */ - -static bool -maybe_lisp_pointer (void *p) -{ - return (uintptr_t) p % GC_OBJECT_ALIGNMENT_MINIMUM =3D=3D 0; -} - /* If P points to Lisp data, mark that as live if it isn't already - marked. */ + marked. If SYMBOL_ONLY, mark it only if it is a symbol. */ =20 static void -mark_maybe_pointer (void *p) +mark_maybe_pointer (void *p, bool symbol_only) { + char *cp =3D p; struct mem_node *m; =20 #ifdef USE_VALGRIND VALGRIND_MAKE_MEM_DEFINED (&p, sizeof (p)); #endif =20 - if (!maybe_lisp_pointer (p)) - return; - if (pdumper_object_p (p)) { + p =3D cp - (uintptr_t) cp % GCALIGNMENT; int type =3D pdumper_find_object_type (p); - if (pdumper_valid_object_type_p (type)) + if (pdumper_valid_object_type_p (type) + && (type =3D=3D Lisp_Symbol || !symbol_only)) mark_object (type =3D=3D Lisp_Symbol ? make_lisp_symbol (p) : make_lisp_ptr (p, type)); @@ -4755,11 +4725,13 @@ mark_maybe_pointer (void *p) break; =20 case MEM_TYPE_CONS: - obj =3D live_cons_holding (m, p); + if (!symbol_only) + obj =3D live_cons_holding (m, p); break; =20 case MEM_TYPE_STRING: - obj =3D live_string_holding (m, p); + if (!symbol_only) + obj =3D live_string_holding (m, p); break; =20 case MEM_TYPE_SYMBOL: @@ -4767,13 +4739,14 @@ mark_maybe_pointer (void *p) break; =20 case MEM_TYPE_FLOAT: - if (live_float_p (m, p)) - obj =3D make_lisp_ptr (p, Lisp_Float); + if (!symbol_only && live_float_p (m, p)) + obj =3D make_lisp_ptr (cp - (uintptr_t) cp % GCALIGNMENT, Lisp_Floa= t); break; =20 case MEM_TYPE_VECTORLIKE: case MEM_TYPE_VECTOR_BLOCK: - obj =3D live_vector_holding (m, p); + if (!symbol_only) + obj =3D live_vector_holding (m, p); break; =20 default: @@ -4830,12 +4803,10 @@ mark_memory (void const *start, void const *end) =20 for (pp =3D start; (void const *) pp < end; pp +=3D GC_POINTER_ALIGNME= NT) { - mark_maybe_pointer (*(void *const *) pp); - - verify (alignof (Lisp_Object) % GC_POINTER_ALIGNMENT =3D=3D 0); - if (alignof (Lisp_Object) =3D=3D GC_POINTER_ALIGNMENT - || (uintptr_t) pp % alignof (Lisp_Object) =3D=3D 0) - mark_maybe_object (*(Lisp_Object const *) pp); + char *p =3D *(char *const *) pp; + mark_maybe_pointer (p, false); + p +=3D (intptr_t) lispsym; + mark_maybe_pointer (p, true); } } =20 --=20 2.17.1 --------------0687281901E7D082DDD4D59C--