From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#23529: Request for fixing randomize_va_space build issues Date: Sat, 10 Sep 2016 09:06:27 +0300 Message-ID: <83wpik8f18.fsf@gnu.org> References: <573C2601.3030308@cs.ucla.edu> <54b89449-083a-a906-986a-f284dbbf482a@cs.ucla.edu> <02c57124-ef39-bc30-89ba-998986d070fc@cs.ucla.edu> <834m5tapuu.fsf@gnu.org> <83twdsalbk.fsf@gnu.org> <83r38vaiyh.fsf@gnu.org> <83eg4vab5o.fsf@gnu.org> <831t0va8br.fsf@gnu.org> <478e7c97-9339-5072-f9c1-ec67a45113aa@cs.ucla.edu> <83h99p8wbh.fsf@gnu.org> <0f0bb7ef-a588-93c3-5d27-c7329c667ed8@cs.ucla.edu> <83d1kd8qb7.fsf@gnu.org> <838tv18mnj.fsf@gnu.org> <83eg4sap3t.fsf@gnu.org> <3fe2aff8-d34e-afa3-a2bf-6e42394d7be6@cs.ucla.edu> Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1473487651 448 195.159.176.226 (10 Sep 2016 06:07:31 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 10 Sep 2016 06:07:31 +0000 (UTC) Cc: p.stephani2@gmail.com, philippe.vaucher@gmail.com, 23529@debbugs.gnu.org To: Paul Eggert Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Sep 10 08:07:25 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bibRc-0006z4-98 for geb-bug-gnu-emacs@m.gmane.org; Sat, 10 Sep 2016 08:07:16 +0200 Original-Received: from localhost ([::1]:33322 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bibRa-00060d-5w for geb-bug-gnu-emacs@m.gmane.org; Sat, 10 Sep 2016 02:07:14 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47384) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bibRT-00060J-Mu for bug-gnu-emacs@gnu.org; Sat, 10 Sep 2016 02:07:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bibRO-0006Py-0i for bug-gnu-emacs@gnu.org; Sat, 10 Sep 2016 02:07:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:57353) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bibRN-0006Pu-Ts for bug-gnu-emacs@gnu.org; Sat, 10 Sep 2016 02:07:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bibRN-0007lQ-MY for bug-gnu-emacs@gnu.org; Sat, 10 Sep 2016 02:07:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 10 Sep 2016 06:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23529 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 23529-submit@debbugs.gnu.org id=B23529.147348760329816 (code B ref 23529); Sat, 10 Sep 2016 06:07:01 +0000 Original-Received: (at 23529) by debbugs.gnu.org; 10 Sep 2016 06:06:43 +0000 Original-Received: from localhost ([127.0.0.1]:55065 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bibR4-0007kq-M3 for submit@debbugs.gnu.org; Sat, 10 Sep 2016 02:06:42 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:56356) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bibR3-0007kd-33 for 23529@debbugs.gnu.org; Sat, 10 Sep 2016 02:06:41 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bibQt-0006HS-FL for 23529@debbugs.gnu.org; Sat, 10 Sep 2016 02:06:36 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56241) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bibQt-0006Gf-Bp; Sat, 10 Sep 2016 02:06:31 -0400 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1745 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bibQr-0004U2-DC; Sat, 10 Sep 2016 02:06:29 -0400 In-reply-to: <3fe2aff8-d34e-afa3-a2bf-6e42394d7be6@cs.ucla.edu> (message from Paul Eggert on Fri, 9 Sep 2016 12:59:16 -0700) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:123134 Archived-At: > Cc: p.stephani2@gmail.com, philippe.vaucher@gmail.com, 23529@debbugs.gnu.org > From: Paul Eggert > Date: Fri, 9 Sep 2016 12:59:16 -0700 > > On 09/09/2016 11:45 AM, Eli Zaretskii wrote: > > All of those data structures are memory allocated for Lisp objects and > > their supporting structures, with known structures, so we know exactly > > which pointers need fixing. > > Of course. But it's not trivial to fix them. It can be done, but it will > take code that will be hard to maintain portably. I fail to see why it would be hard to maintain that portably. Those data structures are entirely our design and implementation, their differences between platforms are almost non-existent. Finding all the pointers in them is almost trivial. > > gmalloc is already implemented > > Yes, and its problems are prompting this discussion. gmalloc was a fine > design for the 1980s but is not now. temacs is not a program that needs to run for prolonged time intervals, its only purpose is to produce the data that the un-dumped Emacs will use. So whether its malloc implementation is strong enough by today's standards is not a relevant question. What matters is is it good enough for what temacs should do before it exits. > > If there are libc's out there that allow the application to define its > > own sbrk, then we could use that (we do on Windows). > > The sbrk model is becoming less and less plausible. Or whatever other back-end is used by malloc implementations, sbrk is not an important detail. > > If not, gmalloc > > will be good enough for the temacs run; emacs will of course use the > > normal libc allocators. > > This would give up on redumping, no? Not necessarily, we could have a variable that would force using the pre-dump malloc in emacs. > Plus, it assumes sbrk, which is backward-looking. What part assumes sbrk? > POSIX has withdrawn support for sbrk and there is > movement to deprecate it in C libraries due to security/robustness > concerns. This is something we should encourage, not run away from. This is a wrong tree to bark up. What we need is a malloc back-end that will allow to allocate memory off an implementation-specified memory block, that's all. If we cannot have that (which would surprise me, since MS-Windows does provide such a feature), we can still implement undump using a data file, but it will make our job slightly more complex, as we'd need to collect the data allocated off the heap before dumping it. Not rocket science, either. > > What is a "block" in this context? Surely, a data structure with > > linked pointers cannot be distributed between different "blocks", > > since a linker will not know how to fixup each address, because it > > doesn't understand the data structure. > > It can be distributed between different "blocks", because we can tell > the compiler and linker the data structure. Here's a quick example with > two small "blocks" dX and dY (the actual code would differ, this is just > a proof of concept): > > /* Simplified version of lisp.h. */ > #include > typedef intptr_t Lisp_Object; > enum { Lisp_Int0 = 2, Lisp_Cons = 3 /* ... */}; > #define make_number(n) (((n) << 2) + Lisp_Int0) > #define TAG_PTR(tag, ptr) ((intptr_t) (ptr) + (tag)) > #define Qnil ((Lisp_Object) 0) > struct Lisp_Cons { Lisp_Object car, cdr; }; > > /* Define a statically-allocated pair x that is equal to (10). */ > struct Lisp_Cons dX = { make_number (10), Qnil }; > #define x TAG_PTR (Lisp_Cons, &dX) > > /* Use x to build a statically-allocated list y that is equal to (5 > 10). */ > struct Lisp_Cons dY = { make_number (5), x }; > #define y TAG_PTR (Lisp_Cons, &dY) But we don't do these things in our code, so how is this relevant to this discussion? What I had in mind is the data structures we use to support maintenance of Lisp objects. One example is string_blocks, which we use to maintain Lisp strings. Surely, this structure will be in a single "block" under memory randomization, right? > > We won't be able to use them as just compilers and linkers. We will > > be using them for a job that is quite a bit more complex and > > different. > > No, this sort of thing is something that compilers and linkers do all > the time. We won't know for sure until this is fully implemented. Anyway, my take from this discussion is that we shouldn't give up so easily on dumping data as a binary file, as that approach sounds to me more future-proof than relying (again) on external technologies that were not meant for this specific job.