From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Newsgroups: gmane.lisp.guile.devel Subject: Re: thinking out loud: wip-rtl, ELF, pages, and mmap Date: Wed, 24 Apr 2013 18:20:27 -0400 Message-ID: <20130424222027.7PBZE.76491.root@cdptpa-web09-z01> References: <874nevhe4a.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1366842033 27964 80.91.229.3 (24 Apr 2013 22:20:33 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 24 Apr 2013 22:20:33 +0000 (UTC) To: guile-devel , Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Thu Apr 25 00:20:38 2013 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UV83J-0004nc-Nj for guile-devel@m.gmane.org; Thu, 25 Apr 2013 00:20:37 +0200 Original-Received: from localhost ([::1]:46208 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UV83J-0007oj-8P for guile-devel@m.gmane.org; Wed, 24 Apr 2013 18:20:37 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:47924) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UV83D-0007ng-96 for guile-devel@gnu.org; Wed, 24 Apr 2013 18:20:33 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UV83C-0005aa-6F for guile-devel@gnu.org; Wed, 24 Apr 2013 18:20:31 -0400 Original-Received: from cdptpa-omtalb.mail.rr.com ([75.180.132.120]:49094) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UV83C-0005Z8-2C for guile-devel@gnu.org; Wed, 24 Apr 2013 18:20:30 -0400 Authentication-Results: cdptpa-omtalb.mail.rr.com smtp.user=dsmich@roadrunner.com; auth=pass (LOGIN) X-Authority-Analysis: v=2.0 cv=Xpd4yC59 c=1 sm=0 a=NvObMiEFBl0A:10 a=zTVDa7HKqxcA:10 a=IkcTkHD0fZMA:10 a=doupyKFmAAAA:8 a=i4U0hfbkl98A:10 a=ybZZDoGAAAAA:8 a=d-nbKA4GY6fgNxTo7rUA:9 a=QEXdDO2ut3YA:10 a=qIVjreYYsbEA:10 a=UJpe99hltnTXGk3FIeyrjA==:117 X-Cloudmark-Score: 0 X-Authenticated-User: dsmich@roadrunner.com Original-Received: from [10.127.132.100] ([10.127.132.100:52330] helo=cdptpa-web09-z01) by cdptpa-oedge01.mail.rr.com (envelope-from ) (ecelerity 2.2.3.46 r()) with ESMTPA id DA/2B-02984-BAA58715; Wed, 24 Apr 2013 22:20:27 +0000 In-Reply-To: <874nevhe4a.fsf@pobox.com> X-Priority: 3 (Normal) Sensitivity: Normal X-Originating-IP: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 75.180.132.120 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:16294 Archived-At: ---- Andy Wingo wrote: ... > Then I realized: why am I doing all of this if the kernel has a virtual > memory system already that does all this for me? > > So I have a new plan, I think. I'll change the linker to always emit > sections and segments that correspond exactly in their on-disk layout > and in their in-memory layout. (In ELF terms: segments are contiguous, > with p_memsz == p_filesz.) I'll put commonly needed things at the > beginning, and debugging info and the section table at the end. Then > I'll just map the whole thing with PROT_READ, and set PROT_WRITE on > those page-aligned segments that need it. (Obviously in the future, > PROT_EXEC as well.) > > Then I'll just record a list of ELF objects that have been loaded. > Simple bisection will map IP -> ELF, and from there we have the section > table in memory (lazily paged in by the virtual memory system) and can > find the symtab and other debug info. > > So that's the plan. It's a significant change, and I wondered if folks > had some experience or reactions. > > Note that we have a read()-based fallback if mmap is not available. > This strategy also makes the read-based fallback easier. > > Thoughts? Sounds like a great idea! I think it's good to use the facilities that already exist instead of reinventing the wheel. A thought was that this would artificially bloat the size of the process when the extra info isn't being used. But so what? The *real* memory won't be consumed. And it's not like we are running out of addressable memory space anyway. It will be a bit ickier for systems without virtual memory. What about non-ELF systems? You are using your own ELF reader and mmap, not relying on the system to load the files right? So that's not an issue? -Dale