From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Philipp Stephani Newsgroups: gmane.emacs.devel Subject: Re: compiled lisp file format (Re: Skipping unexec via a big .elc file) Date: Sun, 28 May 2017 12:43:22 +0000 Message-ID: References: <8A8DA980-13A7-4F8B-9D07-391728C673C9@raeburn.org> <83inmq53xk.fsf@gnu.org> <96D35768-314C-43F5-BD5E-B12187759DCA@raeburn.org> <123104DD-447F-4CDB-B3A0-CED80E3AC8C9@raeburn.org> <20170403165736.GA2851@acm> <2497A2D5-FDB1-47FF-AED3-FD4ABE2FE144@raeburn.org> <83lgrhpalq.fsf@gnu.org> <0D99B4FE-FEEF-4565-87D6-E230A05DEF3C@raeburn.org> <86lgrc4vob.fsf@molnjunk.nocrew.org> <834ly0oew1.fsf@gnu.org> <968E8F50-92F6-43C7-B7E4-EE8378943087@raeburn.org> <83wpawmj4d.fsf@gnu.org> <1e397033-8291-1625-8b78-a1e1c200aea5@gmail.com> <18196f08-408d-8b17-423e-8be54507bb84@gmail.com> <8360hkkcgj.fsf@gnu.org> <26b35c16-33e7-0e08-9cc5-6f9b81e40968@cs.ucla.edu> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="001a113cdf42156bd2055094ebea" X-Trace: blaine.gmane.org 1495975423 32081 195.159.176.226 (28 May 2017 12:43:43 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 28 May 2017 12:43:43 +0000 (UTC) Cc: Emacs developers To: Ken Raeburn , Paul Eggert Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun May 28 14:43:38 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dExXm-0008EK-1N for ged-emacs-devel@m.gmane.org; Sun, 28 May 2017 14:43:38 +0200 Original-Received: from localhost ([::1]:43864 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dExXr-0004F7-Hp for ged-emacs-devel@m.gmane.org; Sun, 28 May 2017 08:43:43 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48966) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dExXj-0004Er-PW for emacs-devel@gnu.org; Sun, 28 May 2017 08:43:37 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dExXi-0005oV-HE for emacs-devel@gnu.org; Sun, 28 May 2017 08:43:35 -0400 Original-Received: from mail-oi0-x22a.google.com ([2607:f8b0:4003:c06::22a]:34487) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dExXi-0005ne-9k for emacs-devel@gnu.org; Sun, 28 May 2017 08:43:34 -0400 Original-Received: by mail-oi0-x22a.google.com with SMTP id b204so54750155oii.1 for ; Sun, 28 May 2017 05:43:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HWT9nUUN15c+ZCpVNfKsXkgLW5odBWwZytzZQ4rhBdI=; b=RDi5juS21WrIomI75OyfNQSsNgKC9H+oSFt/z3iOpGKnj+2FtW3Z3QSUyf42s151rC 3oHpPsgw6rH480/OhZoIBTR8hHD9rYQucpMouDPO8+aOUfQvBYpILVuHPUJikHpdb9AA eAZxKyTPZ6n0UJ83Ma6Mb5H9g527O6Sxk6W0RD+gtdzaCzqxKPKYEZD42HvZz1uj/6j+ l04Ah5SvtFmAPL0A+Jm8Umw8vfC3tziK4vNRxGUydl8gXew8O2kKOpA8Qzm5oJbnE1HC lK1QalXvSQJUQeJ2CGlyxe/XwMKVKPbbi1yJP4o8Teg1o0GiWwJWJH/3Lyd/rhm2LhTb GfAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HWT9nUUN15c+ZCpVNfKsXkgLW5odBWwZytzZQ4rhBdI=; b=Ri/YTddpvLjLUAUsKSI+FV6l47xonjpWIus9wQi7Yz5rnNdZVm6KIU5g8Qe5oOTH73 Sujp5WKKuCs0zjKxMbiQp0SvFGdd6Fmh2nglAl+/4piO6Tq28Xd0Ywxly6ljozLZrVDv R0ZXU0gPK9odwaQOANiT9ps9A7CGxv/LZTUzKeelV15nXobKtwe5GwfcO17e+nQJqZSv d/NNxRBbxd5g9AZCWcpgc1M+C+6Zw9S+ivDHhykPtbeNTonZh46BkrxQp/6A4GcvBfG4 i4ztKl4wxQuC988vR/ERR8ejhmYOSjJKENNXrJ9ixgjgef4T0jXxmvneM+fuYLRGjztb zbxg== X-Gm-Message-State: AODbwcANWn+rOhFkaEb52uD39wmU0Jvpp4FLsUSE62Dzj5wRvsN/6bnb ll0/1MYH0EzA4YvL6vLIjTglU1P9Jg== X-Received: by 10.202.61.84 with SMTP id k81mr4133848oia.25.1495975412983; Sun, 28 May 2017 05:43:32 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4003:c06::22a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:215282 Archived-At: --001a113cdf42156bd2055094ebea Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ken Raeburn schrieb am So., 28. Mai 2017 um 13:07 Uhr= : > > On May 21, 2017, at 04:53, Paul Eggert wrote: > > > Ken Raeburn wrote: > >> The Guile project has taken this idea pretty far; they=E2=80=99re gene= rating > ELF object files with a few special sections for Guile objects, using the > standard DWARF sections for debug information, etc. While it has a certa= in > appeal (making C modules and Lisp files look much more similar, maybe bei= ng > able to link Lisp and C together into one executable image, letting GDB > understand some of your data), switching to a machine-specific format wou= ld > be a pretty drastic change, when we can currently share the files across > machines. > > > > Although it does indeed sound like a big change, I don't see why it > would prevent us from sharing the files across machines. Emacs can use > standard ELF and DWARF format on any platform if Emacs is doing the > loading. And there should be some software-engineering benefit in using t= he > same format that Guile uses. > > Sorry for the delay in responding. > > The ELF format has header fields indicating the word size, endianness, > machine architecture (though there=E2=80=99s a value for =E2=80=9Cnone=E2= =80=9D), and OS ABI. Some > fields vary in size or order depending on whether the 32-bit or 64-bit > format is in use. Some other format details (e.g., relocation types, > interpretation of certain ranges of values in some fields) are > architecture- or OS-dependent; we might not care about many of those > details, but relocations are likely needed if we want to play linking gam= es > or use DWARF. > > I think Guile is using whatever the native word size and architecture > are. If we do that for Emacs, they=E2=80=99re not portable between platf= orms. > Currently it works for me to put my Lisp files, both source and compiled, > into ~/elisp and use them from different kinds of machines if my home > directory is NFS-mounted. > > We could instead pick fixed values (say, architecture =E2=80=9Cnone=E2=80= =9D, > little-endian, 32-bit), but then there=E2=80=99s no guarantee that we cou= ld use any > of the usual GNU tools on them without a bunch of work, or that we=E2=80= =99d ever > be able to use non-GNU tools to treat them as object files. Then again, = we > couldn=E2=80=99t expect to do the latter portably anyway, since some of t= he > platforms don=E2=80=99t even use ELF. > > Is there any significant advantage of using ELF, or could this just use one of the standard binary serialization formats (protobuf, flatbuffer, ...)? --001a113cdf42156bd2055094ebea Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


Ken Ra= eburn <raeburn@raeburn.org>= ; schrieb am So., 28. Mai 2017 um 13:07=C2=A0Uhr:

On May 21, 2017, at 04:53, Paul Eggert <eggert@cs.ucla.edu> wrote:

> Ken Raeburn wrote:
>> The Guile project has taken this idea pretty far; they=E2=80=99re = generating ELF object files with a few special sections for Guile objects, = using the standard DWARF sections for debug information, etc.=C2=A0 While i= t has a certain appeal (making C modules and Lisp files look much more simi= lar, maybe being able to link Lisp and C together into one executable image= , letting GDB understand some of your data), switching to a machine-specifi= c format would be a pretty drastic change, when we can currently share the = files across machines.
>
> Although it does indeed sound like a big change, I don't see why i= t would prevent us from sharing the files across machines. Emacs can use st= andard ELF and DWARF format on any platform if Emacs is doing the loading. = And there should be some software-engineering benefit in using the same for= mat that Guile uses.

Sorry for the delay in responding.

The ELF format has header fields indicating the word size, endianness, mach= ine architecture (though there=E2=80=99s a value for =E2=80=9Cnone=E2=80=9D= ), and OS ABI.=C2=A0 Some fields vary in size or order depending on whether= the 32-bit or 64-bit format is in use.=C2=A0 Some other format details (e.= g., relocation types, interpretation of certain ranges of values in some fi= elds) are architecture- or OS-dependent; we might not care about many of th= ose details, but relocations are likely needed if we want to play linking g= ames or use DWARF.

I think Guile is using whatever the native word size and architecture are.= =C2=A0 If we do that for Emacs, they=E2=80=99re not portable between platfo= rms.=C2=A0 Currently it works for me to put my Lisp files, both source and = compiled, into ~/elisp and use them from different kinds of machines if my = home directory is NFS-mounted.

We could instead pick fixed values (say, architecture =E2=80=9Cnone=E2=80= =9D, little-endian, 32-bit), but then there=E2=80=99s no guarantee that we = could use any of the usual GNU tools on them without a bunch of work, or th= at we=E2=80=99d ever be able to use non-GNU tools to treat them as object f= iles.=C2=A0 Then again, we couldn=E2=80=99t expect to do the latter portabl= y anyway, since some of the platforms don=E2=80=99t even use ELF.


Is there any significant advantage of = using ELF, or could this just use one of the standard binary serialization = formats (protobuf, flatbuffer, ...)?=C2=A0
--001a113cdf42156bd2055094ebea--