unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Andrea Corallo <akrl@sdf.org>
To: Eli Zaretskii <eliz@gnu.org>
Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
Subject: Re: On elisp running native
Date: Sat, 28 Dec 2019 14:35:46 +0000	[thread overview]
Message-ID: <xjfh81kqzzx.fsf@sdf.org> (raw)
In-Reply-To: <83png8o9pz.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 28 Dec 2019 15:34:00 +0200")

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
>> Date: Sat, 28 Dec 2019 11:57:36 +0000
>> 
>> Every .el file is compiled into a .eln.  This is technically a shared
>> library.
>> 
>> Into the .eln Emacs is expecting to find a bunch of symbols.  Two of
>> these are used for relocating objects and Emacs primitive functions.  So
>> Emacs essentially writes some pointers into them.
>
> Can you tell more about this part?  What does this relocation include,
> and to what extent the details depend on the .eln file being in ELF
> format?

Sure.  About these two:

- A pointer has to be set in the shared library to point to the array
  containing all the function pointers of Emacs C primitives and
  functions used to support the run-time.  Lets call it 'function link
  table'.

  There is only one copy of this structure and is malloced by Emacs.

  This is used by the code into the .eln file in order to jump into
  Emacs.  All elns share the same 'function link table'.

- The other structure is an array of lisp objects that are the objects
  coming from the 'constant vectors' of every byte-compiled function.

  This is allocated as static in every eln and initialized during the
  eln load process.

  To recreate the objects Emacs read a string from the eln it-self and
  run the reader.  When the objects are revived into memory these are
  set into the array one by one.

I do not depend on ELF.  I'm already using the functions wrapped by our
'dynlib' in order to execute this machinery.

>> Then there's 'top_level_run'.  This is a real function that is responsible for
>> modifying the environment as the execution of the various top level forms
>> on the original .el file would do.
>> 
>> Emacs jumps into 'top_level_run' and this will call back into Emacs for
>> evaluating forms and/or defining native functions (effectively new
>> subrs).  This function is obviously not called when resuming from an
>> image dump.
>
> So when Emacs calls a Lisp function foo-bar, that call will produce a
> call to 'top_level_run' function of the corresponding .eln file, and
> then 'top_level_run' of that .eln will call the code produced from
> 'foo-bar'?  How does Emacs know which .eln "provides" foo-bar?

Sorry I wasn't clear.

'top_level_run' is run once during file load.  'top_level_run' calls
back into Emacs to register 'foo-bar' (as a lightly special subr).

So after that 'foo-bar' is called as every other subr is.

> And how would the code in the .eln know to call back into Emacs?  When

See above

> you compile the native code, do you leave the symbols of Emacs
> functions unresolved, and rely on the dynamic linker to resolve them?
> If so, this doesn't work on Windows: you will get link errors.

I just use dynlib_sym to obtain the addresses of these few symbols.  The
dynamic linker does not come into play and the rest is handled by the
described mechanism.  The fact that the list of primitives expected by
the eln matches the one provided by the system is checked comparing two
hashes duiring the load process.  If the eln is not compatible an error
is raised.

> Thanks.

I guess it's easier to read in code than to explain (is really not
much).  In case you can find 'load_comp_unit' into:

https://gitlab.com/koral/gccemacs/raw/dev/src/comp.c

Please let me know if I can better clarify some of these points.

Andrea

-- 
akrl@sdf.org



  reply	other threads:[~2019-12-28 14:35 UTC|newest]

Thread overview: 210+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-28  6:54 On elisp running native Andrea Corallo
2019-11-28 11:31 ` Óscar Fuentes
2019-11-28 14:07   ` Andrea Corallo
2019-11-28 12:47 ` Lars Ingebrigtsen
2019-11-28 20:15   ` Andrea Corallo
2019-11-29  6:03     ` Lars Brinkhoff
2019-11-28 16:04 ` Stefan Monnier
2019-11-28 17:28   ` Andrea Corallo
2019-11-28 21:06     ` Stefan Monnier
2019-11-28 22:52       ` Andrea Corallo
2019-11-29  0:20       ` Eric Abrahamsen
2019-11-29 14:59       ` Andrea Corallo
2019-11-29 15:10         ` Eli Zaretskii
2019-11-29 15:30           ` Stefan Monnier
2019-12-01 10:34             ` Andrea Corallo
2019-12-01 15:27               ` Stefan Monnier
2019-11-29 16:43           ` Andrea Corallo
2019-11-29 19:26             ` Eli Zaretskii
2019-11-29 21:18               ` Andrea Corallo
2019-12-08 20:18 ` Andrea Corallo
2019-12-26 13:05   ` Andrea Corallo
2019-12-26 17:30     ` Eli Zaretskii
2019-12-26 18:14       ` Andrea Corallo
2019-12-26 20:37         ` Eli Zaretskii
2019-12-26 22:10           ` Andrea Corallo
2019-12-27  9:12             ` Eli Zaretskii
2019-12-27 10:46               ` Andrea Corallo
2019-12-27 14:03                 ` Eli Zaretskii
2019-12-28  1:17               ` Richard Stallman
2019-12-28  7:50                 ` Eli Zaretskii
2019-12-28  8:17                 ` Andrea Corallo
2019-12-29  0:08                   ` Richard Stallman
2019-12-29  9:29                     ` Andrea Corallo
2019-12-28 13:07               ` Dmitry Gutov
2019-12-28 13:28                 ` Eli Zaretskii
2019-12-28 19:56                 ` Óscar Fuentes
2019-12-29 19:59                 ` chad
2019-12-30  8:17                   ` arthur miller
2019-12-30 14:09                     ` Dmitry Gutov
2019-12-30 15:25                     ` Eli Zaretskii
2019-12-30 22:03                       ` arthur miller
2019-12-31  3:34                         ` Eli Zaretskii
2019-12-30 13:56                   ` Dmitry Gutov
2019-12-30 14:04                     ` Juanma Barranquero
2019-12-30  8:47                 ` Achim Gratz
2019-12-27 15:53             ` Stefan Monnier
2019-12-27 21:02               ` Andrea Corallo
2019-12-27 22:19                 ` Stefan Monnier
2019-12-28  8:34                 ` Eli Zaretskii
2019-12-28  8:56                   ` Andrea Corallo
2019-12-28 10:06                     ` Eli Zaretskii
2019-12-28 11:57                       ` Andrea Corallo
2019-12-28 13:34                         ` Eli Zaretskii
2019-12-28 14:35                           ` Andrea Corallo [this message]
2019-12-28 18:38                             ` Eli Zaretskii
2019-12-28 22:13                               ` Andrea Corallo
2019-12-31  9:55                               ` Andrea Corallo
2019-12-31 15:49                                 ` Eli Zaretskii
2020-02-24 23:21                       ` Andrea Corallo
2020-02-25  6:14                         ` yyoncho
2020-02-25  7:29                           ` Eli Zaretskii
2020-02-25 12:59                             ` Andrea Corallo
2020-02-28  4:34                               ` Eli Zaretskii
2020-02-28  8:26                                 ` Andrea Corallo
2020-02-28  9:23                                   ` yyoncho
2020-04-26 12:00                           ` Andrea Corallo
2020-04-27  4:41                             ` yyoncho
2019-12-26 20:26     ` Stefan Monnier
2019-12-26 20:58       ` dick.r.chiang
2019-12-26 21:57       ` Andrea Corallo
2019-12-27  8:22         ` Eli Zaretskii
2019-12-27 10:49           ` Andrea Corallo
2019-12-28 16:10           ` dick.r.chiang
2019-12-28 17:31             ` Eli Zaretskii
2019-12-31 10:24           ` Andrea Corallo
2019-12-31 15:51             ` Eli Zaretskii
2019-12-31 16:21               ` Andrea Corallo
2019-12-31 16:47                 ` Eli Zaretskii
2019-12-31 22:15                   ` Óscar Fuentes
2020-01-01 12:41                   ` Andrea Corallo
2020-01-01 15:14                     ` Stefan Monnier
2020-01-01 18:42                       ` Andrea Corallo
2020-01-02  7:55                         ` arthur miller
2020-01-02 11:31                           ` Andrea Corallo
2020-01-02 13:53                             ` Eli Zaretskii
2020-01-04 18:17                           ` Stefan Monnier
2020-01-05  1:33                             ` Jean-Christophe Helary
2020-01-05  4:51                               ` Stefan Monnier
2020-01-05  5:18                                 ` Jean-Christophe Helary
2020-01-05  6:36                                   ` Stefan Monnier
2020-01-02 13:47                         ` Eli Zaretskii
2020-01-04 18:14                         ` Stefan Monnier
2020-01-05 10:00                           ` Andrea Corallo
2020-01-02 11:47                       ` Andrea Corallo
2020-01-02 16:52                         ` Ergus
2020-01-02 17:55                           ` Andrea Corallo
2020-01-11 14:10                         ` Andrea Corallo
2020-02-04 11:28                           ` Andrea Corallo
2020-02-04 15:30                             ` Eli Zaretskii
2020-02-04 18:39                               ` Andrea Corallo
2020-02-04 18:58                                 ` Eli Zaretskii
2020-02-04 19:00                                 ` Paul Eggert
2020-02-04 20:30                                   ` Andrea Corallo
2020-02-05 10:15                                     ` Andrea Corallo
2020-02-05 14:24                                       ` Stefan Monnier
2020-02-15  9:29                                         ` Andrea Corallo
2020-02-16 18:18                                           ` Andrea Corallo
2020-02-22 10:15                                             ` On elisp running native - deferred compilation Andrea Corallo
2020-02-22 14:19                                               ` Stefan Monnier
2020-03-04 19:34                                             ` On elisp running native Andrea Corallo
2020-03-04 20:21                                               ` Michael Albinus
2020-03-04 22:16                                                 ` Andrea Corallo
2020-03-05 19:33                                                   ` Michael Albinus
2020-03-06 21:44                                                     ` Andrea Corallo
2020-03-07  9:09                                                       ` Michael Albinus
2020-03-07  9:33                                                         ` Andrea Corallo
2020-03-07  9:41                                                           ` Michael Albinus
     [not found]                                                             ` <xjf36akjx8e.fsf@sdf.org>
     [not found]                                                               ` <87y2sca2ex.fsf@gmx.de>
     [not found]                                                                 ` <xjfy2scigxn.fsf@sdf.org>
     [not found]                                                                   ` <xjftv30hytf.fsf@sdf.org>
2020-03-07 17:57                                                                     ` Michael Albinus
2020-03-07 18:49                                                                       ` Andrea Corallo
2020-03-07 18:54                                                                         ` Michael Albinus
2020-03-07 19:19                                                                           ` Andrea Corallo
2020-03-08 15:03                                                                             ` Andrea Corallo
2020-03-08 20:16                                                                               ` Michael Albinus
2020-03-08 20:55                                                                                 ` Andrea Corallo
2020-03-08 21:09                                                                                   ` Michael Albinus
2020-03-08 21:29                                                                                     ` Andrea Corallo
2020-03-08 21:32                                                                                       ` Michael Albinus
2020-03-08 21:37                                                                                         ` Andrea Corallo
2020-03-05 10:54                                               ` Adam Porter
2020-03-05 12:43                                                 ` Gong-Yi Liao 廖宮毅
2020-03-05 13:27                                                   ` Andrea Corallo
2020-03-08 14:37                                                     ` Gong-Yi Liao 廖宮毅
2020-03-08 15:13                                                       ` Andrea Corallo
2020-03-15 19:00                                                     ` Gong-Yi Liao 廖宮毅
2020-03-15 20:00                                                       ` Andrea Corallo
2020-03-05 13:19                                                 ` Andrea Corallo
2020-03-05 14:42                                                   ` Adam Porter
2020-03-05 19:46                                                     ` Andrea Corallo
2020-03-06 21:15                                                   ` Adam Porter
2020-03-06 21:23                                                     ` Adam Porter
2020-03-06 21:52                                                       ` Andrea Corallo
2020-03-11 22:04                                                 ` Andrea Corallo
2020-03-12 17:10                                                   ` Adam Porter
2020-03-12 17:34                                                   ` Stefan Monnier
2020-03-12 18:03                                                     ` Andrea Corallo
2020-03-23 21:36                                               ` On elisp running native - Update7 Andrea Corallo
2020-03-24  2:59                                                 ` Gong-Yi Liao 廖宮毅
2020-03-24  8:44                                                   ` Andrea Corallo
2020-05-16 15:24                                                 ` On elisp running native - Update 8 Andrea Corallo
2020-05-16 16:21                                                   ` Eli Zaretskii
2020-05-16 20:44                                                     ` Andrea Corallo
2020-05-16 18:22                                                   ` Gregor Zattler
2020-05-16 19:59                                                     ` Andrea Corallo
2020-05-17  5:26                                                       ` Gregor Zattler
2020-06-14 13:23                                                   ` On elisp running native - Update 9 Andrea Corallo
2020-06-14 17:23                                                     ` arthur miller
2020-06-15  5:04                                                     ` Gong-Yi Liao 廖宮毅
2020-06-15  7:41                                                       ` Andrea Corallo
2020-06-17 23:54                                                     ` Stefan Monnier
2020-06-18  6:55                                                       ` Andrea Corallo
2020-07-16 20:00                                                     ` On elisp running native - Update 10 Andrea Corallo via Emacs development discussions.
2020-08-17 18:43                                                       ` On elisp running native - Update 11 Andrea Corallo via Emacs development discussions.
2020-08-18  5:13                                                         ` Arthur Miller
2020-08-18  8:17                                                           ` Andrea Corallo via Emacs development discussions.
2020-09-26 10:35                                                         ` On elisp running native - Update 12 Andrea Corallo via Emacs development discussions.
2020-09-26 14:29                                                           ` T.V Raman
2020-09-26 14:56                                                             ` Andrea Corallo via Emacs development discussions.
2020-09-26 15:07                                                               ` T.V Raman
2020-09-28 12:22                                                                 ` Lars Brinkhoff
2020-09-28 14:03                                                                   ` T.V Raman
2020-09-27 10:35                                                           ` Arthur Miller
2020-09-27 18:28                                                             ` Andrea Corallo via Emacs development discussions.
2020-09-27 18:30                                                               ` Lars Ingebrigtsen
2020-09-27 20:39                                                                 ` T.V Raman
2020-09-27 22:10                                                                 ` Stefan Monnier
2020-09-28  6:06                                                                 ` Eli Zaretskii
2020-09-28 10:28                                                                   ` Andrea Corallo via Emacs development discussions.
2020-09-28 10:45                                                                     ` Eli Zaretskii
2020-09-28 11:02                                                                       ` Andrea Corallo via Emacs development discussions.
2020-09-27 20:37                                                               ` T.V Raman
2020-09-28 14:10                                                           ` Yuan Fu
2020-09-28 14:51                                                             ` Andrea Corallo via Emacs development discussions.
2020-09-28 15:20                                                               ` Yuan Fu
2020-09-28 15:32                                                                 ` Andrea Corallo via Emacs development discussions.
2020-09-28 15:19                                                           ` Caio Henrique
2020-09-28 15:39                                                             ` Andrea Corallo via Emacs development discussions.
2020-09-28 16:12                                                               ` Caio Henrique
2020-09-28 17:21                                                                 ` Andrea Corallo via Emacs development discussions.
2021-01-01 18:37                                                           ` On elisp running native - Update 13 Andrea Corallo via Emacs development discussions.
2021-01-01 20:07                                                             ` tomas
2020-01-01 15:19                     ` On elisp running native Óscar Fuentes
2020-01-01 18:48                       ` Andrea Corallo
2020-01-01 14:59             ` Stefan Monnier
2020-03-15 14:21 ` Spenser Truex
2020-03-15 20:10   ` Andrea Corallo
2020-03-18 14:58     ` Lars Brinkhoff
2020-03-18 17:14       ` Andrea Corallo
  -- strict thread matches above, loose matches on Subject: below --
2020-01-02 21:47 arthur miller
2020-03-06 13:28 Zhu Zihao
2020-03-06 17:51 ` Andrea Corallo
2020-03-09 17:55   ` Zhu Zihao
2020-03-09 18:52     ` Andrea Corallo
2020-03-09 19:14       ` Zhu Zihao
2020-03-09 19:41         ` Andrea Corallo
2020-03-09 21:06         ` Stefan Monnier
2020-03-09 19:18       ` Eli Zaretskii
2020-03-09 19:42         ` Andrea Corallo
2020-04-05 21:48         ` Andrea Corallo
2020-03-10 16:15   ` Zhu Zihao
2020-03-10 17:46     ` Andrea Corallo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xjfh81kqzzx.fsf@sdf.org \
    --to=akrl@sdf.org \
    --cc=eliz@gnu.org \
    --cc=emacs-devel@gnu.org \
    --cc=monnier@iro.umontreal.ca \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).