From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo Newsgroups: gmane.emacs.devel Subject: Re: On elisp running native Date: Sat, 28 Dec 2019 14:35:46 +0000 Message-ID: References: <838smzq9iz.fsf@gnu.org> <8336d6rfgy.fsf@gnu.org> <83woagonl9.fsf@gnu.org> <83sgl4ojci.fsf@gnu.org> <83png8o9pz.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="49476"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Dec 28 15:36:29 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ilDCf-000CjP-D9 for ged-emacs-devel@m.gmane.org; Sat, 28 Dec 2019 15:36:29 +0100 Original-Received: from localhost ([::1]:43792 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ilDCe-000420-6Z for ged-emacs-devel@m.gmane.org; Sat, 28 Dec 2019 09:36:28 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42829) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ilDC3-0003Yj-Ox for emacs-devel@gnu.org; Sat, 28 Dec 2019 09:35:53 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ilDC2-00076o-9r for emacs-devel@gnu.org; Sat, 28 Dec 2019 09:35:51 -0500 Original-Received: from mx.sdf.org ([205.166.94.20]:61454) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ilDC0-0006rd-52; Sat, 28 Dec 2019 09:35:48 -0500 Original-Received: from sdf.org (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id xBSEZlMG006794 (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits) verified NO); Sat, 28 Dec 2019 14:35:47 GMT Original-Received: (from akrl@localhost) by sdf.org (8.15.2/8.12.8/Submit) id xBSEZkSu009304; Sat, 28 Dec 2019 14:35:46 GMT In-Reply-To: <83png8o9pz.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 28 Dec 2019 15:34:00 +0200") X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 205.166.94.20 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:243732 Archived-At: Eli Zaretskii writes: >> From: Andrea Corallo >> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org >> Date: Sat, 28 Dec 2019 11:57:36 +0000 >> >> Every .el file is compiled into a .eln. This is technically a shared >> library. >> >> Into the .eln Emacs is expecting to find a bunch of symbols. Two of >> these are used for relocating objects and Emacs primitive functions. So >> Emacs essentially writes some pointers into them. > > Can you tell more about this part? What does this relocation include, > and to what extent the details depend on the .eln file being in ELF > format? Sure. About these two: - A pointer has to be set in the shared library to point to the array containing all the function pointers of Emacs C primitives and functions used to support the run-time. Lets call it 'function link table'. There is only one copy of this structure and is malloced by Emacs. This is used by the code into the .eln file in order to jump into Emacs. All elns share the same 'function link table'. - The other structure is an array of lisp objects that are the objects coming from the 'constant vectors' of every byte-compiled function. This is allocated as static in every eln and initialized during the eln load process. To recreate the objects Emacs read a string from the eln it-self and run the reader. When the objects are revived into memory these are set into the array one by one. I do not depend on ELF. I'm already using the functions wrapped by our 'dynlib' in order to execute this machinery. >> Then there's 'top_level_run'. This is a real function that is responsible for >> modifying the environment as the execution of the various top level forms >> on the original .el file would do. >> >> Emacs jumps into 'top_level_run' and this will call back into Emacs for >> evaluating forms and/or defining native functions (effectively new >> subrs). This function is obviously not called when resuming from an >> image dump. > > So when Emacs calls a Lisp function foo-bar, that call will produce a > call to 'top_level_run' function of the corresponding .eln file, and > then 'top_level_run' of that .eln will call the code produced from > 'foo-bar'? How does Emacs know which .eln "provides" foo-bar? Sorry I wasn't clear. 'top_level_run' is run once during file load. 'top_level_run' calls back into Emacs to register 'foo-bar' (as a lightly special subr). So after that 'foo-bar' is called as every other subr is. > And how would the code in the .eln know to call back into Emacs? When See above > you compile the native code, do you leave the symbols of Emacs > functions unresolved, and rely on the dynamic linker to resolve them? > If so, this doesn't work on Windows: you will get link errors. I just use dynlib_sym to obtain the addresses of these few symbols. The dynamic linker does not come into play and the rest is handled by the described mechanism. The fact that the list of primitives expected by the eln matches the one provided by the system is checked comparing two hashes duiring the load process. If the eln is not compatible an error is raised. > Thanks. I guess it's easier to read in code than to explain (is really not much). In case you can find 'load_comp_unit' into: https://gitlab.com/koral/gccemacs/raw/dev/src/comp.c Please let me know if I can better clarify some of these points. Andrea -- akrl@sdf.org