From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Dynamic loading progress Date: Sun, 15 Feb 2015 19:32:41 +0200 Message-ID: <83oaovull2.fsf@gnu.org> References: <85oapy5kt6.fsf@stephe-leake.org> <83y4oiiw81.fsf@gnu.org> <838ugdf251.fsf@gnu.org> <54D80098.3020209@cs.ucla.edu> <54D85304.1030600@cs.ucla.edu> <54D9AC29.2020603@cs.ucla.edu> <54DA8539.1020905@cs.ucla.edu> <87zj8ktq8f.fsf@lifelogs.com> <54DD6413.1000403@cs.ucla.edu> <83wq3m436s.fsf@gnu.org> <834mqowbnh.fsf@gnu.org> <85egpsgf5z.fsf@stephe-leake.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1424021580 11404 80.91.229.3 (15 Feb 2015 17:33:00 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 15 Feb 2015 17:33:00 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stephen Leake Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Feb 15 18:32:54 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YN33u-0002EI-BM for ged-emacs-devel@m.gmane.org; Sun, 15 Feb 2015 18:32:54 +0100 Original-Received: from localhost ([::1]:36096 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YN33t-0007zH-Io for ged-emacs-devel@m.gmane.org; Sun, 15 Feb 2015 12:32:53 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:56452) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YN33l-0007z4-GQ for emacs-devel@gnu.org; Sun, 15 Feb 2015 12:32:50 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YN33g-0000Kh-4W for emacs-devel@gnu.org; Sun, 15 Feb 2015 12:32:45 -0500 Original-Received: from mtaout24.012.net.il ([80.179.55.180]:56216) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YN33f-0000Ig-M1 for emacs-devel@gnu.org; Sun, 15 Feb 2015 12:32:40 -0500 Original-Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0NJT00300PFFSJ00@mtaout24.012.net.il> for emacs-devel@gnu.org; Sun, 15 Feb 2015 19:24:19 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NJT000P3POJJ230@mtaout24.012.net.il>; Sun, 15 Feb 2015 19:24:19 +0200 (IST) In-reply-to: <85egpsgf5z.fsf@stephe-leake.org> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.179.55.180 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:183095 Archived-At: > From: Stephen Leake > Date: Sat, 14 Feb 2015 19:02:48 -0600 > > I took a stab at creating emacs_module_api.h; attached. > > To do this, I replaced "#include " > in modules/curl/curl.c with "#include ", and copied > stuff from lisp.h and other headers to emacs_module_api.h until it > compiled. Note the FIXMEs; I could not figure out what was going on in > some places, so I just kludged it for now. Thanks. I started to write comments to that file, but gave up in despair. My overall conclusion from reviewing it is that, if we want the tightly-coupled version of modules, we might as well compile them as any other C source file in the Emacs tree, i.e. we could simply include lisp.h in its entirety, and all the other headers that it pulls in. I see no other workable alternative that would let modules use Lisp objects transparently (as opposed to using them as opaque objects), call Lisp primitives directly like we do in Emacs sources, etc. > > This already presents at least 2 problems. The first is 32- vs > > 64-bit issues. We need some way of checking that a 32-bit module > > isn't loaded by a 64-bit Emacs, and vice versa. > Good point. That's a special case of "don't load a module compiled for > processor X in an Emacs compiled for processor Y". gnu binutils has > facilities for that. Not sure how Binutils could help here. AFAIK, dlopen and dlsym (and their equivalents) don't invoke Binutils on any platform. > > The other problem is more serious: Emacs can be built with or > > without --check-lisp-object-type, and with or without --wide-int; > > how will the module know which implementation to use? > > Similar check when the module is loaded. Which means we need some > metainfo for each module, and a standard function to retrieve it. That "metainfo", whatever it will be, will also have to support compilation of modules: we need to know at compile time what values of USE_LSB_TAG, ENABLE_CHECKING, WIDE_EMACS_INT, optimization flags, etc. were used when Emacs was built, because those affect the layout of objects inside Emacs and also the availability of certain functions in the Emacs binary. > > . Functions to access values of Lisp objects. We shouldn't rely on C > > macros like XINT and XWINDOW for that, because macros track too > > closely the internals of each object. So I propose to use > > functions that will be exposed via the API. > > If we follow Stefan's suggestion, then either this function approach is > not viable, or we need another #ifdef for module vs emacs. If we don't use the function approach, we can only have modules that are tightly coupled with the version of Emacs for which they were written and compiled. We have no hope of binary compatibility with other versions of Emacs, except by sheer luck, and even source-level compatibility will be a challenge. > > They are currently written as if the code were an integral part of > > Emacs on one of the Emacs C files. Going that way means that modules > > will have to be recompiled from sources for each Emacs version, and > > practically will have to live in the Emacs tree. Maybe this is what we > > want, I don't know. > > This is the model I had in mind. Since I need max speed with mixed > Ada/lisp code, I need tight integration with the core. The people who > need that speed for Ada mode will simply have to recompile the module > for each Emacs release; they are coders, so that's not a problem. Being an Ada programmer (or even a C programmer) doesn't necessarily mean you know how to fix compile- and link-time problems with Emacs. You've just bumped into that yourself. In practice, I think this means maintainers of modules will have to adapt their modules to each Emacs version, and perhaps also keep several branches, one each for every version they want to support. > Rather than splitting out emacs_module_api.h from lisp.h, we could add > __declspec(dllexport) to a subset of the functions in lisp.h; I suspect > that will be a small number. What for? If we are going to expose most or all of lisp.h to modules, we are actually saying that all of Emacs internals should be exposed as well, so all of the lisp.h stuff should be externally visible, as it is used all over the place in Emacs sources, and modules will want to be able to do the same, under the "tightly-coupled" concept. > Is there an equivalent on Linux (I don't recall ever seeing one)? You want '__attribute__((visibility ("default")))', and you want to use -fvisibility=hidden on the command line. See https://gcc.gnu.org/wiki/Visibility But this looks futile wrt lisp.h, see above. Perhaps we could use that for other public functions in other headers, but I expect that paradigm to collapse very quickly, since modules will want to call Lisp primitives directly, not through funcall. Look at the modules already on the branch: they include buffer.h and character.h, and call stuff like Fgethash, Fputhash, and Fmake_hash_table. How much time will pass before we see modules call Finsert_file_contents, Fmake_frame, and Fredisplay? How can we explain to module developers that they need to go through funcall? And then "more internal" internals will quickly follow: scan_words, for example, or recenter_overlay_lists. It's a slippery slope, but I don't see how we could prevent the slip if we are willing to go with "tightly-coupled" modules.