From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Dynamic loading progress Date: Tue, 17 Feb 2015 17:43:12 +0200 Message-ID: <83h9uk7ddb.fsf@gnu.org> References: <838ugdf251.fsf@gnu.org> <87bnl1vmqf.fsf@lifelogs.com> <87vbj8tow4.fsf@lifelogs.com> <87r3twtagf.fsf@lifelogs.com> <85siebl7ws.fsf@stephe-leake.org> <85a90ilwmm.fsf@stephe-leake.org> <83386a6f7z.fsf@gnu.org> <85h9upjz7v.fsf@stephe-leake.org> <83wq3k3kl4.fsf@gnu.org> <85bnkwil1c.fsf@stephe-leake.org> <83pp9cwky8.fsf@gnu.org> <85a90ggf2d.fsf@stephe-leake.org> <54E0A40F.5080603@dancol.org> <83sie7un20.fsf@gnu.org> <54E0D181.2080802@dancol.org> <83r3trulse.fsf@gnu.org> <54E0D7E0.305@[87.69.4.28]> <83h9unukbg.fsf@gnu.org> <54E0DEF8.7020901@dancol> <83egpruiyp.fsf@gnu.org> <54E0FF93.2000104@dancol.org> <833865vp4d.fsf@gnu.org> <54E2355A.90@[87.69.4.28]> <83vbj1u020.fsf@gnu.org> <54E24CA4.9020601@dancol.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1424187809 14029 80.91.229.3 (17 Feb 2015 15:43:29 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 17 Feb 2015 15:43:29 +0000 (UTC) Cc: stephen_leake@stephe-leake.org, emacs-devel@gnu.org To: Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Feb 17 16:43:20 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YNkIr-0002Hf-Uo for ged-emacs-devel@m.gmane.org; Tue, 17 Feb 2015 16:43:14 +0100 Original-Received: from localhost ([::1]:45953 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YNkIr-0003AY-AE for ged-emacs-devel@m.gmane.org; Tue, 17 Feb 2015 10:43:13 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44616) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YNkIn-0003AE-8D for emacs-devel@gnu.org; Tue, 17 Feb 2015 10:43:10 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YNkIk-0008CJ-0T for emacs-devel@gnu.org; Tue, 17 Feb 2015 10:43:09 -0500 Original-Received: from mtaout27.012.net.il ([80.179.55.183]:56713) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YNkIj-0008Bk-Hy for emacs-devel@gnu.org; Tue, 17 Feb 2015 10:43:05 -0500 Original-Received: from conversion-daemon.mtaout27.012.net.il by mtaout27.012.net.il (HyperSendmail v2007.08) id <0NJX00O009RWI000@mtaout27.012.net.il> for emacs-devel@gnu.org; Tue, 17 Feb 2015 17:37:23 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout27.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NJX00LATA2BZI30@mtaout27.012.net.il>; Tue, 17 Feb 2015 17:37:23 +0200 (IST) In-reply-to: <54E24CA4.9020601@dancol.org> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.179.55.183 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:183195 Archived-At: > Date: Mon, 16 Feb 2015 12:01:40 -0800 > From: Daniel Colascione > CC: stephen_leake@stephe-leake.org, emacs-devel@gnu.org > > >>> This approach requires us to change the size each time we change the > >>> layout, even if the change itself leaves the size intact. Using a > >>> version field doesn't have this disadvantage. > >> > >> Emacs is allowed to pass a new version of the runtime and context > >> structures to modules expecting old versions. That can only work if the > >> structures are append-only, so `size' is adequate here. > > > > I understand that, but appending members does not necessarily increase > > the size of the struct, due to alignment, bit fields, etc. > > If it's just a table of function pointers and everything has natural > alignment, adding members will increase the size. Having to include only function pointers is also a kind of limitation inherent to this method. I guess I just don't understand why you didn't go with a version field. What are its disadvantages? The value can be generated automatically, so it isn't more error-prone than the size. > Say do_frobricate has address 0xA3B41234. Implementation-wise, the value > `frobricate` that `make_function' returned is a Lisp closure that > essentially looks like this: > > (lambda (&rest args) (%do-module-call frobricate-module 0xA3B41234 2 2 > args)) > > %do-module-call then clears the pending-error information, packs its > arguments into a C array, and calls 0xA3B41234 like this: > > emacs_subr module_function = (module_function) 0xA3B41234; > result = module_function(some_env, args, nargs); It's the arguments that I was talking about. AFAIU, you are saying that %do-module-call takes arguments, which are normal Lisp_Object values, generates the indirection table for them, which stores _pointers_ to the actual arguments, then puts those pointers into a C array, and calls 0xA3B41234. Is that correct? If so, I guess I don't see why we would need all that complexity and extra processing. Why not put the original Lisp_Object values into the args[] array to begin with? > >>> What about the doc string? > >> > No: that's silly. I'm saying that creating a globally-named function in > Emacs has two parts: 1) create the actual _function object_. 2) bind > that function object to a name by setting some symbol's function slot to > that function object. We can do step #2 entirely in Lisp code that the > module calls, and we can set a documentation property at this time. It's > only step #1 that requires a magical incantation. > > I've only described the magical, ABI-relevant bits of the interface. In > order to make a module actually usable, we'll need glue, but since we > can implement this glue in Lisp and call it from module code using the > narrow interface I've described here, I'm not worried about it. Sorry, I'm still not out of the woods. Providing a doc string has nothing to do with calling the function. You need to have the doc string available even if you never call the function. The trigger for accessing the doc string is _outside_ of the module, by one of the help commands. So how does all this work in your scenario above, where we called make_function, which returned 'frobnicate', which is a value in the function slot of some Lisp object? Say now Emacs calls 'documentation-property' on that Lisp object or it calls 'documentation' on 'frobnicate' -- how will that produce a doc string? > >>> Do we need 'unintern' as well? > >> > >> Modules can call unintern through Lisp. > > > > Then why provide 'intern'? It, too, can be called from Lisp. > > Modules have no way other than `intern' of getting an emacs_value that > corresponds to a known symbol. They can call `unintern' by first > interning the symbol "unintern", then calling it. You mean, modules cannot use funcall to call Lisp functions, without some complicated dance with 'intern' before that? Why is that needed? why not let modules call Lisp functions by their string name, and do the 'intern' stuff on the Emacs side of the interface? > >>>> emacs_value (*type_of)( > >>>> emacs_env* env, > >>>> emacs_value value); > >>>> > >>>> Like Lisp type-of: returns a symbol. > >>> > >>> What is a "symbol", from the module's C code POV? > >> > >> It's an emacs_value. You can compare symbols (by calling `eq'), call > >> them as functions, or use them as function arguments. That's sufficient. > >> > >> Come to think of it, though, we do need a C-level `eq'. > > > > Why not simply return a C enumeration type? > > Why make the set of elisp primitive types part of the ABI? An enum is just a list of named integer values. Additions to that list make more values valid, that's all. How is that "part of the ABI" that could introduce incompatibility? OTOH, making the life of module writers harder on every turn is not a good way of encouraging modules. > >>> . not sure how will a module "provide" its feature > >> > >> It doesn't. You don't require a module: you load it. Most modules will > >> come with a small Lisp wrapper, just like Python modules. > > > > So do we _require_ a Lisp wrapper? Otherwise, how would Emacs know > > whether the module is loaded or not? > > By path, I imagine. What path is that? Where and how will Lisp code find it? > Why don't you have this problem with (load "myfile.el")? Because myfile.el normally does a (provide 'myfile), and then I can use featurep. > I'm not opposed to hooking modules into the provide-and-require > machinery, but I don't see a particular need either. The need is to have an easy way of checking that a module is available, without risking loading it twice, or having to jump through hoops. > >>> One thing that's inconvenient is the need to drag the environment > >>> pointer through all the calls. Why exactly is that needed? > >> > >> Modules have no other way of accessing Emacs internals. Dragging one > >> parameter through the system isn't that bad and lets us sanely track > >> which module is doing what. > > > > The environment must be known to the module, that's for sure. But why > > does it have to be plugged into every call to every interface > > function? > > How else are the interface functions supposed to know the module for > which they're being called? Why would the interface functions care? Which parts of the environment are needed for the interface to know about the module in order to do its job?