From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Pascal J. Bourguignon" Newsgroups: gmane.emacs.devel Subject: Re: FFI in Emacs Date: Tue, 12 Mar 2013 23:40:14 +0100 Organization: Informatimago Message-ID: <87d2v4s0mp.fsf@kuiper.lan.informatimago.com> References: <83d2v4jnih.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1363130533 21618 80.91.229.3 (12 Mar 2013 23:22:13 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 12 Mar 2013 23:22:13 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Mar 13 00:22:38 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UFYWj-0004vJ-UY for ged-emacs-devel@m.gmane.org; Wed, 13 Mar 2013 00:22:38 +0100 Original-Received: from localhost ([::1]:56004 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UFYWN-0008H6-Iu for ged-emacs-devel@m.gmane.org; Tue, 12 Mar 2013 19:22:15 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:48676) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UFYWH-0008DV-SM for emacs-devel@gnu.org; Tue, 12 Mar 2013 19:22:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UFYWB-0001jg-4t for emacs-devel@gnu.org; Tue, 12 Mar 2013 19:22:09 -0400 Original-Received: from plane.gmane.org ([80.91.229.3]:36909) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UFXry-0005hQ-Om for emacs-devel@gnu.org; Tue, 12 Mar 2013 18:40:30 -0400 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1UFXsH-0006HD-5b for emacs-devel@gnu.org; Tue, 12 Mar 2013 23:40:49 +0100 Original-Received: from amontsouris-651-1-209-205.w92-140.abo.wanadoo.fr ([92.140.0.205]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 12 Mar 2013 23:40:49 +0100 Original-Received: from pjb by amontsouris-651-1-209-205.w92-140.abo.wanadoo.fr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 12 Mar 2013 23:40:49 +0100 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 67 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: amontsouris-651-1-209-205.w92-140.abo.wanadoo.fr Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAQMAAABtzGvEAAAABlBMVEUAAAD///+l2Z/dAAAA oElEQVR4nK3OsRHCMAwF0O8YQufUNIQRGIAja9CxSA55AxZgFO4coMgYrEDDQZWPIlNAjwq9 033pbOBPtbXuB6PKNBn5gZkhGa86Z4x2wE67O+06WxGD/HCOGR0deY3f9Ijwwt7rNGNf6Oac l/GuZTF1wFGKiYYHKSFAkjIo1b6sCYS1sVmFhhhahKQssRjRT90ITWUk6vvK3RsPGs+M1RuR mV+hO/VvFAAAAABJRU5ErkJggg== X-Accept-Language: fr, es, en User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (gnu/linux) Cancel-Lock: sha1:MGNlNmQxYzhhNGE4MjRhYzE5YTJiZGFjMzkyMDg5OTU0Y2Y5ZmQyNA== X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:157802 Archived-At: Eli Zaretskii writes: >> Date: Tue, 12 Mar 2013 20:16:42 +0100 >> From: Aurélien Aptel >> Cc: emacs-devel@gnu.org >> >> a) load the shared library dynamically >> b) get the address of any symbol in it (be it a global var or a function). >> >> Then, to call a function: >> c) load the argument according to user-provided type and ABI >> d) make the call >> e) retrieve eventual return value (also depends on ABI) >> >> Steps a and b can be done portably using libltdl [1]; c,d and e with >> libffi [2] or libffcall [3]. > > Do we really need these libraries? If the Posix hosts can do with > dlopen, dlsym, dlclose, and dlerror, then it's very easy to emulate > that on platforms that don't have these in the system libraries. What > else is needed, and why? > >> a) a function can allocate memory that has to be freed > > At least on Windows, this cannot be done safely, so please don't > design the interface based on the assumption this is doable. If a > shared object allocates memory, it should be responsible for freeing > it, or provide an API for telling it to free it. > > Likewise with file descriptors -- they cannot be safely shared across > the interface, because the shared library could have used a different > runtime for opening files. Is emacs garbage collector of the copying kind? If so, then objects move in memory during garbage collection, and therefore any reference given to a foreign library become invalid once the garbage collector runs. And vice-versa, managing resources allocated in the library (even if we use library calls to do so) is quite complex, when those resources have to be fred when the lisp object that has a reference is collected. (Do we have finalizer in emacs objects?) Soon the easiest solution is to copy a lot of data, between the lisp and C side, which is even more costly. A lot of libraries are written in C++ too, and they're a bitch to write a FFI for, in general (ie. as soon as they use templates, smartptr anybody?), not counting the compiler-dependent name mangling which is relatively a simple problem to solve. I mean, even if emacs FFI was perfect (the FFI of some CL implementations are good enough), you would still have those two problems, of dealing with C++ libraries, and of crashing into C code, which makes me choose often to just rewrite the thing in lisp, or to write an external wrapper program in C, communicating with the lisp process with whatever IPC is practical (files, sockets, etc). Also, I guess a lot of modern libraries don't need anything fancy from C/C++ either, so perhaps the best solution would be to have actually a C/C++ compiler targetting the emacs VM, and instead of doing FFI to an external binary library, we'd just compile it into the emacs VM. But perhaps writing a whole new compiler is more work than hacking half a FFI. -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}.