From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: arthur miller Newsgroups: gmane.emacs.devel Subject: Sv: as for Calc and the math library Date: Wed, 14 Aug 2024 08:45:57 +0000 Message-ID: References: <864j7qhup6.fsf@gnu.org> <87a5hi0yts.fsf@valhala.localdomain> <86y152ge0b.fsf@gnu.org> <875xs60wmc.fsf@valhala.localdomain> <86wmklho4m.fsf@gnu.org> <86bk1whb9v.fsf@gnu.org> <86ttfng066.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="_000_DU2PR02MB101098E947A3DB1D87E4EC7A296872DU2PR02MB10109eu_" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36146"; mail-complaints-to="usenet@ciao.gmane.io" Cc: "nicolas@n16f.net" , "emacs-devel@gnu.org" To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Aug 14 11:16:52 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1seA7e-00098Y-7z for ged-emacs-devel@m.gmane-mx.org; Wed, 14 Aug 2024 11:16:51 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1seA6w-0004yG-2H; Wed, 14 Aug 2024 05:16:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1se9dv-0001Lj-3r for emacs-devel@gnu.org; Wed, 14 Aug 2024 04:46:07 -0400 Original-Received: from mail-northeuropeazolkn190120001.outbound.protection.outlook.com ([2a01:111:f403:d200::1] helo=DU2PR03CU002.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1se9dp-0006Ke-GE; Wed, 14 Aug 2024 04:46:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WmeDhMHDAwPEW1z6JMwZ/6yFxIFQ9LpZCaLhZi8oqHFBSSEK9P35DRjqPAA0w7K8cpgd+yzjIvcQCdA901YmIrQyjiSlC9luos1CR7/ofBfbVuwONGZPLYoaDL2Ulc5NF7ASkNFbGqqoyobdmc8MOMh7TSL3buv5baK65LfPMOhOrpAuZUz2VRjtz2qHuwl+FbYWDdaz0v1xBP6M2a73kbO2TmWDD63QjYwy7Z6GMB7B8nLev7zwuWvUa7iLNyrXVypc+krzlLCZ9Wo+YxgYMhdq3BLkjh6dsjmE/Sofx9OZWKTnDv2AlEW3cP75yZ6/xNTNMVJ/gfzhp1ou8mgsYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BPRz+rgesBvdUsDcZrCrMaA7qjMjIq5gDEw3sqZ9rGY=; b=gynGyrzB5rI8xLpoP8ifsfnl7OrmB3FFLgKY4+oNuRnb6uutFrJUXtMyISLz4l7VkrUALQhRQrX+owuzg3jbSJ7V0e5nKDKlWsUO2e+D9vDGBNU/+sVt+FZoYlDTKHafmE5UTeeeN2PwCJsMI8utjCg3oi4bPHeedceaL3vD0872vKextonC17iOUFp5pFPOoN2RmmcWZZrLnUSeBj5RxwYDSDDJmY47Jq/hhlgKkuaXSXcVnRZbUvNaBexbxcJsiXo2oDnGarPuAkXvk9hrSbaTPo+ESCmZLtWh16zF03Rw01KEDcsfo1djJWc4PzLhesRjttE+BRWK1nFYx+BEew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPRz+rgesBvdUsDcZrCrMaA7qjMjIq5gDEw3sqZ9rGY=; b=kHtZssjuIK2HHfARjVAltYCG1ofB5yDVpAxUs9WNtVry0Y83u4+KDN2wT2WFEyPU26MuUgenY7TnB6fWh5AxTFGUGrF85wFYUQgB4U0yJx0fecPWJcGVXvx7KY4TAt8E+VZkbjtGH+j5UnD/2NpQDYZO4FBi5tSqywoKde7kgL4i9W5DYgdxyrAKFB0D+2mHPqD82hRG2Y5ZlkPN+Dnq+ZUPIvWv3ipNtzmG2DYT+fbPuEhXGV1IZDw14ocMdGSVISAksJ1VA9t7td6eUvcUXZT7qj2nygpnFX6hVQFiqDQ/nD918ROGboNTbpRtEqdHkJyKQnxkOB6DYB0lSlcJ8Q== Original-Received: from DU2PR02MB10109.eurprd02.prod.outlook.com (2603:10a6:10:497::14) by AS8PR02MB6903.eurprd02.prod.outlook.com (2603:10a6:20b:2e3::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22; Wed, 14 Aug 2024 08:45:57 +0000 Original-Received: from DU2PR02MB10109.eurprd02.prod.outlook.com ([fe80::f3c9:d4cb:290:d487]) by DU2PR02MB10109.eurprd02.prod.outlook.com ([fe80::f3c9:d4cb:290:d487%4]) with mapi id 15.20.7875.016; Wed, 14 Aug 2024 08:45:57 +0000 Thread-Topic: as for Calc and the math library Thread-Index: AQHa7HhyCr4HNzb0tkSg9b4Ke04UmbIjdNFDgAAGX1aAAAZxKoAABt/dgAAT+BqAASgZ9IAAVrozgACZUz2AAILo4YAAPCLp In-Reply-To: <86ttfng066.fsf@gnu.org> Accept-Language: sv-SE, en-US Content-Language: sv-SE x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [7IAwmDLHoiToH8nNFE94MnevxqBm+3Jr] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DU2PR02MB10109:EE_|AS8PR02MB6903:EE_ x-ms-office365-filtering-correlation-id: 3895ac7b-e4bd-4eff-1f15-08dcbc3d8415 x-microsoft-antispam: BCL:0; ARA:14566002|15080799003|15030799003|19110799003|8060799006|461199028|1602099012|3412199025|4302099013|440099028|102099032; x-microsoft-antispam-message-info: z/izYafRzG3g0Hkli7MH83GrBzdkd1AdnyBOhWgyNF0WO1HnPGyfHT5pDvMN2nqpGGPypGyKurOq6LNnWZMCGjwKUdyczbdLg+sA14ykMbZMn4v4duuFp8HsoaP8hnAuoJfeBw4ZRv0OWr1k7k3SdpDq8xde54Bn1TiwKDA4hYgxhjwKW2RyXy2b7JWoSLO+OK+qfSjPbCdquWHwBpyKW+KrNNW/Nhmxv/3y2FEW5NYko4QsXZcObmpJs2PMSq915SbDfAZzJHy9L7AyNgDoWVj+V0eD/FIoPbR4hcJTqhViQrDCl6k+bePZAUO0VEgesASudYsVjeePGvIogohrhLL5Ll0OM8CtAOBPjM0tmyIMnLlqk7QcBnx/avqt0GWlesl966etlI7/sRbZsSZH22vslbWm8oab4UAFklstz1zH1R0lMV811jhQnHf4jE2+yb/CLtCiSeqEBw9BXuga9+xzsFkK3H0DjplVM/NjyA4ShKpGrWb2F606teTXpjc5GRQU2vgyE1fkc/ZXO7gqrlIp7Aa4yRIQh0Nwe7C1ywvb7cNEx8/HSs87au1iA/5916hEs03WrX5kNPsn7nZy2guiEbmYz6yMZYz7jRNuqe4qRso0HI51iFzUgl9J50bzGEoWGgWXYWufpHISHW6U98GvQTwo4X2KEAH27yBCVGORrX4r0BU5vwAHP0qyDQ9r84F163eQmIOw2919NVKlKUTRudJRs6cz1S1oK4wWvdCPlA3CPc/Kgeb03Tok5 oCxLmUneVNAJZdprz6ZldQTW5qRCtRuV4yMiy6mEPTA16SX10mxfxSKJPyeecNUQs54 x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?Bqg0i5Uragx+X+pbRLQTgiOyHBRzV3rStQ0s4zk9fDg7+8UCn/NvyH3cXE?= =?iso-8859-1?Q?MPGxtYhuFU8mGW+9v7ol9gAF176EMeCA4qTdC1CQ2I3T5W+DTXw9ZotoaG?= =?iso-8859-1?Q?2GvziPGD9cs1oEHprFFfdWTNJmjpNuKZJQPufVFSIC9ifuZXNZafIQKhsq?= =?iso-8859-1?Q?by1QYi8K6bBlP+CImY2JEBhLNh3umZ6MpEiqckVUCkTa8rFySFqTwenwPO?= =?iso-8859-1?Q?J6TwHNPCR671XP4fNAbugitICYdRN/IMcNaexiSE/U6ku0X8kuuKItYn0K?= =?iso-8859-1?Q?lKjRTio+Fr1lTmLIuHlG/L6EQLv1sDP6WuUNQSFBKYcBWEx8jW946D36Cy?= =?iso-8859-1?Q?mTxy5EwwJrLyWU2hLr+AAqKeMoq3fJeU2hFVNKhIoe0OlVwh2qpZjAgZUT?= =?iso-8859-1?Q?2aXCSYvl2Acwj7e6FjIKH+teBnTPTjC5aj8aPw8FXYtf5+LIDdyByy7FqP?= =?iso-8859-1?Q?eI4QcKErbWdrUJzN6Ub/59poEq3kKQ2VUEYUMsSrS0ko8V9jljVFf42QzF?= =?iso-8859-1?Q?GYDqB9eqpFNdS6po3XwM+JYhcXxD3Nn9jr0gjBHVmtLuwXtBuRDE3iAwxh?= =?iso-8859-1?Q?BFe+ZqUsZOxqusxiv1Z5jqVlxC+/BIFJrLZIPD3reSLm2roZXS3SMWgDNF?= =?iso-8859-1?Q? X-OriginatorOrg: sct-15-20-7828-19-msonline-outlook-12d23.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DU2PR02MB10109.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 3895ac7b-e4bd-4eff-1f15-08dcbc3d8415 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Aug 2024 08:45:57.4353 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR02MB6903 Received-SPF: pass client-ip=2a01:111:f403:d200::1; envelope-from=arthur.miller@live.com; helo=DU2PR03CU002.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 14 Aug 2024 05:16:01 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:322737 Archived-At: --_000_DU2PR02MB101098E947A3DB1D87E4EC7A296872DU2PR02MB10109eu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable > > > > > > IMO, licenses are to restrict the usage, not arbitrary technical > > > > limitations. We could similary have a token declaration in FFI inte= rface, > > > > when > > > > loading a library, no? > > > > > > You are basically reiterating what Nicolas already said, and I > > > answered that. I see no reason to repeat my answers to these > > > arguments, they are still the same. > > > > Just justifiying the statement that Emacs could have something similar = as > > GPL_COMPATIBLE token in Lisp API, just as there is in module API. I don= 't > > think I have seen anyone suggesting that before, at least not what I k= now of. > > Then I don't understand what you are suggesting here. What do you > mean by "token in Lisp API"? And if you mean something like what we I suggest exactly that, as it sounds: a compliance token, as you have for m= odules. Something like: (defun load-library (library-name, gpl-compatible) "Load a shared object with LIBRARY-NAME. The module has to be compatible wi= th GPL licence, for the deatils see https://www.gnu.org/licenses/gpl-3.0.txt, = which is asserted by GPL-COMPATIBLE been non-nil." ( ... )) > have in the emacs-module machinery, then how is that different from > what we already have? It is not different at all; it is the main point and *the* reason why I sug= gest it. Being in Lisp is obviously a lesser obstacle than being in C, but the s= ame argument(s) you presented for module code, and possible violations of the licence, are valid for the Lisp code as well. By the way I was about to snitch FFI from SXEmacs, but I see now Tromeys co= de is in Emacs, thanks to Madhus mail. I know Tom made one, but I didn't know it = was still included in git sources. > > > > You are preventing people not familiar with programming who can't = write a > > > > simple C wrapper to load a proprietary library, but it ain't stop a= ny > > > > malicious > > > > company anyway. > > > > > > People not familiar with programming will be unable to use FFI for > > > anything serious anyway. > > > > I think you have misunderstand what I am saying: my point was that this > > situation is making it more tedious for experienced people to work with= Emacs > > while possibly being a stopper *only* for unexperienced programmers. > > I understood what you were saying, and I disagree with your assertion > that using emacs-module API makes the job harder. If anything, it > makes it somewhat easier: it gives the module a way to call Emacs > primitives and provides various auxiliary facilities, without which > adjusting a library to Emacs would be a very hard job, because Emacs > has its own object system that is quite different from that of any > programming language that can be used to code a module. I suggest to > re-read the "Writing Dynamically-Loaded Modules" appendix to the ELisp > manual and imagine how hard it would be to use FFI without many of > those facilities provided specifically for dynamic modules. I understand what you say as well. From my relatively short experience of writing some FFI code for SBCL, I find it easier to work iteratively from a= Lisp repl, than writing a C code, recompile, reload and test. I can just C-x C-e= in Slime and have my glue code re-evaled and re-installed, instead of recompil= ing and reloading everything. > > However, pragmatically, I don't think putting an obstacle is the best w= ay. On > > the > > contrary, I think, making it as easy as possible to use and extend Ema= cs for > > whichever purpose, gives a positive incentive for people to see qualit= ies in > > Emacs > > in particular, and implicitly in GNU project. > > > > As with every important decision, there is a balance between the gain a= nd the > > damage. In this case, considering how easy is to work with C libraries = in some > > other Lisp implementations, and how faster it would be to write extensi= ons and > > experiment with Emacs, the gain is perhaps bigger than the possible los= s. > > There are examples out there what happens when projects take the route > you suggest. I cannot say I'm encouraged with LLVM being used for > proprietary closed-code extensions and targets, nor am I overwhelmed > with the popularity of Guile, which does have unrestricted FFI. So I > see no reason to believe what you suggest will bring such significant > advantages to Emacs that we should seriously consider lifting the > GPL-compatibility restriction. I don't say you should allow unrestricted access. Just to furntiture around= a bit, and allow access to a library from Lisp, not just from C. > > > > There is a plethora of MIT licensed math libraries, with big API su= rfaces, > > > > well optimized for many architectures which users could bring into = Emacs > > > > themselves. > > > > > > AFAIU, there should be no reasons not to be able to load MIT licensed > > > libraries via the emacs-module machinery. > > > > Nobody is stopping anyone to implement anything as modules, outside of = Emacs > > core, and I didn't say anything against modules either. I just say FFI = would > > make it easier and faster. > > See above: I think you are mistaken in your assessment. I don't see > any reasons why using FFI would be so much easier than using the > existing modules API. Given some simple boilerplate (like the DEFUN > macro you can see in test/src/emacs-module-resources/mod-test.c, and > other utility functions there), the job becomes quite simple. By > contrast, using FFI the programmer would need to figure all this out > on their own, and that requires very good familiarity with Emacs > internals. > > > > For the record: there are other GNU projects that use the same > > > "restrictions" on plugins: Gawk, GNU Make, and GCC, to mention those = I > > > know about. So it isn't like Emacs is alone in this. > > > > I am not sure if it is relevant with non-lisp projects. > > How so? GNU Make has its own scripting language, and Gawk implements Actually two scripting lanugages. More recent one is Guile. I have never fe= lt need to extend make, so no idea if Guile FFI is usable from make. > the Awk programming language, which is a full-fledged specialized > language for data processing. How is this "not relevant" to our case? I didn't thought it is relevant since neither make, nor awk, nor gcc are us= ed interactively to write Emacs applications, to solve computational problems = and similar in the way Emacs is used. > Gawk even comes with a small set of extensions written using the > plug-in protocol, and I invite you to review them to see how useful > they are, even though they are relatively small and do simple tasks. I have basically stopped using Bash & Co (tr, awk, sed etc) for any scripti= ng. I don't think those are bad, but as you have seen from my numerous posts on R= eddit promoting Elisp and debugging in Elisp, I do honestly find scripting in Lis= p and Emacs much more enjoyable than debugging shell scripts. I don't think it is= even comparable. Perhaps Emanuel could write an awk implementation in Elisp, that works on E= macs buffers, as a (one of) higher level interface. It would be probably more us= eful and enjoybale for text processing than extending awk with C++ modules or implementing some sort of DSL based on a bunch of cryptic acronyms. At leas= t awk is well documented. Just my opinion, I hope Emanuel is not angry on me for reflecting and suggesting it. By the way, a regression (sorry), perhaps someone more literally gifted tha= n me and better used to English language, could add a few words to Emacs manual = about how Emacs API works, regarding text processing and buffer management at least. Something like this: ___________________________________________________________________________= _____ Emacs Lisp uses a simple procedural model to manipulate windows, files, buf= fers, and other objects. In Emacs API, we typically manipulate a current object, whether it is a buf= fer, point, window, frame, etc. It means we act on a selected buffer, selected window, frame which we then manipulate by calling an appropriate function. = Which one we need is best looked up in the manual. The details on the calling arguments and returned results are easiset looked up via C-h f function-nam= e RET. Some of functions can take an optional object to act on, instead of acting = on the current-one, but the basic model of working with Emacs API buffer, wind= ow and frame management, as well as text processing, is to select something (m= ake it current), and than acting on it. ___________________________________________________________________________= _____ Something in that style, I don't see anything similar in introduction to th= e manual, and I think it might be useful to clarify it. At least I have under= stand Emacs API so. One can also speculate how API could look like, and for illustrative purpos= es, we can compare for example to similar API model like PostScript or OpenGL v= s Xlib or win32, or to iterator based design as suggested by Yuri. But I don'= t think it belongs to the manual or emacs-devel list. > > There are lots of > > projects with different needs, some are less, others are more self-cont= ained. > > It > > all depends how the software is used. Lisp(s), Emacs Lisp included, are > > usually > > in somewhat special situation of being relatively isolated islands when= it > > comes > > to available software. Common Lisp perhaps being a biggest exception th= ere. > > An easy to use FFI makes it easier to take advantage of already existin= g > > software, > > such as mathematical libraries for example. > > I think you greatly underestimate the complexity of using FFI-style > API to allow Emacs to use a mathematical library. For starters, > floating-point values have a very specific representation in Emacs, so > you cannot just take a value from the library and use it. Next, most > such libraries allocate memory for its data structures (matrices > etc.), so using them will need to provide GC glue for freeing the > resources when an instance of a data structure is no longer needed. > And that is just the tip of a very large iceberg. Agree. I haven't seen Tromeys code yet, but I have seen and tried FFI in SXEmacs, and as said in SBCL. SBCL does not use libffi under the hood (unlike more portable CFFI), and th= ey do give one access to raw pointers and raw memory allocated by C runtime, as w= ell as the responsibility to free that allocated memory. For the good and for t= he bad. Now, Emacs also has an interface to GCC, I wonder how hard/easy is to expos= e C runtime via native compiler to Lisp? I am not familiar with libgccjit, but = as I udnerstand eln files are shared objects under the hood? Does it not make th= e ffi implementation easier? Sorry if it is an uninformed question, I haven't loo= ked that up myself. > > Anyway, thanks for the arguments; I think it is useful to re-think the > > arguments, > > regardless on which side of the arguments I am. > > As a practical example, I suggest to take a library and integrate it > with Emacs using the modules API, if you haven't done that already. > It is sometimes amazing how many little details need to be figured out > and coded for that, when the target is Emacs Lisp. > I have done some FFI with SBCL, and yes, I agree there are lots of details = to take care of. Error checking via Lisp is not trivial, compared to simple if-statement in C. But still, if I f-up in SBCL with my glue, I can just re= write and C-x C-e in Slime and switch to repl window to test. As said above, I do= n't think it is comparable to writing C code. ________________________________ Fr=E5n: Eli Zaretskii Skickat: den 14 augusti 2024 07:10 Till: arthur miller Kopia: nicolas@n16f.net ; emacs-devel@gnu.org =C4mne: Re: as for Calc and the math library > From: arthur miller > CC: "nicolas@n16f.net" , "emacs-devel@gnu.org" > > Date: Tue, 13 Aug 2024 21:43:00 +0000 > > > > IMO, licenses are to restrict the usage, not arbitrary technical > > > limitations. We could similary have a token declaration in FFI interf= ace, when > > > loading a library, no? > > > > You are basically reiterating what Nicolas already said, and I > > answered that. I see no reason to repeat my answers to these > > arguments, they are still the same. > > Just justifiying the statement that Emacs could have something similar as > GPL_COMPATIBLE token in Lisp API, just as there is in module API. I don't > think I have seen anyone suggesting that before, at least not what I kno= w of. Then I don't understand what you are suggesting here. What do you mean by "token in Lisp API"? And if you mean something like what we have in the emacs-module machinery, then how is that different from what we already have? > > > You are preventing people not familiar with programming who can't wr= ite a > > > simple C wrapper to load a proprietary library, but it ain't stop any > > > malicious > > > company anyway. > > > > People not familiar with programming will be unable to use FFI for > > anything serious anyway. > > I think you have misunderstand what I am saying: my point was that this > situation is making it more tedious for experienced people to work with E= macs > while possibly being a stopper *only* for unexperienced programmers. I understood what you were saying, and I disagree with your assertion that using emacs-module API makes the job harder. If anything, it makes it somewhat easier: it gives the module a way to call Emacs primitives and provides various auxiliary facilities, without which adjusting a library to Emacs would be a very hard job, because Emacs has its own object system that is quite different from that of any programming language that can be used to code a module. I suggest to re-read the "Writing Dynamically-Loaded Modules" appendix to the ELisp manual and imagine how hard it would be to use FFI without many of those facilities provided specifically for dynamic modules. > However, pragmatically, I don't think putting an obstacle is the best way= . On the > contrary, I think, making it as easy as possible to use and extend Emacs= for > whichever purpose, gives a positive incentive for people to see qualitie= s in Emacs > in particular, and implicitly in GNU project. > > As with every important decision, there is a balance between the gain and= the > damage. In this case, considering how easy is to work with C libraries in= some > other Lisp implementations, and how faster it would be to write extension= s and > experiment with Emacs, the gain is perhaps bigger than the possible loss. There are examples out there what happens when projects take the route you suggest. I cannot say I'm encouraged with LLVM being used for proprietary closed-code extensions and targets, nor am I overwhelmed with the popularity of Guile, which does have unrestricted FFI. So I see no reason to believe what you suggest will bring such significant advantages to Emacs that we should seriously consider lifting the GPL-compatibility restriction. > > > There is a plethora of MIT licensed math libraries, with big API surf= aces, > > > well optimized for many architectures which users could bring into Em= acs > > > themselves. > > > > AFAIU, there should be no reasons not to be able to load MIT licensed > > libraries via the emacs-module machinery. > > Nobody is stopping anyone to implement anything as modules, outside of Em= acs > core, and I didn't say anything against modules either. I just say FFI wo= uld > make it easier and faster. See above: I think you are mistaken in your assessment. I don't see any reasons why using FFI would be so much easier than using the existing modules API. Given some simple boilerplate (like the DEFUN macro you can see in test/src/emacs-module-resources/mod-test.c, and other utility functions there), the job becomes quite simple. By contrast, using FFI the programmer would need to figure all this out on their own, and that requires very good familiarity with Emacs internals. > > For the record: there are other GNU projects that use the same > > "restrictions" on plugins: Gawk, GNU Make, and GCC, to mention those I > > know about. So it isn't like Emacs is alone in this. > > I am not sure if it is relevant with non-lisp projects. How so? GNU Make has its own scripting language, and Gawk implements the Awk programming language, which is a full-fledged specialized language for data processing. How is this "not relevant" to our case? Gawk even comes with a small set of extensions written using the plug-in protocol, and I invite you to review them to see how useful they are, even though they are relatively small and do simple tasks. > There are lots of > projects with different needs, some are less, others are more self-contai= ned. It > all depends how the software is used. Lisp(s), Emacs Lisp included, are u= sually > in somewhat special situation of being relatively isolated islands when i= t comes > to available software. Common Lisp perhaps being a biggest exception ther= e. > An easy to use FFI makes it easier to take advantage of already existing = software, > such as mathematical libraries for example. I think you greatly underestimate the complexity of using FFI-style API to allow Emacs to use a mathematical library. For starters, floating-point values have a very specific representation in Emacs, so you cannot just take a value from the library and use it. Next, most such libraries allocate memory for its data structures (matrices etc.), so using them will need to provide GC glue for freeing the resources when an instance of a data structure is no longer needed. And that is just the tip of a very large iceberg. > Anyway, thanks for the arguments; I think it is useful to re-think the ar= guments, > regardless on which side of the arguments I am. As a practical example, I suggest to take a library and integrate it with Emacs using the modules API, if you haven't done that already. It is sometimes amazing how many little details need to be figured out and coded for that, when the target is Emacs Lisp. --_000_DU2PR02MB101098E947A3DB1D87E4EC7A296872DU2PR02MB10109eu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
> >
> > > > IMO, licenses are to restrict the usage, not arbitrary = technical
> > > > limitations. We could similary have a token declaration= in FFI interface,
> > > > when
> > > > loading a library, no?
> > >
> > > You are basically reiterating what Nicolas already said, and= I
> > > answered that.  I see no reason to repeat my answers to= these
> > > arguments, they are still the same.
> >
> > Just justifiying the statement that Emacs could have something si= milar as
> > GPL_COMPATIBLE token in Lisp API, just as there is in module API.= I don't
> >  think I have seen anyone suggesting that before, at least n= ot what I know of.
>
> Then I don't understand what you are suggesting here.  What do yo= u
> mean by "token in Lisp API"?  And if you mean something= like what we

I suggest exactly that, as it sounds: a compliance token, as you have for m= odules.

Something like:

(defun load-library (library-name, gpl-compatible)
"Load a shared object with LIBRARY-NAME. The module has to be compatib= le with
GPL licence, for the deatils see https://www.gnu.org/licenses/gpl-3.0.txt, = which
is asserted by GPL-COMPATIBLE been non-nil."
 ( ... ))

> have in the emacs-module machinery, then how is that different from
> what we already have?

It is not different at all; it is the main point and *the* reason why I sug= gest
it. Being in Lisp is obviously a lesser obstacle than being in C, but the s= ame
argument(s) you presented for module code, and possible violations of the
licence, are valid for the Lisp code as well.

By the way I was about to snitch FFI from SXEmacs, but I see now Tromeys co= de is
in Emacs, thanks to Madhus mail. I know Tom made one, but I didn't know it = was
still included in git sources.

> > > > You are preventing  people not familiar with progr= amming who can't write a
> > > > simple C wrapper to load a proprietary library, but it = ain't stop any
> > > > malicious
> > > > company anyway.
> > >
> > > People not familiar with programming will be unable to use F= FI for
> > > anything serious anyway.
> >
> > I think you have misunderstand what I am saying: my point was tha= t this
> > situation is making it more tedious for experienced people to wor= k with Emacs
> > while possibly being a stopper *only* for unexperienced programme= rs.
>
> I understood what you were saying, and I disagree with your assertion<= /div>
> that using emacs-module API makes the job harder.  If anything, i= t
> makes it somewhat easier: it gives the module a way to call Emacs
> primitives and provides various auxiliary facilities, without which
> adjusting a library to Emacs would be a very hard job, because Emacs
> has its own object system that is quite different from that of any
> programming language that can be used to code a module.  I sugges= t to
> re-read the "Writing Dynamically-Loaded Modules" appendix to= the ELisp
> manual and imagine how hard it would be to use FFI without many of
> those facilities provided specifically for dynamic modules.

I understand what you say as well. From my relatively short experience of
writing some FFI code for SBCL, I find it easier to work iteratively from a= Lisp
repl, than writing a C code, recompile, reload and test. I can just C-x C-e= in
Slime and have my glue code re-evaled and re-installed, instead of recompil= ing
and reloading everything.

> > However, pragmatically, I don't think putting an obstacle is the = best way. On
> > the
> >  contrary, I think, making it as easy as possible to use and= extend Emacs for
> >  whichever purpose, gives a positive incentive for people to= see qualities in
> > Emacs
> >  in particular, and implicitly in GNU project.
> >
> > As with every important decision, there is a balance between the = gain and the
> > damage. In this case, considering how easy is to work with C libr= aries in some
> > other Lisp implementations, and how faster it would be to write e= xtensions and
> > experiment with Emacs, the gain is perhaps bigger than the possib= le loss.
>
> There are examples out there what happens when projects take the route=
> you suggest.  I cannot say I'm encouraged with LLVM being used fo= r
> proprietary closed-code extensions and targets, nor am I overwhelmed
> with the popularity of Guile, which does have unrestricted FFI.  = So I
> see no reason to believe what you suggest will bring such significant<= /div>
> advantages to Emacs that we should seriously consider lifting the
> GPL-compatibility restriction.

I don't say you should allow unrestricted access. Just to furntiture around= a
bit, and allow access to a library from Lisp, not just from C.

> > > > There is a plethora of MIT licensed math libraries, wit= h big API surfaces,
> > > > well optimized for many architectures which users could= bring into Emacs
> > > >  themselves.
> > >
> > > AFAIU, there should be no reasons not to be able to load MIT= licensed
> > > libraries via the emacs-module machinery.
> >
> > Nobody is stopping anyone to implement anything as modules, outsi= de of Emacs
> > core, and I didn't say anything against modules either. I just sa= y FFI would
> > make it easier and faster.
>
> See above: I think you are mistaken in your assessment.  I don't = see
> any reasons why using FFI would be so much easier than using the
> existing modules API.  Given some simple boilerplate (like the DE= FUN
> macro you can see in test/src/emacs-module-resources/mod-test.c, and
> other utility functions there), the job becomes quite simple.  By=
> contrast, using FFI the programmer would need to figure all this out
> on their own, and that requires very good familiarity with Emacs
> internals.
>
> > > For the record: there are other GNU projects that use the sa= me
> > > "restrictions" on plugins: Gawk, GNU Make, and GCC= , to mention those I
> > > know about.  So it isn't like Emacs is alone in this.
> >
> > I am not sure if it is relevant with non-lisp projects.
>
> How so?  GNU Make has its own scripting language, and Gawk implem= ents

Actually two scripting lanugages. More recent one is Guile. I have never fe= lt
need to extend make, so no idea if Guile FFI is usable from make.

> the Awk programming language, which is a full-fledged specialized
> language for data processing.  How is this "not relevant&quo= t; to our case?

I didn't thought it is relevant since neither make, nor awk, nor gcc are us= ed
interactively to write Emacs applications, to solve computational problems = and
similar in the way Emacs is used.

> Gawk even comes with a small set of extensions written using the
> plug-in protocol, and I invite you to review them to see how useful
> they are, even though they are relatively small and do simple tasks.

I have basically stopped using Bash & Co (tr, awk, sed etc) for any scr= ipting. I
don't think those are bad, but as you have seen from my numerous posts on R= eddit
promoting Elisp and debugging in Elisp, I do honestly find scripting in Lis= p and
Emacs much more enjoyable than debugging shell scripts. I don't think it is= even
comparable.

Perhaps Emanuel could write an awk implementation in Elisp, that works on E= macs
buffers, as a (one of) higher level interface. It would be probably more us= eful
and enjoybale for text processing than extending awk with C++ modules or
implementing some sort of DSL based on a bunch of cryptic acronyms. At leas= t awk
is well documented. Just my opinion, I hope Emanuel is not angry on me for<= /div>
reflecting and suggesting it.

By the way, a regression (sorry), perhaps someone more literally gifted tha= n me
and better used to English language, could add a few words to Emacs manual = about
how Emacs API works, regarding text processing and buffer management at
least. Something like this:
___________________________________________________________________________= _____
Emacs Lisp uses a simple procedural model to manipulate windows, files, buf= fers,
and other objects.

In Emacs API, we typically manipulate a current object, whether it is a buf= fer,
point, window, frame, etc. It means we act on a selected buffer, selected
window, frame which we then manipulate by calling an appropriate function. = Which
one we need is best looked up in the manual. The details on the calling
arguments and returned results are easiset looked up via C-h f function-nam= e
RET.

Some of functions can take an optional object to act on, instead of acting = on
the current-one, but the basic model of working with Emacs API buffer, wind= ow
and frame management, as well as text processing, is to select something (m= ake
it current), and than acting on it.
___________________________________________________________________________= _____

Something in that style, I don't see anything similar in introduction to th= e
manual, and I think it might be useful to clarify it. At least I have under= stand
Emacs API so.

One can also speculate how API could look like, and for illustrative purpos= es,
we can compare for example to similar API model like PostScript or OpenGL v= s
Xlib or win32, or to iterator based design as suggested by Yuri. But I don'= t
think it belongs to the manual or emacs-devel list.

> > There are lots of
> > projects with different needs, some are less, others are more sel= f-contained.
> > It
> > all depends how the software is used. Lisp(s), Emacs Lisp include= d, are
> > usually
> > in somewhat special situation of being relatively isolated island= s when it
> > comes
> > to available software. Common Lisp perhaps being a biggest except= ion there.
> > An easy to use FFI makes it easier to take advantage of already e= xisting
> > software,
> > such as mathematical libraries for example.
>
> I think you greatly underestimate the complexity of using FFI-style
> API to allow Emacs to use a mathematical library.  For starters,<= /div>
> floating-point values have a very specific representation in Emacs, so=
> you cannot just take a value from the library and use it.  Next, = most
> such libraries allocate memory for its data structures (matrices
> etc.), so using them will need to provide GC glue for freeing the
> resources when an instance of a data structure is no longer needed.
> And that is just the tip of a very large iceberg.

Agree. I haven't seen Tromeys code yet, but I have seen and tried FFI in
SXEmacs, and as said in SBCL.

SBCL does not use libffi under the hood (unlike more portable CFFI), and th= ey do
give one access to raw pointers and raw memory allocated by C runtime, as w= ell
as the responsibility to free that allocated memory. For the good and for t= he
bad.

Now, Emacs also has an interface to GCC, I wonder how hard/easy is to expos= e C
runtime via native compiler to Lisp? I am not familiar with libgccjit, but = as I
udnerstand eln files are shared objects under the hood? Does it not make th= e ffi
implementation easier? Sorry if it is an uninformed question, I haven't loo= ked
that up myself.

> > Anyway, thanks for the arguments; I think it is useful to re-thin= k the
> > arguments,
> > regardless on which side of the arguments I am.
>
> As a practical example, I suggest to take a library and integrate it
> with Emacs using the modules API, if you haven't done that already.
> It is sometimes amazing how many little details need to be figured out=
> and coded for that, when the target is Emacs Lisp.
>

I have done some FFI with SBCL, and yes, I agree there are lots of details = to
take care of. Error checking via Lisp is not trivial, compared to simple
if-statement in C. But still, if I f-up in SBCL with my glue, I can just re= write
and C-x C-e in Slime and switch to repl window to test. As said above, I do= n't
think it is comparable to writing C code.

Fr=E5n: Eli Zaretskii <e= liz@gnu.org>
Skickat: den 14 augusti 2024 07:10
Till: arthur miller <arthur.miller@live.com>
Kopia: nicolas@n16f.net <nicolas@n16f.net>; emacs-devel@gnu.or= g <emacs-devel@gnu.org>
=C4mne: Re: as for Calc and the math library
 
> From: arthur miller <arthur.miller@live.co= m>
> CC: "nicolas@n16f.net" <nicolas@n16f.net>, "emacs= -devel@gnu.org"
>        <emacs-devel@gnu.org><= br> > Date: Tue, 13 Aug 2024 21:43:00 +0000
>
> > > IMO, licenses are to restrict the usage, not arbitrary techn= ical
> > > limitations. We could similary have a token declaration in F= FI interface, when
> > > loading a library, no?
> >
> > You are basically reiterating what Nicolas already said, and I > > answered that.  I see no reason to repeat my answers to thes= e
> > arguments, they are still the same.
>
> Just justifiying the statement that Emacs could have something similar= as
> GPL_COMPATIBLE token in Lisp API, just as there is in module API. I do= n't
>  think I have seen anyone suggesting that before, at least not wh= at I know of.

Then I don't understand what you are suggesting here.  What do you
mean by "token in Lisp API"?  And if you mean something like= what we
have in the emacs-module machinery, then how is that different from
what we already have?

> > > You are preventing  people not familiar with programmin= g who can't write a
> > > simple C wrapper to load a proprietary library, but it ain't= stop any
> > > malicious
> > > company anyway.
> >
> > People not familiar with programming will be unable to use FFI fo= r
> > anything serious anyway.
>
> I think you have misunderstand what I am saying: my point was that thi= s
> situation is making it more tedious for experienced people to work wit= h Emacs
> while possibly being a stopper *only* for unexperienced programmers.
I understood what you were saying, and I disagree with your assertion
that using emacs-module API makes the job harder.  If anything, it
makes it somewhat easier: it gives the module a way to call Emacs
primitives and provides various auxiliary facilities, without which
adjusting a library to Emacs would be a very hard job, because Emacs
has its own object system that is quite different from that of any
programming language that can be used to code a module.  I suggest to<= br> re-read the "Writing Dynamically-Loaded Modules" appendix to the = ELisp
manual and imagine how hard it would be to use FFI without many of
those facilities provided specifically for dynamic modules.

> However, pragmatically, I don't think putting an obstacle is the best = way. On the
>  contrary, I think, making it as easy as possible to use and exte= nd Emacs for
>  whichever purpose, gives a positive incentive for people to see = qualities in Emacs
>  in particular, and implicitly in GNU project.
>
> As with every important decision, there is a balance between the gain = and the
> damage. In this case, considering how easy is to work with C libraries= in some
> other Lisp implementations, and how faster it would be to write extens= ions and
> experiment with Emacs, the gain is perhaps bigger than the possible lo= ss.

There are examples out there what happens when projects take the route
you suggest.  I cannot say I'm encouraged with LLVM being used for
proprietary closed-code extensions and targets, nor am I overwhelmed
with the popularity of Guile, which does have unrestricted FFI.  So I<= br> see no reason to believe what you suggest will bring such significant
advantages to Emacs that we should seriously consider lifting the
GPL-compatibility restriction.

> > > There is a plethora of MIT licensed math libraries, with big= API surfaces,
> > > well optimized for many architectures which users could brin= g into Emacs
> > >  themselves.
> >
> > AFAIU, there should be no reasons not to be able to load MIT lice= nsed
> > libraries via the emacs-module machinery.
>
> Nobody is stopping anyone to implement anything as modules, outside of= Emacs
> core, and I didn't say anything against modules either. I just say FFI= would
> make it easier and faster.

See above: I think you are mistaken in your assessment.  I don't see any reasons why using FFI would be so much easier than using the
existing modules API.  Given some simple boilerplate (like the DEFUN macro you can see in test/src/emacs-module-resources/mod-test.c, and
other utility functions there), the job becomes quite simple.  By
contrast, using FFI the programmer would need to figure all this out
on their own, and that requires very good familiarity with Emacs
internals.

> > For the record: there are other GNU projects that use the same > > "restrictions" on plugins: Gawk, GNU Make, and GCC, to = mention those I
> > know about.  So it isn't like Emacs is alone in this.
>
> I am not sure if it is relevant with non-lisp projects.

How so?  GNU Make has its own scripting language, and Gawk implements<= br> the Awk programming language, which is a full-fledged specialized
language for data processing.  How is this "not relevant" to= our case?
Gawk even comes with a small set of extensions written using the
plug-in protocol, and I invite you to review them to see how useful
they are, even though they are relatively small and do simple tasks.

> There are lots of
> projects with different needs, some are less, others are more self-con= tained. It
> all depends how the software is used. Lisp(s), Emacs Lisp included, ar= e usually
> in somewhat special situation of being relatively isolated islands whe= n it comes
> to available software. Common Lisp perhaps being a biggest exception t= here.
> An easy to use FFI makes it easier to take advantage of already existi= ng software,
> such as mathematical libraries for example.

I think you greatly underestimate the complexity of using FFI-style
API to allow Emacs to use a mathematical library.  For starters,
floating-point values have a very specific representation in Emacs, so
you cannot just take a value from the library and use it.  Next, most<= br> such libraries allocate memory for its data structures (matrices
etc.), so using them will need to provide GC glue for freeing the
resources when an instance of a data structure is no longer needed.
And that is just the tip of a very large iceberg.

> Anyway, thanks for the arguments; I think it is useful to re-think the= arguments,
> regardless on which side of the arguments I am.

As a practical example, I suggest to take a library and integrate it
with Emacs using the modules API, if you haven't done that already.
It is sometimes amazing how many little details need to be figured out
and coded for that, when the target is Emacs Lisp.
--_000_DU2PR02MB101098E947A3DB1D87E4EC7A296872DU2PR02MB10109eu_--