From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Arthur Miller Newsgroups: gmane.emacs.tangents Subject: Re: Shrinking the C core Date: Thu, 14 Sep 2023 13:35:06 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="30616"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-tangents@gnu.org To: Gerd =?utf-8?Q?M=C3=B6llmann?= Original-X-From: emacs-tangents-bounces+get-emacs-tangents=m.gmane-mx.org@gnu.org Thu Sep 14 13:40:39 2023 Return-path: Envelope-to: get-emacs-tangents@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qgki5-0007jh-Si for get-emacs-tangents@m.gmane-mx.org; Thu, 14 Sep 2023 13:40:38 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qgkhq-0001rp-M9; Thu, 14 Sep 2023 07:40:22 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgkhm-0001rY-Dr for emacs-tangents@gnu.org; Thu, 14 Sep 2023 07:40:18 -0400 Original-Received: from mail-vi1eur05olkn2080c.outbound.protection.outlook.com ([2a01:111:f400:7d00::80c] helo=EUR05-VI1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgkhj-0001wm-0h for emacs-tangents@gnu.org; Thu, 14 Sep 2023 07:40:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z3d7YHoenm6bFpoa/3UJehQyQi6eyq3Q+nNh3FG7cqIVfOYb5De645Ntygx9CjKupSbw6iuFpzaVpsQ/ZuQugwxpQqUKCiY1YujrlrpdIWS75R1OZH5vQimSK54qcVzGx9pJvbpNt50STB3dN3zGidpaKV4vzcTCgKvNc50mhXO9Q+v4MuTJH21jLUS751p7q2//kT0X2gXpHDoygAh3haPHxyT/PMMMcXr2ha+q0ww5Dv2uMB1uYyb82dilHWGWmLxJozNTWLZxnlC8J5meeRz6JO9Mis0cwdbyeH/xOJ0kqQCXPap1AZdx+FLhY4vFGzHlYwt7rPavtj1MpcLBHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Mp1TlqfKPnzc2hwjLoQfTBXatNr8DGqTNrGVtyDMrj4=; b=ZyS8GehhJfKMfAHZkAn27MsETJCXXZ8Dl2tRlqHXuM8f8/Ct+K7cnlk+juSsMKSNn4f7fKGh55EQWT+2/ujNCLIHci+EmmYDE/iusDTtgRmnRSnGarz+cf6pmg4bhParxKzhvyGL6bFwcGw48VlQ5hommmAQCpLK/7UEnxIJRCpqQSjz2Q7ElLrRYX4JjieIGPgp6R9quIAqU/7gHXIkK6Ci0418lZRVe/IgfWUk5cRYu+OaKijEcrINVApM3TeC9TP0yDdyi+/mcJTYSBksbjN56zi/8jqTa5X5MflX9q3YUoQsBi7CUshPpjKyoWRCWO/Y9BTDiDVc+ioFg8Mkzw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mp1TlqfKPnzc2hwjLoQfTBXatNr8DGqTNrGVtyDMrj4=; b=lUXl7eNpKRwWzOXSc5LG/zrrUYWnUqhr61uOhH320wbG6ZpQrfGWQmainobyfnxi0jv3D4Tsc6OfqsJn+HB2oIgqGwg4UvW3vwQxVC6Xnc67jn07EE2AHTYv/m/6LlXxbj0CzFNW+YGTvgvB5WvIVTm+mU4XMuXyqihzmfwQG+Jm/1IzoIBABl8xPYXskUctKTwzpYA0FM/YxXE/i9dqooHlS62IsfkmQwrWfb4RXr8zjCZjwSW5GOrG2c4BHawUPCzu3y/hLbRrxLtzhPduIEVmrsd0MVEi7vEucJjE3gue8Db/MuI7qjzGTPXDi49dsZ/pf3dFeiVduQvkW0i/1A== Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) by AM9PR09MB4721.eurprd09.prod.outlook.com (2603:10a6:20b:283::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20; Thu, 14 Sep 2023 11:35:11 +0000 Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::df56:b1b1:64b1:6122]) by AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::df56:b1b1:64b1:6122%7]) with mapi id 15.20.6792.020; Thu, 14 Sep 2023 11:35:11 +0000 In-Reply-To: ("Gerd =?utf-8?Q?M=C3=B6llmann?= =?utf-8?Q?=22's?= message of "Wed, 13 Sep 2023 08:33:43 +0200") X-TMN: [rRKXUBdZwc4TfWyAoDJxSvXPoAkl5g7q00llPn/BuOs=] X-ClientProxiedBy: MM0P280CA0113.SWEP280.PROD.OUTLOOK.COM (2603:10a6:190:9::19) To AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) X-Microsoft-Original-Message-ID: <878r990yz9.fsf@live.com> X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM9PR09MB4977:EE_|AM9PR09MB4721:EE_ X-MS-Office365-Filtering-Correlation-Id: e2735e85-3581-41e6-c244-08dbb516a6bb X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qnoVzFnbsI5T2Xbp/kiCWjc446gCB5c7Po/Pi+e0sqrXYmHCYX8cB+9EHUzxrrHyPlnBNwD8tCIaFnbHA0Wka0NsHtNSrMiMbChRfc3rZydmd0FV8u4DmEbiPQnGA4kHlrrStmzDBoU0KxAsFY0PXvGgvYL68ioiB59jBqemSxz2bgD9btg1fRZ2lhOpJsMdsyrBTure0GqwmWM7X2AeEPX65rvHtbCtEKC8+RT2WT5MV/0TzXF/2hvTiYK04VQjXZce/WdDMkpSGziu3AfH4cIi6kChqeSw8vXh+P7fEBMrKfNc9gv2zSY99v450WCdNkBvz+x4p9PItE23s9CgPvLWGq1muWMceXFEc+M0XWLYPVAvZVapqNGClKb6wwNR0Ut+4BpIyFyncPm+O7MeljaL1G5NBrMyDkv7kAr8ur00b4ETkJsnv4rafcLoBQ0Kvv53xRqqnwGiKCBVI9WBefuCpmKozmLqqPwTZ8wxIeNto1nuypWqbCRIs1XnYnLT5tbQNtNGP1smhSxM6AP4aeOARfBlhselTGhAIf1GAq07Jfhpk18yyCaUvP4mpnMo+2P/jjPreylYmMBbRWm4pmM415MsTddwCgLZX/llzRA= X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Vmdnc2lmN0JRbFlKcktIK2NIcmlXdlR4bzB6KzM2YUNvakNhUXFTaVpNSjdp?= =?utf-8?B?ZmVZekc1WFNWRHBwZWhqWlpxRmZLWk5HVWQxQnh2aWV2YmM3N242Q29RcFVH?= =?utf-8?B?ZjhNdzhwVmwwbEt1QjhlalpiNmQwUXFlRlFBeGdUUytONzRseER4aENvdGpI?= =?utf-8?B?VmcxandWclYzOXk5dFVESFBnT2xLR09jbUlHcEdoRHlsRUdnNzhJZmwxSGRX?= =?utf-8?B?SHZQdEo4MFVKWWxHeUVSTWVwMCtFTXpGY1Mzb0plaUxwdjJHRFdFS0p3NnVQ?= =?utf-8?B?Zjd3UVZVOEQydEdCQjFkbkZNQmUyMXI4TW9FdGZOa2Z4cmFYaHoyMkducStl?= =?utf-8?B?U1AzT2tJQWNzRUxoOUxYSXlzeUE1ZHY0QzNvdDAzSDRzMUZSZXQ0VnVSQW1l?= =?utf-8?B?ZkpxOWxudkZKOTAvRitQTy9PWUt3QlRFR3B0MkxhWU1LcFlNb0pyZDV0SHN6?= =?utf-8?B?NGh3eFNFVTgyckdVZEkrcmJhZnNld0owcEl3b0hOaXRKb0RrQjZuSmFkNncz?= =?utf-8?B?d1pheWhEV05QWHFoODhBL05jdTFiYWprZmlCWWdhaEtKMk9QRVZNdFl3L1ZE?= =?utf-8?B?d1VKZUgxTmdUVWZXOTZYUFhSNzE5VmZUZXBCeFUvdHR2aU9uMEZFNGVuelFs?= =?utf-8?B?Y1N5TWhkMi9teG83SEQ3UjFXYWE5VnhhQWpjQT X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-64da6.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: e2735e85-3581-41e6-c244-08dbb516a6bb X-MS-Exchange-CrossTenant-AuthSource: AM9PR09MB4977.eurprd09.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2023 11:35:10.9501 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR09MB4721 Received-SPF: pass client-ip=2a01:111:f400:7d00::80c; envelope-from=arthur.miller@live.com; helo=EUR05-VI1-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-tangents@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Emacs news and miscellaneous discussions outside the scope of other Emacs mailing lists List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-tangents-bounces+get-emacs-tangents=m.gmane-mx.org@gnu.org Original-Sender: emacs-tangents-bounces+get-emacs-tangents=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.tangents:1082 Archived-At: Gerd M=C3=B6llmann writes: > Arthur Miller writes: > >>>> IMNSHO, discussing a rewrite of Emacs in _any_ language is waste of >>>> time and energy. We've seen this many times (because people still >>>> insist on bringing this up from time to time). From where I stand, >>>> the main reason is not even the fact that we decided not to do that, >>>> but the fact that such a rewrite will never happen in practice. Such >>>> a rewrite is a massive job which requires very good knowledge of Emacs >>>> internals and features, and a lot of time. People who come close to >>>> the required knowledge level are not interested in doing this job >>>> (because they understand the futility), and those who think it should >>>> be done simply don't know enough and/or don't have enough time on >>>> their hands to pull it through. >>>> >>>> If Emacs will ever be "rewritten", it will not be Emacs, but a >>>> text-processing system with a very different architecture and design, >>>> which will take from the Emacs experience the lessons we learned and >>>> implement them differently, to produce a system whose starting point >>>> is closer to the needs of today's users and whose main technologies >>>> are more modern from the get-go. >>> >>>I couldn't agree more. >>> >>>To me, a rewrite is quatsch, while adding CL facilities to Emacs makes a >>>lot of sense. >> >> I use to say often: either CL will come to Emacs or Emacs to CL, whichev= er >> way around. We need some of features available on CL platforms, sbcl >> notably: built-in concurrency and better garbage collectors from the get= -go; and >> some of the CL language features, namespaces notably, would be very nice= to >> have. > > I agree. Alas, others, who haven't seem the light yet, don't :-). > >> I am not sure which one is easier to achieve, porting elisp to cl, or >> rewriting core to have all those features. > > I don't know either, of course. I guess it depends on the feature. Some > random thoughts: > > I'm pretty sure that CL packages could be added to Emacs as it is, if > some people would work on it. With "CL packages" you mean namespaces? There seem to be already a branch that implements them, but I don't know how well it works, I haven't tried it. I think the reasoning is "with don't want them because we have done 40 years without". But IDK for sure. If there is some technical reason, then they could help the author to implement it the way they like it technically. > I'm also pretty sure that an incremental + generational GC could be > added, at least as an option, because I would have almost done it some > 20+ years ago. It was torpedoed by a patent issue concerning > mostly-copying GC. The patent has since expired. A lot of work, of > course. I think some people do or have done something in this area, but > I don't know details. I am not very familiar wth GC:s implementation more then just some bired-eye overview. SBCL recently started to move towards non-moving GC to help with the speed, notably when calling native code which does not like it's pointers moved underneath, but I am not expert on details there, these what I have got from the paper: https://applied-langua.ge/~hayley/swcl-gc.pdf > I'm not at all sure that non-cooperative multi-threading could be added > to Emacs. But I'm also not sure how a CL core would help here. They are exposing posix threads and have done some work to make at least parts of the Lisp system work well with threads, and it seems it is working well for many applicaitons. http://www.lichteblau.com/sbcl/doc/manual/sbcl/Implementation-_0028Linux-x8= 6_0029.html#Implementation-_0028Linux-x86_0029 I think there is also a missconception in Emacs community that Emacs loop itself has to be parallelized; I am not sure it is needed; I think for many people it would be enough to expose threading in form of "js workers" or something like that. It can be done with processes of course, but people seem to constantly scream about it in disucssions. CL has things like lparallel and green threads built on top of hardware threads, so even there is a bit of job already done. I am sure all that can be done in Emacs too, but I think, both communities would be more helped if we perhaps used sbcl and interested individuals helped make sbcl runtime better intead of reduplication the entire effort. > On the other hand, I'm pretty convinced that an Emacs core written in CL > would have to be close to 100% compatible with the existing C core to be > accepted by users. That includes a CL rewrite of the C Elisp, including > byte code interpreter. Yes, my conclusion too. > accepted by users. That includes a CL rewrite of the C Elisp, including > byte code interpreter. I am not sure how much of byte-interpretter is needed; I was thinking how byte interpretter and native compiler fitt there. Oviously since sbcl is a compiler, with don't need all that stuff, but I am not sure how much of byte code intepretter is needed. I am sure we need to understand all of the syntax, since byte code is a valid elisp, according to the manual; so the reader have to be able to read the syntax I guess, as printed representation, and has to print same stuff back to feed into elisp functions. > That's a massive endeavor. My hair stands up when I remember the > compatibility problems I faced with the new redisplay ages ago. > Multiply that by some factor > 1. But maybe that's a burnt child > dreading the fire :-). Yes, I know. I am fully aware that it is an impossibility for someone alone, even for a very few. I don't think it is a burnt child, since yes, the character renderer of Emacs has to be implemented if Emacs applications will run unchanged. Hopefully it will be possible to implement Emacs stuff as a special kind of terminal/character renderer over some sort of tree/graph structure. I think CLOS and CL have much better tools to refactor that stuff than C, but what do I know, I haven't tried that and I am not sure if I will tbh. I am bolling with the ideas. I have seen what they do in other similar CL software (Hemlock, McClIM, Lem), perhaps there is something that can be reused there, but I don't know how much and what yet. And, yes text properties are a special chapter on its own :). > I'm also not sure how a CL (not Elisp) program would look like using a > CL Emacs core. It would look exactly the same, that is the point :). If I wan't to be able to run elisp applications from elpa/melpa the elisp compatibility has to be 1:1 so to say. I think it is OK to have some "special places", like an empty eieio.el, and bunch of aliases or compiler macros for cl-* stuff, but in general it should be able to read an elisp file without mofications.=20 > CL Emacs core. Is it nice enough, so to speak? Think of Emacs strings, > which couldn't be CL strings because of text properties, buffer-local > variables... Yes, buffer locals have to be implemented in CL, thanks to namespaces we can hav a cl:string and el:string, of which el:string can be a struct with interval and a list of properties or some other I don't know. The hairy stuff with string properties is splitting of intervals and properties and so on. There is much more than just those. I don't know yet how efficient it would, I plan to test with generic functions. There are some people testing with static dispatch for generic functions, but I am not sure if that would work or not yet (works only for compile time strings), I haven't had time to experiment yet. > (Another ansatz might be to make Emacs C core a lib. I haven't given > that much thought, but it could be more promising than rewriting the > whole shit in CL :-).) Perhaps, but I don't think I personally care longer. Once you test and learn how nice it is to be able to just eval a function and test the change, I am not going back to hackig C again, at least not in my spare time. Life is too short for that. >> CFFI would also be nice to have so >> that users can extend Emacs themselves with other libraries and not have= to wait >> for the core devs to do it for them. That would also lessen the burden o= n >> maintaining that stuff in the core. > > FFI for Emacs once existed, I think Dave Love wrote one, for instance. > Don't know what became of that. Might be an issue with interfacing to > non-free libs. I didn't know it ever existed in Emacs.=20 I though it was a political decision to not have it in Emacs. Which I understand completely, but after seeing the recent software development, I believe is hurting "us" much more than "them". As I said in some mail in the context of keyword args, entire life is a struggle between choices to maximize the gain. In this case I believe our net loss is much bigger than gain. Interesting to note is that other GNU projects are not have political dilemmas about CFFI, for example GCL (Gnu Common Lisp) or GNU Guile. I am also aware of emacs-ffi via modules: https://github.com/tromey/emacs-ffi But nobody seems to be using it since modules are not big thing in Emacs community so people barey know about it. That can probably change if someone wrote a "kille app", or in other words an interesting enough package that uses it. Yet another hand, that I haven't seen anywhere, would to load sbcl via compiled module. Sbcl got ability to save itself as a loadable library, so it would be possible to basically expose Emacs runtime to sbcl and export some interface to work with it from within Emacs. I am not sure if that would be a big win over current way of working via external process and sockets, but an idea perhaps? I don't have time to try it myself.