From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.emacs.devel Subject: Re: Emacs rewrite in a maintainable language Date: Mon, 12 Oct 2015 12:20:22 +0200 Message-ID: <87d1wkbd7d.fsf@fencepost.gnu.org> References: <561A19AB.5060001@cumego.com> <87io6dl0h0.fsf@wanadoo.es> <83vbadp69k.fsf@gnu.org> <87egh1kx83.fsf@wanadoo.es> <83r3l1p4fn.fsf@gnu.org> <87a8rpkvsu.fsf@wanadoo.es> <83pp0kq0h7.fsf@gnu.org> <561B3038.8090405@cs.ucla.edu> <87bnc4pgx1.fsf@mbork.pl> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1444645257 14264 80.91.229.3 (12 Oct 2015 10:20:57 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 12 Oct 2015 10:20:57 +0000 (UTC) Cc: emacs-devel@gnu.org To: Marcin Borkowski Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Oct 12 12:20:56 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZlaDv-0002dl-Fc for ged-emacs-devel@m.gmane.org; Mon, 12 Oct 2015 12:20:55 +0200 Original-Received: from localhost ([::1]:54307 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlaDu-00064Z-IV for ged-emacs-devel@m.gmane.org; Mon, 12 Oct 2015 06:20:54 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:45677) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlaDR-00064T-H8 for emacs-devel@gnu.org; Mon, 12 Oct 2015 06:20:27 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZlaDQ-0004xh-3D for emacs-devel@gnu.org; Mon, 12 Oct 2015 06:20:25 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35520) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlaDQ-0004xd-03; Mon, 12 Oct 2015 06:20:24 -0400 Original-Received: from localhost ([127.0.0.1]:49339 helo=lola) by fencepost.gnu.org with esmtp (Exim 4.82) (envelope-from ) id 1ZlaDP-0001gd-77; Mon, 12 Oct 2015 06:20:23 -0400 Original-Received: by lola (Postfix, from userid 1000) id 8DA1FDF3DF; Mon, 12 Oct 2015 12:20:22 +0200 (CEST) In-Reply-To: <87bnc4pgx1.fsf@mbork.pl> (Marcin Borkowski's message of "Mon, 12 Oct 2015 11:36:26 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:191311 Archived-At: Marcin Borkowski writes: > For the record: there was once a project to rewrite TeX in Java (it > was before the mankind learned that Java was not the solution to all > the humanity's problems). NTS -- New Typesetting System. > It even got some funding (as in, real money). The project is now > considered a failure: while they /did/ produce a working, 99.9% > compatible TeX replacement, it was an order of magnitude slower (or > so), and the main goal - to have a TeX replacement written in a more > modular fashion, so that it would be easier to e.g. experiment with > various line/page-breaking algorithms and extensions to TeX - was not > achieved: nobody actually performed those experiments. (There aren't > even many traces on the internet; while you can read about this "New > Typesetting System" in a few places, the actual code is lost. That is > a pity, imho, since I hear that Java is much faster nowadays than 10+ > years ago, so it might work now.) Well, later there was exTeX . The front page of the web site and the FAQ are still accessible, but most parts of the site are dead these days. > Another project, LuaTeX, took off and can be now considered very > successful. This project, however, has different goals (instead of > replacing TeX with something different, they embedded Lua in TeX, and > then opend up a possibility to reimplement many parts of TeX in Lua. > In effect, it seems that the goal of NTS - to be able to easily > experiment with TeX internals - was achieved, though in a completely > different way). It did not as much clone TeX as slowly turn it into a cyborg, putting hooks into the existing works. > Also, it is run by a small group of dedicated individuals, and my > guess is that one of the reasons of its success is Hans Hagen and his > team. One, they are exceptionally smart people. Two, they /think/ > and /design/ before coding, not the other way round. Uh, are we talking about the same project here? The whole point of LuaTeX was _not_ to design anything but rather take the existing design and code and "open it up" by adding hooks into its code and data structures. This most certainly _is_ a "code foremost, to heck with design" project. > (Not to say that Emacs devs don't do it, but the majority of the > software world apparently got this wrong.) Last but not least, Hans' > company is based around ConTeXt and LuaTeX, so he has a /really/ > strong motivation to have this system working smoothly: he /actually > relies on it for his income/. He has a really strong motivation to have _something_ working _today_. And he started with an existing complex format employing the unchanged TeX engine, hooking into TeX where the limited facilities provided by the base TeX engine were making the format rather awkward and inefficient even though basically "everything needed" would have been reasonably close under the hood in the underlying Pascal code. I once translated a video game I wrote for Z80 processors at 4MHz (obviously in assembly language) to C. That was the opportunity to rewrite everything in a modular manner, make the various "monsters" have movement code in an object-oriented manner and so on. In the end, this ended up more or less as a literal translation of the logic of the assembly code, partly using even register assignments as variables. The game was coded very well and readable in spite of 6-character limits for identifiers (uppercase and digits only) and almost all comments consisting in accumulative cycle counts for the sake of the game's timing. To "rewrite" in more modern idioms would have required basically designing the code from scratch, and the existing code was a very good "local minimum" with nothing in the line of "incremental improvement" being in line of sight. The same problem pervades in a larger scale the original code base of TeX. You either end up with redesigning it from scratch, or with just having a clone not inherently different, in particular not any more modular or maintainable. LuaTeX does not even _try_ to touch the TeX "table of equivalence" and its related dynamic scoping mechanisms. From Lua, you just get to touch the current group-local settings of TeX variables but do not get to either read, simulate, or touch the TeX grouping itself. There still isn't any way to embed Lua code in TeX or vice versa: any Lua you want to execute from TeX is first _tokenized_ according to TeX's input conversion rules and then _printed_ back into characters read into Lua. If Lua was a language picky about differences in whitespace (like Python), this would be totally untenable. The reverse is similar: Lua can "print" into TeX's idea of the current input file, and TeX will at some later time when Lua has ceded control pick that input up and interpret it as part of whatever it was doing when it called Lua, of course first turning the characters back into TeX tokens according to whatever catcode regime it might be in. That's not an integrated application as much as it is two different systems conversing over byte-stream pipes with synchronous but non-symmetric flow control. They get results as opposed to the several failed projects preceding them. But the main reason is that they don't bother with a coherent design rather than getting results. In the course of their work, they sometimes throw out some component of TeX altogether and replace its Pascal code with something coded entirely in Lua, in order to make it more accessible to their personal modifications. So there _is_ some "rewrite TeX in Lua" work involved, but it's not really part of a design process but done ad-hoc on an as-needed basis. So the overall message LuaTeX sends to people wanting to redesign systems is rather bleak. -- David Kastrup