Re: we need *modularity* [last problem] (was: Re: as for Calc and the math library)

[arthur miller <arthur.miller@live.com>]

[-- Attachment #1: Type: text/plain, Size: 8662 bytes --]

>> I have seen your previous answers to others, and I do
>> understand the point you make, but I think the argument
>> fails because a module can be just a proxy to
>> a non-free library.
>
>Emacs is notoriously not modular. Everything is everywhere,
>you seem to have everything but if fact you have nothing.
>There is so much stuff you can't find the basics stuff that
>isn't there.
>
>Here take a look: what does this name say?
>
>  cl--defalias
>
>Ah, that's a local function in the Common Lisp package, right?
>
>Well, YES, but ...
>
>It is actually an Elisp defun. And that prefix isn't really
>a package qualifier, it is just a convention, part of
>the name. And the double dash, likewise just a convention, we
>don't have package modularity or any of that kind of data
>hiding, abstraction, encapsulation ... what is it more called?
>Well, modularity!
>
>In Emacs everything is intermixed, Elisp-Emacs,
>non-interactive/interactive, data/data processing/the
>interface (buffer, buffer, and buffer), what is a file, what
>is a package, what is a library, and what is a program - no
>one can tell.
>
>Emacs Lisp isn't is fun, but absolutely not any kind
>of powerhouse. It is inconsistent in itself and on top of that
>all those zillion interfaces, cl-, seq-, slime- (yes, had to
>use that today, for `slime-transpose-lists').
>
>It is pretty fast to get something going but complexity
>skyrockets.
>
>What we should do, in general: modularity! clear divisions, in
>particular, libraries for everything, including those that provide basic
>stuff in a complete and consistent way.
>
>We should get real package modularity, transform the prefixes
>into real package qualifiers, and throw away the ugly
>local ones.
>
>With Elisp, optimize everything for development speed and
>convenience, have consistent interfaces as much as possible,
>acknowledge that Elisp was underpowered, which is why all the
>interfaces came, now, core Elisp and the interfaces should
>make peace, one should identify what it is exactly, what are
>the to 10 things that people use with cl-, pcase-, seq- etc,
>and bring them into a new core Elisp, and the few CL experts
>can still use all the rest.
>
>Now, with native compilation we have speed - we have power
>with the interfaces if one could harvest it, which one
>currently - well, it is very slow and difficult, anyway - the
>very, very last spet into maturity and getting out of the
>Elisp ghetto (and getting visitors!) are modularity, modern
>software principles - by modularity I mean real modularity,
>based on technology - not silly conventions - I forgot where
>I was - the last step is modularity in general, and libraries
>in particular.
>
>Clear cut interfaces. Get away with the ugly prefixes and
>error--prone conventions that don't even do anything.
>
>Elisp 3.0! \o/
>
>PS. This is the problem. This series is completed, I said
>    I would write 10, I wrote 4. Here are the other parts.
>    It is all based on the same theme. Modularity,
>    consistency, libraries, modern software practises.
>
>    https://lists.gnu.org/archive/html/emacs-devel/2024-08/msg00154.html
>    https://lists.gnu.org/archive/html/emacs-devel/2024-08/msg00380.html
>    https://lists.gnu.org/archive/html/emacs-devel/2024-08/msg00388.html
>
>PPS. So who is working on real modularity based on the package
>     with real public and local functions as we speak? I know
>     there are always some guy - at least - working on such
>     a basic idea. I know some way. And I know some day.

Emacs Lisp like all other programming languages has its problems. I am not sure
though all the things you have taken up in those mails are problems. I haven't
answered on previous ones, more than a cursory suggestion about adding some
explanation to the manual in one mail about FFI which seem to be just ingored.
IMO, we have to understand Emacs as application and what it has to do to
achieve its goal of an interactive text editor, in addition to be also a Lisp
implementation.

At low level, there is the central idiom, buffer, implemented as a gap buffer,
which has to be manipulated in some way. IMO, something like "goto-char", sounds
much better than "move-gap" in the context of text editing. They need to have
some low-level primitives on which to implement the higher level ones. Compare
to some other editors, like VSC or whichever. They are all in pretty similar
situation, since very few programming languages do have a "text" data structure
built into the language.

They need some sort of "assembly" to build on. As discussed in the mentioned
mail from yesterday or the day before, I think it could be added to the manual
about the programming model used in low-level text and buffer management: make
an object current, and than act upon it. This is not so uncommon, stack-based
procedural approach. PostScript and OpenGL are two notable languages/API that
use similar approach. There are probably others I am not familiar with. I don't
know if that was popular programming model during 80's when Emacs was designed
or not, but Emacs Lisp is not alone in its design.

I don't know what is a better alternative? Look at Xlib and numerous arguments
their functions take. Win32 uses structures to pass arguments, which leads to
less function arguments passed around, but one has to setup structures before
they are passed to a function. It is also ads verbosity. Yuri mentioned
iterators, but I think iterators would have similar problem as markers: one has
to update all iterators into a buffer if one of them writes to the buffer. I am
not sure, but iterator could be seen as "functional marker", i.e. a functional
interface to markers? Since Emacs Lisp does not have doted notation for
functions that work on objects, like some other languages that use iterators
extensively (C++, Java), I think iterators and markers are basically the same
thing in EmacsLisp? I haven't think of it too much, so I might be wrong.

So, I don't know, while yes, you are correct that the programmatic interface is
a low-level, I don't think it is a bad one, or that they had so much more choice
there. There is though nothing that forbids anyone to make a higher-level
interface on top of the low-level one. I think, as someone mentioned in some
earlier mail, org-mode, or perhaps font-lock, regular expressions, or various
programming modes could be seen as a higher level interfaces. At least
conceptually? They could be implemented on top of goto-char, char-before/after
and point, but for the performance reasons are not (just my guess).

Also Emacs has to implement its own command loop and renderer, and since most of
us would like some control over those from Lisp, they need to expose those parts
to Lisp as well. I don't think that that Emacs rendering and text processing is
all very entangled in some bad way. Emacs does implement a console-like
(character) renderer. The design is (somehwat) similar to other similar
projects. For example MS console API implements a similar design, a 2d matrix
of characters, with text properties describing colors and other rendering
attributes for characters. There is a long comment by Gerd in xdisp.c explaining
the design in Emacs.

I will definitely agree with you that Emacs needs modular development, but now
you are opening another can of worms into an already unwelcome discussion :).

I also agree with you that EmacsLisp and C core could get a big cleanup, the API
surface is big, especially now when Lisp can be compiled to machine code via
GCC. There also seem to be some dissonance with the manual, as seen from my
other mail about subrs and some other functions (which I hope will get an
answer), so I definitely also agree that Elisp manual could get a facelift too.

I actually wrote this explanation, including a small CL program ~100 sloc, that
implements toy version of Emacs text processing API for the illustrative purpose
days before, on your first mail about goto-char, but never sent because of
explicitly not spending time of maintainers on the discussion that they probably
don't care much of, so not to waste their time and understanding that the topic,
while interesting is probably unwelcome, and yet, Eli accused me of delibarately
wasting maintainers time and energy. Personally, I think it is interesting to
talk about low-level implementation details, at least to check and further my
own udnerstanding, but it is probably best to continue the discussion via PM if
you are interested.

best regards
/a

[-- Attachment #2: Type: text/html, Size: 27449 bytes --]