10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Emanuel Berg]

Eli Zaretskii wrote:

> Please, everybody, take the Lisp vs Python argument off this
> list, it is off-topic here. If you must discuss this, please
> use the emacs-tangents@gnu.org mailing list instead.

Sure, but we are allowed to discuss how to make Elisp better?

Since Python has had enormous success, and Lisp hasn't - or if
it had, it lost it - it might be a good ide to analyze what
they (Python) did good.

You might think this is some bängalow discussion just because
certain people are in it. But it doesn't have to be like
that, it can be very hands-on.

Ten things that are annoying with Emacs Lisp from the holster,
and what to do about them:

10. Moving point around all the time. Whatever program, what
    you do is, it feels like, not solving your problem
    algorithmicly, you are just doing endless

    (goto (point-min))
    (prong (end-of-paragraph) (current-column))
    (pos-eol -1)

    (setq exists-after-point (unless (re-search-forward re nil t) (point)))

    Frustration: What has this to do with my problem and
    proposed solution? I understand Emacs grew around its
    function as a text editor, but "everything is in the
    buffer" and "the buffer is the data structure of Emacs",
    that has goon too far and we see a lot of code being
    virtually a very, very long traversal of buffers moving
    around point. So what ought to have been a tolerable
    exception, has become the norm and hailed model to the
    point, as I said earlier, interfaces toward programming
    are so weak it is considered normal that ispell can't even
    to (spell "word") without first outputting it somewhere
    and to the operation on that surface.

    Solution: MVC-ish separation of the interface, the
    computation, and the presentation of the result. To solve
    a problem, get the data into modern data structures with
    modern access methods, apply known algorithms by the book
    known specifically to help this problem rather than
    "Everything goes" Elisp hackin. (Yes you find that stuff
    in libraries, often. Apply on data in data structures, not
    on data as it appears in Emacs buffers.) But how ever well
    one does, it is gonna be _a lot_ of of moving point around
    in Emacs Lisp, so don't worry :)
   
Example of problem from my favorite part of Emacs, ispell.el:

(defun ispell-mime-multipartp (&optional limit)
  "Return multipart message start boundary or nil if none."
  ;; caller must ensure `case-fold-search' is set to t
  (and
   (re-search-forward
    "Content-Type: *multipart/\\([^ \t\n]*;[ \t]*[\n]?[ \t]*\\)+boundary="
    limit t)
   (let (boundary)
     (if (looking-at "\"")
	 (let (start)
	   (forward-char)
	   (setq start (point))
	   (while (not (looking-at "\""))
	     (forward-char 1))
	   (setq boundary (buffer-substring-no-properties start (point))))
       (let ((start (point)))
	 (while (looking-at "[-0-9a-zA-Z'()+_,./:=?]")
	   (forward-char))
	 (setq boundary (buffer-substring-no-properties start (point)))))
     (if (< (length boundary) 1)
	 (setq boundary nil)
       (concat "--" boundary)))))

Moving point around, looking, searching, seeing or not seeing.
This is boring and error prone to write, and error prone to
take over from someone else, or return to after x years.

You don't think in terms of the problem, or the solution for
that matter, you are just somewhere in the buffer and
according the the map you are completely lost!

That is why I have, at place 10 annoying things about Elisp,
excessive movement of point around the buffer, often involving
manual retrieving and editing data from a buffer that, in
execution, might not even look like what you thought it would,
and that was 25 lines ago!

Okay, I know - if only I could be passionate about it.

-- 
underground experts united
https://dataswamp.org/~incal

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Mon, 05 Aug 2024 18:27:35 +0200
> 
> Eli Zaretskii wrote:
> 
> > Please, everybody, take the Lisp vs Python argument off this
> > list, it is off-topic here. If you must discuss this, please
> > use the emacs-tangents@gnu.org mailing list instead.
> 
> Sure, but we are allowed to discuss how to make Elisp better?

Yopu could try, although it is usually not a useful discussion.

> Since Python has had enormous success, and Lisp hasn't - or if
> it had, it lost it - it might be a good ide to analyze what
> they (Python) did good.
> 
> You might think this is some bängalow discussion just because
> certain people are in it. But it doesn't have to be like
> that, it can be very hands-on.
> 
> Ten things that are annoying with Emacs Lisp from the holster,
> and what to do about them:
> 
> 10. Moving point around all the time. Whatever program, what
>     you do is, it feels like, not solving your problem
>     algorithmicly, you are just doing endless
> 
>     (goto (point-min))
>     (prong (end-of-paragraph) (current-column))
>     (pos-eol -1)
> 
>     (setq exists-after-point (unless (re-search-forward re nil t) (point)))
> 
>     Frustration: What has this to do with my problem and
>     proposed solution? I understand Emacs grew around its
>     function as a text editor, but "everything is in the
>     buffer" and "the buffer is the data structure of Emacs",
>     that has goon too far and we see a lot of code being
>     virtually a very, very long traversal of buffers moving
>     around point. So what ought to have been a tolerable
>     exception, has become the norm and hailed model to the
>     point, as I said earlier, interfaces toward programming
>     are so weak it is considered normal that ispell can't even
>     to (spell "word") without first outputting it somewhere
>     and to the operation on that surface.

There's nothing more natural than an editor analyzing text in a
buffer.  Why it frustrates you is beyond me.

Emacs Lisp is not a general-purpose programming language.  It is a
language for implementing Emacs and Emacs extensions.  Thus, comparing
it with Python is, in general, simply wrong.  We can compare a few
specific aspects, but not the languages as a whole, and definitely not
their success rate: the scope of Emacs Lisp is limited to Emacs, which
is orders of magnitude more narrow than the scope of Python (or any
other general-purpose programming language).

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Emanuel Berg]

Eli Zaretskii wrote:

> There's nothing more natural than an editor analyzing text
> in a buffer. Why it frustrates you is beyond me.

Sure, as is analyzing chemical substances in chemistry.

And it is well known that instead of using machines and
well-known methods from the outside to do the job, the
chemists are swimming around in the substances mucking around
with individual molecules giving explicit instructions what
should happen for each case?

Oh, no, all computing is the same, basically, we have of
course specific problems and applications here, but instead of
doing the old thing we should move up one level of abstraction
and be there instead, and focus not on "getting the job done"
but instead getting it done in a way that is much better than
what we - to a large extent - have been doing so far.

Everyone has a problem domain with specific characteristics
that isn't the same as do stuff on the detail level,
"everyone" doesn't do that if that is what you thought.

> Emacs Lisp is not a general-purpose programming language.

It doesn't matter what it is, it can be better, we should aim
for that.

> It is a language for implementing Emacs and Emacs
> extensions. Thus, comparing it with Python is, in general,
> simply wrong.

Yes, Python is incomparable to Emacs Lisp and would probably
win quite even against the collective Lisp world, I'm afraid.

Lisp would probably loose to some other languages as well.

> We can compare a few specific aspects, but not the languages
> as a whole, and definitely not their success rate: the scope
> of Emacs Lisp is limited to Emacs, which is orders of
> magnitude more narrow than the scope of Python (or any other
> general-purpose programming language).

Emacs is the bastion of Lisp, if we care about Lisp we should
do what we can to make Elisp more competitive altho we should
focus on getting better, and not compare us to other languages
as we are way too far behind in many areas, I'm afraid.

-- 
underground experts united
https://dataswamp.org/~incal

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Yuri Khan]

On Mon, 5 Aug 2024 at 23:31, Emanuel Berg <incal@dataswamp.org> wrote:

> 10. Moving point around all the time.[…]
>
>     Frustration: What has this to do with my problem and
>     proposed solution? I understand Emacs grew around its
>     function as a text editor, but "everything is in the
>     buffer" and "the buffer is the data structure of Emacs",
>     that has goon too far and we see a lot of code being
>     virtually a very, very long traversal of buffers moving
>     around point.

In a differently designed API, you[^*] would have iterators into
buffers. And you would want from an iterator everything that is
available from the point, just that there could be many of them
simultaneously and that they wouldn’t affect the real user’s point.
And you would want iterators to be visible in the buffer text when
debugging.

In Elisp, you have save-excursion which protects the user’s point, and
markers which provide multiplicity.

[^*]: I.

10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Christopher Dimech]

> Sent: Tuesday, August 06, 2024 at 5:03 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)
>
> Eli Zaretskii wrote:
>
> > There's nothing more natural than an editor analyzing text
> > in a buffer. Why it frustrates you is beyond me.
>
> Sure, as is analyzing chemical substances in chemistry.
>
> And it is well known that instead of using machines and
> well-known methods from the outside to do the job, the
> chemists are swimming around in the substances mucking around
> with individual molecules giving explicit instructions what
> should happen for each case?
>
> Oh, no, all computing is the same, basically, we have of
> course specific problems and applications here, but instead of
> doing the old thing we should move up one level of abstraction
> and be there instead, and focus not on "getting the job done"
> but instead getting it done in a way that is much better than
> what we - to a large extent - have been doing so far.
>
> Everyone has a problem domain with specific characteristics
> that isn't the same as do stuff on the detail level,
> "everyone" doesn't do that if that is what you thought.
>
> > Emacs Lisp is not a general-purpose programming language.
>
> It doesn't matter what it is, it can be better, we should aim
> for that.
>
> > It is a language for implementing Emacs and Emacs
> > extensions. Thus, comparing it with Python is, in general,
> > simply wrong.
>
> Yes, Python is incomparable to Emacs Lisp and would probably
> win quite even against the collective Lisp world, I'm afraid.

Python is not great.  But people can believe what they want.

Sometimes we discover unpleasant truths. Whenever we do so, we
are in difficulties: suppressing them is scientifically dishonest,
so we must tell them, but telling them, however, will fire back on us.
If the truths are sufficiently impalatable, our audience is psychically
incapable of accepting them and we will be written off as totally
unrealistic, hopelessly idealistic, dangerously revolutionary, foolishly
gullible or what have you.

Most Computer Science Departments have opted for the easy way out, to
pretend that problems do not exist.  Programming is one of the most
difficult branches of applied mathematics; most mathematicians should
better remain pure mathematicians.

It is practically impossible to teach good programming to students that
have had a prior exposure to Python, as potential programmers they are
mentally mutilated beyond hope of regeneration.

In the good old days physicists repeated each other's experiments, just
to be sure. Today they stick to Python, so that they can share each
other's programs, bugs included.

> Lisp would probably loose to some other languages as well.
>
> > We can compare a few specific aspects, but not the languages
> > as a whole, and definitely not their success rate: the scope
> > of Emacs Lisp is limited to Emacs, which is orders of
> > magnitude more narrow than the scope of Python (or any other
> > general-purpose programming language).
>
> Emacs is the bastion of Lisp, if we care about Lisp we should
> do what we can to make Elisp more competitive altho we should
> focus on getting better, and not compare us to other languages
> as we are way too far behind in many areas, I'm afraid.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Emanuel Berg]

> 10. Moving point around all the time. Whatever program, what
>     you do is, it feels like, not solving your problem
>     algorithmicly, you are just doing endless
>
>     (goto (point-min))
>     (progn (end-of-paragraph) (current-column))
>     (pos-eol -1)

It isn't even "end-of-paragraph", is is
`end-of-paragraph-text'.

This brings up another interesting aspect, namely if moving
point around in a future buffer - if that is our whole
business model and everyone tries really hard to convince
themself it is what everybody does - which is completely wrong
by the way - then answer me, how is is that we have so poor
tools to do it?

Just look above.

There isn't even a way to go to the beginning of the buffer
without combining not "goto" (should be `goto-char' of course)
with `point-min'. `pos-eol' is the only one to my liking of
those I just mentioned as they came up, then.

It does say `current-column' but it doesn't matter because
(end-of-paragraph-text) doesn't return point. Such, and many,
many other examples are why Elisp is filled with constructs
such as

( save-mark-and-excursion ( progn ( move the point somewhere )
(point) )

If anyone really believed in that idea, all such cases would
have been streamlined into single functions ages ago instead
of having programmers put together those meaningless towers
all the time just to retrieve information from what is
supposed to be one of our native methods to do just that.

I can complete the list (9 more) but I don't know, maybe leave
the whole thing at that? I don't want to be too negative.
But the endless moving around point in a future buffer one
envision while typing code - yeah, it is a crazy idea

-- 
underground experts united
https://dataswamp.org/~incal

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

      > Solution: MVC-ish separation of the interface, the
      > computation, and the presentation of the result. To solve
      > a problem, get the data into modern data structures with
      > modern access methods, apply known algorithms by the book
      > known specifically to help this problem rather than

That is very abstract and does not propose any change concretely enough
that we could consider it.

I am guessing that "MVC" refers to some Microsoft editor.  Please
don't expect us to have seen it.  Emacs is older than Windows.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Abraham S.A.H. via Emacs development discussions.]

> Moving point around

I thought it's the strengh point of Emacs.  
Why “Everything being a buffer” is bad?  Other than being tired of it?

Sounds like people who talk about their frustration with Unix's “everything is a file” concept.

Can you explain this with more logics and examples?

10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Abraham S.A.H. via Emacs development discussions.]

> Since Python has had enormous success

Just in being popular you mean right?

I would not learn it otherwise. 

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Emanuel Berg]

Richard Stallman wrote:

>> Solution: MVC-ish separation of the interface, the
>> computation, and the presentation of the result. To solve
>> a problem, get the data into modern data structures with
>> modern access methods, apply known algorithms by the book
>> known specifically to help this problem rather than
>
> That is very abstract and does not propose any change
> concretely enough that we could consider it.

MVC is a term from CS theory, it means "Model View Control"
and you can summarize it just by saying that data,
data processing, and the interface, those should be
kept apart.

We have an interesting situation where the processing of data
with Elisp is often done directly in the buffer, i.e. _in_
that same data!

This is why Elisp code often gets completely littered with
commands that move point back and forth, that extracts data,
modifies data, inserts it back again, and so on.

What we should do - well, (1) one should be aware of it and
that it is a problem. So for example, instead of sorting
a region in the buffer, one should ask - what does that region
represent? what outputted it? - and then modify that function,
for example to make it accept an optional alternative sorter
or whatever.

And (2), since it is gonna be like this anyway to a huge
extent, to not have Elisp code being completely overrun by
such stuff, we can provide a much neater, more consistent such
set of functions.

So this is bad

(save-mark-and-excursion (progn (forward-paragraph -2) (point)))

This OTOH is much better

(pos-bol &optional N)
(pos-eol &optional N)

So for every unit, we should have pos-unit, pos-unit-beg,
pos-unit-end, and pos-unit-reg (pos-unit should probably be
the same as pos-unit-beg rather than pos-unit-reg for
practical reasons).

-- 
underground experts united
https://dataswamp.org/~incal

Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Emanuel Berg]

Abraham S.A.H." via "Emacs development discussions. wrote:

>> Moving point around
>
> I thought it's the strengh point of Emacs.

When programming, half the time is moving around in the buffer
instead of solving the actual problem.

Not only that, moving around in the buffer is difficult and
error-prone as you don't know how it will look at the point of
code execution, and also moving around and of course
especially _editing_ it inside it changes it, so it is a crazy
situation to encourage.

> Why "Everything being a buffer" is bad?

Because one would like to separate data, data retrieval, data
processing, and how data is displayed.

This is possible to do in Elisp but people have not payed
attention to it which is why many program including such in
core Emacs are very long programs with a lot of moving around
buffers endlessly and this code is all intermixed with
everything else.

Very ugly, boring to write, difficult to read, error-prone
to maintain.

-- 
underground experts united
https://dataswamp.org/~incal

10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)

[Christopher Dimech]

> Sent: Wednesday, August 07, 2024 at 7:34 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: 10 problems with Elisp, part 10 (was: Re: Emacs website, Lisp, and other)
>
> Abraham S.A.H." via "Emacs development discussions. wrote:
> 
> >> Moving point around
> >
> > I thought it's the strengh point of Emacs.
> 
> When programming, half the time is moving around in the buffer
> instead of solving the actual problem.
> 
> Not only that, moving around in the buffer is difficult and
> error-prone as you don't know how it will look at the point of
> code execution, and also moving around and of course
> especially _editing_ it inside it changes it, so it is a crazy
> situation to encourage.
> 
> > Why "Everything being a buffer" is bad?
> 
> Because one would like to separate data, data retrieval, data
> processing, and how data is displayed.
> 
> This is possible to do in Elisp but people have not payed
> attention to it which is why many program including such in
> core Emacs are very long programs with a lot of moving around
> buffers endlessly and this code is all intermixed with
> everything else.
> 
> Very ugly, boring to write, difficult to read, error-prone
> to maintain.

How the latex and tex modes have been written is an example.
 
> -- 
> underground experts united
> https://dataswamp.org/~incal
> 
> 
>