dolist considered harmful

dolist considered harmful

[Stefan Monnier]

All this pcase-dolist discussion made me take a second look at dolist
and now I wondering whether we should keep it:

- The docstring is incomprehensible:

      Loop over a list.
      Evaluate BODY with VAR bound to each car from LIST, in turn.
      Then evaluate RESULT to get return value, default nil.

  I don't know how to loop over a list.  Did they mean to set the cdr
  of the last element so it loops back to the first element?
  "evaluate" is unclear (is it passed to   `eval`?  Then why not say it?
  What second arg is passed to `eval`?).
  but why evaluate it several times?
  Also, in which order?
  How does RESULT communicate the return value?
  Is it RESULT which defaults to nil or the return value?
  If the return value, then when does it default to nil?

- The docstring says nothing of what happens when LIST is not a list,
  yet a quick grep shows that most uses of dolist use for LIST a simple
  variable rather than a list.  What gives?

Since it can only be used when we're 100% sure that LIST is a LIST,
(and who can be sure of such a thing in a dynamic language like Emacs
Lisp), I think we'd be better off deprecating it and changing all uses
of dolist with a clear while+cdr loop that everyone can understand.


        Stefan

Re: dolist considered harmful

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Tue, 30 Oct 2018 14:39:46 -0400
> 
> - The docstring is incomprehensible:
> 
>       Loop over a list.
>       Evaluate BODY with VAR bound to each car from LIST, in turn.
>       Then evaluate RESULT to get return value, default nil.
> 
>   I don't know how to loop over a list.

I think the rest of the doc string explains how.

>   "evaluate" is unclear (is it passed to   `eval`?  Then why not say it?

What "evaluation" is, is described in the section "Evaluation" of the
ELisp manual.

>   Also, in which order?

It says "each car from LIST, in turn".  I think this answers your
question.

>   How does RESULT communicate the return value?
>   Is it RESULT which defaults to nil or the return value?
>   If the return value, then when does it default to nil?

I think this is clear from the text, I'm unsure what confused you.

> - The docstring says nothing of what happens when LIST is not a list,
>   yet a quick grep shows that most uses of dolist use for LIST a simple
>   variable rather than a list.  What gives?

There's no contradiction here.

> Since it can only be used when we're 100% sure that LIST is a LIST,
> (and who can be sure of such a thing in a dynamic language like Emacs
> Lisp), I think we'd be better off deprecating it and changing all uses
> of dolist with a clear while+cdr loop that everyone can understand.

Patches welcome.

Re: dolist considered harmful

[Jay Kamat]

Stefan Monnier writes:

> Since it can only be used when we're 100% sure that LIST is a LIST,
> (and who can be sure of such a thing in a dynamic language like Emacs
> Lisp), I think we'd be better off deprecating it and changing all uses
> of dolist with a clear while+cdr loop that everyone can understand.

I haven't been following the other discussion, but I think deprecating dolist
is a little extreme. I use dolist a lot because it makes a lot of things I
write a lot simpler (over using the loop macro or mapcar).

I suspect a lot of existing code out there uses dolist, a github search
suggests about 500k uses (although many of those are probably copies).

Re: dolist considered harmful

[Noam Postavsky]

On Tue, 30 Oct 2018 at 19:24, Jay Kamat <jaygkamat@gmail.com> wrote:

> Stefan Monnier writes:
>
> > Since it can only be used when we're 100% sure that LIST is a LIST,
> > (and who can be sure of such a thing in a dynamic language like Emacs
> > Lisp), I think we'd be better off deprecating it and changing all uses
> > of dolist with a clear while+cdr loop that everyone can understand.
>
> I haven't been following the other discussion, but I think deprecating dolist
> is a little extreme.

I think Stefan just forgot to add sarcasm tags to his message.

RE: dolist considered harmful

[Drew Adams]

> I think Stefan just forgot to add sarcasm tags to his message.

That was my interpretation as well.  He used the same
terms and arguments on dolist that were tossed at pcase.

Re: dolist considered harmful

[Garreau, Alexandre]

On 2018-10-30 at 14:39, Stefan Monnier wrote:
> All this pcase-dolist discussion made me take a second look at dolist
> and now I wondering whether we should keep it:

This is still extreme: I never saw anyone suggesting to remove anything
about pcase or its derivative… but I can get quite distracted sometimes
so maybe I missed it.  I think pattern-matching, though I’m not fond of
it, is sometimes necessary, and is a must-be in any high-level language,
and as pcase exists and is pretty used…

> Since it can only be used when we're 100% sure that LIST is a LIST,
> (and who can be sure of such a thing in a dynamic language like Emacs
> Lisp), I think we'd be better off deprecating it and changing all uses
> of dolist with a clear while+cdr loop that everyone can understand.

Before people spoke about sarcasm I really thought this all was serious
and first degree, about removing a such now idiomatic form from elisp,
that also exists in common lisp, to end suggesting such much lower level
style.

Also I dislike `while': it looks pretty not lispy to me, very imperative
(but well all emacs and much lisp is imperative as well…)…  I’d rather
define `dolist' upon `mapc' than `while'.  And if something common and
simple as `dolist' was to be removed, I think I’d end up programming
like in scheme and use recursion everywhere (btw is emacs
tail-recursive?), so in the end that’d divide programming style between
`while' and recursion so all programs becomes half as readable as before
(except recursion-people are afaik more rare, so I’m going to be forced
in `while', and anyway it seems like in the end I made an argument in
favor of `pcase' and against `case' ><).

> - The docstring is incomprehensible:
>
>       Loop over a list.
>       Evaluate BODY with VAR bound to each car from LIST, in turn.
>       Then evaluate RESULT to get return value, default nil.
>
>   I don't know how to loop over a list.

I must say, that this seems true to me.  I regularely prefers recursion
or mapcar/mapc to dolist because they’re simpler and cleaner.  I recall
at a time, each time I realized I needed `dolist', I evaled a test using
it so to be sure I still knew how to properly use it.  Still today,
before writing it, it is unclear in my mind how I would use dolist to do
something (I have to say I find unnatural to repetedly use setq or use
side-effects (especially to set stuff) to achieve something).  Last time
I used `dolist', I first wanted to use recursion before someone noticed
me it was unnatural / strang / unidiomatic in elisp to do that, so I
needed dolist (or seq, which still seems more uncommon to me), while
what I wanted to do was simply to `reduce' composition (I got a list of
functions, I wanted to apply them all to a string, one upon the other:
so if there were a `compose' function, `reduce' it to a sequence of
functions upon a string).

>   Did they mean to set the cdr of the last element so it loops back to
>   the first element?

I have to say that, though it is closer to real, low-level, actual
meaning/behavior, and not difficult to grasp once you know what’s a
list, “to each element of LIST, in turn” would have been more simple to
read, parse (it is a syntactically simpler phrase), understand,
including for, but not limited to, newcomers.

>   "evaluate" is unclear (is it passed to `eval`?  Then why not say it?
>   What second arg is passed to `eval`?).  but why evaluate it several
>   times?  Also, in which order?  How does RESULT communicate the
>   return value?  Is it RESULT which defaults to nil or the return
>   value?  If the return value, then when does it default to nil?

Again I have to say “Then return RESULT evaluation, or nil if omitted.”
is simpler and more unambiguous.

> - The docstring says nothing of what happens when LIST is not a list,
>   yet a quick grep shows that most uses of dolist use for LIST a simple
>   variable rather than a list.  What gives?

a variable… returning a list, so being a list (the symbol is not a list,
but it doesn’t say it doesn’t evaluate the variable).

Again I have to say (this formulation is getting a little too much
redundant here and adding “again” is not hiding it well…) I find it is
true that `dolist' docstring is not explicit enough about what is
quoted, to quote, evaluated or not.  It no longer happens to me, but I
clearly recall having some pain about `dolist' in my first times I used
it, to understand what I needed to quote and what I needed to put as is…
and that required me several empirical tests to be sure: ideally a clear
docstring should spare you that.

Because there’s some inconstancy to explain there: VAR is not evaluated,
never, it *must* be a symbol as `read' would read it, not something
evaluating to a symbol, LIST is evaluated right away, and RESULT is too,
only once, and at the end.

Re: dolist considered harmful

[Noam Postavsky]

On Wed, 31 Oct 2018 at 18:10, Garreau, Alexandre <galex-713@galex-713.eu> wrote:
>
> On 2018-10-30 at 14:39, Stefan Monnier wrote:
> > All this pcase-dolist discussion made me take a second look at dolist
> > and now I wondering whether we should keep it:
>
> This is still extreme: I never saw anyone suggesting to remove anything
> about pcase or its derivative… but I can get quite distracted sometimes
> so maybe I missed it.

Not pcase, but Alan talked about removing pcase-dolist and other derivatives:

https://lists.gnu.org/archive/html/emacs-devel/2018-10/msg00664.html

    If we can't get quality doc strings for pcase-dolist and friends, I say
    we should remove them from Emacs.  There are only around 30 occurrences
    of pcase-dolist.

> (btw is emacs tail-recursive?),

Nope. Which is why recursion is not idiomatic.

> I used `dolist', I first wanted to use recursion before someone noticed
> me it was unnatural / strang / unidiomatic in elisp to do that,

Re: dolist considered harmful

[Garreau, Alexandre]

On 2018-10-31 at 18:44, Noam Postavsky wrote:
> On Wed, 31 Oct 2018 at 18:10, Garreau, Alexandre <galex-713@galex-713.eu> wrote:
>>
>> On 2018-10-30 at 14:39, Stefan Monnier wrote:
>> > All this pcase-dolist discussion made me take a second look at dolist
>> > and now I wondering whether we should keep it:
>>
>> This is still extreme: I never saw anyone suggesting to remove anything
>> about pcase or its derivative… but I can get quite distracted sometimes
>> so maybe I missed it.
>
> Not pcase, but Alan talked about removing pcase-dolist and other derivatives:
>
> https://lists.gnu.org/archive/html/emacs-devel/2018-10/msg00664.html
>
>     If we can't get quality doc strings for pcase-dolist and friends, I say
>     we should remove them from Emacs.  There are only around 30 occurrences
>     of pcase-dolist.

These are cool, and really beautiful examples of what it means to
efficiently (in terms of semantics) generalize a high-level behavior
pattern to outside things (if I wasn’t fearing pattern-matching to
exageratedly spread, I’d even suggest for these to become the default as
replacement of their non-pcase versions, as I already desired once (I
dislike their current syntax but they happen to be potentially
backward-compatible to all others lisp (pattern-)match(ing) that I would
find acceptable)…).

If ever that much needed I’d like to help improving all these docstrings
so they keep existing.  But I feel like there’s a stronger global will
to improve these anyway.

>> (btw is emacs tail-recursive?),
>
> Nope. Which is why recursion is not idiomatic.

Oh.  I once heard emacs was “kind of, too, a naive implementation of
lisp”, and each day I discover it is effectively a bit more than I
initially though.  …so it may end up with bignums before tail-recursion,
I wonder if that’s not to be considered very surprising.