bug#63588: 29.x: dotimes (possible) problem

bug#63588: 29.x: dotimes (possible) problem

[balducci]

hello

the dotimes macro behaves differently in 29.x with respect to previous
versions

Basically: changing the value of the loop variable in the body of
dotimes does not seem to have any effect, where for versions <29.x it
used to.

Here is a minimal stretch of dummy code clarifying the problem I'm
reporting.

emacs-29.0.91 (or 29.0.90)
==========================

(dotimes (ii 10)
(insert (format "%2d " ii))
(when (= ii 4)(setq ii 11))
)
==>  0  1  2  3  4  5  6  7  8  9 

emacs-28.2 (or any version <29.x)
=================================

(dotimes (ii 10)
(insert (format "%2d " ii))
(when (= ii 4)(setq ii 11))
)
==>  0  1  2  3  4 

The ability to jump out of the loop by pushing the loop variable over
the upper limit is something that I happen to use in my scripts, so
29.x breaks them somehow

Of course, there are very many other equivalent ways to
accomplish the same result, but I don't see why this one
shouldn't be supported (any longer)

Is the changed behavior intentional? Am I missing some blatant point here?

AFAICS, changing the value of the loop variable from inside the loop
body is supported by any other language which I know about

thank you very much in advance for any hint/feedback

ciao
-gabriele

bug#63586: 29.x: dotimes (possible) problem

[balducci]

hello

the dotimes macro behaves differently in 29.x with respect to previous
versions

Basically: changing the value of the loop variable in the body of
dotimes does not seem to have any effect, where for versions <29.x it
used to.

Here is a minimal stretch of dummy code clarifying the problem I'm
reporting.

emacs-29.0.91 (or 29.0.90)
==========================

(dotimes (ii 10)
(insert (format "%2d " ii))
(when (= ii 4)(setq ii 11))
)
==>  0  1  2  3  4  5  6  7  8  9 

emacs-28.2 (or any version <29.x)
=================================

(dotimes (ii 10)
(insert (format "%2d " ii))
(when (= ii 4)(setq ii 11))
)
==>  0  1  2  3  4 

The ability to jump out of the loop by pushing the loop variable over
the upper limit is something that I happen to use in my scripts, so
29.x breaks them somehow

Of course, there are very many other equivalent ways to
accomplish the same result, but I don't see why this one
shouldn't be supported (any longer)

Is the changed behavior intentional? Am I missing some blatant point here?

AFAICS, changing the value of the loop variable from inside the loop
body is supported by any other language which I know about

thank you very much in advance for any hint/feedback

ciao
-gabriele

bug#63588: 29.x: dotimes (possible) problem

[Drew Adams]

> changing the value of the loop variable in the body of
> dotimes does not seem to have any effect, where for versions <29.x it
> used to.
...
> The ability to jump out of the loop by pushing the loop variable over
> the upper limit is something that I happen to use in my scripts, so
> 29.x breaks them somehow
> 
> Of course, there are very many other equivalent ways to
> accomplish the same result, but I don't see why this one
> shouldn't be supported (any longer)
> 
> Is the changed behavior intentional? Am I missing some blatant point here?
> 
> AFAICS, changing the value of the loop variable from inside the loop
> body is supported by any other language which I know about

No, it's not.

From CLTL2 [*]:

  "Altering the value of var in the body of the loop
   (by using setq, for example) will have unpredictable,
   possibly implementation-dependent results. A Common
   Lisp compiler may choose to issue a warning if such
   a variable appears in a setq."

That allows an implementation of CL to define some
particular behavior in this regard.  But it also says
that if it does, and if your CL code depends on that
behavior, then it won't necessarily be portable to
other CL implementations.

It's unfortunate that if this Elisp change isn't
backward compatible.  That's indeed a consideration.
But it's wise for your code not to depend on being
able to change the loop variable and get any
meaningful resulting behavior.

Both in CL and Elisp `dotimes' lets you use RETURN
to break out of the loop.  That should speak to one
of your use cases, at least.

(dotimes (ii  100  "*********")
  (when (= 4 ii) (return "4444444444")))

CLTL2 says this:

  "An explicit return statement may be used to
   terminate the loop and return a specified value."

But I don't see where this is documented for Elisp.
Maybe it is documented somewhere.  Should be.
___

[*] https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node89.html

bug#63586: 29.x: dotimes (possible) problem

[Mattias Engdegård]

Hello Gabriele,

> changing the value of the loop variable in the body of dotimes does not seem to have any effect, where for versions <29.x it used to.

Prior to Emacs 29, the `dotimes` macro expanded to different code depending on whether lexical-binding was used or not, implicitly permitting your practice of altering the loop counter in code using dynamic binding.

However, the macro was probably never intended to be used that way, and your example does not work with lexical binding in any version as far as I can tell.

If you want to terminate your loop early, use catch/throw or a different loop construct altogether: while, named-let, or cl-loop, just to name a few.

> changing the value of the loop variable from inside the loop
> body is supported by any other language which I know about

Then you should learn more languages!

And you should use lexical-binding:t in all your elisp code. Always.

bug#63586: 29.x: dotimes (possible) problem

[Eli Zaretskii]

> From: balducci@units.it
> Date: Fri, 19 May 2023 17:10:44 +0200
> 
> the dotimes macro behaves differently in 29.x with respect to previous
> versions
> 
> Basically: changing the value of the loop variable in the body of
> dotimes does not seem to have any effect, where for versions <29.x it
> used to.
> 
> Here is a minimal stretch of dummy code clarifying the problem I'm
> reporting.
> 
> emacs-29.0.91 (or 29.0.90)
> ==========================
> 
> (dotimes (ii 10)
> (insert (format "%2d " ii))
> (when (= ii 4)(setq ii 11))
> )
> ==>  0  1  2  3  4  5  6  7  8  9 
> 
> emacs-28.2 (or any version <29.x)
> =================================
> 
> (dotimes (ii 10)
> (insert (format "%2d " ii))
> (when (= ii 4)(setq ii 11))
> )
> ==>  0  1  2  3  4 

I can only reproduce the behavior you see in 28.2 in Emacs 26.3.  All
the later versions, starting from 27.1, behave like Emacs 29 does.

> The ability to jump out of the loop by pushing the loop variable over
> the upper limit is something that I happen to use in my scripts, so
> 29.x breaks them somehow
> 
> Of course, there are very many other equivalent ways to
> accomplish the same result, but I don't see why this one
> shouldn't be supported (any longer)
> 
> Is the changed behavior intentional? Am I missing some blatant point here?
> 
> AFAICS, changing the value of the loop variable from inside the loop
> body is supported by any other language which I know about

Adding Stefan and Mattias who made changes in dotimes recently.

bug#63588: 29.x: dotimes (possible) problem

[Eli Zaretskii]

merge 63588 63586
thanks

> From: balducci@dschgrazlin2.units.it
> Date: Fri, 19 May 2023 10:54:46 +0200
> 
> hello
> 
> the dotimes macro behaves differently in 29.x with respect to previous
> versions

This is an exact duplicate of bug#63586 which you submitted recently.

bug#63586: 29.x: dotimes (possible) problem

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Basically: changing the value of the loop variable in the body of
>> dotimes does not seem to have any effect,

It does, but only within the current iteration, because the variable is
local to the iteration (each iteration gets a fresh new variable).

>> where for versions <29.x it used to.

Indeed it was changed last year for dynamically scoped ELisp to align it
with the semantics used in the lexically scoped ELisp dialect (and thus
simplify the macro).

>> Here is a minimal stretch of dummy code clarifying the problem I'm
>> reporting.
>>
>> emacs-29.0.91 (or 29.0.90)
>> ==========================
>>
>> (dotimes (ii 10)
>> (insert (format "%2d " ii))
>> (when (= ii 4)(setq ii 11))
>> )
>> ==>  0  1  2  3  4  5  6  7  8  9 
>>
>> emacs-28.2 (or any version <29.x)
>> =================================
>>
>> (dotimes (ii 10)
>> (insert (format "%2d " ii))
>> (when (= ii 4)(setq ii 11))
>> )
>> ==>  0  1  2  3  4 
>
> I can only reproduce the behavior you see in 28.2 in Emacs 26.3.  All
> the later versions, starting from 27.1, behave like Emacs 29 does.

AFAIK what the OP describes is new in Emacs-29 (commit
c6c9dfc8670f5698634a8d5853853056ff928974).
But the change only affects code that has not activated
`lexical-binding`: for `lexical-binding`, the behavior has been with us
"forever" (i.e. since Emacs-24, when `lexical-binding` was introduced).

>> AFAICS, changing the value of the loop variable from inside the loop
>> body is supported by any other language which I know about

It's definitely not supported by Common Lisp, which explicitly allows
both our old implementation and our new implementation (i.e. code like
yours may or may not work depending on the CL implementation).
Several other languages (starting with Pascal :-) either disallow or
discourage modifying the iteration variable(s) inside the loop body.

But in any case, the question is not whether it's a good idea or not: it
used to happen to work and now it doesn't work any more.

Maybe we should mention the change in etc/NEWS?


        Stefan

bug#63586: 29.x: dotimes (possible) problem

[Drew Adams]

> But in any case, the question is not whether it's a good idea or not: it
> used to happen to work and now it doesn't work any more.
> 
> Maybe we should mention the change in etc/NEWS?

+1

bug#63586: 29.x: dotimes (possible) problem

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: balducci@units.it,  Mattias Engdegård
>  <mattiase@acm.org>,
>   63586@debbugs.gnu.org
> Date: Fri, 19 May 2023 13:29:14 -0400
> 
> Maybe we should mention the change in etc/NEWS?

Feel free to suggest a NEWS entry, and thanks.

bug#63586: 29.x: dotimes (possible) problem

[Mattias Engdegård]

19 maj 2023 kl. 19.47 skrev Eli Zaretskii <eliz@gnu.org>:

> Feel free to suggest a NEWS entry, and thanks.

There is now a NEWS entry in emacs-29. Do modify it as you like.

bug#63588: 29.x: dotimes (possible) problem

[Stefan Kangas]

Version: 29.1

Mattias Engdegård <mattiase@acm.org> writes:

> 19 maj 2023 kl. 19.47 skrev Eli Zaretskii <eliz@gnu.org>:
>
>> Feel free to suggest a NEWS entry, and thanks.
>
> There is now a NEWS entry in emacs-29. Do modify it as you like.

It seems like that was all that needed doing here, so I'm closing this.