bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Jari Aalto]

Package: emacs
Version: 23.2+1-4
Severity: serious

The current definition of dolist in subr.el breaks lot of things that rely
on the standard 'dolist' behavior:

    $ emacs -Q --debug-init --batch --eval "(dolist (elt '(1 2)) (return elt))"
    Symbol's function definition is void: return

    $ macs -Q -l cl-macs --debug-init --batch --eval "(dolist (elt '(1 2)) (return elt))"
    <no errors>

Please update subr.el::dolist in respect to 'return'.

Jari

-- System Information
Debian Release: squeeze/sid
  APT Prefers testing
  APT policy: (990, testing) (500, unstable) (1, experimental)
Architecture: amd64
Kernel: Linux picasso 2.6.32-5-amd64 #1 SMP Fri Sep 17 21:50:19 UTC 2010 x86_64 GNU/Linux
Locale: LANG=en_DK.UTF-8

-- Versions of packages `emacs depends on'.
Depends:
emacs23         23.2+1-4        GNU Emacs is the extensible self-documenting 
emacs23-lucid   23.2+1-4        GNU Emacs is the extensible self-documenting 
emacs23-nox     23.2+1-4        GNU Emacs is the extensible self-documenting 

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Stefan Monnier]

> The current definition of dolist in subr.el breaks lot of things that rely
> on the standard 'dolist' behavior:

>     $ emacs -Q --debug-init --batch --eval "(dolist (elt '(1 2)) (return elt))"
>     Symbol's function definition is void: return

By standard, you mean "CL", right?  If you want CL behavior, then do
(require 'cl), and it will then work.


        Stefan

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Lennart Borgman]

On Mon, Nov 15, 2010 at 6:22 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> The current definition of dolist in subr.el breaks lot of things that rely
>> on the standard 'dolist' behavior:
>
>>     $ emacs -Q --debug-init --batch --eval "(dolist (elt '(1 2)) (return elt))"
>>     Symbol's function definition is void: return
>
> By standard, you mean "CL", right?  If you want CL behavior, then do
> (require 'cl), and it will then work.


So this is one of the reasons you should do

   (eval-when-compile (require 'cl))

Or? ;-)

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Stefan Monnier]

>>> The current definition of dolist in subr.el breaks lot of things that rely
>>> on the standard 'dolist' behavior:
>> 
>>>     $ emacs -Q --debug-init --batch --eval "(dolist (elt '(1 2)) (return elt))"
>>>     Symbol's function definition is void: return
>> 
>> By standard, you mean "CL", right?  If you want CL behavior, then do
>> (require 'cl), and it will then work.


> So this is one of the reasons you should do

>    (eval-when-compile (require 'cl))

> Or? ;-)

Don't know about "should", but it can be one of the cases where you
might want to do that, yes.  Of course, you can also change your code to
use catch&throw instead (which is what's used by `return' under the
hood, AFAIK).


        Stefan

bug#7408: closed (Re: bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return)

[jari]

On 2010-11-15 17:33, GNU bug Tracking System wrote:
| Your bug report
| 
| #7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return
| 
| which was filed against the emacs package, has been closed.
| 
| The explanation is attached below, along with your original report.
| If you require more details, please reply to 7408@debbugs.gnu.org.

The 'dolist' has existed long before subr.el::dolist was written. Code
that uses dolist may expect "standard" CL behavior of, because that is
what has been previously available in Emacs.

I do not find it appropriate to close the bug report, because the
problem still exists:

	There are two definitions of dolist, which are not compatible
	in current Emacs

Jari

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Glenn Morris]

> 	There are two definitions of dolist, which are not compatible
> 	in current Emacs

subr.el has had a dolist definition since at least Emacs 21.x; ie 9 years.
Therefore this cannot be a major issue in practice.

bug#7408: Linux patchutils: Development of the project?

[MON KEY]

> Show me a piece of code which would work with CL's dolist (but without
> CL's return) and yet doesn't work with subr.el's dolist.

Well, FWIW following is the toplevel/main interface to my elisp
implementation of Steel Bank Common Lisp's `map1' which in turn is
that implementation's high-level driver for its portable Common Lisp
mapping procedures.

Among other things, my port to elisp required adding a cath/throw
inside of a dolist b/c Elisp's `dolist' doesn't have `return'.

Indeed, I was as surprised by the same `dolist' disconnect which Jari
is currently poking at. What I found troubling was the process
required to locate the locus of this disconnect...  In so much as
Elisp code transitions silently between the two versions of dolist it
should do transparently.

BTW - the impetus for this routine was w/re incorporating more cl seq
fun into core but without the keyword overhead...

;;; :COURTESY SBCL :FILE sbcl/src/code/list.lisp
(defun mon-map1 (fun-designator original-arglists accumulate take-car)
  (unless
      (functionp fun-designator)
    (error (concat ":FUNCTION `mon-map1' "
                   "-- arg FUN-DESIGNATOR does not satisfy `functionp'")))
  (let* ((mmp1-arg-lsts (mon-copy-list-mac original-arglists))
         (mmp1-rtn-list (list nil))
         (mmp1-tmp mmp1-rtn-list))
    (do ((mmp1-rslt nil)
         (mmp1-args '() '()))
        ((catch 'is-null ;; :ADDED
           (dolist (mmp1-thrw mmp1-arg-lsts nil)
             (when (null mmp1-thrw) ;; :WAS (return t)))
               (throw 'is-null t))))
         (if accumulate
             (cdr mmp1-rtn-list)
           (car original-arglists)))
      (do ((mmp1-arg-l mmp1-arg-lsts (cdr mmp1-arg-l)))
          ((null mmp1-arg-l))
        (push (if take-car (caar mmp1-arg-l) (car mmp1-arg-l)) mmp1-args)
        (setf (car mmp1-arg-l) (cdar mmp1-arg-l)))
      (setq mmp1-rslt
            (apply fun-designator (nreverse mmp1-args)))
      (case accumulate
        (:nconc (setq mmp1-tmp (last (nconc mmp1-tmp mmp1-rslt))))
        (:list (setcdr mmp1-tmp (list mmp1-rslt))
               (setq mmp1-tmp (cdr mmp1-tmp)))))))

bug#7408: Linux patchutils: Development of the project?

[Jari Aalto]

Glenn Morris <rgm <at> gnu.org>
> subr.el has had a dolist definition since at least Emacs 21.x; ie 9 years.
> Therefore this cannot be a major issue in practice.

In my book it is still a bug, no matter how many years ago this bug were
introduced:

    - To have two different implementations of same function.
    - To not be able to rely on uniform behavior.

Jari

bug#7408: Linux patchutils: Development of the project?

[Stefan Monnier]

>> subr.el has had a dolist definition since at least Emacs 21.x; ie 9 years.
>> Therefore this cannot be a major issue in practice.
> In my book it is still a bug, no matter how many years ago this bug were
> introduced:

>     - To have two different implementations of same function.
>     - To not be able to rely on uniform behavior.

You look at it the wrong way: the problem is not with dolist, it's that
you use `return' which is a special form that's not defined in standard
Elisp.  After (require 'cl), `return' gets defined and things work as
you expect.


        Stefan

bug#7408: Linux patchutils: Development of the project?

[Štěpán Němec]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> subr.el has had a dolist definition since at least Emacs 21.x; ie 9 years.
>>> Therefore this cannot be a major issue in practice.
>> In my book it is still a bug, no matter how many years ago this bug were
>> introduced:
>
>>     - To have two different implementations of same function.
>>     - To not be able to rely on uniform behavior.
>
> You look at it the wrong way: the problem is not with dolist, it's that
> you use `return' which is a special form that's not defined in standard
> Elisp.  After (require 'cl), `return' gets defined and things work as
> you expect.

While I don't share Jari's complaint in general (i.e., the approach "if
you only want the subr.el dolist, you don't care if it's really CL
dolist you end up calling; if you want the CL dolist, you require 'cl
and are done with it" has served me acceptably well), I've always
wondered what the hell was the person adding another function of the
same name thinking. To have a library that clobbers existing definitions
is a no no even outside Emacs, let alone in the core.

Is the explanation (I'm not familiar with the history) that at the time
cl.el was added there was no dolist in core Emacs, so there was no
perceived need to call it dolist* as in other similar cases (mapcar*,
defun* etc)? (In that case my sincere disdain would go for the person
who introduced dolist into subr.el later without addressing the naming
clash.)

Same with `dotimes'.

  Štěpán

bug#7408: Linux patchutils: Development of the project?

[Stefan Monnier]

> Is the explanation (I'm not familiar with the history) that at the time
> cl.el was added there was no dolist in core Emacs, so there was no
> perceived need to call it dolist* as in other similar cases (mapcar*,
> defun* etc)? (In that case my sincere disdain would go for the person
> who introduced dolist into subr.el later without addressing the naming
> clash.)

AFAIK subr.el's dolist and dotimes are 100% compatible with
CL's definition.  Of course, if you want to call `return' in there,
you'll need to define `return', which is only provided by CL so you need
to (require 'cl).

I'd reconsider it only if someone can show me reasonable code that works
without error both with CL loaded and without CL loaded and yet returns
different results.


        Stefan

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Jari Aalto]

2010-11-17 15:39 Stefan Monnier <monnier@IRO.UMontreal.CA>:
>> Is the explanation (I'm not familiar with the history) that at the time
>> cl.el was added there was no dolist in core Emacs, so there was no
>> perceived need to call it dolist* as in other similar cases (mapcar*,
>> defun* etc)? (In that case my sincere disdain would go for the person
>> who introduced dolist into subr.el later without addressing the naming
>> clash.)
>
> AFAIK subr.el's dolist and dotimes are 100% compatible with
> CL's definition.  Of course, if you want to call `return' in there,
> you'll need to define `return', which is only provided by CL so you need
> to (require 'cl).
>
2010-11-17 15:39 Stefan Monnier <monnier@IRO.UMontreal.CA>:
>> The 'return' has been integral part of 'dolist' since the start.
>
> Who cares?

There happens to be people that care.

> Show me a piece of code which would work with CL's dolist (but without
> CL's return) and yet doesn't work with subr.el's dolist.

As per request:

    - "if you want to call `return' in there, you'll need to define `return"
    - "dolist and dotimes are 100% compatible with CL's definition."

Jari

    $ emacs -Q

    (progn
      (autoload 'return "cl-macs" nil nil 'macro)
      (dolist (elt '(1 2))
        (return elt)))

    Debugger entered--Lisp error: (no-catch --cl-block-nil-- 1)
      cl-block-throw(--cl-block-nil-- 1)
      (return-from nil elt)
      (return elt)
      (while --dolist-tail-- (setq elt (car --dolist-tail--)) (return elt) (setq --dolist-tail-- (cdr --dolist-tail--)))
      (let ((--dolist-tail-- ...) elt) (while --dolist-tail-- (setq elt ...) (return elt) (setq --dolist-tail-- ...)))
      (dolist (elt (quote ...)) (return elt))
      (progn (autoload (quote return) "cl-macs" nil nil (quote macro)) (dolist (elt ...) (return elt)))
      eval((progn (autoload (quote return) "cl-macs" nil nil (quote macro)) (dolist (elt ...) (return elt))))
      eval-last-sexp-1(nil)
      eval-last-sexp(nil)
      call-interactively(eval-last-sexp nil nil)

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Stefan Monnier]

>     $ emacs -Q

>     (progn
>       (autoload 'return "cl-macs" nil nil 'macro)
>       (dolist (elt '(1 2))
>         (return elt)))

This causes cl-macs to be run at an unexpected time.  I.e. it's ruled
out for being a contrived example.  E.g. I'd be *really* surprised if it
were an example you bumped into before this discussion.


        Stefan

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[jari]

On 2010-11-21 00:45, Stefan Monnier wrote:
| >     $ emacs -Q
| 
| >     (progn
| >       (autoload 'return "cl-macs" nil nil 'macro)
| >       (dolist (elt '(1 2))
| >         (return elt)))
| 
| This causes cl-macs to be run at an unexpected time.  I.e. it's ruled
| out for being a contrived example. 

Is the above code not valid? Does it not do what it is supposed to do;
to define `return' when it will be used for the first time?

| E.g. I'd be *really* surprised if it were an example you bumped into
| before this discussion.

Surprise or not, it demonstrates the defiency of claimed:

    - "dolist and dotimes are 100% compatible with CL's definition."

The question is not about "missing return", but differing dolist
implementation in subr.el

Jari

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Eli Zaretskii]

> Date: Sun, 21 Nov 2010 11:08:48 +0200
> From: jari <jari.aalto@cante.net>
> Cc: 7408@debbugs.gnu.org
> 
> On 2010-11-21 00:45, Stefan Monnier wrote:
> | >     $ emacs -Q
> | 
> | >     (progn
> | >       (autoload 'return "cl-macs" nil nil 'macro)
> | >       (dolist (elt '(1 2))
> | >         (return elt)))
> | 
> | This causes cl-macs to be run at an unexpected time.  I.e. it's ruled
> | out for being a contrived example. 
> 
> Is the above code not valid?

Breaking news: you can easily hang yourself with a rope made of valid
ELisp code.

bug#7408: 23.2.1 dolist -- subr.el and cl-macs differ with nil-block return

[Stefan Monnier]

> | >     $ emacs -Q
> | >     (progn
> | >       (autoload 'return "cl-macs" nil nil 'macro)
> | >       (dolist (elt '(1 2))
> | >         (return elt)))
> | This causes cl-macs to be run at an unexpected time.  I.e. it's ruled
> | out for being a contrived example. 
> Is the above code not valid?

Doesn't matter: lots of vlid Elisp code doesn't do what you'd expect.

> Does it not do what it is supposed to do;

Apparently it doesn't for you.

> to define `return' when it will be used for the first time?

The normal way to use CL features like `return' is with (require 'cl)
somewhere at the top-level.  Anything else is poor code that needs to
be improved.

> | E.g. I'd be *really* surprised if it were an example you bumped into
> | before this discussion.
> Surprise or not, it demonstrates the defiency of claimed:
>     - "dolist and dotimes are 100% compatible with CL's definition."

You don't need to teach me this.  I know full well that 100%
compatibility between two pieces of code in Elisp is *never* true,
unless the two are `eq'.


        Stefan