return

return

[Lars Magne Ingebrigtsen]

Wouldn't it be nice if Emacs Lisp had a workable early-return mechanism?

(This is where Andreas chimes in with "What's wrong with <foo>?")

While debugging and fixing stuff, I often find myself in the situation
of adding more if/cond statements to a function and pushing the original
body further in.  I think this usually leads to less clear code.

Like with 

(defun foo ()
  ... lots of code)

the options are usually

(defun foo ()
  (if zot
      t
    ... lots of code))

or (the horror!)

(defun foo ()
  (block nil
    (when zot
       (return t))
    ... lots of code))

or

(defun foo ()
  (if zot
      t
    (foo-1)))

(defun foo-1 ()
  ... lots of code)

and so on.  I think the ideal way to deal with this is to be more C-ish,
and say

(defun foo ()
  (when zot
    (return t))
  ... lots of code)

That is, I think it would probably be pretty nice if every defun
establishes a nil block.  So this wouldn't be a Common Lisp-ism, but
better!  :-)

(If we want to avoid confusion with the CL return, then we could call it
something else, like freturn.)

This isn't the only use case, of course.  A lot of functions in Emacs go
through pretty awkward contortions to end loops when certain conditions
occur.  A simple return from the middle of a complex series of loops can
often make the code a lot more readable.  (And probably faster.)

Thoughts?
  
-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: return

[Tassilo Horn]

On Friday 26 November 2010 09:57:21 Lars Magne Ingebrigtsen wrote:
> Wouldn't it be nice if Emacs Lisp had a workable early-return
> mechanism?

I guess so.  One option you didn't list is using catch/throw:

(defun foo (n)
  (catch 'val
    (dostuff)
    (when zot
      (throw 'val t))))

That's at least not something you need the cl package for.

Bye,
Tassilo

Re: return

[Miles Bader]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> That is, I think it would probably be pretty nice if every defun
> establishes a nil block.  So this wouldn't be a Common Lisp-ism, but
> better!  :-)
>
> (If we want to avoid confusion with the CL return, then we could call it
> something else, like freturn.)

Doesn't every defun in CL establish a nil block?

What you're proposing looks pretty much exactly like CL...

(by "CL", note that I mean "Common Lisp", not "cl.el")

Which is fine of course, CL's block mechanism is nice.

Of course one question is:  what about returns out of lambdas?
Would/should they be supported, given that they need an entirely
different mechanism internally?

-Miles

-- 
"An atheist doesn't have to be someone who thinks he has a proof that
there can't be a god.  He only has to be someone who believes that the
evidence on the God question is at a similar level to the evidence on
the werewolf question."  [John McCarthy]

Re: return

[Lars Magne Ingebrigtsen]

Miles Bader <miles@gnu.org> writes:

> Doesn't every defun in CL establish a nil block?

No, it establishes a named block, so you can say

(defun foo ()
  (when zot
    (return-from foo
      t))
  ... stuff)

Which is kinda annoying, in my opinion.  Especially if you have long
function names and a lot of return-from forms.  Or you rename function
names...

But that's also an option if we think it's better not to confuse Common
Lisp peeps. 
  
> Of course one question is:  what about returns out of lambdas?
> Would/should they be supported, given that they need an entirely
> different mechanism internally?

Hm...  I haven't thought much about that, but my immediate feeling is
that a syntactically clearer return functionality isn't as necessary in
lambda forms as in defuns.  Lambda forms are (in practise) quite short
and sweet, in my experience.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: return

[Tassilo Horn]

On Friday 26 November 2010 10:24:04 Miles Bader wrote:

Hi Miles,

> Doesn't every defun in CL establish a nil block?

No, it implicitly establishes a block named accordind to the "function
block name" of the function name, which in general is the function name.

(defun cl-fun ()
  (return "bing"))
==> *** - RETURN-FROM: no block named NIL is currently visible

(defun cl-fun ()
  (return-from cl-fun "bang"))
==> "bang"

Bye,
Tassilo

Re: return

[Miles Bader]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> No, it establishes a named block, so you can say
..
> Which is kinda annoying, in my opinion.  Especially if you have long
> function names and a lot of return-from forms.  Or you rename function
> names...

Hmm, kinda annoying or not, I think it would be silly to add a
slightly-incompatible form of return to elisp (regardless of whether it
uses a new name or not); it's just confusing, for a very small gain.

If we add this, I think we should just do so compatibly with CL, and
then change cl.el to use the new mechanism.

-Miles

-- 
Accordion, n. An instrument in harmony with the sentiments of an assassin.

Re: return

[Lars Magne Ingebrigtsen]

Miles Bader <miles@gnu.org> writes:

> Hmm, kinda annoying or not, I think it would be silly to add a
> slightly-incompatible form of return to elisp (regardless of whether it
> uses a new name or not); it's just confusing, for a very small gain.

I'd rather have return-from in Emacs Lisp than nothing, but I do think
the potential for confusion with, say, freturn, is pretty minimal.  It
does what you'd expect from almost all other languages -- return from
the current function.

I don't think this is a very important point, so I won't argue it
further (after this message), but if we have to possibility to add a new
language feature, I think we're slightly wasting an opportunity if we
don't make the new language feature as compelling as possible.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: return

[Stefan Monnier]

> While debugging and fixing stuff, I often find myself in the situation
> of adding more if/cond statements to a function and pushing the original
> body further in.  I think this usually leads to less clear code.

If this is only "while debugging and fixing stuff", I don't think it's
a serious enough problem to warrant such a change.  Note that my
preference to stick with the current situation in his respect is not
just out of inertia but also because I like a more functional style
of programming: usually/often you can find a formulation that's just as
elegant without the need for an early exit.

> This isn't the only use case, of course.  A lot of functions in Emacs go
> through pretty awkward contortions to end loops when certain conditions
> occur.

I tend to agree that early exit from loops would be sometimes welcome.
Currently, people either use catch/throw, or contortions, or just decide
not to exit early.  Also our while loops have no way to return a value,
so a `return' from a while loop can make the code *more* functional.
In functional programming language early exit constructs are rarely
needed because loops are written using recursion, so it's easy to exit
early: just don't recurse.  One constructs which would improve the
situation (without adding early exits) would be to add an `until': it's
just like `while' except that the condition is reversed *and* that the
non-nil final condition is returned as value of the `until' expression.
Very often (until (progn foo bar) baz) works well.  But I'd be OK with
adding a `return' from while loops: after all `while' loops are
naturally imperative, so such a `return' would not make things
less imperative.


        Stefan

Re: return

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> If this is only "while debugging and fixing stuff", I don't think it's
> a serious enough problem to warrant such a change.

Well, not, that was just what I was doing moments before writing the
email, so it just reminded me.  But the style of doing

(defun foo ()
  (unless condition
    (return bar))
  .. compute a bit more
  (unless other-condition
    (return zot))
  ... do the real foo work here)

feels kinda natural.  It's what some people calls "guard based
programming".  Without it, you end up with

(defun foo ()
  (if (not condition)
      bar
    .., compute a bit more
    (if (not other-condition)
        zot
      ... do the real work)))

where you have to read quite a lot of the code to find out what the exit
points are.  I feel like the former style often makes it easier to read
the code.
  
> Note that my preference to stick with the current situation in his
> respect is not just out of inertia but also because I like a more
> functional style of programming: usually/often you can find a
> formulation that's just as elegant without the need for an early exit.

Yeah, you're right, early returns aren't very functional in style.

> I tend to agree that early exit from loops would be sometimes welcome.

[...]

> But I'd be OK with adding a `return' from while loops: after all
> `while' loops are naturally imperative, so such a `return' would not
> make things less imperative.

while, dolist, dotimes...

The temptation to rewrite loops using the cl-macs `loop' macro can
sometimes be overwhelming, since you can return from the loop easier.
But using `loop' doesn't seem to be in the Emacs Lisp style, I think --
it's not used a lot, which kinda leads me to think that most Emacs Lisp
programmers don't really know how it works.

And it's not always contortion-free, either.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: return

[Stefan Monnier]

>> If this is only "while debugging and fixing stuff", I don't think it's
>> a serious enough problem to warrant such a change.
> Well, not, that was just what I was doing moments before writing the
> email, so it just reminded me.  But the style of doing

> (defun foo ()
>   (unless condition
>     (return bar))
>   .. compute a bit more
>   (unless other-condition
>     (return zot))
>   ... do the real foo work here)

> feels kinda natural.  It's what some people calls "guard based
> programming".  Without it, you end up with

> (defun foo ()
>   (if (not condition)
>       bar
>     .., compute a bit more
>     (if (not other-condition)
>         zot
>       ... do the real work)))

> where you have to read quite a lot of the code to find out what the exit
> points are.  I feel like the former style often makes it easier to read
> the code.

Of course, that's what `cond' is for.  Now, I know it doesn't always cut
it, but I'm still unconvinced that the rare cases where it would really
improve the code are worth all the cases where people will end up using
it instead of using a cleaner functional style.

>> But I'd be OK with adding a `return' from while loops: after all
>> `while' loops are naturally imperative, so such a `return' would not
>> make things less imperative.
> while, dolist, dotimes...

Yes, same thing.

> The temptation to rewrite loops using the cl-macs `loop' macro can
> sometimes be overwhelming, since you can return from the loop easier.

You can try the `do' macro, which is not as flexible as `loop', but does
correspond fairly simply to a functional recursive loop.  We could also
provide (and encourage the use of) Scheme's named-let, although it tends
to be "too powerful" (not inherently tail-recursive, so making it
efficient (CPU-wise and stack-space-wise) would require a good bit more
work).

> But using `loop' doesn't seem to be in the Emacs Lisp style, I think --
> it's not used a lot, which kinda leads me to think that most Emacs Lisp
> programmers don't really know how it works.

Indeed.


        Stefan

Re: return

[MON KEY]

> just out of inertia but also because I like a more functional style
> of programming: usually/often you can find a formulation that's just
> as elegant without the need for an early exit.

I don't see how (of itself) this preference is sufficiently applicable.

W/re the current inertia of the Emacs lisp VM, it seems whatever
"elegance" gained by the "functional" style it provides is per the
dynamic scoping, and even so it still _requires_ dynamic non-local
exits with catch/throw.

How do you anticipate reconciling this preference in terms of lexbind
integration? The impression I'm given from the CL ANSI spec is that
the `block'/`return-from' special forms (despite their "non-elegance")
were desirable/needed in lieu of dynamic shadowing other dynamic
shadowing concerns w/re transfer of control on exit, e.g.

Section 3.1.5 "Shadowing" of the Common Lisp ANSI spec:
(URL `http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/sec_3-1-5.html')

Note, in particular that the above should not be considered without
consideration of the impact this hs w/re to Common Lisps treatment of
Closures and Lexical Binding e.g. the immediately preceding section 3.1.4.

and the X3J13 Exit Extent Issue:

,----
|
|     The extent of an exit is not the same thing as the scope of the
|     designator by which the exit is identified. For example, a BLOCK
|     name has lexical scope but the extent of its exit is dynamic; the
|     scope of a CATCH tag could differ from the extent of the CATCH's
|     return point. (That's part of what is at issue here.)
|
`----
:SEE (URL `http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Issues/iss152-writeup.html')

In reading the second link above it's pretty clear that this issue had
an immediate affect on `unwind-protect'. Indeed, to the extent :P this
is so, discussion/dismissals of block/return-from should take into the
historic tension between Maclisp derived lisps dialects and the Scheme
derived lisp dialects around `unwind-protect'.

IOW, It almost certainly wasn't simply as a matter of personal
preference that the Common Lisp people incorporated and subsequently
debated the correct implementation of the things.

> We could also provide (and encourage the use of) Scheme's named-let,
> although it tends to be "too powerful" (not inherently
> tail-recursive, so making it efficient (CPU-wise and
> stack-space-wise) would require a good bit more work).

So, how then is `named-let' a somehow better/cleaner solution than an
alternative first-class implementation of a `block'/`return-from'
regime?

--
/s_P\

Re: return

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> (defun foo ()
>>   (unless condition
>>     (return bar))
>>   .. compute a bit more
>>   (unless other-condition
>>     (return zot))
>>   ... do the real foo work here)

[...]

> Of course, that's what `cond' is for.

Really?  You would write

(defun foo ()
  (cond
    ((not condition) bar)
    ((progn
      ..compute a bit more
      (not other-condition))
      zot)
    (t
     ... do the real foo work here)))

?

I think that's slightly, er, disgusting.  :-)

> You can try the `do' macro, which is not as flexible as `loop', but does
> correspond fairly simply to a functional recursive loop.

Nobody understands `do'.  The `do' syntax makes, well, everything else
look obvious.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: return

[Stefan Monnier]

> I think that's slightly, er, disgusting.  :-)

Well, I often find the "..compute a bit more" part, er, disgusting in
the first place anyway.


        Stefan

Re: return

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I think that's slightly, er, disgusting.  :-)
>
> Well, I often find the "..compute a bit more" part, er, disgusting in
> the first place anyway.

OK, I'll try harder in the future not to suggest Emacs-level features
(that seeming every other languages and environments have) while Emacs
shuffles towards a single-digit number of contributors.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: return

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> OK, I'll try harder in the future not to suggest Emacs-level features
> (that seeming every other languages and environments have)

As Tassilo Horn explained, catch/throw does exactly what you wanted.

Re: return

[Miles Bader]

Chong Yidong <cyd@stupidchicken.com> writes:
>> OK, I'll try harder in the future not to suggest Emacs-level features
>> (that seeming every other languages and environments have)
>
> As Tassilo Horn explained, catch/throw does exactly what you wanted.

I presume he wants something without the overhead tho...

-Miles

-- 
Run away!  Run away!

Re: return

[Chong Yidong]

Miles Bader <miles@gnu.org> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>>> OK, I'll try harder in the future not to suggest Emacs-level features
>>> (that seeming every other languages and environments have)
>>
>> As Tassilo Horn explained, catch/throw does exactly what you wanted.
>
> I presume he wants something without the overhead tho...

You mean the performance overhead from adding an extra internal_catch?
I doubt there's a free lunch here; adding a "return" or "return-from"
mechanism would also add overhead, and that overhead would apply to
every single funcall.  Still, it's a worthwhile experiment to implement
"return"/"return-from" and see how big the performance impact is.

Re: return

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

> You mean the performance overhead from adding an extra internal_catch?
> I doubt there's a free lunch here; adding a "return" or "return-from"
> mechanism would also add overhead, and that overhead would apply to
> every single funcall.  Still, it's a worthwhile experiment to implement
> "return"/"return-from" and see how big the performance impact is.

I did a quick experiment, and turns out built-in blocking is a little
faster than an explicit `catch', mostly because of reduced consing.  I
tested with a function that runs 500,000 tight `while' loops:

  (defun test-loop-with-catch ()
    (dotimes (ii 500000)
      (let ((ll '(1 2 3 4 5 6 7 8 9 10)))
        (catch 'exit
	  (while ll (setq ll (cdr ll)))))))

The run time is 1.164s, as opposed to 1.084s with the `catch' omitted.
So an explicit `catch' adds about 10 percent to the run time.

If I hack Fwhile to perform a catch internally, the runtime for the test
function (with the `catch' omitted) is 1.057s, within the margin of
error of the unhacked Emacs.

This (very limited) test indicates that adding built-in support for
block, return, and return-from should have little performance impact.
(Though the block tags ought to use a specialized obarray instead of
what cl-macs.el does, which is to intern them as "--cl-block-%s--".)

Does anyone see a deeper problem with providing such functions?

Re: return

[Stefan Monnier]

>> You mean the performance overhead from adding an extra internal_catch?
>> I doubt there's a free lunch here; adding a "return" or "return-from"
>> mechanism would also add overhead, and that overhead would apply to
>> every single funcall.  Still, it's a worthwhile experiment to implement
>> "return"/"return-from" and see how big the performance impact is.

> I did a quick experiment, and turns out built-in blocking is a little
> faster than an explicit `catch', mostly because of reduced consing.  I
> tested with a function that runs 500,000 tight `while' loops:

>   (defun test-loop-with-catch ()
>     (dotimes (ii 500000)
>       (let ((ll '(1 2 3 4 5 6 7 8 9 10)))
>         (catch 'exit
> 	  (while ll (setq ll (cdr ll)))))))

> The run time is 1.164s, as opposed to 1.084s with the `catch' omitted.
> So an explicit `catch' adds about 10 percent to the run time.

> If I hack Fwhile to perform a catch internally, the runtime for the test
> function (with the `catch' omitted) is 1.057s, within the margin of
> error of the unhacked Emacs.

A few questions:
- how do you explain that Fwhile with internal catch is faster (1.057 <
  1.084) than without an internal catch?  Or is that what you mean by
  "within the margin of error"?
- You seem to be measuring time for the interpreted code, is that right?
  If so, I think it would be more interesting to measure the time for
  byte-code.

The little tests I've performed seem to indicate that for interpreted
code the extra `catch' doesn't make much of a difference, but for the
compiled version of your test, the difference is around 20%.

> This (very limited) test indicates that adding built-in support for
> block, return, and return-from should have little performance impact.
> (Though the block tags ought to use a specialized obarray instead of
> what cl-macs.el does, which is to intern them as "--cl-block-%s--".)

If we add support for C-like exit statements (break/return, that only
work locally and hence don't add any extra run-time cost) to the
byte-compiler, that would be nice.


        Stefan

Re: return

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

> This (very limited) test indicates that adding built-in support for
> block, return, and return-from should have little performance impact.
> (Though the block tags ought to use a specialized obarray instead of
> what cl-macs.el does, which is to intern them as "--cl-block-%s--".)
>
> Does anyone see a deeper problem with providing such functions?

Thinking about this some more, I do see a couple of snags.  First, the
`dotimes' and `dolist' macros use `while' internally, so if we introduce
a built-in `return' that exits from `while', it's hard to make that same
`return' exit correctly from `dotimes'/`dolist'.  One solution might be
to introduce something like `while-noblock' and use that in `dotimes'
and `dolist'.

Also, block tags should have lexical scope, so in order to implement
`block' properly we probably ought to wait for the lexical binding
changes.  I think it is currently possible to implement `return' for
exiting unnamed blocks, and leave `block'/`return-from' for the future,
but I dunno if doing it like that is worth our while.

Re: return

[Chong Yidong]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> A few questions:
> - how do you explain that Fwhile with internal catch is faster (1.057 <
>   1.084) than without an internal catch?  Or is that what you mean by
>   "within the margin of error"?
> - You seem to be measuring time for the interpreted code, is that right?
>   If so, I think it would be more interesting to measure the time for
>   byte-code.
>
> The little tests I've performed seem to indicate that for interpreted
> code the extra `catch' doesn't make much of a difference, but for the
> compiled version of your test, the difference is around 20%.

Hmm, yes, I was testing on interpreted code.  Here is a new test using a
byte-compiled file:

(defun test-loop-nocatch ()
  (dotimes (ii 2000000)
    (let ((ll '(1 2 3 4 5 6 7 8 9 10)))
      (while ll
	(setq ll (cdr ll))))))

Averaging over ten runs, this takes 1.351 +- 0.022s without an internal
catch, and 1.371 +- 0.034s with an internal catch.

Using the Emacs without an internal-catch, I tested on a byte-compiled
version of a function with a catch inserted manually into the loop:

(defun test-loop-catch ()
  (dotimes (ii 2000000)
    (let ((ll '(1 2 3 4 5 6 7 8 9 10)))
      (while ll
        (catch 'exit
          (setq ll (cdr ll)))))))

The result is 1.725 +- 0.033s.

The simplistic "internal catch" implementation I used is this:

*** src/eval.c	2010-10-26 22:23:09 +0000
--- src/eval.c	2010-12-03 22:59:10 +0000
***************
*** 1054,1065 ****
    return unbind_to (count, elt);
  }
  
! DEFUN ("while", Fwhile, Swhile, 1, UNEVALLED, 0,
!        doc: /* If TEST yields non-nil, eval BODY... and repeat.
! The order of execution is thus TEST, BODY, TEST, BODY and so on
! until TEST returns nil.
! usage: (while TEST BODY...)  */)
!   (Lisp_Object args)
  {
    Lisp_Object test, body;
    struct gcpro gcpro1, gcpro2;
--- 1054,1061 ----
    return unbind_to (count, elt);
  }
  
! Lisp_Object
! internal_while (Lisp_Object args)
  {
    Lisp_Object test, body;
    struct gcpro gcpro1, gcpro2;
***************
*** 1078,1083 ****
--- 1074,1090 ----
    return Qnil;
  }
  
+ DEFUN ("while", Fwhile, Swhile, 1, UNEVALLED, 0,
+        doc: /* If TEST yields non-nil, eval BODY... and repeat.
+ The order of execution is thus TEST, BODY, TEST, BODY and so on
+ until TEST returns nil.
+ usage: (while TEST BODY...)  */)
+   (Lisp_Object args)
+ {
+   int count = SPECPDL_INDEX ();
+   return internal_catch (Qexit, &internal_while, args);
+ }
+ 
  DEFUN ("macroexpand", Fmacroexpand, Smacroexpand, 1, 2, 0,
         doc: /* Return result of expanding macros at top level of FORM.
  If FORM is not a macro call, it is returned unchanged.

Re: return

[Stefan Monnier]

> Hmm, yes, I was testing on interpreted code.  Here is a new test using a
> byte-compiled file:

> (defun test-loop-nocatch ()
>   (dotimes (ii 2000000)
>     (let ((ll '(1 2 3 4 5 6 7 8 9 10)))
>       (while ll
> 	(setq ll (cdr ll))))))

> Averaging over ten runs, this takes 1.351 +- 0.022s without an internal
> catch, and 1.371 +- 0.034s with an internal catch.

The Fwhile function is not used in the byte-compiled version of the
`while' special form.  Instead, the byte-compiler turns it into a bunch
of byte-codes like test and jump to label.
So your "with an internal catch" version should perform 100% identically
in this test ;-)


        Stefan

Re: return

[Fren Zeee]

On Fri, Nov 26, 2010 at 1:19 AM, Tassilo Horn <tassilo@member.fsf.org> wrote:
> On Friday 26 November 2010 09:57:21 Lars Magne Ingebrigtsen wrote:
>> Wouldn't it be nice if Emacs Lisp had a workable early-return
>> mechanism?
>
> I guess so.  One option you didn't list is using catch/throw:
>
> (defun foo (n)
>  (catch 'val
>    (dostuff)
>    (when zot
>      (throw 'val t))))
>
> That's at least not something you need the cl package for.
>

Why not just throw the whole elisp and use CL to run emacs with
lexical scoping ?

What particular advantages do there accrue from the dyn scoped elisp
and by incrementally making fixes to this dinosaur ?

Franz Xe

Re: return

[Chong Yidong]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> The Fwhile function is not used in the byte-compiled version of the
> `while' special form.  Instead, the byte-compiler turns it into a bunch
> of byte-codes like test and jump to label.
> So your "with an internal catch" version should perform 100% identically
> in this test ;-)

Ah, I see.  I guess the key problem is the unacceptably large
performance difference, in byte compiled code, when we wrap `while' in
an additional `catch' (1.75s vs 1.4s without the `catch' in my test).
The `catch' byte operation does this:

    case Bcatch:
      {
        Lisp_Object v1;
        BEFORE_POTENTIAL_GC ();
        v1 = POP;
        TOP = internal_catch (TOP, Feval, v1);
        AFTER_POTENTIAL_GC ();
        break;
      }

So presumably it's the Feval that's killing us.  Hmm.

Re: return

[Stephen J. Turnbull]

Fren Zeee writes:

 > Why not just throw the whole elisp and use CL to run emacs with
 > lexical scoping ?

A better question might be "why not build elisp on top of a more
efficient extension language?"  Mike Sperber <sperber@xemacs.org> did
some experiments with Scheme48 several years ago that he claims
indicate that a Scheme48 implementation of elisp is quite competitive
with XEmacs Lisp (mostly written in C).  Experiments done in about
2000 indicated that the XEmacs virtual machine was about 15% faster
than Emacs's, and 10% faster than Python's.  I suppose we *now* lag
behind the state of the art somewhat, but for a Scheme implementation
of Emacs Lisp to be competitive with XEmacs's C implementation at that
time suggests that a well-performing implementation of elisp in
another language suitable for use as an extension language for Emacs
is quite feasible (a "SMOP", as they say).

Note that Python (at least) has adopted this approach, with a core
language definition that is 99% or so independent of implementation
(cf. CPython, Cython nee pyrex, Stackless Python, Jython, IronPython,
and PyPy).

I doubt this will appeal to core Emacs hackers (I hear loud echoes of
YAGNI from the future, especially since Miles's "lexbind" branch is
scheduled for addition to Emacs 24 AIUI), but you might want to look
into it, at least try a couple of implementations of Python on the
same script a few times to see how little it could matter to users,
and then talk to Mike about somehow reviving his project or at least
getting advice (eg, if you prefer a different substrate language --
but I suspect Common Lisp would a lot more work to get decent
performance than a Scheme).

Speaking of Common Lisp, since it has a package scheme, you might be
able to implement the elisp interpreter by using the elisp
interpreter, and put in in a package that uses elisp eval instead of
Common Lisp's eval.  (Maybe I'm just smoking too much pakalolo. :-)

 > What particular advantages do there accrue from the dyn scoped elisp
 > and by incrementally making fixes to this dinosaur ?

Read Richard's screed about Emacs and its dependence on dynamic
scoping for why dynamic scoping is a feature in a language designed
for implementing editors.  (It's an MIT AI Lab working paper but it's
publicly available online somewhere.)  Of course like Common Lisp you
could default to lexical scoping and require declaration of
dynamically-scoped variables; Richard has said that on this list so
there's no disagreement on that point.  However, some form of dynamic
scoping is needed, and needed frequently even in modern Emacs,
according to him.  OTOH, AIUI he doesn't think dynamic scoping causes
problems all that often, lexical scoping is a bugaboo of minds that
have taken too many language design courses. :-)

The pragmatic advantage to reusing the code base based on the now-
archaic Emacs Lisp is that Emacs hackers rarely ;-) write code
designed to isolate use of dynamic scoping to where it's needed, so it
would be a huge effort to (1) find all those dynamically-scoped
variables, (2) identify the ones that can be lexical, (3) change them,
and (4) debug the product.

Like knowledge based on "obsolete" scientific paradigms, code
developed using "obsolete" programming paradigms is too valuable in
use to throw away.  It gets gradually replaced over time, and only
rarely "just because we can't bear to look at that any more!"  The
Emacs-Lisp-in-a-better-extension-language idea is attractive because
it makes that evolution possible.

Re: return

[Leo]

On 2010-12-04 06:18 +0000, Stephen J. Turnbull wrote:
> A better question might be "why not build elisp on top of a more
> efficient extension language?"

That would be many people's dream comes true. Looks like guile has
already implemented elisp. I wonder if the work to move emacs over guile
will take off after guile 2.0.

Cheers,
Leo

Re: return

[Helmut Eller]

* Chong Yidong [2010-12-03 22:44] writes:

> Also, block tags should have lexical scope, so in order to implement
> `block' properly we probably ought to wait for the lexical binding
> changes.  I think it is currently possible to implement `return' for
> exiting unnamed blocks, and leave `block'/`return-from' for the future,
> but I dunno if doing it like that is worth our while.

What's an "unnamed block"?  Is that the same as a block with name nil?

The compiler should recognize when a block tag doesn't escape and use
simple byte-goto instructions for that case.  Would that be difficult?

Helmut

Re: Return

[MON KEY]

,----
| Note that Python (at least) has adopted this approach, with a core
| language definition that is 99% or so independent of implementation
| (cf. CPython, Cython nee pyrex, Stackless Python, Jython, IronPython,
| and PyPy).
`----

Pshaw... Doesn't GVR eschew ++Lambda++ in p3k? Heresy!

,----
| Speaking of Common Lisp, since it has a package scheme, you might be
| able to implement the elisp interpreter by using the elisp
| interpreter, and put in in a package that uses elisp eval instead of
| Common Lisp's eval.
`----

Sam Steingold has explored this in the CLOCC.

There is also LICE.

Pretty sure Closure has an "Emacs like" editor for Mac environs.

PJB maintains quite a bit of Elisp<-->Common-Lisp code.

FWIW I've been collecting various bits and pieces from the above mentioned
sources into a personal package for the last ~6 months.

I'm sure many Common Lispers do similarly.

Also, as mentioned there is Climacs which is quite functional with McClim/CLX.

Likewise, there is Hemlock. (AIUI someone is currently porting it to Qt...)

Also, there are any number of legacy "Emacs like" editors from the
Lispm/TI/Symbolics/BBN/Xerox... implementations. Source is generally available
for all of these.

Contemporary commercial sources like Lispworks and Allegro provide "Emacs like"
editors.

Given Emacs' Maclisp lineage and the number of Lisp2 users whom actively use
GNU/Xemacs (or some equivalent) and the presence of a formal Common Lisp
specification implemented on myriad platforms and multiple
implementations of said
specification, why on earth should a switch to either a Scheme/Python VM warrant
consideration.

FWIW I find it quite odd that discussions of moving the Emacs VM to
Guile are bandied
about when Clisp is available as a GNU project (since forever), has an existing
C interface, is easily as portable across platformas as Guile scheme, counts
among its core authors a formidable contributor to various high-level
GNU projects
(Bruno Haible) and has the admirable quality that it doesn't get confused
between nil/empty-list/falsehood in the way that a Scheme will.

--
/s_P\

Re: Return

[Stephen J. Turnbull]

MON KEY writes:
 > ,----
 > | Note that Python (at least) has adopted this approach, with a core
 > | language definition that is 99% or so independent of implementation
 > | (cf. CPython, Cython nee pyrex, Stackless Python, Jython, IronPython,
 > | and PyPy).
 > `----
 > 
 > Pshaw... Doesn't GVR eschew ++Lambda++ in p3k? Heresy!

No, YAGNI.  Even more so in Emacs Lisp:

(defmacro lambda (&rest forms) `(defun ,(gensym) ,@forms))

or something like that. ;-)

 > ,----
 > | Speaking of Common Lisp, since it has a package scheme, you might be
 > | able to implement the elisp interpreter by using the elisp
 > | interpreter, and put in in a package that uses elisp eval instead of
 > | Common Lisp's eval.
 > `----
 > 
 > Sam Steingold has explored [...]

I'll take a look.  But ...

 > Also, there are any number of legacy "Emacs like" editors

Indeed.  For unacceptably small values of "like", though.  AFAIK the
people you would expect to actually be working on those things (eg,
you mention Bruno Haible as well as Sam Steingold) use "real" Emacs
instead for production work.  I wonder why? ;-)

 > why on earth should a switch to either a Scheme/Python VM warrant
 > consideration.

I'm not suggesting switching to the Python VM, I'm proposing Python as
an example of a language with several implementations targeting
multiple VMs.  The reason for using a Scheme is that it's much lighter
weight than a Common Lisp implementation.  That may not butter your
bread, but it matters to Richard, I believe, and to the people who
work on those implementations.

 > FWIW I find it quite odd that discussions of moving the Emacs VM to
 > Guile are bandied about

"Bandied about" is quite the odd term to use for a decision that AIUI
was made more than a decade ago.  Emacs just has a long gestation
period for features. :-)

Re: Return

[MON KEY]

On Sun, Dec 5, 2010 at 8:48 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
 >
 > Pshaw... Doesn't GVR eschew ++Lambda++ in p3k? Heresy!

,----
| No, YAGNI.  Even more so in Emacs Lisp:
`----

Indeed, one often doesn't miss what one never knew wasn't there to be missed.
GVM has no doubt leveraged this against future Python initiates.

Fortunately a good many lispers are too accustomed w/ lambda by virtue of
defmacro and Lisp's terse parenthetical syntax to not miss it.

,----
| (defmacro lambda (&rest forms) `(defun ,(gensym) ,@forms))
|
| or something like that. ;-)
`----

Perhaps, but quite a bit more happens at compile time around lambda and will do
more so if/when the lexbind is merged. :)

 > ,----
 > | Speaking of Common Lisp, since it has a package scheme, you might be
 > | able to implement the elisp interpreter by using the elisp
 > | interpreter, and put in in a package that uses elisp eval instead of
 > | Common Lisp's eval.
 > `----
 >
 > Sam Steingold has explored [...]

,----
| I'll take a look.  But ...
`----

FWIW I've found the old LispM sources to be the most enlightening.

 > Also, there are any number of legacy "Emacs like" editors

,----
| Indeed.  For unacceptably small values of "like", though.
|
| AFAIK the people you would expect to actually be working on those things (eg,
| you mention Bruno Haible as well as Sam Steingold) use "real" Emacs
| instead for production work.  I wonder why? ;-)
`----

I've no idea why either of those individuals might do what they do.

What would be interesting to learn is whether there is any will for a formal
incorporation of the C in Clisp w/ the E in elisp?

I imagine there are still a good many Common Lisp hacks that would contribute
their valuable skills to such an effort were it not for the closed/proprietary
control maintained on the FSF Emacs source tree.

Regardless, the "real" Emacsen we are using today are not fundamentally so far
removed from the legacy CLTL based systems they were once built upon that we can
describe the contemporary Emacsen as somehow more "real" than its ancestor.

Indeed, as you have framed it, mine was a comparison of Emacs to TECO. I would
offer that the situation may be inverted, and that in many ways todays Emacsen
are the TECOs to a certain breed of ancestral Emacsen.  No doubt you're well
aware of the lineage ;-)

Indeed, when browsing through the LispM sources it is quite surprising to learn
how much of the underlying Emacs VM (now C based) was once CLTL based. In some
cases one can still catch glimpes of entire blocks of text which nearly
superimpose one another (syntax differences aside)...

 > why on earth should a switch to either a Scheme/Python VM warrant
 > consideration.

,----
| I'm not suggesting switching to the Python VM, I'm proposing Python as
| an example of a language with several implementations targeting
| multiple VMs.
`----

Great! I misunderstood.

,----
| The reason for using a Scheme is that it's much lighter
| weight than a Common Lisp implementation.
`----

This is an awfully general assertion.
Which Scheme implementation?
Which Scheme with which R[456]RS?
And for any given implementation, which shared/linked libraries are required?
Which CL implementation?

,----
| That may not butter your bread, but it matters to Richard, I believe, and to
| the people who work on those implementations.
`----

Butter aside, it isn't at all clear what the matter that matters is and to and
for whom.

 > FWIW I find it quite odd that discussions of moving the Emacs VM to
 > Guile are bandied about

,----
| "Bandied about" is quite the odd term to use for a decision that AIUI
| was made more than a decade ago.
`----

Which decision, the decision to fork Emacs' CLTL code base from the LispM or the
decision that Guile might eventually grow into its shoes enough to become a
worthy consideration as the Emacs scripting engine?

,----
| Emacs just has a long gestation period for features. :-)
`----

And enough hubris to squelch many such features which should never have been
removed during the initial LispM fork 25+ yrs ago.

--
/s_P\

Re: Return

[Stephen J. Turnbull]

MON KEY writes:

 > Indeed, one often doesn't miss what one never knew wasn't there to be missed.
 > GVM has no doubt leveraged this against future Python initiates.

Not at all.  Python is intended to have functional programming
features, but it's also intended to primarily be an imperative
language, not a functional language.  GvR is well aware of the uses of
lambda and there are a lot of fans of lambda (as well as of "first-
class anonymous blocks" which may or may not be the same as lambdas)
among Pythonistas.  He just doesn't think such features belong in
Python, and I have to admit I don't miss them.

 > ,----
 > | (defmacro lambda (&rest forms) `(defun ,(gensym) ,@forms))
 > |
 > | or something like that. ;-)
 > `----
 > 
 > Perhaps, but quite a bit more happens at compile time around lambda and will do
 > more so if/when the lexbind is merged. :)

That was a wink because of course I'm cheating like hell: in a typical
interpreted Lisp, defun is just a feature-creepy way of associating a
lambda with a name (aka symbol), and my macro is an inefficient way of
creating an anonymous function.  IOW, no, nothing special happens at
compile time and no, lexbind won't change that.  Any semantic changes
that happen to lambda will happen to defun and vice versa.

 > What would be interesting to learn is whether there is any will for a formal
 > incorporation of the C in Clisp w/ the E in elisp?

Hrvoje Niksic and Martin Buchholz advocated a Common Lisp as the
appropriate way to evolve elisp, and IIRC Erik Naggum was in that camp
too.  But they've all long since left the development community.  I
think most of the people with an interest in it are either hacking
elisp directly (like Miles) or are Schemers (like Mike "R6RS"
Sperber).

 > the closed/proprietary control maintained on the FSF Emacs source
 > tree.

Hardly closed *or* proprietary.  Remember XEmacs in your prayers, and
rest assured that any work you do on Emacs remains free for others to
use, whether GNU chooses to distribute it or not.  If they choose not,
you can always do it yourself, as we do.

But ... I wouldn't bet that you'll have more luck peddling your warez
at xemacs.org or sxemacs.org for that matter.  That's the nature of a
distribution, that somebody decides what to distribute.  Typically by
rejecting your proposals, c'est la vie. :-)

Re: Return

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> MON KEY writes:
>
>  > Indeed, one often doesn't miss what one never knew wasn't there to
>  > be missed.
>  > GVM has no doubt leveraged this against future Python initiates.
>
> Not at all.  Python is intended to have functional programming
> features, but it's also intended to primarily be an imperative
> language, not a functional language.

If you take a look at Elisp programs, you'll notice that in practice it
is primarily used as an imperative language, not a functional language
(that is: the focus is on executing actions, not on constructing
values).  It doesn't really help that the fundamental data structure,
the list, is not an abstract data type but realised via a rather
low-level pointer to a statically allocated pair of user-accessible
values, cutting right through the idea of "functional programming" where
the output is a mathematic function/transformation of the input.

Lisp does not have separate syntaxes for imperative execution and
expressions (and things like the C dichotomy of if/else/?: are somewhat
ugly, especially considering that expressions count as statements if you
tack a semicolon on).  But particularly the Emacs code base is not all
that "functional".

-- 
David Kastrup

Re: return

[Davis Herring]

> + {
> +   int count = SPECPDL_INDEX ();
> +   return internal_catch (Qexit, &internal_while, args);
> + }

Just for education, what is the purpose of `count' here in the absence of
an `unbind_to'?

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: return

[Chong Yidong]

"Davis Herring" <herring@lanl.gov> writes:

>> + {
>> +   int count = SPECPDL_INDEX ();
>> +   return internal_catch (Qexit, &internal_while, args);
>> + }
>
> Just for education, what is the purpose of `count' here in the absence of
> an `unbind_to'?

It was a typo.

Re: Return

[Stefan Monnier]

> creating an anonymous function.  IOW, no, nothing special happens at
> compile time and no, lexbind won't change that.  Any semantic changes
> that happen to lambda will happen to defun and vice versa.

;-)

Actually, I'm considering to disallow non-top-level defuns in
lexical-scope mode, just because it's a good opportunity to introduce
such "breakage" and because non-top-level defuns are bugs in 99% of
the cases (in Elisp).


        Stefan

Re: Return

[Stefan Monnier]

> If you take a look at Elisp programs, you'll notice that in practice it
> is primarily used as an imperative language, not a functional language
> (that is: the focus is on executing actions, not on constructing
> values).  It doesn't really help that the fundamental data structure,
> the list, is not an abstract data type but realised via a rather
> low-level pointer to a statically allocated pair of user-accessible
> values, cutting right through the idea of "functional programming" where
> the output is a mathematic function/transformation of the input.

Actually, I think that the cons cells are relatively harmless in this
regard (setcar/setcdr are not used that often, tho maybe slightly more
so via delq and friends) compared to the blow incurred by the
inefficiency of Elisp recursion.


        Stefan

Re: Return

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Actually, I'm considering to disallow non-top-level defuns in
> lexical-scope mode, just because it's a good opportunity to introduce
> such "breakage" and because non-top-level defuns are bugs in 99% of
> the cases (in Elisp).

There are quite a number of packages, many external to Emacs, that use
non-top-level defuns for compatibility functions.

Re: Return

[Stefan Monnier]

>> Actually, I'm considering to disallow non-top-level defuns in
>> lexical-scope mode, just because it's a good opportunity to introduce
>> such "breakage" and because non-top-level defuns are bugs in 99% of
>> the cases (in Elisp).

> There are quite a number of packages, many external to Emacs, that use
> non-top-level defuns for compatibility functions.

But as long as they don't use lexical-scope it's a non-issue.


        Stefan

Re: Return

[MON KEY]

On Mon, Dec 6, 2010 at 2:20 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> MON KEY writes:
>
>  > Indeed, one often doesn't miss what one never knew wasn't there to be missed.
>  > GVM has no doubt leveraged this against future Python initiates.
>
>
> Not at all.  Python is intended to have functional programming
> features, but it's also intended to primarily be an imperative
> language, not a functional language.  GvR is well aware of the uses of
> lambda and there are a lot of fans of lambda (as well as of "first-
> class anonymous blocks" which may or may not be the same as lambdas)
> among Pythonistas.  He just doesn't think such features belong in
> Python, and I have to admit I don't miss them.
>

I'e no clue whether this is good or bad for Python(istas) my
concern is simply that this viewpoint not encroach upon the Emacs Lisp
VM without qualification. :)

In any event, AIUI much of GvR's objection w/re Python's lambda was
that it won't parse cleanly given the syntactic reliance on
whitespace.

Whichever, I was mostly pointing out that with lisp source we don't
rely on whitespace to formally convey syntax and that (with lisp2's at
least) there is ample opportunity to make use of imperative style
block/go/return/return-from forms as well... No doubt this is obvious
to most here but it is all too easily forgotten when discussion of
alternative non-lispy languages are brought to bare on the efficacy of
lispy languages.

>
>  > ,----
>  > | (defmacro lambda (&rest forms) `(defun ,(gensym) ,@forms))
>  > |
>  > | or something like that. ;-)
>  > `----
>  >
>  > Perhaps, but quite a bit more happens at compile time around lambda and will do
>  > more so if/when the lexbind is merged. :)
>
> That was a wink because of course I'm cheating like hell: in a typical
> interpreted Lisp, defun is just a feature-creepy way of associating a
> lambda with a name (aka symbol), and my macro is an inefficient way of
> creating an anonymous function.

Likely you know better than I and the wink wasn't lost on me. :)

However, as you suggest with your caveat "in a typical interpreted
Lisp" this may not apply to compiled code... Which is to say, doesn't
your example dance around what happens around defun/defmacro/lambda at
compile time in any lexically scoped implementation of
<substitute your non-interpreted lexically scoped modern lispy
dialect of choice here>?

> IOW, no, nothing special happens at
> compile time and no, lexbind won't change that.  Any semantic changes
> that happen to lambda will happen to defun and vice versa.
>
>  > What would be interesting to learn is whether there is any will for a formal
>  > incorporation of the C in Clisp w/ the E in elisp?
>
> {...}
> Hrvoje Niksic and Martin Buchholz advocated a Common Lisp as the
> appropriate way to evolve elisp, and IIRC Erik Naggum was in that camp
> too.  But they've all long since left the development community.

The slime users/developers are _actively_ incorporating Common Lisp
(of various implementations) with Emacs/Emacs lisp (unfortunately they
have to resort to a different RPC per implementation to do
so). Regardless, the Slime user/devel community is hardly an
insignificant group.

This said, I suspect most of them only actively engage elisp in lieu
of Common Lisp out of necessity not preference.

>
> I think most of the people with an interest in it are either hacking
> elisp directly (like Miles) or are Schemers (like Mike "R6RS"
> Sperber).
>

Don't forget the Guile Schemers whose efforts to incorporate the R6RS
with that implementation will eventually bring yet another Scheme that
much closer to Greenspun's 10th...

In which case, assuming further future integration of Guile with (GNU)
Emacs we might find that much of what the Common Lisp crowd has sought
for years will (finally) be made available to the Emacs VM.

If such does happen, it will be highly amusing to witness the clamor
by Schemers for elisp incorporation of `#:keywords-styled-thusly'
esp. when these too are dismissed as "ugly" , "inefficient to
implement", "confusing to users", etc.

Meanwhile burgeoning adepts of the GNU Findutils toolbox will no doubt
chuckle nervously at the thought of trying to invoke a `find' command
(with its 70+ flags) with only "By Order of Argument" at their
disposal...

>
>  > the closed/proprietary control maintained on the FSF Emacs source
>  > tree.
>
> Hardly closed *or* proprietary.

Tomato/Potato[1]

> Remember XEmacs in your prayers, and
> rest assured that any work you do on Emacs remains free for others to
> use, whether GNU chooses to distribute it or not.

No doubt. :)

>  If they choose not, you can always do it yourself, as we do.

Yes but there is an accord to maintain some mutual consensus. AIUI your
presence on this list is indicative of such an effort no?
To paraphrase JWZ,  "Now you have two problems."

>
> But ... I wouldn't bet that you'll have more luck peddling your warez
> at xemacs.org or sxemacs.org for that matter.
>

I'm not aware of peddling either here or there... what is occurring
here is more akin to carpet-bagging than to commerce in snake-oil.

>
> That's the nature of a distribution, that somebody decides what to distribute.
> Typically by rejecting your proposals, c'est la vie. :-)
>

If there is angst here it is not around the rejection of any
particular proposal (certainly not my own), but rather about the
persistent inability to engage in reasoned/meaningful/intentioned
discussion w/re incorporating of a particular set of Lisp features by
either ignoring and/or dismissing the utility these features do
otherwise provide both Emacs user community and the greater community
of Lisp dialect users despite a general acknowledgment by both of
these communities that the particular set of features are
wanted/needed and FTMP can be reasonably implemented.

[1] The long term evidence of this inability IMO suggests that the Emacs
project(s) are closed and proprietary project (whether their source be
free or not). That [SX]Emacs does remain reasonably compatible with
GNU Emacs suggest that it too abides this inability (whether tacitly
or otherwise).

--
/s_P\

Re: Return

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> Actually, I'm considering to disallow non-top-level defuns in
> lexical-scope mode, just because it's a good opportunity to introduce
> such "breakage" and because non-top-level defuns are bugs in 99% of
> the cases (in Elisp).

I presume by "non-top-level defun" you mean "defun inside a function",
not "defun inside a form"...

-Miles

-- 
97% of everything is grunge

Re: Return

[David Kastrup]

Miles Bader <miles@gnu.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> Actually, I'm considering to disallow non-top-level defuns in
>> lexical-scope mode, just because it's a good opportunity to introduce
>> such "breakage" and because non-top-level defuns are bugs in 99% of
>> the cases (in Elisp).
>
> I presume by "non-top-level defun" you mean "defun inside a function",
> not "defun inside a form"...

There are lots of reasons for doing a defun inside of a function.  One
important point of Lisp is making it easy to create code
programmatically.

I don't understand the "bugs in 99% of the cases", I could hardly
imagine any situation where a defun is used inside of a form
unintentionally, without the interpreter/compiler barfing anyway because
of unmatched parens and/or producing totally non-working code that is
straightforward to figure out.

-- 
David Kastrup

Re: Return

[Stephen J. Turnbull]

Stefan Monnier writes:

 > Actually, I'm considering to disallow non-top-level defuns in
 > lexical-scope mode, just because it's a good opportunity to introduce
 > such "breakage" and because non-top-level defuns are bugs in 99% of
 > the cases (in Elisp).

So I guess you don't expect to be adding to any hooks in lexical mode?

(add-hook captain-hook (defun sm-or-dak?-style-foo () (foo bar baz)))

;-)

Re: Return

[Stephen J. Turnbull]

MON KEY writes:

 > > [GvR] just doesn't think such [lambda or anon blocks] belong in
 > > Python, and I have to admit I don't miss them.
 > >
 > 
 > I'e no clue whether this is good or bad for Python(istas) my
 > concern is simply that this viewpoint not encroach upon the Emacs Lisp
 > VM without qualification. :)

No, it won't.  However, I must say (as a maintainer) that use of
lambdas often imposes extra work on debuggers.  I almost always give
lambdas a nickname when I'm reading code that uses them, unless
they're simply implementing partial evaluation.

 > In any event, AIUI much of GvR's objection w/re Python's lambda was
 > that it won't parse cleanly given the syntactic reliance on
 > whitespace.

Not at all.  That's an excuse that some lambda supporters will buy,
but he just doesn't think that the scope for appropriate use of
lambdas is that great, and especially not in Python.  He'd rather add
new specialized syntax for major use cases.  He's almost certainly
right for Python, and he doesn't claim any more generality than that.

 > No doubt this is obvious to most here but it is all too easily
 > forgotten when discussion of alternative non-lispy languages are
 > brought to bare on the efficacy of lispy languages.

Proposing a non-Lispy language wasn't my intention at all.

 > However, as you suggest with your caveat "in a typical interpreted
 > Lisp" this may not apply to compiled code... Which is to say, doesn't
 > your example dance around what happens around defun/defmacro/lambda at
 > compile time in any lexically scoped implementation of
 > <substitute your non-interpreted lexically scoped modern lispy
 > dialect of choice here>?

I'm not sure what you think happens there.  All that defun ever does
(that matters to this discussion) is create a function object and
assign a name (a symbol) to it.  In a Lispy language when that symbol
is encountered as the car of a list to be evalled it is dereferenced
(the function cell in Lisp-2s and the value cell in Lisp-1s), and the
resulting function object applied to the arguments.  Otherwise the car
must be a function object.  And what is a function object?  Either a
lambda or equivalent compiled code (yes, Virginia, a list starting
with a subr can be eval'ed).

So all the interesting questions of lexical scope vs. compilation have
to do with argument passing (including the special semantics of the
environment "variable", however that is defined and implemented).
However, AFAICS this doesn't (and mustn't, otherwise you'd be able to
tell the difference between eval'ing interpreted code and eval'ing
compiled code, which is verboten) change depending on whether the
compiled code object was defined directly as a lambda expression, or
via defun.  If you know differently, I'd rather learn than teach. :-)

 > > But [the CL advocates] all long since left the [Emacs]
 > > development community.
 > 
 > The slime users/developers are _actively_ incorporating Common Lisp
 > (of various implementations) with Emacs/Emacs lisp (unfortunately they
 > have to resort to a different RPC per implementation to do
 > so). Regardless, the Slime user/devel community is hardly an
 > insignificant group.

Nobody says they are.  But unless they hang out here, which they only
seem to do when they've got a bug report, they're not going to have
much influence on the future of the language.  Given that RMS is
generally not in favor of incorporating Common Lisp features, I grant
that could be a huge investment of time, effort, code, and persuasive
engineering.  I'm not criticizing them, I'm telling you that my view
is Emacs Lisp will turn into an implementation of Common Lisp sometime
later than Real Soon Now.

 > >  If they choose not, you can always do it yourself, as we do.
 > 
 > Yes but there is an accord to maintain some mutual consensus. AIUI your
 > presence on this list is indicative of such an effort no?
 > To paraphrase JWZ,  "Now you have two problems."

One of my problems is an addiction to email.  The other is an
affection for David Kastrup.  That is what my presence here
indicates.

More to the point, I long ago gave up on effecting a mutual consensus.
We either do it Emacs's way, or we don't.  That's just the way it has
to be.  I advocate things here that I think would be good for Emacs on
Emacs's own terms.  I don't always get that right, but I'm definitely
not here to advocate the XEmacs way of thinking.

 > persistent inability to engage in reasoned/meaningful/intentioned
 > discussion w/re incorporating of a particular set of Lisp features
 > by either ignoring and/or dismissing the utility these features do
 > otherwise provide both Emacs user community and the greater
 > community of Lisp dialect users despite a general acknowledgment by
 > both of these communities that the particular set of features are
 > wanted/needed and FTMP can be reasonably implemented.

Frankly, I don't see any such general acknowledgement in the general
Emacs community.  There is a significant subset of developers who
would like that, yes.  But the non-developer users could care less,
and many developers either prefer a different style of language or
fear that incorporation of more features would lead to a deterioration
of performance for typical use-cases or instability in use, which
really isn't acceptable.

Take lexbind, for example.  It's been on the agenda for a long time,
and AFAIK RMS *never* objected to it in principle (at least once it
was made clear that dynamic scope would continue to be supported).
However, he does like dynamic scope, and insists that features like
lexbind be added in a way that doesn't get in the way of ordinary
users of existing Emacs features or cause the whole program to become
unstable.

 > [1] The long term evidence of this inability IMO suggests that the Emacs
 > project(s) are closed and proprietary project (whether their source be
 > free or not). That [SX]Emacs does remain reasonably compatible with
 > GNU Emacs suggest that it too abides this inability (whether tacitly
 > or otherwise).

All it proves is that we're lazy, and want to take advantage of the
improvements to *Emacs* technology more than we want to take advantage
of improvements to *Lisp* technology.  When all is said and done,
\(S?X\)?Emacs is as much an application as it is a language, and most
of our developers have far more interest in the application than the
language.

Eg, Aidan Kehoe has been busily improving our Common Lisp support,
even down to rewriting some of the docs (will wonders never cease?!) 
and using some standard set of tests for the features he's
adding/upgrading/conformancing.  Has he released any code that
*depends* on these features?  Not yet, and I bet not soon.  Does he
release (non-CL-implementation) code?  Sure, an' he does.  But it's
all bug fixes and conventional features.  You may think that's
unfortunate, but that's the way the world works.

Re: Return

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Stefan Monnier writes:
>
>  > Actually, I'm considering to disallow non-top-level defuns in
>  > lexical-scope mode, just because it's a good opportunity to introduce
>  > such "breakage" and because non-top-level defuns are bugs in 99% of
>  > the cases (in Elisp).
>
> So I guess you don't expect to be adding to any hooks in lexical mode?
>
> (add-hook captain-hook (defun sm-or-dak?-style-foo () (foo bar baz)))

Eek.  No point in using the return value of a defun (is it even
defined?)  like that if you can equally well use the function symbol.

-- 
David Kastrup

Re: Return

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> MON KEY writes:
>
>  > > [GvR] just doesn't think such [lambda or anon blocks] belong in
>  > > Python, and I have to admit I don't miss them.
>  > >
>  > 
>  > I'e no clue whether this is good or bad for Python(istas) my
>  > concern is simply that this viewpoint not encroach upon the Emacs Lisp
>  > VM without qualification. :)
>
> No, it won't.  However, I must say (as a maintainer) that use of
> lambdas often imposes extra work on debuggers.  I almost always give
> lambdas a nickname when I'm reading code that uses them, unless
> they're simply implementing partial evaluation.

A nickname doesn't really apply well with closures.  For debugging
purposes, a "named lambda" might be helpful.  Conceivably one could
supply something like that using a DOC string.

> Nobody says they are.  But unless they hang out here, which they only
> seem to do when they've got a bug report, they're not going to have
> much influence on the future of the language.  Given that RMS is
> generally not in favor of incorporating Common Lisp features,

I don't think anybody minds the features.  The problem is the cost in
syntactic complexity and language intransparency.  cl is a large and
invasive hack that is not particularly nice during debugging,
decompiling and code design and optization.  lexbind is a first step in
a direction where some Common Lisp features become less costly.

>  > Yes but there is an accord to maintain some mutual consensus. AIUI
>  > your presence on this list is indicative of such an effort no?  To
>  > paraphrase JWZ, "Now you have two problems."
>
> One of my problems is an addiction to email.  The other is an
> affection for David Kastrup.  That is what my presence here indicates.

Doing all the right things for ostensibly awfully bad reasons sounds
like a form of compassionate nihilism designed to get out of the line of
appreciation.  After all, it could get you into serious trouble at
/home/xemacs if somebody were as foolish as to, say, nominate you for
the Free Software Award for minimizing, for whatever reason, the
consequences of the great schism without losing face or focus.  And it
would look decidedly fishy if I were to do such a thing after this
announcement.

> More to the point, I long ago gave up on effecting a mutual consensus.
> We either do it Emacs's way, or we don't.  That's just the way it has
> to be.  I advocate things here that I think would be good for Emacs on
> Emacs's own terms.  I don't always get that right, but I'm definitely
> not here to advocate the XEmacs way of thinking.

You are advocating the Turnbull way of thinking, here and "there".  At
least here, mixed success is nothing to be worried about.

> Frankly, I don't see any such general acknowledgement in the general
> Emacs community.  There is a significant subset of developers who
> would like that, yes.  But the non-developer users could care less,
> and many developers either prefer a different style of language or
> fear that incorporation of more features would lead to a deterioration
> of performance for typical use-cases or instability in use, which
> really isn't acceptable.

I see much more the danger in bit rot.  I've been considering helping
with a dedicated lilypond-mode building on lyqi.el (there is a git repo
for that).  The code is an exercise in cl use and makes my eyes glaze
over.  I can't sensibly contribute, because it is utterly beyond me.
Many times in Emacs programming, I give up on the documentation and just
dig through the sources to see what something does deep down.

cl does not lend itself to that approach.  Its code is highly complex
and utterly underdocumented.  You have to trust it to do the right
thing.  I hate doing that.

-- 
David Kastrup

Re: Return

[Stephen J. Turnbull]

David Kastrup writes:
 > "Stephen J. Turnbull" <stephen@xemacs.org> writes:

 > > (add-hook captain-hook (defun sm-or-dak?-style-foo () (foo bar baz)))

In view of David's reaction, it looks like

    (add-hook 'captain-hook (defun sm-style-foo () foo bar baz))

is a better guess, and fixes a syntax error in passing.

 >  > Eek.  No point in using the return value of a defun (is it even
 > defined?)  like that if you can equally well use the function symbol.

Better yet, use both, which is what that idiom does.  (defun returns
the defun'ed function symbol.)

(defun sm-style-foo () foo bar baz)
=> sm-style-foo

I use this in my init file in contexts like

(add-hook 'text-mode-hook
          ;; yes I know this function is already defined
          (defun auto-fill-mode-on () (auto-fill-mode 1)))

Note that the defun serves a real purpose here; it makes the init file
"just enough" idempotent in the face of changes to the hook function
being added.  If that were a lambda, not only would the definition of
the hook function be changed, but the original definition would remain
on the hook.

Embedding the defun in the argument might be considered excessively
cute.

Re: Return

[Stephen J. Turnbull]

David Kastrup writes:

 > I don't think anybody minds the features.

IIRC rms has recently declared his dislike for CL-style keyword
arguments.  I suppose that's part of the "syntactic complexity" you
mention, but MON KEY OTOH points out cases where he'd like to use
them.  So there are some fundamental disagreements here.

There are also people who would like full-blown CL conformance in
Emacs.  You could argue they could get that with Hemlock, but then
they don't get full-blown Emacs conformance.  (I'm not sure why they
think it would be easier to turn Emacs Lisp into Common Lisp, than to
turn Hemlock into a fully compatible implementation of GNU Emacs, but
there you have it.)

 > cl does not lend itself to that approach.  Its code is highly complex
 > and utterly underdocumented.  You have to trust it to do the right
 > thing.  I hate doing that.

I think everybody agrees with all of those points, except that some
people are happy to trust now and debug later.

Re: Return

[Stefan Monnier]

>>> Actually, I'm considering to disallow non-top-level defuns in
>>> lexical-scope mode, just because it's a good opportunity to introduce
>>> such "breakage" and because non-top-level defuns are bugs in 99% of
>>> the cases (in Elisp).
>> I presume by "non-top-level defun" you mean "defun inside a function",
>> not "defun inside a form"...

Yes.

> There are lots of reasons for doing a defun inside of a function.  One
> important point of Lisp is making it easy to create code
> programmatically.

There's `fset' for that.

> I don't understand the "bugs in 99% of the cases", I could hardly
> imagine any situation where a defun is used inside of a form
> unintentionally,

That's because you understand Elisp.  many Elisp programmers don't.


        Stefan

Re: Return

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Dec 08, 2010 at 01:14:34AM +0900, Stephen J. Turnbull wrote:

[...]

> Embedding the defun in the argument might be considered excessively
> cute.

Hey, but thanks for the idea, anyway :-)

- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFM/mq2Bcgs9XrR2kYRAtMUAJ0cL5nQ88DXknGGv0mTNJ+9bg1wxACfQC0D
6juOh8SKxd8Kv3lHxccg3fw=
=ZukB
-----END PGP SIGNATURE-----

Re: Return

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> Actually, I'm considering to disallow non-top-level defuns in
>>>> lexical-scope mode, just because it's a good opportunity to introduce
>>>> such "breakage" and because non-top-level defuns are bugs in 99% of
>>>> the cases (in Elisp).
>>> I presume by "non-top-level defun" you mean "defun inside a function",
>>> not "defun inside a form"...
>
> Yes.
>
>> There are lots of reasons for doing a defun inside of a function.  One
>> important point of Lisp is making it easy to create code
>> programmatically.
>
> There's `fset' for that.
>
>> I don't understand the "bugs in 99% of the cases", I could hardly
>> imagine any situation where a defun is used inside of a form
>> unintentionally,
>
> That's because you understand Elisp.  many Elisp programmers don't.

I think you are confusing "unintentionally" with "unwisely".  I think it
is presumptuous to assume as a compiler writer that one can make a
compiler make better decisions than any other programmer, without even
knowing the problem in advance.  That's somewhat above believing to be
able to write a program passing the Turing test.

-- 
David Kastrup

Re: Return

[MON KEY]

On Tue, Dec 7, 2010 at 9:34 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> MON KEY writes:
>
> No, it won't.  However, I must say (as a maintainer) that use of
> lambdas often imposes extra work on debuggers.  I almost always give
> lambdas a nickname when I'm reading code that uses them, unless
> they're simply implementing partial evaluation.

I'm sorry, but it isn't clear, for which language?

> Proposing a non-Lispy language wasn't my intention at all.
>

OK, great!

>  > your example dance around what happens around defun/defmacro/lambda at
>  > compile time in any lexically scoped implementation of
>  > <substitute your non-interpreted lexically scoped modern lispy
>  > dialect of choice here>?
>
> I'm not sure what you think happens there.

The question is (apropos the above) when will what happen where? ;-)

> All that defun ever does (that matters to this discussion) is create a
> function object and assign a name (a symbol) to it.

AFAIK the discussion is still re return/return-from/block and presumably by
proxy go/tagbody etc. with the ostensible lexbind sprinkled on top.

>
> compiled code, which is verboten) change depending on whether the
> compiled code object was defined directly as a lambda expression, or
> via defun.  If you know differently, I'd rather learn than teach. :-)
>

Thank you for taking the time to communicate your knowledge.

I am curious to learn how you understand all of what you say re lexcial scoping
to jibe with lambda, closures, and macroexpansion at compile time, and more
particularly with consideration as to why Common Lisp's `lambda-lists-keywords'
contains an &environment and &whole?

>
> Frankly, I don't see any such general acknowledgement in the general
> Emacs community.
>

Most likely those who might otherwise caucus more vocally have left
with a whimper.

--
/s_P\

Re: Return

[Samuel Bronson]

On Mon, Dec 6, 2010 at 9:42 PM, MON KEY <monkey@sandpframing.com> wrote:
> On Mon, Dec 6, 2010 at 2:20 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
>> MON KEY writes:

>>  > What would be interesting to learn is whether there is any will for a formal
>>  > incorporation of the C in Clisp w/ the E in elisp?
>>
>> {...}
>> Hrvoje Niksic and Martin Buchholz advocated a Common Lisp as the
>> appropriate way to evolve elisp, and IIRC Erik Naggum was in that camp
>> too.  But they've all long since left the development community.
>
> The slime users/developers are _actively_ incorporating Common Lisp
> (of various implementations) with Emacs/Emacs lisp (unfortunately they
> have to resort to a different RPC per implementation to do
> so). Regardless, the Slime user/devel community is hardly an
> insignificant group.

I thought they used a common CL-side codebase known as "swank" for all
of the Common Lisp implementations, and that something else was only
needed when talking with some other kind of Lisp (or a non-Lisp)?

> This said, I suspect most of them only actively engage elisp in lieu
> of Common Lisp out of necessity not preference.


>>  > the closed/proprietary control maintained on the FSF Emacs source
>>  > tree.
>>
>> Hardly closed *or* proprietary.
>
> Tomato/Potato[1]

Obviously the meaning here was not precisely "the opposite of Open
Source / Free Software", which *is* what those words most often mean
when they describe software in these parts, but note that here they
describe the *control* over a *particular* source tree, which is a
distinctly different thing, though many effects may be similar.

(Thankfully, not all: no matter how much of MS's .NET Framework source
code may be available for me to (hypothetically) single-step through,
I doubt I will ever be permitted to fork it or incorporate it into my
own programs beyond trivial-fair-use copy/paste/modify of tiny bits.
Not that I use .NET or anything; it's just the only example of a
totally proprietary codebase which nevertheless has more-or-less
publicly-visible code that I could think of -- where public == people
with access to Windows systems w/ sufficient free hard drive space to
install a recent enough Visual Foo Express Edition (at least, I think
it's supposed to work on Express Editions), admittedly.)

>> Remember XEmacs in your prayers, and
>> rest assured that any work you do on Emacs remains free for others to
>> use, whether GNU chooses to distribute it or not.
>
> No doubt. :)
>
>>  If they choose not, you can always do it yourself, as we do.
>
> Yes but there is an accord to maintain some mutual consensus. AIUI your
> presence on this list is indicative of such an effort no?
> To paraphrase JWZ,  "Now you have two problems."

You mean, maintaining the feature and dealing with the pain that is
non-core autoloads?

>> But ... I wouldn't bet that you'll have more luck peddling your warez
>> at xemacs.org or sxemacs.org for that matter.
>>
>
> I'm not aware of peddling either here or there... what is occurring
> here is more akin to carpet-bagging than to commerce in snake-oil.
>
>>
>> That's the nature of a distribution, that somebody decides what to distribute.
>> Typically by rejecting your proposals, c'est la vie. :-)
>
> If there is angst here it is not around the rejection of any
> particular proposal (certainly not my own), but rather about the
> persistent inability to engage in reasoned/meaningful/intentioned
> discussion w/re incorporating of a particular set of Lisp features by
> either ignoring and/or dismissing the utility these features do
> otherwise provide both Emacs user community and the greater community
> of Lisp dialect users despite a general acknowledgment by both of
> these communities that the particular set of features are
> wanted/needed and FTMP can be reasonably implemented.

I seem to recall there being a very significant incident of this that
lead to someone forking Emacs ... if only I could remember what the
fork was called!

> [1] The long term evidence of this inability IMO suggests that the Emacs
> project(s) are closed and proprietary project (whether their source be
> free or not). That [SX]Emacs does remain reasonably compatible with
> GNU Emacs suggest that it too abides this inability (whether tacitly
> or otherwise).

Compatibility and restriction are two different things. That [S]XEmacs
does not incorporate more of the RMS-rejected features probably says
more about the development effort available to [S]XEmacs vs. that
required to implement/port/submit the features than anything else.
Probably, in the case of add-on packages, it does not help that the
XEmacs packaging machinery is only intended for use with packages
stored in its own CVS tree...

Re: Return

[Stephen J. Turnbull]

MON KEY writes:
 > On Tue, Dec 7, 2010 at 9:34 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
 > > MON KEY writes:
 > >
 > > No, it won't.  However, I must say (as a maintainer) that use of
 > > lambdas often imposes extra work on debuggers.  I almost always give
 > > lambdas a nickname when I'm reading code that uses them, unless
 > > they're simply implementing partial evaluation.
 > 
 > I'm sorry, but it isn't clear, for which language?

Any language that has lambdas.  Especially Ruby, where lambda (aka
anonymous block) abuse is reaching levels that demand protective
legislation for the poor little critters.

 > The question is (apropos the above) when will what happen where?
 > ;-)

Once you have the answer for lambda, you have the answer for defun,
and vice versa, modulo the symbol deref required for a defun'ed
symbol.  The answer for lambda (resp. defun) is left as an exercise
for the reader.

 > AFAIK the discussion is still re return/return-from/block and presumably by
 > proxy go/tagbody etc. with the ostensible lexbind sprinkled on top.

You're confused, then.  Sorry, I guess I should have changed the
subject to "defun vs. lambda". :-)  Of the concepts you belatedly
reintroduce here, only "lexbind" has been mentioned in the last 4 or 5
messages in this subthread, and the paragraph above applies perfectly
well to all that other stuff anyway.

 > I am curious to learn how you understand all of what you say re lexcial scoping
 > to jibe with lambda, closures, and macroexpansion at compile time, and more
 > particularly with consideration as to why Common Lisp's `lambda-lists-keywords'
 > contains an &environment and &whole?

It should be clear: I haven't even thought about it since the
publication of CLTL1, and even then I considered it a spectator sport.

But I don't need to think about it, in the question of lambda
vs. defun.

 > > Frankly, I don't see any such general acknowledgement in the general
 > > Emacs community.
 > 
 > Most likely those who might otherwise caucus more vocally have left
 > with a whimper.

I predict that's what you will do, too.  You're trying to score
debating points, but that cuts no ice here.  What counts are code and
user requirements explained in a way that makes sense to the Emacs
maintainers.  All three (code, requirement, making sense) must be
present or your feature won't make it to the distributed Emacs.

Re: Return

[Stephen J. Turnbull]

Samuel Bronson writes:

 > when they describe software in these parts, but note that here they
 > describe the *control* over a *particular* source tree, which is a
 > distinctly different thing,

Nobody has control over "the" source tree in open source or free
software.  What individuals can control are disk space and maintainer
time, and perhaps "mind share".

 > though many effects may be similar.

FUD.  For heaven's sake, we all use dVCSes, so everybody clearly has
"control" over "the" source tree in the sense of disk space.  And for
distribution, there are dozens of zero-fee outlets.  Within the free
software and open source communities[1], only mind share matters, but
that is independent of "control of a source tree".

Richard, Linus, Theo, and I, inter alia, have some say over mind share
because of past service to the community, present effort, and the
assessment of our abilities by our communities.  Not because of our
control of certain disk space, but because people *want to use* the
trees on the disks we control.  N.B. I make that comparison
deliberately in all humility[sic], because it demonstrates that there
is a continuum of "control" over mind share (and it's
multidimensional, too).  Some are more equal than others.  There are
reasons for that (perhaps none that justify my own exalted position --
and so it goes :-).

 > Probably, in the case of add-on packages, it does not help that the
 > XEmacs packaging machinery is only intended for use with packages
 > stored in its own CVS tree...

That is false.  If somebody wants to use our packaging machinery for
out of tree packaging, they are certainly welcome to do so.  VM does
it, for example.  I among others would spend some effort to make that
easier; I know it's rather awkward at the moment.

The point is that what people have consistently asked for is *not*
that the machinery be usable out of tree.  It turns out that lots of
people have figured out how to make the machinery work out of tree.
What they want is for *us to distribute* packages made out of tree.
That's quite different, and I don't see any way it's going to happen,
any more than Emacs Lisp is going to become Common Lisp without
convincing Richard of the need.

If someone doesn't like that, all they need to change it is mind
share.  They most likely have everything else already, and if not,
it's easy to acquire.

Footnotes: 
[1]  The point being that outside of those communities, control over
trees is indeed possible and powerful.  In fact, the GNU GPL exploits
that power.

Re: Return

[Richard Stallman]

    IIRC rms has recently declared his dislike for CL-style keyword
    arguments.

I don't mind them in things that are very heavyweight anyway, such as
setting frame attributes.  What I dislike about Common Lisp is
introducing them into familiar basic Lisp functions such as `member'.
Common Lisp also changes `member' incompatibly to use `eql' as the
default comparison function.  So you must use a keyword argument to
get the existing `member' behavior.


-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: Return

[Fren Zeee]

On Tue, Dec 7, 2010 at 1:17 AM, David Kastrup <dak@gnu.org> wrote:
> Miles Bader <miles@gnu.org> writes:
>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> Actually, I'm considering to disallow non-top-level defuns in
>>> lexical-scope mode, just because it's a good opportunity to introduce
>>> such "breakage" and because non-top-level defuns are bugs in 99% of
>>> the cases (in Elisp).
>>
>> I presume by "non-top-level defun" you mean "defun inside a function",
>> not "defun inside a form"...
>
> There are lots of reasons for doing a defun inside of a function.  One
> important point of Lisp is making it easy to create code
> programmatically.
>
> I don't understand the "bugs in 99% of the cases", I could hardly
> imagine any situation where a defun is used inside of a form
> unintentionally, without the interpreter/compiler barfing anyway because
> of unmatched parens and/or producing totally non-working code that is
> straightforward to figure out.
>

LOL

He wants to go back to PASCAL of Niklaus ... from one level style of C and LISP.

Franz Xe

Re: Return

[MON KEY]

On Wed, Dec 8, 2010 at 2:28 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> MON KEY writes:
>  > On Tue, Dec 7, 2010 at 9:34 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
>  > > MON KEY writes:
>  > >
>  > > No, it won't.  However, I must say (as a maintainer) that use
>  > > of lambdas often imposes extra work on debuggers.  I almost
>  > > always give lambdas a nickname when I'm reading code that uses
>  > > them, unless they're simply implementing partial evaluation.
>  >
>  > I'm sorry, but it isn't clear, for which language?
>
> Any language that has lambdas.  Especially Ruby, where lambda (aka
> anonymous block) abuse is reaching levels that demand protective
> legislation for the poor little critters.
>
>  > The question is (apropos the above) when will what happen where?
>  > ;-)
>
> Once you have the answer for lambda, you have the answer for defun,
> and vice versa, modulo the symbol deref required for a defun'ed
> symbol.  The answer for lambda (resp. defun) is left as an exercise
> for the reader.
>

Is such an exercise so trivial a thing?

I recall there being a seminal series of paper written in the late
1970s with the short form titles matching the pattern:
 "Lambda the Ultimate .*"
e.g. the "The Lambda Papers" http://library.readscheme.org/page1.html

Note my use of the word "seminal" above. It was "left as an exercise"
to the now highly accomplished minds of CS to figure this out. I don't
pretend to understand/grasp or even have read much of what was
published on the topic. Should I assume this sort of stuff is obvious?

It apparently wasn't all that obvious for someone at DARPA circa
197[0-9] or they probably wouldn't have funded the work that produced
the deliverable largely responsible for the contemporary Lisps...

I would point out though that AI Memo 443[+] has an appendix section
entitled "Notes" with a subsection entitled "{Note Step Variables}" at
page 19 which discusses lexical scope and what might happen at runtime
with non-local exits inside a labeled recursive looping construct.

[+] Guy Lewis Steele, Jr.. "Debunking the "Expensive Procedure Call"
Myth, or Procedure Call Implementations Considered Harmful, or LAMBDA,
the Ultimate GOTO". ACM Conference Proceedings. October 1977.

>
>  > AFAIK the discussion is still re return/return-from/block and
>  > presumably by proxy go/tagbody etc. with the ostensible lexbind
>  > sprinkled on top.
>
> You're confused, then.

Most likely I am confused.

I don't see how `return', `return-from', `block', `tagbody', `go',
`unwind-protect', etc. can't not be affected by the `lexbind' nor how
their implementation in elisp (now or eventual) won't affect the
semantics of lambda/defun/defmacro.

> Sorry, I guess I should have changed the subject to "defun vs. lambda". :-)

No, that would not have been useful for readers navigating the
Internet time machine.

> Of the concepts you belatedly reintroduce here, only "lexbind" has

It strikes me that the concept of "belated reintroduction" is exactly
what is under consideration here. :)

What happens where in the `lexbind' will most definitely have
consequences w/re unwind-protect and I would suggest an eventual (an
inevitable plea for) attempts to integrate some form of
return/return-from/block/tagbody/go/ etc.

> been mentioned in the last 4 or 5 messages in this subthread, and
> the paragraph above applies perfectly well to all that other stuff
> anyway.
>

It is good to know that _you_ are sure, because apparently such
non-trivial issues as whether the lexbind will allow non-toplevel
defun's is not yet a "finalized" matter.

>  > I am curious to learn how you understand all of what you say re
>  > lexcial scoping to jibe with lambda, closures, and macroexpansion
>  > at compile time, and more particularly with consideration as to
>  > why Common Lisp's `lambda-lists-keywords' contains an
>  > &environment and &whole?

>
> It should be clear: I haven't even thought about it since the
> publication of CLTL1, and even then I considered it a spectator
> sport.
>

Can I assume then that the nature of my query re &environment and
&whole wasn't entirely lost on you?

What is clear (to me at least) is that CLTL2 pre-supposed changes to
Common Lisp which were sometimes subtle and sometimes not so subtle
ways. Moreover, given that the nearest contemporary lisp dialect to
Emacs lisp is Common Lisp (and this is the thrust of my concern) there
is a tremendously useful archive from the discussion of the CL ANSI
committee which addressed many of these issues and which I think
remain relevant to considerations w/re the lexbind.

I'm reasonably sure that only Miles/Stephan will have a hand in the
initial integration of the core of the lexbind with GNU Emacs, and
this will happen without too much discussion here. What I'm interested
to understand is how once that initial integration occurs others will
react, adapt, adjust, complain, lobby etc.

At the very least, the X3J13s formal discussion of and rationale for
how/why certain issues brought before them were resolved/addressed may
be able to inform future discussions re the elisp lexbind.

This is for example why I asked about use of &environment &whole.
There were more than one X3J13 votes around these.  See for example
"Issue &ENVIRONMENT-BINDING-ORDER Writeup" the CLHS.

From a CL perspective those issues often appear to have had an impact
on what happened w/ macroexpansion.

I find this both relevant and interesting w/re the GNU Emacs `lexbind'
and non-local exits because (as currently implemented in the lexbind
branch) it redefines what happens with `macroexpand' `macroexpand-all'
at compile time... this in turn influences `byte-compile-closure' and
`byte-compile-make-closure'.

It is left as an exercise for all of us to figure out how this stuff
will interact w/ things elisp's `define-modify-macro', `defsetf', etc.

> But I don't need to think about it, in the question of lambda
> vs. defun.
>

This was never the crux of _my_ question.

As a reminder, apropos the YAGNI re an alternative elisp VM you tossed
out a caveatted example of defmacro'ing `lambda' by way of
defun/gensym, e.g.:

   ,----
   | (defmacro lambda (&rest forms) `(defun ,(gensym) ,@forms))
   |
   | or something like that. ;-)
   `----

My question was:

    "doesn't your example dance around what happens around
    defun/defmacro/lambda at compile time in any lexically scoped
    implementation of <substitute your non-interpreted lexically
    scoped modern lispy dialect of choice here>?"

>
> maintainers.  All three (code, requirement, making sense) must be
> present or your feature won't make it to the distributed Emacs.
>

There is no feature.

--
/s_P\

Re: Return

[Samuel Bronson]

On Wed, Dec 8, 2010 at 3:06 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:

>  > Probably, in the case of add-on packages, it does not help that the
>  > XEmacs packaging machinery is only intended for use with packages
>  > stored in its own CVS tree...
>
> That is false.  If somebody wants to use our packaging machinery for
> out of tree packaging, they are certainly welcome to do so.  VM does
> it, for example.  I among others would spend some effort to make that
> easier; I know it's rather awkward at the moment.

Er, I guess I should have made that clearer... I actually meant that
it seems that (more or less) needs to put *something* in the
xemacs-packages tree, and this is (I assume) not particularly fun
without CVS commit access. At least, none of my DVCSes seem to be able
to handle branching from CVS...

Re: Return

[Stephen J. Turnbull]

MON KEY writes:

 > > Once you have the answer for lambda, you have the answer for defun,
 > > and vice versa, modulo the symbol deref required for a defun'ed
 > > symbol.  The answer for lambda (resp. defun) is left as an exercise
 > > for the reader.
 > >
 > 
 > Is such an exercise so trivial a thing?

No, it's nontrivial.  But it's not the question I was answering; the
question I was answering was the trivial one.  As Spivak points out,
answering trivial questions is the ultimate in mathematical elegance.

 > > Sorry, I guess I should have changed the subject to "defun vs. lambda". :-)
 > 
 > No, that would not have been useful for readers navigating the
 > Internet time machine.

Of course it would be useful.  Those who are interested in what you
always wanted to know about Lisp but were afraid to ask would have
known to avoid this subthread.  Me too, for that matter. :-)

 > > Of the concepts you belatedly reintroduce here, only "lexbind" has
 > 
 > It strikes me that the concept of "belated reintroduction" is exactly
 > what is under consideration here. :)

Cute!

 > Can I assume then that the nature of my query re &environment and
 > &whole wasn't entirely lost on you?

Yes.  That will be the first valid assumption you've made about what I
know or am thinking in this conversation, most likely. :-)

 > What happens where in the `lexbind' will most definitely have
 > consequences w/re unwind-protect and I would suggest an eventual (an
 > inevitable plea for) attempts to integrate some form of
 > return/return-from/block/tagbody/go/ etc.

Sure, but you complained about lack of lambda in a language (Python)
which has had to deal with all those issues, too, and has done so
fairly successfully.  If you wanted to discuss something else, you
should have made your point in different words.  The discussion about
lambda among developers of Python is *entirely* about the ugliness
and/or inconvenience of having to choose a name for a function used
only once.  Not about missing functionality.  You seem to have a
misconception about that, which I wanted to correct.

I would imagine that Lisp researchers were the first to come to grips
with these issues in many cases, and perhaps in the most generality.
But it is *not* specious to draw analogies to Python in the context of
*Emacs* Lisp.  Both languages are pragmatically designed, Emacs Lisp
as a substrate for implementing powerful editor/operating systems, and
Python as a powerful generic scripting language.  While Python does
provide solutions for all these problems, it does so in a very
pragmatic way.  Rather than try to come up with a generic syntax that
works for all functions, Python provides a (limited) variety of
syntaxes that are very efficient for situations commonly encountered
by developers using Python.  It seems reasonable to suppose that such
an approach could be useful for Emacs Lisp as well.

Of course that's debatable.  But I at least find it as plausible as
pleas for implementation of all these Common Lisp features.

Re: Return

[Stefan Monnier]

> There are lots of reasons for doing a defun inside of a function.

Give me examples and I'll tell you if I consider them as
valuable enough.

> One important point of Lisp is making it easy to create
> code programmatically.

Could be, but that's unrelated: I don't intend to rule out (eval
`(defun ...)).


        Stefan

Re: Return

[Stephen J. Turnbull]

Stefan Monnier writes:
 > > There are lots of reasons for doing a defun inside of a function.
 > 
 > Give me examples and I'll tell you if I consider them as
 > valuable enough.

Anything you would use `require' for inside a function.  Betcha find a
dozen or more instances of require-in-a-function in Gnus alone.

 > Could be, but that's unrelated: I don't intend to rule out (eval
 > `(defun ...)).

How about (eval '(defun ...))?

But the whole idea is gross, Stefan.  Please rethink this, at least
until April 1.  This is a job for the documentation, not the syntax.

Re: Return

[Stefan Monnier]

>> > There are lots of reasons for doing a defun inside of a function.
>> Give me examples and I'll tell you if I consider them as
>> valuable enough.
> Anything you would use `require' for inside a function.  Betcha find a
> dozen or more instances of require-in-a-function in Gnus alone.

Irrelevant: the defuns in the loaded file are defined at the top-level
even if the require is called from within a function.  We're calling
about "static top-level", not "top of the recursion stack".


        Stefan

Re: Return

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 919 bytes --]

On 12/7/10 8:30 AM, Stephen J. Turnbull wrote:
> David Kastrup writes:
> 
>  > I don't think anybody minds the features.
> 
> IIRC rms has recently declared his dislike for CL-style keyword
> arguments.  I suppose that's part of the "syntactic complexity" you
> mention, but MON KEY OTOH points out cases where he'd like to use
> them.  So there are some fundamental disagreements here.

I'd just like to add my support for keyword arguments. Functions like
write-region are both horrible and brittle because their parameters are
both numerous and overloaded; specific functionality can be more simply
expressed with using keyword arguments. Precedent can be seen in
play-sound, defcustom, and elsewhere. The performance arguments opposing
keyword arguments don't seem to be supported by benchmarks, and in any
case, most functions, especially ones with rich functionality, aren't on
the fast path.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Return

[Stephen J. Turnbull]

Stefan Monnier writes:
 > >> > There are lots of reasons for doing a defun inside of a function.
 > >> Give me examples and I'll tell you if I consider them as
 > >> valuable enough.
 > > Anything you would use `require' for inside a function.  Betcha find a
 > > dozen or more instances of require-in-a-function in Gnus alone.
 > 
 > Irrelevant: the defuns in the loaded file are defined at the top-level
 > even if the require is called from within a function.

Irrelevant: the defuns in the function are defined at the top-level
even though defun is called from within a function.

My point is that you could just as well do the defuns inside the
function as in a require'd file.  Much of the time it makes sense to
split them out into a file, of course, but I don't see any good reason
why that should be enforced if the author would prefer to put her
defuns in a function.

defun is always at top level, right?  If I want a local function
(which I do quite frequently) I use flet.

Re: Return

[Fren Zeee]

On Fri, Dec 10, 2010 at 12:28 AM, Stephen J. Turnbull
<stephen@xemacs.org> wrote:
> Stefan Monnier writes:
>  > >> > There are lots of reasons for doing a defun inside of a function.
>  > >> Give me examples and I'll tell you if I consider them as
>  > >> valuable enough.
>  > > Anything you would use `require' for inside a function.  Betcha find a
>  > > dozen or more instances of require-in-a-function in Gnus alone.
>  >
>  > Irrelevant: the defuns in the loaded file are defined at the top-level
>  > even if the require is called from within a function.
>
> Irrelevant: the defuns in the function are defined at the top-level
> even though defun is called from within a function.

In this case it is helpful to allow nested defuns.

Most code is developed in build-fix method as Paul Graham has
explained very well.

For making modules, one puts simply a wrapper and want the minimal to
edit and move things around. An auxilary function then gets nested. I
do this often in a language that is lisp under a different label.

> My point is that you could just as well do the defuns inside the
> function as in a require'd file.  Much of the time it makes sense to
> split them out into a file, of course, but I don't see any good reason
> why that should be enforced if the author would prefer to put her
> defuns in a function.
>
> defun is always at top level, right?  If I want a local function
> (which I do quite frequently) I use flet.
>
>
>