sending function arguments to recursive function calls

sending function arguments to recursive function calls

[Gauthier Östervall]

I am very new to e-lisp and lisp, and I expect the answer to my
question to be quite obvious when I see it.

I am puzzled by the function text-scale-adjust in lisp/face-remap.el.
The function takes (inc) as input parameter, and calls and passes this
(inc) to itself.

If I copy this function to *scratch* and evaluate the defun with C-x
C-e, I expect not to have broken anything. What happens instead is
that the function's call to itself breaks.
The line (lambda () (interactive) (text-scale-adjust (abs inc))))))
complains that inc is not defined:
"Symbol's value as variable is void: inc"

If I return to the original function in face-remap.el and evaluate the
defun there again with C-x C-e, the function starts working again.

What is the difference between the defun in face-remap.el, and its
copied version in *scratch*, that makes the propagation of inc work in
the first case but not in the second?

I vaguely suspect it has to do with autoloads, but mainly because this
is what is most obscure to me at this point.

RE: sending function arguments to recursive function calls

[Drew Adams]

> I am very new to e-lisp and lisp, and I expect the answer to my
> question to be quite obvious when I see it.

Newbie or not, you gave a great description of what you encountered.

> I am puzzled by the function text-scale-adjust in lisp/face-remap.el.
> The function takes (inc) as input parameter, and calls and passes this
> (inc) to itself.

Actually, it does not call itself.  It sets up a keymap so that when you hit -,
0, or + it gets called again with (abs INC) as its arg.  That's not the same
thing as a recursive call.  (But this is not relevant to the problem.)

> If I copy this function to *scratch* and evaluate the defun with C-x
> C-e, I expect not to have broken anything. What happens instead is
> that the function's call to itself breaks.
> The line (lambda () (interactive) (text-scale-adjust (abs inc))))))
> complains that inc is not defined:
> "Symbol's value as variable is void: inc"
> 
> If I return to the original function in face-remap.el and evaluate the
> defun there again with C-x C-e, the function starts working again.
> 
> What is the difference between the defun in face-remap.el, and its
> copied version in *scratch*, that makes the propagation of inc work in
> the first case but not in the second?
> 
> I vaguely suspect it has to do with autoloads, but mainly because this
> is what is most obscure to me at this point.

Nope.  Welcome to the joys of Emacs Lisp's way of mixing lexical and dynamic
binding/scope.

The key to the puzzle is this little declaration in the first comment of the
file:

;;; face-remap.el --- Functions for ... -*- lexical-binding: t -*-

That `lexical-binding t' tells Emacs that the code in this file is meant to be
understood with the variable `lexical-binding' bound to t (locally).

If you add and evaluate the following sexp to your *scratch* buffer then you
will get the same effect as for the file:

(set (make-local-variable 'lexical-binding) t)

The problem was that the code you evaluated was not interpreted using lexical
bindings, so the lambda form did not contain any environment for looking up the
value of variable INC.

An alternative to using a lexical binding here would be to simply use this:

 `(lambda () (interactive) (text-scale-adjust (abs ',inc)))

That substitutes the value of INC from the initial call to `text-scale-adjust'
into the lambda.  So instead of there being a variable INC to look up there is a
literal value (e.g. 2).

Good question, BTW.

Re: sending function arguments to recursive function calls

[Gauthier Östervall]

On Sat, May 4, 2013 at 5:30 PM, Drew Adams <drew.adams@oracle.com> wrote:
> The key to the puzzle is this little declaration in the first comment of the
> file:
>
> ;;; face-remap.el --- Functions for ... -*- lexical-binding: t -*-
>
> That `lexical-binding t' tells Emacs that the code in this file is meant to be
> understood with the variable `lexical-binding' bound to t (locally).

Thanks for the kind words and the explanation. I do not feel very
confident about having a function that relies on a file scope setting
of lexical-binding, which as in this case happens 300 lines earlier.
Wouldn't it be better to have functions that work whatever the value
of lexical-binding, if possible?

> An alternative to using a lexical binding here would be to simply use this:
>
>  `(lambda () (interactive) (text-scale-adjust (abs ',inc)))

If this alternative works whatever lexical-binding, it seems superior
to me. Isn't it?

RE: sending function arguments to recursive function calls

[Drew Adams]

> > The key to the puzzle is this little declaration in the 
> > first comment of the file:
> >
> > ;;; face-remap.el --- Functions for ... -*- lexical-binding: t -*-
> >
> > That `lexical-binding t' tells Emacs that the code in this 
> > file is meant to be understood with the variable
> `lexical-binding' bound to t (locally).
> 
> Thanks for the kind words and the explanation. I do not feel very
> confident about having a function that relies on a file scope setting
> of lexical-binding, which as in this case happens 300 lines earlier.
> Wouldn't it be better to have functions that work whatever the value
> of lexical-binding, if possible?

I agree.

Sometimes, however, you specifically want the function definition to take
advantage of one kind of scoping/binding or the other.

IOW, when it makes no difference, then yes, it would be nice not to be bothered
with the difference. ;-)  When it does make a difference, that is, when you want
it to make a difference, then it should be easy to choose and it should be clear
what is going on.

Emacs Lisp does not particularly make things clear, IMO.  Common Lisp does a
pretty good job of helping lexical and dynamic binding cohabit.  Emacs Lisp does
not do such a great job, IMO.  As Stefan pointed out:

  Reality is that there are 2 Emacs Lisps.  One is the
  dynamically scoped Elisp, and the other is the lexically
  scoped Elisp.  They are very similar, but they are two
  different languages.

> > An alternative to using a lexical binding here would be to 
> > simply use this:
> >  `(lambda () (interactive) (text-scale-adjust (abs ',inc)))
> 
> If this alternative works whatever lexical-binding, it seems superior
> to me. Isn't it?

Yes.  That's my opinion, anyway.  In this case (and in many others), there is no
_logical_ reason to pass a variable and have that variable be looked up
lexically when the lambda is invoked.  All that is needed is the variable's
value at the time the lambda is _constructed_.  Anything else is overkill.  

But logic is not the only thing involved here.  Disadvantages of substituting
the value can include: (a) backquote syntax, or its equivalent, is less readable
and (b) the byte compiler might not be able to compile the lambda in a
performant way.

I'm no expert on Emacs byte compilation, but it's possible that the byte
compiler just punts on a dynamically constructed sexp such as the one above,
even though it might be able to figure out that the result in this case is
straightforward.  Dunno.

Lexical binding was added to Emacs relatively recently.  With time, perhaps it
and dynamic binding will become better roommates.  And certainly, with time,
users will become more used to lexical binding and to looking out for
lexical/dynamic binding gotchas.

For one thing, you will no doubt be reflexively looking for a local-variable
binding declaration in the file, right? ;-)  And you will be looking for free
variables, and when you notice one you will ask yourself whether it will be
evaluated using lexical or dynamic binding.  IOW, you will internalize things
like this help thread.

Re: sending function arguments to recursive function calls

[Stefan Monnier]

> Emacs Lisp does not particularly make things clear, IMO.  Common Lisp
> does a pretty good job of helping lexical and dynamic binding cohabit.

Actually, Emacs's lexical-binding variant of Elisp is pretty close to
Common Lisp: the default binding is lexical but you can use dynamic
binding for specific cases.

The difference is that Emacs has to keep the old compatibility mode
where lexical-binding is nil, whereas Common-Lisp never had such a thing.

> Lexical binding was added to Emacs relatively recently.  With time,
> perhaps it and dynamic binding will become better roommates.

When lexical-binding is t, they're very good roomates already.
Hopefully with time, the "lexical-binding = nil" case can be dropped.


        Stefan

Re: sending function arguments to recursive function calls

[Gauthier Östervall]

On Wed, May 8, 2013 at 2:21 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> Actually, Emacs's lexical-binding variant of Elisp is pretty close to
> Common Lisp: the default binding is lexical but you can use dynamic
> binding for specific cases.
>
> The difference is that Emacs has to keep the old compatibility mode
> where lexical-binding is nil, whereas Common-Lisp never had such a thing.
>
>> Lexical binding was added to Emacs relatively recently.  With time,
>> perhaps it and dynamic binding will become better roommates.
>
> When lexical-binding is t, they're very good roomates already.
> Hopefully with time, the "lexical-binding = nil" case can be dropped.

If I have an existing file with lexical-binding = nil, and I want to
add more functions to that file, what is then the preferred way to do
it?  Should I write the new function with lexical-binding = t?  Isn't
it risky or wrong to set lexical-binding at the top of the file (like
face-remap.el does), what would happen to the other functions, which
earlier had lexical-binding = nil?

Is there a way (and is it better) to set lexical-binding = t locally
for the new function?

Or is it better to make the new function work with lexical-binding =
nil, despite the caveats described by Pascal?

Re: sending function arguments to recursive function calls

[Stefan Monnier]

> Is there a way (and is it better) to set lexical-binding = t locally
> for the new function?

No.  But converting existing dynamically scoped code to lexical scoping
is usually pretty easy: set lexical-binding to t at the top of the file,
then byte-compile it, then fix the warnings, and you should be good to go.

Sometimes there's some extra tricks undetected by the simple checks
performed by the byte-compiler, so it doesn't always work as smoothly,
but in most cases it's really a straightforward process.


        Stefan

Re: sending function arguments to recursive function calls

[Gauthier Östervall]

[-- Attachment #1: Type: text/plain, Size: 1170 bytes --]

On Wed, May 8, 2013 at 2:21 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> When lexical-binding is t, they're very good roomates already.
> Hopefully with time, the "lexical-binding = nil" case can be dropped.

Does this imply that if I want to give for example window.el
lexical-binding (I do have an ulterior motive), no one would mind
having it that way?

> [...]  But converting existing dynamically scoped code to lexical scoping
> is usually pretty easy: set lexical-binding to t at the top of the file,
> then byte-compile it, then fix the warnings, and you should be good to go.
>
> Sometimes there's some extra tricks undetected by the simple checks
> performed by the byte-compiler, so it doesn't always work as smoothly,
> but in most cases it's really a straightforward process.

I have tried to do just that to window.el. It seemed indeed
straightforward to me, but I would greatly appreciate feedback on the
change. I really just added the lexical-binding at the top, and
removed seemingly unused variables. I don't know why these variables
were left (were they really unused?), but byte-compiling gives no
warning, at least.

See attached bundle.

[-- Attachment #2: make-window-lexical.txt --]
[-- Type: text/plain, Size: 4466 bytes --]

# Bazaar merge directive format 2 (Bazaar 0.90)
# revision_id: gauthier@ostervall.se-20130512130720-hpthrwn048nrj4ao
# target_branch: file:///media/sf_prog/emacs/trunk/
# testament_sha1: 288aa595c17d0d7923aa72d1cbe25a3a3b081380
# timestamp: 2013-05-12 15:09:54 +0200
# base_revision_id: rgm@gnu.org-20130430101735-ccy06l3ndx4vrj18
# 
# Begin patch
=== modified file 'lisp/window.el' (properties changed: -x to +x)
--- lisp/window.el	2013-04-13 14:37:20 +0000
+++ lisp/window.el	2013-05-12 13:07:20 +0000
@@ -1,4 +1,4 @@
-;;; window.el --- GNU Emacs window commands aside from those written in C
+;;; window.el --- GNU Emacs window commands aside from those written in C  -*- lexical-binding: t -*-
 
 ;; Copyright (C) 1985, 1989, 1992-1994, 2000-2013 Free Software
 ;; Foundation, Inc.
@@ -685,11 +685,9 @@
 This function may be called only if no window on SIDE exists yet.
 The new window automatically becomes the \"major\" side window on
 SIDE.  Return the new window, nil if its creation window failed."
-  (let* ((root (frame-root-window))
-	 (left-or-right (memq side '(left right)))
+  (let* ((left-or-right (memq side '(left right)))
 	 (major (window--major-side-window side))
-	 (selected-window (selected-window))
-	 (on-side (cond
+         (on-side (cond
 		   ((eq side 'top) 'above)
 		   ((eq side 'bottom) 'below)
 		   (t side)))
@@ -698,8 +696,7 @@
 	 ;; parent window unless needed.
 	 (window-combination-resize 'side)
 	 (window-combination-limit nil)
-	 (new (split-window major nil on-side))
-	 fun)
+         (new (split-window major nil on-side)))
     (when new
       ;; Initialize `window-side' parameter of new window to SIDE.
       (set-window-parameter new 'window-side side)
@@ -749,8 +746,7 @@
   A positive value means use a slot following (that is, below or
   on the right of) the middle slot.  The default is zero."
   (let ((side (or (cdr (assq 'side alist)) 'bottom))
-	(slot (or (cdr (assq 'slot alist)) 0))
-	new)
+        (slot (or (cdr (assq 'slot alist)) 0)))
     (cond
      ((not (memq side '(top bottom left right)))
       (error "Invalid side %s specified" side))
@@ -776,9 +772,8 @@
 		  ((eq side 'right) 2)
 		  ((eq side 'bottom) 3))
 		 window-sides-slots))
-	   (selected-window (selected-window))
-	   window this-window this-slot prev-window next-window
-	   best-window best-slot abs-slot new-window)
+           window this-window this-slot prev-window next-window
+           best-window best-slot abs-slot)
 
       (cond
        ((and (numberp max-slots) (<= max-slots 0))
@@ -5713,7 +5708,7 @@
 		   0)
 		  (display-buffer-reuse-frames 0)
 		  (t (last-nonminibuffer-frame))))
-	 entry best-window second-best-window window)
+         best-window second-best-window window)
     ;; Scan windows whether they have shown the buffer recently.
     (catch 'best
       (dolist (window (window-list-1 (frame-first-window) 'nomini frames))

# Begin bundle
IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWedecm4AAeVfgAaQUPP//1qB
kgC////wUAQ+e2A7zVroA0DQpB6jTEHqZPUBppkHlAAAAjSJpqep5TyIyAAAAAAADRT0TRgiahmk
YTE2hGATTJkxCKfqmSPU2SB6mJk00ABgIDTQ0CSIRqbSMIEwmFPRqPUZGkekyYaiIvrvtah8WXqt
OO3Qj96nDbPuxEtdU906bq4tuhKEs4gdL3ef37/wQnX+TOz7O/0Qafx9gM89hRR0vAfQOQg7wAyg
4JxuiNaisWWRddptKfJkoQEuAygQPDyORisUUT0DoF05YXlz1NgQyLIRNcgaDacUxRzIAgfsIwTm
ah2/WBLepi/eRjuqOyiLZP9eKflzkQ3jL+M7WPkJaUe2ZVJYysv3/OS97MyL3f4JNz5wBkRcQyqi
O0QirBOKxMKwu7MtcyDZE8XhRBWygx3yPHq6jgWfG/BWKMtwp5bhxGIvWM1l/SjguNPlVdRenH2W
UL8xmz3mirRS1LomYRirBMEyemsm2rUZj6YZrQz2HdE34ZIYXJ1kEGcZYHsY5FSVe6VIKXz2qWqL
6mJ5FPJLfBR0aiupRoq4LJlkQFhBoKZTYK8q9rTNo5rRnWYhSYKzadOUUt4riqllFEz1PlC2xxkp
oUYc8KOARMoyYoFeSJSy2qBI6ehPM0eJguCEggJmlMjHL9nrpkpbFA+eHqpAw4uqBIYroQmRI6JV
XAgvrgGKKzPU1YkNr3yW6XrLHJOmrknkphXNvQ/s1t20U66yDN0/x49F0SnUAY/XIYy9JF9VTxYQ
1a0BgVj08KsfptNrN1QCsZMm4gXUqrbci9vDMidnrWtXNL38+eGo8LPz9verze164INs2GyHpYJ8
nC2txyPW5RtC0A9D7r8lFyxq1FQCYgsyJ0nAkIMzS8rlBcdJ5NgSdxQtI06niPx5vfHrbZSzX1ZK
VcCiZatqSvKIOY+SCi2fFIReuZyF99h1fKD7JJG4LJlDhw3Qglr5cogeumq2oGhpohusE81LgZeW
/6+C8NUR1G4/fO7Mx1bIPDBN9T4DRnmRYVr+B1NgaJ7NSR5CYXYxQOYUOybGv/laD8gJ7Ssr2KyR
rX3DgdgC6w89miN4XHQOaw1yhBBu2GoM5Lc6koJOHBTCEAkYteO2gv4Wb2QY/O1FvargvWmfQyvt
uugcaqGNdQ9Kx2knbaJqmo7khIf6KHlbVUubzXyFYiTKi0QXPKLIsHbcSttdQO0iX3n0C9UWuC5e
KW1Gnu6AG98drMnJa5drdc52iqydVSlG7mcQwc4Its5WlSgpV9NNXDwWUVWRadWTLKTAYidX749+
LPZof+T2pS1pymOSJM5uIonFA7avZlGmEGRIBN/QAQCqIetAnNkpmqUfBqMTlExi97swFDvkA67h
Ty8riYUCDK03J1pupXHbQDD0oBX5rNNVBUoOyS1lTB9mddvNoWC4qV69zKwGYJK5Tuzkv/F3JFOF
CQ515ybg

Re: sending function arguments to recursive function calls

[Stefan Monnier]

>> When lexical-binding is t, they're very good roomates already.
>> Hopefully with time, the "lexical-binding = nil" case can be dropped.
> Does this imply that if I want to give for example window.el
> lexical-binding (I do have an ulterior motive), no one would mind
> having it that way?

Yup.  My local Emacs is currently pretty broken because I started to
compile all files as "lexical-binding = t".

> I don't know why these variables were left (were they really unused?)

That's the question that the byte-compiler can't answer: maybe they
really weren't used (so you can remove them), or maybe they were used
elsewhere via dynamic scoping (in which case you need to add a (defvar
<var>) to force the use of dynamic scoping for this variable).

It's very common to have variables that aren't used (since the
byte-compiler didn't warn about it).


        Stefan

Re: sending function arguments to recursive function calls

[Gauthier Östervall]

On Mon, May 13, 2013 at 4:55 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> That's the question that the byte-compiler can't answer: maybe they
> really weren't used (so you can remove them), or maybe they were used
> elsewhere via dynamic scoping (in which case you need to add a (defvar
> <var>) to force the use of dynamic scoping for this variable).

All the variables that the byte-compiler complained about were defined
in the first parameter of a let or a let* expression.

My understanding of let and let* is that the first parameter defines
local variables to be used in the second parameter (or later in the
first parameter in the case of let*).

The unused variables I had to remove were not referenced in the last
parameter of the let or let* (nor later in the first parameter of
let*), so I assumed they were not used anywhere.

Is there a way that such variables may be used elsewhere (via dynamic
scoping)? If yes, how?
Couldn't it be the case only if they were defined with setq (so called
free variables)?

Re: sending function arguments to recursive function calls

[Dmitry Gutov]

Gauthier Östervall <gauthier@ostervall.se> writes:

> On Mon, May 13, 2013 at 4:55 PM, Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
>> That's the question that the byte-compiler can't answer: maybe they
>> really weren't used (so you can remove them), or maybe they were used
>> elsewhere via dynamic scoping (in which case you need to add a (defvar
>> <var>) to force the use of dynamic scoping for this variable).
>
> All the variables that the byte-compiler complained about were defined
> in the first parameter of a let or a let* expression.
>
> My understanding of let and let* is that the first parameter defines
> local variables to be used in the second parameter (or later in the
> first parameter in the case of let*).
>
> The unused variables I had to remove were not referenced in the last
> parameter of the let or let* (nor later in the first parameter of
> let*), so I assumed they were not used anywhere.
>
> Is there a way that such variables may be used elsewhere (via dynamic
> scoping)? If yes, how?
> Couldn't it be the case only if they were defined with setq (so called
> free variables)?

This is the ugly side of dynamic scoping.

(defun foo ()
  (let ((bar 42))
    (baz)))

(defun baz ()
  bar)

(foo) ; => 42

baz ; => void-variable error

RE: sending function arguments to recursive function calls

[Drew Adams]

> This is the ugly side of dynamic scoping.
> (defun foo () (let ((bar 42)) (baz)))
> (defun baz () bar)
> (foo) ; => 42
> baz ; => void-variable error

Huh?  I guess you meant to write
(baz) ; => "void-variable bar" error

There is nothing ugly about that behavior.

The `let' binds variable `bar' for the dynamic extent of the call to `foo'.
There is no other binding of `bar' or assignment to it here, so `(baz)' refers
to an unbound variable `bar'.

What happens with lexical scoping?
(foo) ; => "void-variable bar" error
(baz) ; => "void-variable bar" error

Which is also not ugly and not unusual.  There is no binding of `bar' lexically
visible in `baz'.

Dynamic and lexical binding are very different.  That's all.  Each has its
advantages.

Lexical binding is generally cleaner (correct for funargs etc.), so it is
simpler to understand (WYSIWYG, where the `S' is all about lexical scope).  As
such, it can often allow compilation to more efficient code.  And it can
facilitate program proving and transformation, but mainly for "pure"
(referentially transparent) languages, not full Lisp.

Dynamic binding facilitates user extension ("monkey patching").  And yes, this
is particularly important for a dynamic user environment like Emacs.

It is easy to find references lauding the benefits of lexical binding (most
languages use only lexical binding).  Stallman explains well why dynamic binding
is important for Emacs:
http://www.gnu.org/software/emacs/emacs-paper.html#SEC17.

--

Some other background/discussion:

http://www.emacswiki.org/emacs/DynamicBindingVsLexicalBinding

http://en.wikipedia.org/wiki/Static_scoping#Dynamic_scoping

http://stackoverflow.com/questions/321000/what-are-the-advantages-of-dynamic-sco
ping

http://stackoverflow.com/questions/2979428/uses-for-dynamic-scope

http://c2.com/cgi/wiki?DynamicScoping

http://academic.udayton.edu/saverioperugini/courses/cps343/lecture_notes/scope.h
tml

http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html

http://www.codinghorror.com/blog/2008/07/monkeypatching-for-humans.html

http://devblog.avdi.org/2008/02/23/why-monkeypatching-is-destroying-ruby/

Re: sending function arguments to recursive function calls

[Dmitry Gutov]

On 17.05.2013 18:31, Drew Adams wrote:
>> This is the ugly side of dynamic scoping.
>> (defun foo () (let ((bar 42)) (baz)))
>> (defun baz () bar)
>> (foo) ; => 42
>> baz ; => void-variable error
>
> Huh?  I guess you meant to write
> (baz) ; => "void-variable bar" error

No, I meant exactly to write variable reference, not function call, to 
illustrate that it's not defined globally.

> There is nothing ugly about that behavior.

It's ugly because this kind of code is hard to reason about and, 
consequently, hard to modify. Suppose I want to rewrite `foo' (and 
suppose it's longer than this one line).

Can I rename `bar' to something else? No idea: to be absolutely sure, I 
have to search the definitions of all functions that `foo' calls, and if 
I find a `bar' reference in any of them, I'll now have to search for any 
other functions that call them, etc. IOW, this makes for terrible 
composability.

The behavior is ugly because it allows the code to be written this way.

A worse example is when `bar' is one of the arguments to `foo' (ugh).

> The `let' binds variable `bar' for the dynamic extent of the call to `foo'.
> There is no other binding of `bar' or assignment to it here, so `(baz)' refers
> to an unbound variable `bar'.
>
> What happens with lexical scoping?
> (foo) ; => "void-variable bar" error
> (baz) ; => "void-variable bar" error

As it should. Contrast this with the situation when `bar' has been 
`defvar'ed in advance. Both functions would know that this var is 
global, so if it's renamed in some place, it definitely should be 
renamed in all functions that reference it.

This is what I can call the light side of dynamic scoping, and it's how 
the term "dynamic binding" is often defined.

> Dynamic binding facilitates user extension ("monkey patching").  And yes, this
> is particularly important for a dynamic user environment like Emacs.
>
> It is easy to find references lauding the benefits of lexical binding (most
> languages use only lexical binding).  Stallman explains well why dynamic binding
> is important for Emacs:
> http://www.gnu.org/software/emacs/emacs-paper.html#SEC17.

I could offer some criticism for the paper, but there's really no need.

Just recall that Emacs is on track to eventually replace dynamic scoping 
with lexical scoping everywhere, with exceptions for defvar'ed vars 
(controlled dynamic binding), and nobody is really arguing that Emacs 
will become too hard to customize as a result. Nobody reasonably 
well-informed, anyway.

RE: sending function arguments to recursive function calls

[Drew Adams]

> Just recall that Emacs is on track to eventually replace dynamic scoping
> with lexical scoping everywhere, with exceptions for defvar'ed vars
> (controlled dynamic binding), and nobody is really arguing that Emacs
> will become too hard to customize as a result. Nobody reasonably
> well-informed, anyway.

Which is just to say that Emacs Lisp now aims to end up like Common Lisp (30 years later): lexical scoping unless the variable is "special" (which for Emacs Lisp means defvar'd, so far).

IOW, nothing new here.  No rejection of dynamic scoping.  Reasonable cohabitation of lexical and dynamic scoping.  That's still the aim, AFAIU, and a good one.

Re: sending function arguments to recursive function calls

[Pascal J. Bourguignon]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 17.05.2013 18:31, Drew Adams wrote:
>>> This is the ugly side of dynamic scoping.
>>> (defun foo () (let ((bar 42)) (baz)))
>>> (defun baz () bar)
>>> (foo) ; => 42
>>> baz ; => void-variable error
>>
>> Huh?  I guess you meant to write
>> (baz) ; => "void-variable bar" error
>
> No, I meant exactly to write variable reference, not function call, to
> illustrate that it's not defined globally.

Then you meant to write:

    bar

not:

    baz


>> There is nothing ugly about that behavior.
>
> It's ugly because this kind of code is hard to reason about and,
> consequently, hard to modify. Suppose I want to rewrite `foo' (and
> suppose it's longer than this one line).

Yes. That's why you should adopt the CL convention of naming all the
special variables (those with dynamic binding) with stars:

(defun foo ()
  (let ((*bar* 42)) 
     (declare (special *bar*))
     (baz)))

(defun baz ()
   (declare (special *bar*))
   *bar*)


(foo) ; => 42

*bar* ; Debugger entered--Lisp error: (void-variable *bar*)



> Can I rename `bar' to something else? 

Yes.  You should name it *bar*, and declare it special locally.  Right,
for now (declare (special *bar*)) has no effect in emacs lisp since it's
the default, but it states your intent!


> No idea: to be absolutely sure,
> I have to search the definitions of all functions that `foo' calls,
> and if I find a `bar' reference in any of them, I'll now have to
> search for any other functions that call them, etc. IOW, this makes
> for terrible composability.

Definitely.  That's why the default is lexical binding, and you have to
declare specially variables with dynamic binding, either with declare
special, or globally with defvar or defparameter.


> The behavior is ugly because it allows the code to be written this way.
>
> A worse example is when `bar' is one of the arguments to `foo' (ugh).

Global or local special declarations are still possible, even for
parameters.


>> The `let' binds variable `bar' for the dynamic extent of the call to `foo'.
>> There is no other binding of `bar' or assignment to it here, so `(baz)' refers
>> to an unbound variable `bar'.
>>
>> What happens with lexical scoping?
>> (foo) ; => "void-variable bar" error
>> (baz) ; => "void-variable bar" error
>
> As it should. Contrast this with the situation when `bar' has been
> defvar'ed in advance. Both functions would know that this var is
> global, so if it's renamed in some place, it definitely should be
> renamed in all functions that reference it.
>
> This is what I can call the light side of dynamic scoping, and it's
> how the term "dynamic binding" is often defined.
>
>> Dynamic binding facilitates user extension ("monkey patching").  And yes, this
>> is particularly important for a dynamic user environment like Emacs.
>>
>> It is easy to find references lauding the benefits of lexical binding (most
>> languages use only lexical binding).  Stallman explains well why dynamic binding
>> is important for Emacs:
>> http://www.gnu.org/software/emacs/emacs-paper.html#SEC17.
>
> I could offer some criticism for the paper, but there's really no need.
>
> Just recall that Emacs is on track to eventually replace dynamic
> scoping with lexical scoping everywhere, with exceptions for defvar'ed
> vars (controlled dynamic binding), and nobody is really arguing that
> Emacs will become too hard to customize as a result. Nobody reasonably
> well-informed, anyway.

Of course.  As long as dynamic binding is still available for the few
cases where it's needed, as in Common Lisp.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.
You can take the lisper out of the lisp job, but you can't take the lisp out
of the lisper (; -- antifuchs

Re: sending function arguments to recursive function calls

[Dmitry Gutov]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:
> Then you meant to write:
>
>     bar
>
> not:
>
>     baz
>

Right. Sorry.

>> It's ugly because this kind of code is hard to reason about and,
>> consequently, hard to modify. Suppose I want to rewrite `foo' (and
>> suppose it's longer than this one line).
>
> Yes. That's why you should adopt the CL convention of naming all the
> special variables (those with dynamic binding) with stars:
>

I agree it's a good convention in CL (and Clojure), but Elisp has its
own convention: name the variable starting with the package prefix.
Until we have real modularity, it should be good enough.

>> Can I rename `bar' to something else? 
>
> Yes.  You should name it *bar*, and declare it special locally.  Right,
> for now (declare (special *bar*)) has no effect in emacs lisp since it's
> the default, but it states your intent!

Yes, I think, currently only byte-compiler looks at it, and warns the
user when it sees undeclared variables.

>> No idea: to be absolutely sure,
>> I have to search the definitions of all functions that `foo' calls,
>> and if I find a `bar' reference in any of them, I'll now have to
>> search for any other functions that call them, etc. IOW, this makes
>> for terrible composability.
>
> Definitely.  That's why the default is lexical binding, and you have to
> declare specially variables with dynamic binding, either with declare
> special, or globally with defvar or defparameter.
>
>
>> The behavior is ugly because it allows the code to be written this way.
>>
>> A worse example is when `bar' is one of the arguments to `foo' (ugh).
>
> Global or local special declarations are still possible, even for
> parameters.

...in CL, right? Your earmuff example didn't work for me in Emacs with
lexical-binding t, `(declare (special *bar*))' doesn't replace the need
for `defvar'ing it.

I'm not sure how I feel about special local variables and parameters.
The problem with tracking down callers/callees graph applies to both of
these types, as far as I can tell, so each local variable of this kind
might as well be declared global.

During the few years I've been using Emacs, I think I've only used the
ability to change the value of a parameter in a caller function once,
deep inside the Helm codebase. Sure enough, in a few months, the caller
function code has changed, and my function broke.

Re: sending function arguments to recursive function calls

[Pascal J. Bourguignon]

Dmitry Gutov <dgutov@yandex.ru> writes:

> During the few years I've been using Emacs, I think I've only used the
> ability to change the value of a parameter in a caller function once,
> deep inside the Helm codebase. Sure enough, in a few months, the caller
> function code has changed, and my function broke.

Definitely.  

But then any kind of patching is doomed, if not integrated in the
upstream.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.
You can take the lisper out of the lisp job, but you can't take the lisp out
of the lisper (; -- antifuchs

Re: sending function arguments to recursive function calls

[Pascal J. Bourguignon]

Gauthier Östervall <gauthier@ostervall.se> writes:

> On Sat, May 4, 2013 at 5:30 PM, Drew Adams <drew.adams@oracle.com> wrote:
>> The key to the puzzle is this little declaration in the first comment of the
>> file:
>>
>> ;;; face-remap.el --- Functions for ... -*- lexical-binding: t -*-
>>
>> That `lexical-binding t' tells Emacs that the code in this file is meant to be
>> understood with the variable `lexical-binding' bound to t (locally).
>
> Thanks for the kind words and the explanation. I do not feel very
> confident about having a function that relies on a file scope setting
> of lexical-binding, which as in this case happens 300 lines earlier.
> Wouldn't it be better to have functions that work whatever the value
> of lexical-binding, if possible?
>
>> An alternative to using a lexical binding here would be to simply use this:
>>
>>  `(lambda () (interactive) (text-scale-adjust (abs ',inc)))
>
> If this alternative works whatever lexical-binding, it seems superior
> to me. Isn't it?

No, it is inferior.

Here, you have to duplicate the code of the function for each occurence.

With closures (lexical binding) you only need to duplicate the enclosed
environment, the code of the function being the same for all the
closures.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: sending function arguments to recursive function calls

[Pascal J. Bourguignon]

"Drew Adams" <drew.adams@oracle.com> writes:

>> > An alternative to using a lexical binding here would be to 
>> > simply use this:
>> >  `(lambda () (interactive) (text-scale-adjust (abs ',inc)))
>> 
>> If this alternative works whatever lexical-binding, it seems superior
>> to me. Isn't it?
>
> Yes.  That's my opinion, anyway.  In this case (and in many others), there is no
> _logical_ reason to pass a variable and have that variable be looked up
> lexically when the lambda is invoked.  All that is needed is the variable's
> value at the time the lambda is _constructed_.  Anything else is overkill.  
>
> But logic is not the only thing involved here.  Disadvantages of substituting
> the value can include: (a) backquote syntax, or its equivalent, is less readable
> and (b) the byte compiler might not be able to compile the lambda in a
> performant way.
>
> I'm no expert on Emacs byte compilation, but it's possible that the byte
> compiler just punts on a dynamically constructed sexp such as the one above,
> even though it might be able to figure out that the result in this case is
> straightforward.  Dunno.

Well, the thing is that emacs lisp accepts as function any list starting
with the symbol lambda, and when calling it, it just interprets
it. Since the function is quoted not by function, then it won't be
compiled (unless explicitely requested).


 (defun make-scale-adjuster (inc)
   `(lambda () (interactive) (text-scale-adjust (abs ',inc))))

 (byte-compile 'make-scale-adjuster)
 --> #[(inc) "\301\302\303\304\305\306\bDDDF\207" [inc lambda nil
       (interactive) text-scale-adjust abs quote] 7]

 (make-scale-adjuster 42)
 --> (lambda nil (interactive) (text-scale-adjust (abs (quote 42))))

 (eql (make-scale-adjuster 1) (make-scale-adjuster 2))
 --> nil

 (functionp (make-scale-adjuster 33))
 --> t

 (funcall (make-scale-adjuster 33))
 --> (((#:t keymap (67108912 . #[0 #1="\301\302\300!!\207" [33 text-scale-adjust abs] 3 #2="

(fn)" nil]) (67108925 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]) (67108907 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]) (67108909 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]) (48 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]) (61 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]) (43 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]) (45 . #[0 #1# [33 text-scale-adjust abs] 3 #2# nil]))))
 ;; Oops, I shouldn't have done that…




You could compile the lambda:

 (defun make-scale-adjuster (inc)
   (byte-compile `(lambda () (interactive) (text-scale-adjust (abs ,inc)))))

which makes you lose that compilation time each time you
callmake-scale-adjuster:

 (make-scale-adjuster 42)
  --> #[nil "\300\301\302!!\207" [text-scale-adjust abs 42] 3 nil nil]

but hopefully, you'll get back this time when calling the generated
function:

  (loop with scale-adjuster = (make-scale-adjuster 42)
        repeat a-lot
        do (funcall scale-adjuster))


Nonetheless, each time you call make-scale-adjuster, it has to return
duplicated code:

 (make-scale-adjuster 42)
  --> #[nil "\300\301\302!!\207" [text-scale-adjust abs 42] 3 nil nil]
 (make-scale-adjuster 33)
  --> #[nil "\300\301\302!!\207" [text-scale-adjust abs 33] 3 nil nil]

while only one constant changes…


 (setf lexical-binding t)
 (defun make-scale-adjuster (inc)
   (lambda () (interactive) (text-scale-adjust (abs inc))))

  (make-scale-adjuster 42)
  --> (closure ((inc . 42) t) nil (interactive) (text-scale-adjust (abs inc)))
  (make-scale-adjuster 33)
  --> (closure ((inc . 33) t) nil (interactive) (text-scale-adjust (abs inc)))


  (byte-compile 'make-scale-adjuster)
  --> #[(inc) "\300\207" [#[nil "\301\302\b!!\207" [inc text-scale-adjust abs] 3 nil nil]] 1]

  (make-scale-adjuster 42)
  --> #[nil "\301\302\b!!\207" [inc text-scale-adjust abs] 3 nil nil]
  (make-scale-adjuster 33)
  --> #[nil "\301\302\b!!\207" [inc text-scale-adjust abs] 3 nil nil]

  (eq (make-scale-adjuster 42)
      (make-scale-adjuster 33)) 
  --> t

Here when compiling the function the closure inside is compiled at the
same time, and when calling the compiled function, the same closed
function is returned.  (It is a little strange that they're eq, since
they should be different closures, when calling them we get different
results, but until emacs lisp becomes Common Lisp, we can't expect too
much of it, can we).



> Lexical binding was added to Emacs relatively recently.  With time, perhaps it
> and dynamic binding will become better roommates.  And certainly, with time,
> users will become more used to lexical binding and to looking out for
> lexical/dynamic binding gotchas.

Too bad in 2013, users must suffer those gotchas that have been debugged
in the 70's by scheme users, and integrated in the 80's by all lispers
(thru the common lisp).  


> For one thing, you will no doubt be reflexively looking for a local-variable
> binding declaration in the file, right? ;-)  And you will be looking for free
> variables, and when you notice one you will ask yourself whether it will be
> evaluated using lexical or dynamic binding.  IOW, you will internalize things
> like this help thread.

A good trick is to wrap all your special^W dynamic variables in stars:

    (defvar *dynamic-variable* 'dyn)

    (let ((lexical-variable 'lex))
      (defun f ()           
         (list *dynamic-variable* lexical-variable)))



-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
You can take the lisper out of the lisp job, but you can't take the lisp out
of the lisper (; -- antifuchs

Re: sending function arguments to recursive function calls

[Stefan Monnier]

>   (make-scale-adjuster 42)
>   --> #[nil "\301\302\b!!\207" [inc text-scale-adjust abs] 3 nil nil]
>   (make-scale-adjuster 33)
>   --> #[nil "\301\302\b!!\207" [inc text-scale-adjust abs] 3 nil nil]
>   (eq (make-scale-adjuster 42)
>       (make-scale-adjuster 33)) 
--> t

> Here when compiling the function the closure inside is compiled at the
> same time, and when calling the compiled function, the same closed
> function is returned.  (It is a little strange that they're eq, since

They're `eq' because you compiled them in dynamic-binding mode
(i.e. they won't work).

> A good trick is to wrap all your special^W dynamic variables in stars:
>     (defvar *dynamic-variable* 'dyn)

That's the Common-Lisp convention, but Elisp uses another convention,
which is to add a "package prefix" such as `mypackage-myvar'.


        Stefan

Re: sending function arguments to recursive function calls

[Stefan Monnier]

> If I copy this function to *scratch* and evaluate the defun with C-x
> C-e, I expect not to have broken anything.  What happens instead is

Reality is that there are 2 Emacs Lisps.  One is the dynamically scoped
Elisp, and the other is the lexically scoped Elisp.
They are very similar, but they are two different languages.

When evaluating code (e.g. with C-x C-e), Emacs has to "guess" which
in Elisp this is supposedly written.  It does that by checking the value
of `lexical-binding' variable.  In face-remap.el, this variable is set
to t, whereas in *scratch* it is set to nil, so by moving the code from
one buffer to the other, you asked Emacs to interpret this code in "the
other Elisp".  And this failed because the code you copied relies on
behavior specific to lexical scoping (a lot of Elisp code doesn't care
either way).


        Stefan