RE: Can't M-x compile-defun `edebug' because dynamic variables are falsely taken as lexical.

[Drew Adams <drew.adams@oracle.com>]

> > Permanently is not the opposite of locally.  Instead
> > of "permanently", shouldn't that text say "globally"?
> 
> Feel free to provide a patch that tries to clarify.

That will have to come from someone who understands
this better than I.  I'm hoping to understand it a
bit better from this thread, and then, hopefully,
from an updated doc.

> Part of the issue
> is that (defvar VAR VAL) is just like (defvar VAR) when it's "processed"
> by the byte-compiler: it won't affect bindings of VAR everywhere but
> only within the corresponding lexical scope.  But when (defvar VAR VAL)
> is *executed* then it subsequently will affect all interpreted code and
> all the code that is subsequently compiled in that same Emacs session,
> whereas *executing* (defvar VAR VAL) only affects the code subsequently
> interpreted, and only within the current lexical scope.

I'm not sure how that relates to permanently vs globally,
but OK, thanks.

> >>     if @var{value} is omitted then the variable is only
> >>     marked special locally (i.e.@: within the current
> >>     lexical scope, or file if at the top-level).
> > This feature doesn't exist in Common Lisp, AFAIK.
> 
> AFAIK, in CL you can get something similar with
> (declare (special foo)).

Yes, similar, but not exactly the same.  In CL, such
a declaration can only be used, I think, within
(certain) special forms.

In particular, I don't think there's any equivalent
for Emacs's file-local scope (other than putting the
declaration in a top-level `let', or similar - a
top-level `(locally (declare (special foo)))' perhaps).
But maybe I'm wrong about this.

The real point was about `defvar'.  CL's `defvar' is
not Emacs's, AFAIU.

> > In CL, `defvar' always proclaims the variable special,
> > whether or not an initial value is provided.
> 
> Indeed, Elisp is slightly different in this respect.

The key word there is "proclaim".  Function `proclaim':

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node104.html.

  "The function `proclaim' takes a DECL-SPEC as its
   argument and puts it into effect globally."

  "As a special case (so to speak), `proclaim' treats a
   `special' DECL-SPEC as applying to all bindings as
   well as to all references of the mentioned variables."

  "For example, after `(proclaim '(special x))' in a
   function definition such as `(defun example (x) ...)'
   the parameter X will be bound as a special (dynamic)
   variable rather than as a lexical (static) variable."

That's what CL `defvar' does - it declares the var to
be special, i.e., dynamic, everywhere.  There's no
shadowing (unlike the case for other proclamations).

  "`(defvar variable)' proclaims variable to be
   special (see `proclaim'), and may perform other
   system-dependent bookkeeping actions."

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node67.html

> > And I think that for CL, a variable that is ever
> > special is always and everywhere special, i.e., in
> > all contexts.  It's always bound dynamically.
> 
> In Common Lisp
> 
>    (list
>      (lambda (x)
>        (let ((y x))
>          (declare (special y))
>          (lambda (z) (+ y z))))
>      (lambda (x)
>        (let ((y x))
>          (lambda (z) (+ y z)))))
> 
> gives you two functions that don't behave the same because the `y`
> binding in the first is dynamically scoped whereas that same `y`
> binding is statically scoped in the second.

In CL parlance, those are not only not the same binding,
they are not the same variable `y'.  That's the point.

The same symbol (name) is used for those two different
`y' variables, but they're different vars (for CL).

The first variable named `y' is special, which means
that it's special everywhere (indefinite scope), and
it's duration is the duration of the executing code
that uses it (dynamic extent).

> I'm saying that there's more to it than just either always or
> never.  E.g. in Common Lisp after the system has *compiled* a chunk of
> code with (defvar VAR ...), VAR may or may not be special (i.e. the
> `defvar` must have affected the compiled code, but not necessarily the
> state of the system that compiled the file).

Again, I think maybe you're thinking of VAR being
occurrences of VAR as a symbol, as opposed to the
variable that's declared ("created" in a sense, but
not necessarily assigned a value - it might have
no value, i.e., in CL-speak, be "unbound").

The effect on the compiled code gets to the use
of the actual variable that was declared special.
And yes, occurrences of other, different variables
with the same name are not special.  Not only
"might not be special", but cannot be special -
there is only ever one special variable with a
given name.

(Again, this is my understanding.  It may be wrong.)

> [ In Elisp, the situation is somewhat similar, tho a bit
>   less undefined. ]
> 
> >> Does Common Lisp offer something like `special-variable-p`?
> > Not that I know of.
> 
> Indeed.  And for good reasons: being a function it's hard to make it
> provide the kind of semantic you expect from it, given the complexity
> and scope-sensitivity of "specialness".

I fully agree about that (as far as my understanding goes).
 
> > But it also doesn't use `defvar' with and without second arg to
> > declare specialness.
> 
> That's just a superficial syntactic difference.
> 
> Maybe the slightly deeper difference is that Common Lisp doesn't have
> a way to say "all bindings of VAR under the current scope should be
> dynamic", like (defvar VAR) does.  Instead it has (declare (special VAR))
> which only talks about one particular binding.

I don't think that's right, but perhaps I don't
follow you correctly.

CL's `defvar' _proclaims_ the variable to be
special (dynamic) - and the scope of that is
indefinite - unlimited.  It's not a lexical
scope (this binding or this file).

> But both Elisp's `defvar` and CL's `declare` introduce
> the same issue that specialness is not all-or-nothing

Dunno what that means, but I'm guessing that the
difference in how we're talking about this is
maybe the difference between how CL talks about
a "variable".

For the CL doc, a special variable is special
everywhere - no lexical scoping for it at all,
and its extent (lifetime) is dynamic.  And as a
consequence, there is only ever a single special
variable with any given name (symbol).  Its
bindings are always dynamic. Its scope remains
indefinite.

> and that a plain function like `special-variable-p`
> can't give quite the answer we'd ideally want.

On that we agree.
___


Which brings us back to the beginning: 

Maybe remove `special-variable-p' altogether?
Or if, as is apparently the case, it's needed for
the byte-compiler implementation, at least rename
it to indicate that it's internal, and remove it
from the manual?

I'm not trying to remake the Emacs design of
lexical scoping - or anything like that.  I'm
not even trying to change how it's presented in
the doc (though maybe from this thread you'll
have some slight changes you'd like to make).

This is really only about `special-variable-p'.
Both its name and its doc are currently quite
misleading.  And it doesn't seem like something
that users will ever use or should ever try to
use.  I've asked about that use case...