bug#18059: 24.3.92; defvar and special variables

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Hello,

From eval.c:

    /* A simple (defvar foo) with lexical scoping does "nothing" except
       declare that var to be dynamically scoped *locally* (i.e. within
       the current file or let-block).  */

This behavior isn't obvious, so it should be mentioned in the docstring
of "defvar" and in

   (info "(elisp) Defining Variables")


Thanks,

Michael.



In GNU Emacs 24.3.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2014-07-17 on drachen
Windowing system distributor `The X.Org Foundation', version 11.0.11599904
System Description:	Debian GNU/Linux testing (jessie)

Important settings:
  value of $LC_ALL: de_DE.utf8
  value of $LC_COLLATE: C
  value of $LC_TIME: C
  value of $LANG: de_DE.utf8
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

tags 18059 + patch
quit

Michael Heerdegen <michael_heerdegen@web.de> writes:

>>From eval.c:
>
>     /* A simple (defvar foo) with lexical scoping does "nothing" except
>        declare that var to be dynamically scoped *locally* (i.e. within
>        the current file or let-block).  */
>
> This behavior isn't obvious, so it should be mentioned in the docstring
> of "defvar" and in
>
>    (info "(elisp) Defining Variables")

How about this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 2218 bytes --]

From c284028ba554fb4292d3a3ef275351d5f2dda80e Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 10 Feb 2018 14:06:05 -0500
Subject: [PATCH] Explain more about (defvar foo) form (Bug#18059)

* doc/lispref/variables.texi (Defining Variables):
* doc/lispref/compile.texi (Compiler Errors): Emphasize that omitting
VALUE for `defvar' marks the variable special only locally.
---
 doc/lispref/compile.texi   |  4 ++--
 doc/lispref/variables.texi | 10 +++++++---
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/doc/lispref/compile.texi b/doc/lispref/compile.texi
index 0e39866d34..77d35be1ba 100644
--- a/doc/lispref/compile.texi
+++ b/doc/lispref/compile.texi
@@ -500,8 +500,8 @@ Compiler Errors
 @item
 Likewise, you can tell the compiler that a variable is defined using
 @code{defvar} with no initial value.  (Note that this marks the
-variable as special, i.e.@: dynamically bound.)  @xref{Defining
-Variables}.
+variable as special, i.e.@: dynamically bound, but only within the
+file.)  @xref{Defining Variables}.
 @end itemize
 
   You can also suppress any and all compiler warnings within a certain
diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index e025d3fd10..cd9e1afc97 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -442,9 +442,13 @@ Defining Variables
 evaluated and @var{symbol} is set to the result.  But if @var{symbol}
 is not void, @var{value} is not evaluated, and @var{symbol}'s value is
 left unchanged.  If @var{value} is omitted, the value of @var{symbol}
-is not changed in any case.  Using @code{defvar} with no value is one
-method of suppressing byte compilation warnings, see @ref{Compiler
-Errors}.
+is not changed in any case.
+
+Note that specifying a value, even @code{nil}, marks the variable as
+special permanently.  Whereas if @var{value} is omitted then the
+variable is only marked special locally (i.e. within the current file
+or let-block).  This can be useful for suppressing byte compilation
+warnings, see @ref{Compiler Errors}.
 
 If @var{symbol} has a buffer-local binding in the current buffer,
 @code{defvar} acts on the default value, which is buffer-independent,
-- 
2.11.0

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Hello Noam,

thanks for working on this.

> +Note that specifying a value, even @code{nil}, marks the variable as
> +special permanently.  Whereas if @var{value} is omitted then the
> +variable is only marked special locally (i.e. within the current file
> +or let-block).  This can be useful for suppressing byte compilation
> +warnings, see @ref{Compiler Errors}.

After reading this and thinking about it, I'm confused now what the
extent of `defvar' without a specified value is.

In the case of a file, does it mean that the variable is considered
special for the rest of the file, or for the whole file?

And for locally special variables, when you eval

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(defvar testfun nil)

(let ((x 1) f g)
  (defvar x)
  (setq testfun (lambda () x)))

(funcall testfun)
#+end_src

you get 1, but OTOH

#+begin_src emacs-lisp
;; -*- lexical-binding: t -*-

(defvar testfun nil)

(progn
  (defvar x)
  (let ((x 1) f g)
    (setq testfun (lambda () x))))

(funcall testfun)
#+end_src

gives you an error.  I don't really see the connection to a `let' block:
in the first example, the defvar is in a let block, but the variable is
not treated as special in the block.  In the second example, it's
outside, but the variable is treated as special (though the extent is
surely not limited to any `let' block).


Michael.

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> > +Note that specifying a value, even @code{nil}, marks the variable as
> > +special permanently.  Whereas if @var{value} is omitted then the
> > +variable is only marked special locally (i.e. within the current file
> > +or let-block).  This can be useful for suppressing byte compilation
> > +warnings, see @ref{Compiler Errors}.
> 
> After reading this and thinking about it, I'm confused now what the
> extent of `defvar' without a specified value is.
> 
> In the case of a file, does it mean that the variable is considered
> special for the rest of the file, or for the whole file?

Apologies for not following this. Question: is some change
in behavior being discussed here or is it just a question
about improving documentation of the longstanding behavior?

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Drew Adams <drew.adams@oracle.com> writes:

> Apologies for not following this. Question: is some change
> in behavior being discussed here or is it just a question
> about improving documentation of the longstanding behavior?

This is only about documentation.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> After reading this and thinking about it, I'm confused now what the
> extent of `defvar' without a specified value is.
>
> In the case of a file, does it mean that the variable is considered
> special for the rest of the file, or for the whole file?

The rest of the file, I believe.  AFAIK, both interpretation and
compilation are 1-pass.

> ;; -*- lexical-binding: t -*-
>
> (defvar testfun nil)
>
> (let ((x 1) f g)
>   (defvar x)
>   (setq testfun (lambda () x)))
>
> (funcall testfun)

> [...] you get 1

> I don't really see the connection to a `let' block:
> the defvar is in a let block, but the variable is
> not treated as special in the block.

Yes, but the let-binding happened before the defvar.  

> ;; -*- lexical-binding: t -*-
>
> (defvar testfun nil)
>
> (progn
>   (defvar x)
>   (let ((x 1) f g)
>     (setq testfun (lambda () x))))
>
> (funcall testfun)

> [...] gives you an error.  

> [the defvar is] outside, but the variable is treated as special
> (though the extent is surely not limited to any `let' block).

Right, because the defvar is not inside any let-block, so it applies to
the rest of the file.

Compare

    ;; -*- lexical-binding: t -*-

    (progn
      (defvar x))

    (let ((x 1))
      (setq testfun (lambda () x)))

    (message "%S" (funcall testfun))

vs

    ;; -*- lexical-binding: t -*-

    (let (_)
      (defvar x))

    (let ((x 1))
      (setq testfun (lambda () x)))

    (message "%S" (funcall testfun))

I noticed doing (let () (defvar x)) seems to be the same as (progn
(defvar x)), which may be a bug.

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> Right, because the defvar is not inside any let-block, so it applies to
> the rest of the file.
>
> Compare
>
>     ;; -*- lexical-binding: t -*-
>
>     (progn
>       (defvar x))
>
>     (let ((x 1))
>       (setq testfun (lambda () x)))
>
>     (message "%S" (funcall testfun))
>
> vs
>
>     ;; -*- lexical-binding: t -*-
>
>     (let (_)
>       (defvar x))
>
>     (let ((x 1))
>       (setq testfun (lambda () x)))
>
>     (message "%S" (funcall testfun))

Thanks for the examples.  Maybe - and this is my personal opinion - such
examples in the manual would not be too bad.  I find the issue hard to
understand without such examples, though it might be a secondary
subject.


> I noticed doing (let () (defvar x)) seems to be the same as (progn
> (defvar x)), which may be a bug.

I guess because the interpreter doesn't create a new lexical environment
in this case?  But indeed it seems to contradict the doc.


Michael.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Thanks for the examples.  Maybe - and this is my personal opinion - such
> examples in the manual would not be too bad.  I find the issue hard to
> understand without such examples, though it might be a secondary
> subject.

Yeah, it's just a bit tricky where to put them without cluttering up the
main explanation with this fairly (IMO) niche use-case.

>> I noticed doing (let () (defvar x)) seems to be the same as (progn
>> (defvar x)), which may be a bug.
>
> I guess because the interpreter doesn't create a new lexical environment
> in this case?  But indeed it seems to contradict the doc.

Although, maybe that means we should just change the doc.  Stefan, thoughts?

bug#18059: 24.3.92; defvar and special variables

[Stefan Monnier]

> I noticed doing (let () (defvar x)) seems to be the same as (progn
> (defvar x)), which may be a bug.

Both interpretations would make sense, so given the lack of
documentation about the intention, we could consider it as a feature as
much as a bug.

Have you checked whether the behavior is the same with the
byte-compiler?


        Stefan

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> I noticed doing (let () (defvar x)) seems to be the same as (progn
>> (defvar x)), which may be a bug.
>
> Both interpretations would make sense, so given the lack of
> documentation about the intention, we could consider it as a feature as
> much as a bug.
>
> Have you checked whether the behavior is the same with the
> byte-compiler?

Yes, it's the same.

I see that a lambda form scopes the defvar even if it has no parameters
though.  I.e., the following prints 1:


    ;; -*- lexical-binding: t -*-

    (lambda ()
      (defvar x))

    (let ((x 1))
      (setq testfun (lambda () x)))

    (message "%S" (funcall testfun))

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> I see that a lambda form scopes the defvar even if it has no
> parameters though.  I.e., the following prints 1:
>
>
>     ;; -*- lexical-binding: t -*-
>
>     (lambda ()
>       (defvar x))
>
>     (let ((x 1))
>       (setq testfun (lambda () x)))
>
>     (message "%S" (funcall testfun))

That defvar doesn't get evaluated, or do I miss something?


Michael.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>>     (lambda ()
>>       (defvar x))

> That defvar doesn't get evaluated, or do I miss something?

Oops, you are absolutely correct.  After putting a funcall around that,
I see the same behaviour with lambda and let (i.e., both only scope the
defvar if there is at least one variable bound), both interpreted and
compiled.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
>>> I noticed doing (let () (defvar x)) seems to be the same as (progn
>>> (defvar x)), which may be a bug.
>>
>> Both interpretations would make sense, so given the lack of
>> documentation about the intention, we could consider it as a feature as
>> much as a bug.
>>
>> Have you checked whether the behavior is the same with the
>> byte-compiler?
>
> Yes, it's the same.

It's the same when let-binding a lexical variable, but not when binding
a dynamic one.

    ;; -*- lexical-binding: t -*-

    (defvar foo-dynamic 'foo)

    (let ((foo-dynamic 99))
      (defvar x))

    (let ((x 1))
      (setq testfun (lambda () x)))

    (message "%S" (funcall testfun))

gives

    Symbol’s value as variable is void: x

when running the interpreted version, and

    1

when running the compiled version.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 1806 bytes --]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> Noam Postavsky <npostavs@users.sourceforge.net> writes:
>
>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>>
>>>> I noticed doing (let () (defvar x)) seems to be the same as (progn
>>>> (defvar x)), which may be a bug.
>>>
>>> Both interpretations would make sense, so given the lack of
>>> documentation about the intention, we could consider it as a feature as
>>> much as a bug.
>>>
>>> Have you checked whether the behavior is the same with the
>>> byte-compiler?
>>
>> Yes, it's the same.
>
> It's the same when let-binding a lexical variable, but not when binding
> a dynamic one.
>
>     ;; -*- lexical-binding: t -*-
>
>     (defvar foo-dynamic 'foo)
>
>     (let ((foo-dynamic 99))
>       (defvar x))
>
>     (let ((x 1))
>       (setq testfun (lambda () x)))
>
>     (message "%S" (funcall testfun))
>
> gives
>
>     Symbol’s value as variable is void: x
>
> when running the interpreted version, and
>
>     1
>
> when running the compiled version.

This let-block thing feels more like a bug to me, so I'm inclined to
leave it out of the manual.  I'm also not so sure about putting in
examples into the manual; the shortest and simplest I could come up with
is this, but I don't know that it's worth putting in.

    This example demonstrates the effects of using @code{defvar} without
    an initial value:

    @example
    @group
    (let ((x 'lexical))
      (defun get-lexical-x ()
        x))
    (defvar x)
    (defun get-dynamic-x ()
      x)

    (let ((x 'dynamic))
      (list (get-dynamic-x)
            (get-lexical-x)))
         @result{} (dynamic lexical)
    @end group
    @end example

Here's the patch I have in my local repo right now:


[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 2435 bytes --]

From 20757fcabb82828e63096bcd02faee2ec54e3afb Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 10 Feb 2018 14:06:05 -0500
Subject: [PATCH v2] Explain more about (defvar foo) form (Bug#18059)

* doc/lispref/variables.texi (Defining Variables):
* doc/lispref/compile.texi (Compiler Errors): Emphasize that omitting
VALUE for `defvar' marks the variable special only locally.
---
 doc/lispref/compile.texi   |  4 ++--
 doc/lispref/variables.texi | 11 ++++++++---
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/doc/lispref/compile.texi b/doc/lispref/compile.texi
index 0e39866d34..77d35be1ba 100644
--- a/doc/lispref/compile.texi
+++ b/doc/lispref/compile.texi
@@ -500,8 +500,8 @@ Compiler Errors
 @item
 Likewise, you can tell the compiler that a variable is defined using
 @code{defvar} with no initial value.  (Note that this marks the
-variable as special, i.e.@: dynamically bound.)  @xref{Defining
-Variables}.
+variable as special, i.e.@: dynamically bound, but only within the
+file.)  @xref{Defining Variables}.
 @end itemize
 
   You can also suppress any and all compiler warnings within a certain
diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index e025d3fd10..a6df16d03f 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -442,9 +442,13 @@ Defining Variables
 evaluated and @var{symbol} is set to the result.  But if @var{symbol}
 is not void, @var{value} is not evaluated, and @var{symbol}'s value is
 left unchanged.  If @var{value} is omitted, the value of @var{symbol}
-is not changed in any case.  Using @code{defvar} with no value is one
-method of suppressing byte compilation warnings, see @ref{Compiler
-Errors}.
+is not changed in any case.
+
+Note that specifying a value, even @code{nil}, marks the variable as
+special permanently.  Whereas if @var{value} is omitted then the
+variable is only marked special locally (i.e.@: for the rest of the
+current file).  This can be useful for suppressing byte compilation
+warnings, see @ref{Compiler Errors}.
 
 If @var{symbol} has a buffer-local binding in the current buffer,
 @code{defvar} acts on the default value, which is buffer-independent,
@@ -488,6 +492,7 @@ Defining Variables
 
 The @code{defvar} form returns @var{symbol}, but it is normally used
 at top level in a file where its value does not matter.
+
 @end defspec
 
 @cindex constant variables
-- 
2.11.0

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Noam Postavsky <npostavs@gmail.com> writes:

> This let-block thing feels more like a bug to me, so I'm inclined to
> leave it out of the manual.

Maybe somebody could clarify the rationale behind the behavior?

It would not be good to leave this undocumented - it is something people
actually do.


> I'm also not so sure about putting in examples into the manual; the
> shortest and simplest I could come up with is this, but I don't know
> that it's worth putting in.
>
>     This example demonstrates the effects of using @code{defvar} without
>     an initial value:
>
>     @example
>     @group
>     (let ((x 'lexical))
>       (defun get-lexical-x ()
>         x))
>     (defvar x)
>     (defun get-dynamic-x ()
>       x)
>
>     (let ((x 'dynamic))
>       (list (get-dynamic-x)
>             (get-lexical-x)))
>          @result{} (dynamic lexical)
>     @end group
>     @end example

Obviously I would like something like this being added.


Michael.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

On Sun, Feb 18, 2018 at 8:44 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:
> Noam Postavsky <npostavs@gmail.com> writes:
>
>> This let-block thing feels more like a bug to me, so I'm inclined to
>> leave it out of the manual.
>
> Maybe somebody could clarify the rationale behind the behavior?
>
> It would not be good to leave this undocumented - it is something people
> actually do.

Can I ask them to stop? ;)

Seriously though, I honestly can't see much use for a non-toplevel
defvar. Can you point to any examples?

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> Seriously though, I honestly can't see much use for a non-toplevel
> defvar. Can you point to any examples?

Seriously?

(when (> emacs-major-version 24)
  (defvar foo 42 "The answer."))

(when (fboundp 'fooness)
  (let ((the_answer  (fooness)))
    (defvar foo the_answer "The answer.")))

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

On Wed, Feb 21, 2018 at 11:08 AM, Drew Adams <drew.adams@oracle.com> wrote:
>> Seriously though, I honestly can't see much use for a non-toplevel
>> defvar. Can you point to any examples?
>
> Seriously?
>
> (when (> emacs-major-version 24)
>   (defvar foo 42 "The answer."))
>
> (when (fboundp 'fooness)
>   (let ((the_answer  (fooness)))
>     (defvar foo the_answer "The answer.")))

I meant real life examples.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

On Wed, Feb 21, 2018 at 11:23 AM, Noam Postavsky <npostavs@gmail.com> wrote:
> On Wed, Feb 21, 2018 at 11:08 AM, Drew Adams <drew.adams@oracle.com> wrote:
>>> Seriously though, I honestly can't see much use for a non-toplevel
>>> defvar. Can you point to any examples?
>>
>> Seriously?
>>
>> (when (> emacs-major-version 24)
>>   (defvar foo 42 "The answer."))
>>
>> (when (fboundp 'fooness)
>>   (let ((the_answer  (fooness)))
>>     (defvar foo the_answer "The answer.")))
>
> I meant real life examples.

Also, to clarify, I meant specifically non-toplevel (defvar foo), not
(defvar foo <value>).

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> Also, to clarify, I meant specifically non-toplevel (defvar foo), not
> (defvar foo <value>).

Too bad I didn't get your reply until after I spent time
digging out the latter, not the former.

I have FAR MORE examples of the former (vacuous defvars)
if you really need them.

Frankly, I find it hard to believe that someone would
doubt that there would be a use for non top-level
defvars.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

On Wed, Feb 21, 2018 at 12:15 PM, Drew Adams <drew.adams@oracle.com> wrote:
>> Also, to clarify, I meant specifically non-toplevel (defvar foo), not
>> (defvar foo <value>).
>
> Too bad I didn't get your reply until after I spent time
> digging out the latter, not the former.

Yeah, I broke my rule about not sending a reply immediately. If I had
waited with the draft a bit more, I probably would have included the
clarification in the first message.

> Not to mention lots of vacuous defvars to quiet the byte-compiler:
>
> (unless (> emacs-major-version 22)
>   (defvar display-buffer-reuse-frames))

So here, what is the use of putting it below the toplevel? Why don't
you just write

(defvar display-buffer-reuse-frames) ; For Emacs 22 and earlier.

> I have FAR MORE examples of the former (vacuous defvars)
> if you really need them.

I'd like to see more examples only if they would come with different
answers to my question above (why not write it at toplevel).
Otherwise, no need to repeat yourself.

Do you have any examples of the form (let (...) (defvar foo) ...)?
That's really where the troublesome behaviour that I'm hesitating to
document comes in.

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> Do you have any examples of the form (let (...) (defvar foo) ...)?
> That's really where the troublesome behaviour that I'm hesitating to
> document comes in.

No, I don't think I do.  But I can imagine that it can be
useful for code that generally has non-nil `lexical-binding'.

bug#18059: 24.3.92; defvar and special variables

[Stefan Monnier]

> Do you have any examples of the form (let (...) (defvar foo) ...)?

Look for calendar-dlet*


        Stefan

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> > Do you have any examples of the form (let (...) (defvar foo) ...)?
>
> Look for calendar-dlet*

Ah, fluid-let.  We also have a similar thing in
cedet/semantic/wisent/comp.el: `wisent-with-context'.

I wonder if it would be better to just provide fluid-let and give up the
special handled `defvar' case instead.


Michael.

bug#18059: 24.3.92; defvar and special variables

[Stefan Monnier]

> I wonder if it would be better to just provide fluid-let and give up the
> special handled `defvar' case instead.

This `defvar` case was devised specifically as a way to make it possible
to provide something like fluid-let without having to introduce a new
special form.


        Stefan

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> This `defvar` case was devised specifically as a way to make it
> possible to provide something like fluid-let without having to
> introduce a new special form.

Then we should document that use case.


Michael.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
>> This `defvar` case was devised specifically as a way to make it
>> possible to provide something like fluid-let without having to
>> introduce a new special form.
>
> Then we should document that use case.

Yes, you're right.  Since this case really relates more to the lexical
vs dynamic binding, I think it would make more sense to add the example
in a later section (with a link, of course).  I still don't think I can
claim with a straight face that (let (_) (defvar foo) ...) being
different than (let () (defvar foo) ...) is not a bug though.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 3601 bytes --]

From 3e62f1d6d892b26993ad5553ee17a2aaeae67d8b Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 10 Feb 2018 14:06:05 -0500
Subject: [PATCH v3] Explain more about (defvar foo) form (Bug#18059)

* doc/lispref/variables.texi (Defining Variables)
(Using Lexical Binding):
* doc/lispref/compile.texi (Compiler Errors): Emphasize that omitting
VALUE for `defvar' marks the variable special only locally.
---
 doc/lispref/compile.texi   |  5 +++--
 doc/lispref/variables.texi | 38 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 39 insertions(+), 4 deletions(-)

diff --git a/doc/lispref/compile.texi b/doc/lispref/compile.texi
index 0e39866d34..70da9727ee 100644
--- a/doc/lispref/compile.texi
+++ b/doc/lispref/compile.texi
@@ -500,8 +500,9 @@ Compiler Errors
 @item
 Likewise, you can tell the compiler that a variable is defined using
 @code{defvar} with no initial value.  (Note that this marks the
-variable as special, i.e.@: dynamically bound.)  @xref{Defining
-Variables}.
+variable as special, i.e.@: dynamically bound, but only for the rest
+of the current binding form, or file if at top-level.)
+@xref{Defining Variables}.
 @end itemize
 
   You can also suppress any and all compiler warnings within a certain
diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index e025d3fd10..9acdb210e1 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -442,8 +442,13 @@ Defining Variables
 evaluated and @var{symbol} is set to the result.  But if @var{symbol}
 is not void, @var{value} is not evaluated, and @var{symbol}'s value is
 left unchanged.  If @var{value} is omitted, the value of @var{symbol}
-is not changed in any case.  Using @code{defvar} with no value is one
-method of suppressing byte compilation warnings, see @ref{Compiler
+is not changed in any case.
+
+Note that specifying a value, even @code{nil}, marks the variable as
+special permanently.  Whereas if @var{value} is omitted then the
+variable is only marked special locally (i.e.@: for the rest of the
+current binding form, or file if at the top-level).  This can be
+useful for suppressing byte compilation warnings, see @ref{Compiler
 Errors}.
 
 If @var{symbol} has a buffer-local binding in the current buffer,
@@ -488,6 +493,9 @@ Defining Variables
 
 The @code{defvar} form returns @var{symbol}, but it is normally used
 at top level in a file where its value does not matter.
+
+For a more elaborate example of using @code{defvar} without a value,
+@xref{Local defvar example}.
 @end defspec
 
 @cindex constant variables
@@ -1164,6 +1172,32 @@ Using Lexical Binding
 (@pxref{Defining Variables}).  All other variables are subject to
 lexical binding.
 
+@anchor{Local defvar example}
+Using @code{defvar} without a value, it is possible to bind a variable
+dynamically just in one file, or in just one part of a file while
+still binding it lexically elsewhere.  For example:
+
+@example
+@group
+(let (_)
+  (defvar x)      ; @r{Let-bindings of @code{x} will be dynamic within this let.}
+  (let ((x -99))  ; @r{This is a dynamic binding of @code{x}.}
+    (defun get-dynamic-x ()
+      x)))
+
+(let ((x 'lexical)) ; @r{This is a lexical binding of @code{x}.}
+  (defun get-lexical-x ()
+    x))
+
+(let (_)
+  (defvar x)
+  (let ((x 'dynamic))
+    (list (get-lexical-x)
+          (get-dynamic-x))))
+    @result{} (lexical dynamic)
+@end group
+@end example
+
 @defun special-variable-p symbol
 This function returns non-@code{nil} if @var{symbol} is a special
 variable (i.e., it has a @code{defvar}, @code{defcustom}, or
-- 
2.11.0

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Noam Postavsky <npostavs@gmail.com> writes:

> From 3e62f1d6d892b26993ad5553ee17a2aaeae67d8b Mon Sep 17 00:00:00 2001
> From: Noam Postavsky <npostavs@gmail.com>
> Date: Sat, 10 Feb 2018 14:06:05 -0500
> Subject: [PATCH v3] Explain more about (defvar foo) form (Bug#18059)

Thanks for working on this, Noam.

> +variable as special, i.e.@: dynamically bound, but only for the rest
> +of the current binding form, or file if at top-level.)

Maybe we can be more precise than "current binding form" - "lexical
environment" maybe?


Michael.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>> +variable as special, i.e.@: dynamically bound, but only for the rest
>> +of the current binding form, or file if at top-level.)
>
> Maybe we can be more precise than "current binding form" - "lexical
> environment" maybe?

Hmm, how would that work?  I don't think "the rest of the lexical
environment" makes sense.  So maybe

    dynamically bound, until the current lexical environment ends, or
    file if at top-level.)

bug#18059: 24.3.92; defvar and special variables

[Michael Heerdegen]

Noam Postavsky <npostavs@gmail.com> writes:

> > Maybe we can be more precise than "current binding form" - "lexical
> > environment" maybe?
>
> Hmm, how would that work?  I don't think "the rest of the lexical
> environment" makes sense.  So maybe
>
>     dynamically bound, until the current lexical environment ends, or
>     file if at top-level.)

Sounds ok.  I don't have any better wording.


Michael.

bug#18059: 24.3.92; defvar and special variables

[Stefan Monnier]

>>     dynamically bound, until the current lexical environment ends, or
>>     file if at top-level.)
> Sounds ok.  I don't have any better wording.

I think the term to use here might be "lexical scope" rather than
"lexical environment" [ usually, an "environment" refers to a list of
bindings, whereas "scope" refers to an area of the program.  ]


        Stefan

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> I think the term to use here might be "lexical scope" rather than
> "lexical environment" [ usually, an "environment" refers to a list of
> bindings, whereas "scope" refers to an area of the program.  ]

So does "end of the current lexical scope" make sense?  I think I've
been looking at this for too long, it's hard for me to tell by now.

    Likewise, you can tell the compiler that a variable is defined using
    @code{defvar} with no initial value.  (Note that this marks the
    variable as special, i.e.@: dynamically bound, but only until the end
    of the current lexical scope, or file if at top-level.)

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> So does "end of the current lexical scope" make sense?  I think I've
> been looking at this for too long, it's hard for me to tell by now.
> 
>     Likewise, you can tell the compiler that a variable is defined using
>     @code{defvar} with no initial value.  (Note that this marks the
>     variable as special, i.e.@: dynamically bound, but only until the end
>     of the current lexical scope, or file if at top-level.)

 ... but only _within_ the current lexical scope,
 or within the current file if at top-level.

Lexical scope is defined lexically.  It can have two
limits or "ends", but it's best to avoid speaking of
beginning and end, as that can suggest temporal end.

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Drew Adams <drew.adams@oracle.com> writes:

>> So does "end of the current lexical scope" make sense?  I think I've
>> been looking at this for too long, it's hard for me to tell by now.
>> 
>>     Likewise, you can tell the compiler that a variable is defined using
>>     @code{defvar} with no initial value.  (Note that this marks the
>>     variable as special, i.e.@: dynamically bound, but only until the end
>>     of the current lexical scope, or file if at top-level.)
>
>  ... but only _within_ the current lexical scope,
>  or within the current file if at top-level.
>
> Lexical scope is defined lexically.  It can have two
> limits or "ends", but it's best to avoid speaking of
> beginning and end, as that can suggest temporal end.

I phrased it that way to rule out the interpretation where the defvar
could affect statements occurring earlier in the file, though still
within the scope.  I guess there can be wrong interpretations for any
phrasing though...

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18059#15:

    In the case of a file, does it mean that the variable is considered
    special for the rest of the file, or for the whole file?

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

tags 18059 fixed
close 18059 26.1
quit

Noam Postavsky <npostavs@gmail.com> writes:

> I guess there can be wrong interpretations for any
> phrasing though...

I went back to "within", pushed to emacs-26.

[1: 10b1f2fdd5]: 2018-03-23 08:19:09 -0400
  Explain more about (defvar foo) form (Bug#18059)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=10b1f2fdd5465407766790131b2ead3500d0798c

bug#18059: 24.3.92; defvar and special variables

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Date: Sun, 04 Mar 2018 18:27:54 -0500
> Cc: 18059@debbugs.gnu.org, Stefan Monnier <monnier@IRO.UMontreal.CA>
> 
> > Then we should document that use case.
> 
> Yes, you're right.  Since this case really relates more to the lexical
> vs dynamic binding, I think it would make more sense to add the example
> in a later section (with a link, of course).  I still don't think I can
> claim with a straight face that (let (_) (defvar foo) ...) being
> different than (let () (defvar foo) ...) is not a bug though.

Thanks.

> +For a more elaborate example of using @code{defvar} without a value,
> +@xref{Local defvar example}.

@xref is inappropriate in the middle of a sentence; please use "see
@ref" instead.

bug#18059: 24.3.92; defvar and special variables

[Noam Postavsky]

Noam Postavsky <npostavs@gmail.com> writes:

> On Wed, Feb 21, 2018 at 12:15 PM, Drew Adams <drew.adams@oracle.com> wrote:
>> Not to mention lots of vacuous defvars to quiet the byte-compiler:
>>
>> (unless (> emacs-major-version 22)
>>   (defvar display-buffer-reuse-frames))
>
> So here, what is the use of putting it below the toplevel? Why don't
> you just write
>
> (defvar display-buffer-reuse-frames) ; For Emacs 22 and earlier.

Hey Drew, I am actually interested in your answer to this question,
sorry if it came across as rhetorical.

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> >> Not to mention lots of vacuous defvars to quiet the byte-compiler:
> >>
> >> (unless (> emacs-major-version 22)
> >>   (defvar display-buffer-reuse-frames))
> >
> > So here, what is the use of putting it below the toplevel? Why don't
> > you just write
> >
> > (defvar display-buffer-reuse-frames) ; For Emacs 22 and earlier.
> 
> Hey Drew, I am actually interested in your answer to this question,
> sorry if it came across as rhetorical.

I didn't take it wrong, Noam; don't worry.

I don't have a good answer, in terms of "I _need_ to do it this
way because otherwise...".

My arguments:

1. Some users _will_ do it this way.  Nothing says they shouldn't.
   If that means they lose out on some functionality then they will
   need to know that.  And some users will have already done it,
   so they will need to change.

   Is that really necessary?  Should it be?  What's the real need
   here?

2. As one user, I _prefer_ to do it the way I do it, for facility
   of code maintenance, i.e., to document to myself which versions
   need what.  Yes, I could do that using only comments (but I find
   this way clearer, as it's the same way I do it where it really
   matters).

HTH - D.

bug#18059: 24.3.92; defvar and special variables

[Drew Adams]

> > (when (> emacs-major-version 24)
> >   (defvar foo 42 "The answer."))
> >
> > (when (fboundp 'fooness)
> >   (let ((the_answer  (fooness)))
> >     (defvar foo the_answer "The answer.")))
> 
> I meant real life examples.

Do these count?

(when (> emacs-major-version 24)
  (defvar bmkp-eww-history () "History for EWW bookmarks."))

(when (< emacs-major-version 23)
  (defvar bookmark-make-record-function 'bookmark-make-record-default
    "Function called with no arguments, to create a bookmark record.
...")

(when (or (featurep 'bookmark+-lit)
          (and (fboundp 'diredp-highlight-autofiles-mode)
               (featurep 'highlight)))
  (defvar bmkp-bmenu-highlight-menu (make-sparse-keymap "Highlight")
    "`Highlight' submenu for menu-bar `Bookmark+' menu.")
  ...)

(when (< emacs-major-version 21)
  (defvar custom-raised-buttons nil))

(when (fboundp 'dired-hide-details-mode) ; Emacs 24.4+
  (defvar diredp-hide-details-last-state
          diredp-hide-details-initially-flag
    "Last `dired-hide-details-mode' value. ...")

  (defvar diredp-hide-details-toggled nil
    "Non-nil means you have already ...")
  ...)

(when (fboundp 'epa-dired-do-encrypt)   ; Emacs 23+
  (defvar diredp-single-encryption-menu
          (make-sparse-keymap "Encryption")))

(when (require 'bookmark+ nil t)
  (defvar diredp-single-bookmarks-menu
          (make-sparse-keymap "Bookmark")))

(when (boundp 'Info-virtual-files)      ; Emacs 23.2+
  (defvar Info-indexed-file "*Indexed*"
    "Info file for virtual manual from ....")

  (defvar Info-indexed-nodes ()
    "Alist of cached nodes with matching index entries.))

(defun icicle-cmd2-after-load-hexrgb ()
  "..."
  (defvar icicle-named-colors () "Named colors.")
  ...)

;; The name changed during development of Emacs 23.
;; They aliased it for 23.1, but removed it for 23.2.
;; Use the new name and alias the old, but don't declare
;; old obsolete (let Emacs 23 do that.)
;;
(when (and (boundp 'minibuffer-local-must-match-filename-map)  
           (fboundp 'defvaralias)) ; Emacs 22
  (defvar minibuffer-local-filename-must-match-map
          minibuffer-local-must-match-filename-map
    "Local keymap for ....")
  (defvaralias 'minibuffer-local-must-match-filename-map
               'minibuffer-local-filename-must-match-map))

(eval-after-load "crm"
  '(progn
    ...
    (defvar icicle-ORIG-crm-local-completion-map
            crm-local-completion-map
      "Original CRM completion map.")
    ...))

(defun icicle-define-icicle-maps () "..."
  ...
  (unless icicle-menu-map
    ...
    (cond ((not icicle-touche-pas-aux-menus-flag)
           (defvar icicle-options-menu-map (make-sparse-keymap)
             "`Options' > `Icicles' submenu.")
           ...)
          (t
           (defvar icicle-options-menu-map (make-sparse-keymap)
             "`Icicles' > `Icicles Options' submenu.")
           ...))
    ...
    (defvar icicle-options-toggle-menu-map (make-sparse-keymap)
      "`Toggle' submenu of Icicles options menu.")
    ...
    (defvar icicle-options-choose-menu-map (make-sparse-keymap)
      "`Choose' submenu of Icicles options menu.")

    ... ; LOTS more
    ))

(when (> emacs-major-version 23)
  (defvar icicle-file-name-completion-table
    (completion-table-in-turn
      #'icicle-completion--embedded-envvar-table
      #'completion-file-name-table)
    "Completion table used for file-name completion."))

(when (require 'kmacro nil t) ; Emacs 22+
  (defvar icicle-kmacro-alist nil "Alist with elements ....")
  ...)

(when (and (fboundp 'read-char-by-name) ; Emacs 23-25
           (< emacs-major-version 26))
  (defvar icicle-read-char-history () "..."))

(when (require 'kmacro nil t); Emacs 22+
  (defvar icicle-saved-kmacro-ring-max kmacro-ring-max
    "Saved ...."))

(when (> emacs-major-version 23)        ; Emacs 24+
  (unless (boundp 'isearch-lax-whitespace) ; Emacs 24.1, 24.2.
    (defvar isearch-lax-whitespace t "...")
    (defvar isearch-regexp-lax-whitespace nil "..."))
    ...)
  ...)

(when (< emacs-major-version 24)
  (defvar isearch-face 'isearch "Face used to ...."))

(when (or (> emacs-major-version 24) ; Emacs 24.4+
          (and (= emacs-major-version 24)
               (> emacs-minor-version 3)))
  ...
  (defvar isearchp-lazy-regexp-level-overlays nil
    "Overlays used to ....")
  ...
  (defvar isearchp-filter-map nil "Keymap ....")
  ...
  (defvar isearchp-regexp-level-overlays nil
    "Overlays used to ....")

  (defvar isearchp-ffap-max-region-size 1024 ; See bug #25243.
    "Max size of active region used to ....")

  ... ; Plenty more
  )

(unless (boundp 'isearch-update-post-hook)
  (defvar isearch-update-post-hook () "Function(s) ...."))

(when (boundp 'isearch-lazy-highlight)  ; Emacs 22+
  (defvar isearchp-lazy-highlight-face 'lazy-highlight
    "Face ....")
  ...)

(when (or (featurep 'doremi-frm) (featurep 'doremi-cmd))
  (defvar menu-bar-doremi-menu (make-sparse-keymap "Do Re Mi"))
  ...)

(when (or (featurep 'frame-cmds) (featurep 'fit-frame))
  (defvar menu-bar-frames-menu (make-sparse-keymap "Frames"))
  ...)

(unless (< emacs-major-version 22)
  (defvar menu-bar-i-search-menu
    (make-sparse-keymap "Incremental Search"))
  ...)

(when (boundp 'pre-redisplay-function)  ; Emacs 24.4+
  ...
  (defvar mlw-pre-redisplay-selected-window nil
    "Window selected before redisplay.")
  ...
  (defvar mlw-orig-mode-line-buf-id
          (default-value 'mode-line-buffer-identification)
    "Original default value of ....")
  ...)

(when (featurep 'xemacs)
  ;; XEmacs which has no built-in minibuffer history truncation.
  (defvar history-length 100))

And plenty, plenty more.

Not to mention lots of vacuous defvars to quiet the byte-compiler:

(unless (> emacs-major-version 22)
  (defvar display-buffer-reuse-frames))

bug#18059: 24.3.92; defvar and special variables

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Sun, 11 Feb 2018 00:59:10 +0100
> Cc: 18059@debbugs.gnu.org
> 
> Hello Noam,
> 
> thanks for working on this.

Seconded.

> > +Note that specifying a value, even @code{nil}, marks the variable as
> > +special permanently.  Whereas if @var{value} is omitted then the
> > +variable is only marked special locally (i.e. within the current file

A minor Texinfo nit: if you want to use "i.e." without a following
comma, you need to follow it with "@:", to tell TeX this period
doesn't end a sentence.