bug#23781: 25.0.95; read-string with HIST lexically bound

bug#23781: 25.0.95; read-string with HIST lexically bound

[Tino Calancha]

Hello,

recently i have noticed that arg HIST in read-from-minibuffer/read-string
needs to be dynamically bound.  If the file has the local variable
lexical-binding non-nil those functions just pick up one element of HIST.

In case this is a feature, it would be worth to mention it on the manual.


Following example reproduce the issue:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

I)

Create two files 'test-dynamic.el', 'test-lexical.el'
as follows:

*) test-dynamic.el:
(defun my-test-dynamic ()
   "Test of dynamic-binding."
   (interactive)
   (let ((values '("a" "b" "c")))
     (read-string "dynamic: " (car values) '(values . 1))))


*) test-lexical.el:
;; -*- lexical-binding: t; -*-

(defvar my-test-var nil)

(defun my-test-lexical-defvar ()
   "Test of lexical-binding with defvar."
   (interactive)
     (let ((values '("a" "b" "c")))
       (setq my-test-var values))
     (read-string "lexical-defvar: " (car my-test-var) '(my-test-var . 1)))

(defun my-test-lexical ()
   "Test of lexical-binding."
   (interactive)
   (let ((values '("a" "b" "c")))
     (read-string "lexical: " (car values) '(values . 1))))

II)
emacs -Q -l test-dynamic.el -l test-lexical.el

II.1)

M-x my-test-dynamic RET
;; minibuffer shows: dynamic: a
M-p
;; minibuffer shows: dynamic: b
M-p
;; minibuffer shows: dynamic: c

II.2) (Similar than II.1)

M-x my-test-lexical-defvar RET
;; minibuffer shows: lexical-defvar: a
M-p
;; minibuffer shows: lexical-defvar: b
M-p
;; minibuffer shows: lexical-defvar: c

II.3) (Different)
M-x my-test-lexical RET
;; minibuffer shows: lexical: a
M-p
;; user-error: Beginning of history; no preceding item


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

In GNU Emacs 25.0.95.1 (x86_64-pc-linux-gnu, GTK+ Version 3.20.6)
  of 2016-06-15 built on calancha-pc
Repository revision: d7084f2260943287cdfb5e3021ac33aab6a14c6d
Windowing system distributor 'The X.Org Foundation', version 11.0.11803000
System Description:    Debian GNU/Linux testing (stretch)

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
   tooltip-mode: t
   global-eldoc-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Quit [2 times]
user-error: Beginning of history; no preceding item [2 times]
Quit

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 86343 6478)
  (symbols 48 19751 0)
  (miscs 40 38 121)
  (strings 32 14375 4876)
  (string-bytes 1 409631)
  (vectors 16 11718)
  (vector-slots 8 428578 4129)
  (floats 8 164 84)
  (intervals 56 193 0)
  (buffers 976 11)
  (heap 1024 28246 991))

bug#23781: 25.0.95; read-string with HIST lexically bound

[Michael Heerdegen]

Tino Calancha <f92capac@gmail.com> writes:

> (defun my-test-lexical ()
>   "Test of lexical-binding."
>   (interactive)
>   (let ((values '("a" "b" "c")))
>     (read-string "lexical: " (car values) '(values . 1))))
>
> [...]
> II.3) (Different)
> M-x my-test-lexical RET
> ;; minibuffer shows: lexical: a
> M-p
> ;; user-error: Beginning of history; no preceding item

AFAICT this is expected and not specific to `read-string': A quoted
symbol can never refer to a lexical binding - and `symbol-value' or
`eval' always return the dynamic binding.

Some time ago, Stefan gave a good explanation about this topic (in
emacs-dev, I think).


Michael.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Noam Postavsky]

On Fri, Jun 17, 2016 at 11:25 AM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:
>
> AFAICT this is expected and not specific to `read-string': A quoted
> symbol can never refer to a lexical binding - and `symbol-value' or
> `eval' always return the dynamic binding.

Right, it's sort of hinted at in the elisp manual (11.9.3 Lexical Binding):

    Note that functions like ‘symbol-value’, ‘boundp’, and ‘set’ only
    retrieve or modify a variable’s dynamic binding (i.e., the
    contents of its symbol’s value cell).

I think we should be a little more specific, not
just give examples, something like:

    Note that functions which take a symbol argument (like
    ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
    variable’s dynamic binding (i.e., the contents of its symbol’s
    value cell).

Sort of releated: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21185

bug#23781: 25.0.95; read-string with HIST lexically bound

[Drew Adams]

> > AFAICT this is expected and not specific to `read-string': A quoted
> > symbol can never refer to a lexical binding - and `symbol-value' or
> > `eval' always return the dynamic binding.
> 
> Right, it's sort of hinted at in the elisp manual (11.9.3 Lexical Binding):
> 
>     Note that functions like ‘symbol-value’, ‘boundp’, and ‘set’ only
>     retrieve or modify a variable’s dynamic binding (i.e., the
>     contents of its symbol’s value cell).
> 
> I think we should be a little more specific, not
> just give examples, something like:
> 
>     Note that functions which take a symbol argument (like
>     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>     variable’s dynamic binding (i.e., the contents of its symbol’s
>     value cell).
> 
> Sort of releated: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21185

Be even more specific: A Lisp symbol is a dynamic thing.
It is an object.  Lexical binding has nothing to do with symbols.
A given _name_ in code can sometimes be lexically bound.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Noam Postavsky]

On Thu, Jun 23, 2016 at 7:18 PM, Drew Adams <drew.adams@oracle.com> wrote:
>> I think we should be a little more specific, not
>> just give examples, something like:
>>
>>     Note that functions which take a symbol argument (like
>>     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>>     variable’s dynamic binding (i.e., the contents of its symbol’s
>>     value cell).
>
> Be even more specific: A Lisp symbol is a dynamic thing.
> It is an object.  Lexical binding has nothing to do with symbols.
> A given _name_ in code can sometimes be lexically bound.

Hmm, this threatens to get a little philosophical, but that seems to
contradict earlier text in the same node:

       Here is how lexical binding works.  Each binding construct
    defines a “lexical environment”, specifying the symbols that are
    bound within the construct and their local values.  When the Lisp
    evaluator wants the current value of a variable, it looks first in
    the lexical environment; if the variable is not specified in
    there, it looks in the symbol’s value cell, where the dynamic
    value is stored.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Stephen Berman]

On Fri, 24 Jun 2016 20:26:45 -0400 Noam Postavsky <npostavs@users.sourceforge.net> wrote:

> On Thu, Jun 23, 2016 at 7:18 PM, Drew Adams <drew.adams@oracle.com> wrote:
>>> I think we should be a little more specific, not
>>> just give examples, something like:
>>>
>>>     Note that functions which take a symbol argument (like
>>>     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>>>     variable’s dynamic binding (i.e., the contents of its symbol’s
>>>     value cell).
>>
>> Be even more specific: A Lisp symbol is a dynamic thing.
>> It is an object.  Lexical binding has nothing to do with symbols.
>> A given _name_ in code can sometimes be lexically bound.
>
> Hmm, this threatens to get a little philosophical, but that seems to
> contradict earlier text in the same node:
>
>        Here is how lexical binding works.  Each binding construct
>     defines a “lexical environment”, specifying the symbols that are
>     bound within the construct and their local values.

I think it's more a question of definition than philosophy: AFAIU using
the word "symbols" here is strictly speaking incorrect; it should be
"variables".

Steve Berman

bug#23781: 25.0.95; read-string with HIST lexically bound

[Noam Postavsky]

On Sat, Jun 25, 2016 at 6:12 AM, Stephen Berman <stephen.berman@gmx.net> wrote:
> On Fri, 24 Jun 2016 20:26:45 -0400 Noam Postavsky <npostavs@users.sourceforge.net> wrote:
>
>> On Thu, Jun 23, 2016 at 7:18 PM, Drew Adams <drew.adams@oracle.com> wrote:
>>>> I think we should be a little more specific, not
>>>> just give examples, something like:
>>>>
>>>>     Note that functions which take a symbol argument (like
>>>>     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>>>>     variable’s dynamic binding (i.e., the contents of its symbol’s
>>>>     value cell).
>>>
>>> Be even more specific: A Lisp symbol is a dynamic thing.
>>> It is an object.  Lexical binding has nothing to do with symbols.
>>> A given _name_ in code can sometimes be lexically bound.
>>
>> Hmm, this threatens to get a little philosophical, but that seems to
>> contradict earlier text in the same node:
>>
>>        Here is how lexical binding works.  Each binding construct
>>     defines a “lexical environment”, specifying the symbols that are
>>     bound within the construct and their local values.
>
> I think it's more a question of definition than philosophy: AFAIU using
> the word "symbols" here is strictly speaking incorrect; it should be
> "variables".

Yes, perhaps the implementation details leaked a bit too much into the
description. I think it remains the case that the name of a variable
is a symbol (as opposed to a string) though.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Stephen Berman]

On Sat, 25 Jun 2016 12:53:18 -0400 Noam Postavsky <npostavs@users.sourceforge.net> wrote:

> On Sat, Jun 25, 2016 at 6:12 AM, Stephen Berman <stephen.berman@gmx.net> wrote:
>> On Fri, 24 Jun 2016 20:26:45 -0400 Noam Postavsky
>> <npostavs@users.sourceforge.net> wrote:
>>
>>> On Thu, Jun 23, 2016 at 7:18 PM, Drew Adams <drew.adams@oracle.com> wrote:
>>>>> I think we should be a little more specific, not
>>>>> just give examples, something like:
>>>>>
>>>>>     Note that functions which take a symbol argument (like
>>>>>     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>>>>>     variable’s dynamic binding (i.e., the contents of its symbol’s
>>>>>     value cell).
>>>>
>>>> Be even more specific: A Lisp symbol is a dynamic thing.
>>>> It is an object.  Lexical binding has nothing to do with symbols.
>>>> A given _name_ in code can sometimes be lexically bound.
>>>
>>> Hmm, this threatens to get a little philosophical, but that seems to
>>> contradict earlier text in the same node:
>>>
>>>        Here is how lexical binding works.  Each binding construct
>>>     defines a “lexical environment”, specifying the symbols that are
>>>     bound within the construct and their local values.
>>
>> I think it's more a question of definition than philosophy: AFAIU using
>> the word "symbols" here is strictly speaking incorrect; it should be
>> "variables".
>
> Yes, perhaps the implementation details leaked a bit too much into the
> description. I think it remains the case that the name of a variable
> is a symbol (as opposed to a string) though.

Well, the info node `(elisp) Variables' says:

   In Lisp, each variable is represented by a Lisp symbol (see
   Symbols::).  The variable name is simply the symbol’s name [...]

and a symbol's name is a string (according to `symbol-name').  But maybe
we should leave the bike shed before this gets too philosophical ;-)

Steve Berman 

bug#23781: 25.0.95; read-string with HIST lexically bound

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

On Sat, Jun 25, 2016 at 2:53 PM, Stephen Berman <stephen.berman@gmx.net> wrote:
> But maybe
> we should leave the bike shed before this gets too philosophical ;-)

Ugh, yes please.

So can we agree on this updated wording? (as shown in attached patch)

       Note that unlike dynamic variables which are tied to the symbol
    object itself, the relationship between lexical variables and
    symbols is only present in the interpreter (or compiler).
    Therefore, functions which take a symbol argument (like
    ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
    variable’s dynamic binding (i.e., the contents of its symbol’s
    value cell).  Also, the code in the body of a ‘defun’ or
    ‘defmacro’ cannot refer to surrounding lexical variables.

Should it be updated any further? (if yes, please reply with concrete proposals)

[-- Attachment #2: 0001-Clarify-lexical-binding-with-symbol-args-behavior.patch --]
[-- Type: text/x-patch, Size: 2145 bytes --]

From 151c2837dc5c82073465f6b68e0216a0e8149cf7 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Fri, 24 Jun 2016 20:39:24 -0400
Subject: [PATCH] Clarify lexical binding with symbol args behavior

* doc/lispref/variables.texi (Lexical Binding): Clarify that symbol
arguments always refer to dynamic values (Bug #23781).
---
 doc/lispref/variables.texi | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index 6c53e9b..b4d6857 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -974,7 +974,7 @@ Lexical Binding
 
 @cindex lexical environment
   Here is how lexical binding works.  Each binding construct defines a
-@dfn{lexical environment}, specifying the symbols that are bound
+@dfn{lexical environment}, specifying the variables that are bound
 within the construct and their local values.  When the Lisp evaluator
 wants the current value of a variable, it looks first in the lexical
 environment; if the variable is not specified in there, it looks in
@@ -1032,11 +1032,14 @@ Lexical Binding
 time we evaluate the closure, it increments @code{x}, using the
 binding of @code{x} in that lexical environment.
 
-  Note that functions like @code{symbol-value}, @code{boundp}, and
-@code{set} only retrieve or modify a variable's dynamic binding
-(i.e., the contents of its symbol's value cell).  Also, the code in
-the body of a @code{defun} or @code{defmacro} cannot refer to
-surrounding lexical variables.
+  Note that unlike dynamic variables which are tied to the symbol
+object itself, the relationship between lexical variables and symbols
+is only present in the interpreter (or compiler).  Therefore,
+functions which take a symbol argument (like @code{symbol-value},
+@code{boundp}, and @code{set}) can only retrieve or modify a
+variable's dynamic binding (i.e., the contents of its symbol's value
+cell).  Also, the code in the body of a @code{defun} or
+@code{defmacro} cannot refer to surrounding lexical variables.
 
 @node Using Lexical Binding
 @subsection Using Lexical Binding
-- 
2.8.0

bug#23781: 25.0.95; read-string with HIST lexically bound

[Stephen Berman]

On Sat, 25 Jun 2016 15:46:38 -0400 Noam Postavsky <npostavs@users.sourceforge.net> wrote:

> So can we agree on this updated wording? (as shown in attached patch)
>
>        Note that unlike dynamic variables which are tied to the symbol
>     object itself, the relationship between lexical variables and
>     symbols is only present in the interpreter (or compiler).
>     Therefore, functions which take a symbol argument (like
>     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>     variable’s dynamic binding (i.e., the contents of its symbol’s
>     value cell).  Also, the code in the body of a ‘defun’ or
>     ‘defmacro’ cannot refer to surrounding lexical variables.

This sounds clearer to me, thanks.

> Should it be updated any further? (if yes, please reply with concrete proposals)

I don't feel competent enough to judge that; however, Drew pointed out
that the `(elisp) Variables' node I quoted from earlier and other places
in the manual haven't been updated for lexical binding.  Anyway, these
questions would be more properly assigned to a separate bug report.

Steve Berman

bug#23781: 25.0.95; read-string with HIST lexically bound

[Michael Heerdegen]

Stephen Berman <stephen.berman@gmx.net> writes:

> >  Also, the code in the body of a ‘defun’ or
> >  ‘defmacro’ cannot refer to surrounding lexical variables.


This seems a bit unclear to me.  It sounds like something like this
would not work:

;; -*- lexical-binding: t -*-
(let ((x 1))
  (defun f () x))

(f) ==> 1


The term "surrounding" can describe different things here (mostly,
"lexically" vs. "dynamically" surrounding).


Michael.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

On Sat, Jun 25, 2016 at 7:42 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:
> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> >  Also, the code in the body of a ‘defun’ or
>> >  ‘defmacro’ cannot refer to surrounding lexical variables.
>
>
> This seems a bit unclear to me.  It sounds like something like this
> would not work:
>
> ;; -*- lexical-binding: t -*-
> (let ((x 1))
>   (defun f () x))
>
> (f) ==> 1

That's indeed what it meant, but this restriction was lifted sometime
after 24.1, see
http://help-gnu-emacs.gnu.narkive.com/uspqRdsq/surrounding-lexical-variable-reference-in-the-body-of-defun
and http://stackoverflow.com/questions/12026137/emacs-the-code-in-the-body-of-a-defun-or-defmacro-cannot-refer-to-surrounding-l.
Updated patch to remove that sentence.

[-- Attachment #2: v2-0001-Add-to-elisp-completion-at-point-s-docstring.patch --]
[-- Type: text/x-patch, Size: 2087 bytes --]

From 3f08a8a535e8fb5dced8bf1474659244ed358125 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 25 Jun 2016 16:23:04 -0400
Subject: [PATCH v2] Add to elisp-completion-at-point's docstring

* lisp/progmodes/elisp-mode.el (elisp-completion-at-point): Document
position dependent behavior.  Remove mention of obsolete restriction
regarding lexical binding for defun and defmacro, this no longer applies
since 61b108cc 2012-05-29 "* lisp/emacs-lisp/byte-run.el (defmacro,
defun): Move from C..." (Bug #19854).
---
 doc/lispref/variables.texi   | 3 +--
 lisp/progmodes/elisp-mode.el | 6 +++++-
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index b4d6857..a2d6481 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -1038,8 +1038,7 @@ Lexical Binding
 functions which take a symbol argument (like @code{symbol-value},
 @code{boundp}, and @code{set}) can only retrieve or modify a
 variable's dynamic binding (i.e., the contents of its symbol's value
-cell).  Also, the code in the body of a @code{defun} or
-@code{defmacro} cannot refer to surrounding lexical variables.
+cell).
 
 @node Using Lexical Binding
 @subsection Using Lexical Binding
diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index 7ad8871..850ebb4 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -455,7 +455,11 @@ elisp--company-location
      ((facep sym) (find-definition-noselect sym 'defface)))))
 
 (defun elisp-completion-at-point ()
-  "Function used for `completion-at-point-functions' in `emacs-lisp-mode'."
+  "Function used for `completion-at-point-functions' in `emacs-lisp-mode'.
+The returned completions depend on whether point is in a function
+or variable position; in positions where both are
+possible (e.g. quoted symbols), functions are annotated with
+\"<f>\" via the `:annotation-function' property."
   (with-syntax-table emacs-lisp-mode-syntax-table
     (let* ((pos (point))
 	   (beg (condition-case nil
-- 
2.8.0

bug#23781: 25.0.95; read-string with HIST lexically bound

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 226 bytes --]

On Sat, Jun 25, 2016 at 11:34 PM, Noam Postavsky
<npostavs@users.sourceforge.net> wrote:
> Updated patch to remove that sentence.

I messed up the patch (the change got combined with unrelated commit).
Here's a corrected one.

[-- Attachment #2: v3-0001-Clarify-lexical-binding-with-symbol-args-behavior.patch --]
[-- Type: text/x-patch, Size: 2240 bytes --]

From a669e69972cf6bcb9845a06a2427034b8bd18c0b Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Fri, 24 Jun 2016 20:39:24 -0400
Subject: [PATCH v3] Clarify lexical binding with symbol args behavior

* doc/lispref/variables.texi (Lexical Binding): Clarify that symbol
arguments always refer to dynamic values (Bug #23781).  Remove mention
of obsolete restriction regarding lexical binding for defun and
defmacro, this no longer applies since 61b108cc 2012-05-29 "*
lisp/emacs-lisp/byte-run.el (defmacro, defun): Move from C...".
---
 doc/lispref/variables.texi | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index 6c53e9b..a2d6481 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -974,7 +974,7 @@ Lexical Binding
 
 @cindex lexical environment
   Here is how lexical binding works.  Each binding construct defines a
-@dfn{lexical environment}, specifying the symbols that are bound
+@dfn{lexical environment}, specifying the variables that are bound
 within the construct and their local values.  When the Lisp evaluator
 wants the current value of a variable, it looks first in the lexical
 environment; if the variable is not specified in there, it looks in
@@ -1032,11 +1032,13 @@ Lexical Binding
 time we evaluate the closure, it increments @code{x}, using the
 binding of @code{x} in that lexical environment.
 
-  Note that functions like @code{symbol-value}, @code{boundp}, and
-@code{set} only retrieve or modify a variable's dynamic binding
-(i.e., the contents of its symbol's value cell).  Also, the code in
-the body of a @code{defun} or @code{defmacro} cannot refer to
-surrounding lexical variables.
+  Note that unlike dynamic variables which are tied to the symbol
+object itself, the relationship between lexical variables and symbols
+is only present in the interpreter (or compiler).  Therefore,
+functions which take a symbol argument (like @code{symbol-value},
+@code{boundp}, and @code{set}) can only retrieve or modify a
+variable's dynamic binding (i.e., the contents of its symbol's value
+cell).
 
 @node Using Lexical Binding
 @subsection Using Lexical Binding
-- 
2.8.0

bug#23781: 25.0.95; read-string with HIST lexically bound

[npostavs]

tags 23781 fixed
close 23781 25.1
quit

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> On Sat, Jun 25, 2016 at 11:34 PM, Noam Postavsky
> <npostavs@users.sourceforge.net> wrote:
>> Updated patch to remove that sentence.
>
> I messed up the patch (the change got combined with unrelated commit).
> Here's a corrected one.

I've pushed as 850ba444

bug#23781: 25.0.95; read-string with HIST lexically bound

[Drew Adams]

> > So can we agree on this updated wording? (as shown in attached patch)
> >
> >        Note that unlike dynamic variables which are tied to the symbol
> >     object itself, the relationship between lexical variables and
> >     symbols is only present in the interpreter (or compiler).
> >     Therefore, functions which take a symbol argument (like
> >     ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
> >     variable’s dynamic binding (i.e., the contents of its symbol’s
> >     value cell).  Also, the code in the body of a ‘defun’ or
> >     ‘defmacro’ cannot refer to surrounding lexical variables.
> 
> This sounds clearer to me, thanks.
> 
> > Should it be updated any further? (if yes, please reply with concrete
> proposals)
> 
> I don't feel competent enough to judge that; however, Drew pointed out
> that the `(elisp) Variables' node I quoted from earlier and other places
> in the manual haven't been updated for lexical binding.  Anyway, these
> questions would be more properly assigned to a separate bug report.

My comments were probably too simplistic.  It would be good for
someone to take a close look at all of the doc about symbols,
variables, and lexical/dynamic treatment.  But it is probably
not critical.  If someone does that, s?he probably needs to be
careful, and the result, if precise, might be too unreadable
for our manual.

I agree that any such consideration is outside this bug report.

FWIW, the CLTL2 section "3. Scope and Extent" is helpful here,
even though Emacs Lisp is not Common Lisp.

You will soon see there, however, that the relation between
symbols and variables is not so straightforward.  You will
right away see "referencable entities", which have both scope
and extent and which become "established" by "the execution
of some language construct".  Referencable entities include
function parameters and other variables, and even `catch'
and `throw' tags.

I recommend a (re-)reading of that section, asking yourself
why this or that term is introduced instead of just referring
to words like "variable".  None of the terms are introduced
gratuitously.

So yes, it would be good to be precise in the Elisp manual,
but some degree of imprecision can sometimes make for more,
not less clarity. ;-)

One thing that could perhaps be made more clear in the doc
is that the Lisp reader creates symbols (and conses and
vectors and...).  I'm not sure that this is made clear.

These created symbols etc. are used when the objects resulting
from reading are later evaluated.  IOW, even a referencable
entity such as a `catch' tag is associated with a symbol when
the `catch' code is read.  Likewise, for function parameters and
other variables, including variables that use lexical binding.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Drew Adams]

> Well, the info node `(elisp) Variables' says:
> 
>    In Lisp, each variable is represented by a Lisp symbol (see
>    Symbols::).  The variable name is simply the symbol’s name [...]
> 
> and a symbol's name is a string (according to `symbol-name').  But maybe
> we should leave the bike shed before this gets too philosophical ;-)

That was written (and was true) back when Emacs had only dynamic
variables, that is, symbols that act as variables, with name
`symbol-name' and value `symbol-value'.

Unfortunately, the developer who added lexical binding to Emacs
Lisp was not very strong or motivated in the Doc department (by
his own admission), so that text was not amended.  At least that's
my guess for why it still says that.

Section `Variable Scoping' was added to the manual to present
lexical binding.  And there is some hint of it in node `Local
Variables'.  The rest of the doc seems to generally be talking
about dynamic variables (even if local, buffer-local, file-local,
etc.).

bug#23781: 25.0.95; read-string with HIST lexically bound

[Drew Adams]

> > I think it's more a question of definition than philosophy: AFAIU using
> > the word "symbols" here is strictly speaking incorrect; it should be
> > "variables".
> 
> Yes, perhaps the implementation details leaked a bit too much into the
> description. I think it remains the case that the name of a variable
> is a symbol (as opposed to a string) though.

The _name_ of a variable is not a symbol.

A symbol is a Lisp object.  It can have a name (`symbol-name'), a
value as a (dynamic) variable (`symbol-value'), a value as a
function (`symbol-function'), and a list of arbitrary Lisp values
(`symbol-plist').

Other kinds of programming variables also exist in Emacs Lisp,
including lexical variables.  A `let' and similar constructs in
the language can bind a lexical variable or it can bind a dynamic
variable, that is, a symbol used as a variable.  In the latter
case, the bound value is available as the symbol's `symbol-value'.

bug#23781: 25.0.95; read-string with HIST lexically bound

[Tino Calancha]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]



On Thu, 23 Jun 2016, Noam Postavsky wrote:

> I think we should be a little more specific, not
> just give examples, something like:
>
>    Note that functions which take a symbol argument (like
>    ‘symbol-value’, ‘boundp’, and ‘set’) can only retrieve or modify a
>    variable’s dynamic binding (i.e., the contents of its symbol’s
>    value cell).
>
> Sort of releated: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21185
+1
I read that part of the manual before opening this bug and i was still 
confuse.