bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Alan Mackenzie]

Hello, Emacs.

Emacs 25.3, Emacs 26.0.91 elisp manual.

In enough places in emacs, we find terms like "generalized variable"
"place form", and "place" being used.  These terms are not defined in
the Elisp manual, or any place where they are used.  This is a bug.

There is a page in elisp which purports to define "generalized
variable", but rather than defining the term, it talks vaguely around
it, saying it is "one of the many places in Lisp memory where values can
be stored".  Does this mean it is different from the other such places?
If so, how does it differ.  WHAT IS IT????

The elisp page then goes on to give examples of "generalized variables",
never defining the term.  It gives no criterion by which the reader can
determine whether some random object is a generalized variable or not.

I want to know whether a function is a "generalized variable".  After a
long time trying to find out, I still don't know.  I've been trying for
over an hour to use add-function, with forms like

    (add-function :before sit-for (lambda () (acm-backtrace 5)))
    (add-function :before 'sit-for (....))
    (add-function :before #'sit-for (.....))
    (add-function :before (symbol-function 'sit-for) (....))

, and got nothing but unhelpful error messages back, such as 

    Symbol's value as variable is void: sit-for

.  The documentation of add-function is likewise vague and unhelpful.
The various inflections of "sit-for" above are at a place in the
add-function form where a "generalized variable" is needed.  Is a
function a generalized variable or not?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Noam Postavsky]

On Wed, Jan 24, 2018 at 3:06 PM, Alan Mackenzie <acm@muc.de> wrote:

>     (add-function :before sit-for (lambda () (acm-backtrace 5)))
>     (add-function :before 'sit-for (....))
>     (add-function :before #'sit-for (.....))
>     (add-function :before (symbol-function 'sit-for) (....))
>
> , and got nothing but unhelpful error messages back, such as
>
>     Symbol's value as variable is void: sit-for

Your last form works for me, but the lambda has to accept the same
arguments as sit-for.

(add-function :before (symbol-function 'sit-for)
          (lambda (&rest _) (acm-backtrace 5)))

I think advice-add would make more sense for that use case though.

(advice-add 'sit-for :before (lambda (&rest _) (acm-backtrace 5)))

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Drew Adams]

> Emacs 25.3, Emacs 26.0.91 elisp manual.
> 
> In enough places in emacs, we find terms like "generalized variable"
> "place form", and "place" being used.  These terms are not defined in
> the Elisp manual, or any place where they are used.  This is a bug.
> 
> There is a page in elisp which purports to define "generalized
> variable", but rather than defining the term, it talks vaguely around
> it, saying it is "one of the many places in Lisp memory where values can
> be stored".  Does this mean it is different from the other such places?
> If so, how does it differ.  WHAT IS IT????
> 
> The elisp page then goes on to give examples of "generalized variables",
> never defining the term.  It gives no criterion by which the reader can
> determine whether some random object is a generalized variable or not.
> 
> I want to know whether a function is a "generalized variable".  After a
> long time trying to find out, I still don't know.  I've been trying for
> over an hour to use add-function, with forms like
> 
>     (add-function :before sit-for (lambda () (acm-backtrace 5)))
>     (add-function :before 'sit-for (....))
>     (add-function :before #'sit-for (.....))
>     (add-function :before (symbol-function 'sit-for) (....))
> 
> , and got nothing but unhelpful error messages back, such as
> 
>     Symbol's value as variable is void: sit-for
> 
> .  The documentation of add-function is likewise vague and unhelpful.
> The various inflections of "sit-for" above are at a place in the
> add-function form where a "generalized variable" is needed.  Is a
> function a generalized variable or not?

Hear, hear.

This is a more general problem, for all of our Common Lisp
emulation or Cl-inspired stuff.  (Not that the Emacs
implementation of generalized vars actually emulates what
Common Lisp has for generalized vars.)

When Emacs has something more or less borrowed or inspired
from Common Lisp, the Emacs doc explaining it (e.g. the
concepts) is sometimes somewhat paltry.  In such cases, you
it can help to (1) know that Emacs was inspired by Common
Lisp for that construct and (2) consult the Common Lisp doc.

To learn about generalized variables I think you need to
consult the Common Lisp doc, which is quite clear about it:

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node80.html

"The concept of variables named by symbols can be generalized to
any storage location that can remember one piece of data, no
matter how that location is named. Examples of such storage
locations are the car and cdr of a cons, elements of an array,
and components of a structure."

When you hear "generalized variable" just think `setf'.

For Emacs, we could conceivably have many more generalized
vars than we have now.  We might make it possible to use
`setf' for functions like these, for instance: `point',
`selected-* (*=...), `point-min|max', `current-*', ...

Not that we need that, but it could sometimes be useful.

In most cases it just provides a uniform way to change
something.  E.g., (setf (current-local-map) my-map)
instead of (use-local-map my-map).  The old Emacs update
functions are often simpler, but using `setf' means you
don't need to know the setter/update function; you just
need to know the getter/access function.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Noam Postavsky]

On Wed, Jan 24, 2018 at 4:25 PM, Drew Adams <drew.adams@oracle.com> wrote:

> For Emacs, we could conceivably have many more generalized
> vars than we have now.  We might make it possible to use
> `setf' for functions like these, for instance: `point',
> `selected-* (*=...), `point-min|max', `current-*', ...

We do seem have such setf functions for several of those, a lot of
them require loading cl-lib (historical reasons, I guess).

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Drew Adams]

> > For Emacs, we could conceivably have many more generalized
> > vars than we have now.  We might make it possible to use
> > `setf' for functions like these, for instance: `point',
> > `selected-* (*=...), `point-min|max', `current-*', ...
> 
> We do seem have such setf functions for several of those, a lot of
> them require loading cl-lib (historical reasons, I guess).

Yes, and in particular we have `cl-letf' (which is misnamed,
since it does not come from Common Lisp - it should be called
`letf').

I meant only that we could conceivably have more generalized
vars than we have now.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Eli Zaretskii]

> Date: Wed, 24 Jan 2018 13:25:27 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> 
> To learn about generalized variables I think you need to
> consult the Common Lisp doc, which is quite clear about it:
> 
> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node80.html
> 
> "The concept of variables named by symbols can be generalized to
> any storage location that can remember one piece of data, no
> matter how that location is named. Examples of such storage
> locations are the car and cdr of a cons, elements of an array,
> and components of a structure."

Maybe I'm missing something, but how is the above significantly
different from what we have in the ELisp manual now:

  A “generalized variable” or “place form” is one of the many places in
  Lisp memory where values can be stored.  The simplest place form is a
  regular Lisp variable.  But the CARs and CDRs of lists, elements of
  arrays, properties of symbols, and many other locations are also places
  where Lisp values are stored.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Drew Adams]

> > To learn about generalized variables I think you need to
> > consult the Common Lisp doc, which is quite clear about it:
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.cs.cmu.edu_Groups_AI_html_cltl_clm_node80.html&d=DwIDaQ&c=RoP1YumCX
> CgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=kI3P6ljGv6CTHIKju0jqInF6AOwMCYRDQUmqX2
> 2rJ98&m=uYx8D-
> JLTyaLnvAeQBooq2REnESUwTiKj48oTomgyLw&s=R0hqNECRUwHIUT4SaPqUG5lRICppWh0FMJr
> C-q7z3qw&e=
> >
> > "The concept of variables named by symbols can be generalized to
> > any storage location that can remember one piece of data, no
> > matter how that location is named. Examples of such storage
> > locations are the car and cdr of a cons, elements of an array,
> > and components of a structure."
> 
> Maybe I'm missing something, but how is the above significantly
> different from what we have in the ELisp manual now:
> 
>   A “generalized variable” or “place form” is one of the many places in
>   Lisp memory where values can be stored.  The simplest place form is a
>   regular Lisp variable.  But the CARs and CDRs of lists, elements of
>   arrays, properties of symbols, and many other locations are also places
>   where Lisp values are stored.

Would it be clearer if I said "To learn MORE about generalized
variables" instead of "To learn about generalized variables"?
Surely it's possible to learn SOMETHING about generalized vars
from the Emacs doc about them.

I did not suggest that the paragraph I quoted was the only
thing that the CL doc says about generalized vars.  I pointed
to the part of the CL doc that provides their explanation.

If you want to compare the Emacs explanation with the CL one,
compare all that is written at the URL I provided.  I think
you'll see a difference - it is, IMO, "significantly different".

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Eli Zaretskii]

> Date: Fri, 26 Jan 2018 08:15:32 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: acm@muc.de, 30241@debbugs.gnu.org
> 
> If you want to compare the Emacs explanation with the CL one,
> compare all that is written at the URL I provided.

I already did, and would expect you to assume that up front.

> I think you'll see a difference - it is, IMO, "significantly
> different".

Actually, no, I didn't.  I do see some additional explanations that
might have helped Alan understand the issue, but nothing
"significant".  So much so that I doubt Alan will find the CL docs
helpful after disliking our docs of the same subject, as he did, based
on his original bug report.  Of course, it's possible that I'm missing
something here.

Therefore, I invite Alan (and anyone else who'd like to chime in) to
please compare the CL docs on this matter with ours, and tell what
parts of the former made the issue "fall into place" (pun intended)
wrt this topic, where our docs don't.  Bonus points for proposing
patches for the relevant parts of the ELisp manual, to make this
subject's documentation "significantly" better.

Thanks.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Eli Zaretskii]

> Date: Sat, 27 Jan 2018 12:34:53 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: acm@muc.de, 30241@debbugs.gnu.org
> 
> > I think you'll see a difference - it is, IMO, "significantly
> > different".
> 
> Actually, no, I didn't.  I do see some additional explanations that
> might have helped Alan understand the issue, but nothing
> "significant".  So much so that I doubt Alan will find the CL docs
> helpful after disliking our docs of the same subject, as he did, based
> on his original bug report.  Of course, it's possible that I'm missing
> something here.
> 
> Therefore, I invite Alan (and anyone else who'd like to chime in) to
> please compare the CL docs on this matter with ours, and tell what
> parts of the former made the issue "fall into place" (pun intended)
> wrt this topic, where our docs don't.  Bonus points for proposing
> patches for the relevant parts of the ELisp manual, to make this
> subject's documentation "significantly" better.

Alan, any inputs on this?  As you disliked the current text so much,
I'd like to fix it if possible for Emacs 26.1.

TIA

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Eli Zaretskii]

> Date: Fri, 02 Feb 2018 12:33:04 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 30241@debbugs.gnu.org
> 
> > Date: Sat, 27 Jan 2018 12:34:53 +0200
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: acm@muc.de, 30241@debbugs.gnu.org
> > 
> > > I think you'll see a difference - it is, IMO, "significantly
> > > different".
> > 
> > Actually, no, I didn't.  I do see some additional explanations that
> > might have helped Alan understand the issue, but nothing
> > "significant".  So much so that I doubt Alan will find the CL docs
> > helpful after disliking our docs of the same subject, as he did, based
> > on his original bug report.  Of course, it's possible that I'm missing
> > something here.
> > 
> > Therefore, I invite Alan (and anyone else who'd like to chime in) to
> > please compare the CL docs on this matter with ours, and tell what
> > parts of the former made the issue "fall into place" (pun intended)
> > wrt this topic, where our docs don't.  Bonus points for proposing
> > patches for the relevant parts of the ELisp manual, to make this
> > subject's documentation "significantly" better.
> 
> Alan, any inputs on this?  As you disliked the current text so much,
> I'd like to fix it if possible for Emacs 26.1.

Ping!

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Alan Mackenzie]

Hello, Eli.

Sorry about the long delay.

On Sat, Jan 27, 2018 at 12:34:53 +0200, Eli Zaretskii wrote:
> > Date: Fri, 26 Jan 2018 08:15:32 -0800 (PST)
> > From: Drew Adams <drew.adams@oracle.com>
> > Cc: acm@muc.de, 30241@debbugs.gnu.org

[ .... ]

> > I think you'll see a difference - it is, IMO, "significantly
> > different".

> Actually, no, I didn't.  I do see some additional explanations that
> might have helped Alan understand the issue, but nothing
> "significant".  So much so that I doubt Alan will find the CL docs
> helpful after disliking our docs of the same subject, as he did, based
> on his original bug report.  Of course, it's possible that I'm missing
> something here.

> Therefore, I invite Alan (and anyone else who'd like to chime in) to
> please compare the CL docs on this matter with ours, and tell what
> parts of the former made the issue "fall into place" (pun intended)
> wrt this topic, where our docs don't.  Bonus points for proposing
> patches for the relevant parts of the ELisp manual, to make this
> subject's documentation "significantly" better.

I've read https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node80.html,
and found it clear indeed.  However it was also long.  After reading it,
the Elisp sections on generalised variables make much more sense.

This suggests that these Elisp sections contain the material, but are
not suitable for readers who don't already understand generalised
variables.

In that CL page, a generalised variable is effectively defined as
something you can use `setf' on within the first three paragraphs.  In
the elisp sections, that identification is not present in the opening
paragraphs - there is no definition on that opening page.  The first
sub-page does not define a generalised variable as something you can use
setf on - it merely says setf is a way to access one.  But that
definition needs to be in the top level page, and it needs to be clear
that it _is_ a definition.

The CL page gives a complete list of forms setf will work with.  The
elisp page merely gives a list, without it being clear whether that list
is complete or not.

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Drew Adams]

> > > I think you'll see a difference - it is, IMO, "significantly
> > > different".
> 
> > Actually, no, I didn't.  I do see some additional explanations that
> > might have helped Alan understand the issue, but nothing
> > "significant".  So much so that I doubt Alan will find the CL docs
> > helpful after disliking our docs of the same subject, as he did, based
> > on his original bug report.  Of course, it's possible that I'm missing
> > something here.
> 
> > Therefore, I invite Alan (and anyone else who'd like to chime in) to
> > please compare the CL docs on this matter with ours, and tell what
> > parts of the former made the issue "fall into place" (pun intended)
> > wrt this topic, where our docs don't.  Bonus points for proposing
> > patches for the relevant parts of the ELisp manual, to make this
> > subject's documentation "significantly" better.
> 
> I've read...,
> and found it clear indeed.  However it was also long.  After reading it,
> the Elisp sections on generalised variables make much more sense.
> 
> This suggests that these Elisp sections contain the material, but are
> not suitable for readers who don't already understand generalised
> variables.
> 
> In that CL page, a generalised variable is effectively defined as
> something you can use `setf' on within the first three paragraphs.  In
> the elisp sections, that identification is not present in the opening
> paragraphs - there is no definition on that opening page.  The first
> sub-page does not define a generalised variable as something you can use
> setf on - it merely says setf is a way to access one.  But that
> definition needs to be in the top level page, and it needs to be clear
> that it _is_ a definition.
> 
> The CL page gives a complete list of forms setf will work with.  The
> elisp page merely gives a list, without it being clear whether that list
> is complete or not.

Good summary.  FWIW, I agree.  It's about `setf', in particular.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I've read...,
  > > and found it clear indeed.  However it was also long.  After reading it,
  > > the Elisp sections on generalised variables make much more sense.

Is it possible a few more cross-references that explanation
would solve the problem?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)
Skype: No way! See https://stallman.org/skype.html.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Eli Zaretskii]

> Date: Sat, 10 Feb 2018 21:58:41 +0000
> Cc: Drew Adams <drew.adams@oracle.com>, 30241@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> I've read https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node80.html,
> and found it clear indeed.  However it was also long.  After reading it,
> the Elisp sections on generalised variables make much more sense.
> 
> This suggests that these Elisp sections contain the material, but are
> not suitable for readers who don't already understand generalised
> variables.
> 
> In that CL page, a generalised variable is effectively defined as
> something you can use `setf' on within the first three paragraphs.  In
> the elisp sections, that identification is not present in the opening
> paragraphs - there is no definition on that opening page.  The first
> sub-page does not define a generalised variable as something you can use
> setf on - it merely says setf is a way to access one.  But that
> definition needs to be in the top level page, and it needs to be clear
> that it _is_ a definition.

It seems strange to me to define something in terms of its uses, but
if people think this is better than the other way around, I don't
mind.

Can you propose how to change what we have now to include what you
think should be there?

> The CL page gives a complete list of forms setf will work with.  The
> elisp page merely gives a list, without it being clear whether that list
> is complete or not.

It's supposed to be a complete list, but if someone knows of other
forms, let them speak up.

Thanks.

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Alan Mackenzie]

Hello, Eli.

Maybe there's still a chance to get this doc fix into Emacs 26.  What do
you say?

On Sun, Feb 11, 2018 at 18:00:43 +0200, Eli Zaretskii wrote:
> > Date: Sat, 10 Feb 2018 21:58:41 +0000
> > Cc: Drew Adams <drew.adams@oracle.com>, 30241@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I've read https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node80.html,
> > and found it clear indeed.  However it was also long.  After reading it,
> > the Elisp sections on generalised variables make much more sense.

> > This suggests that these Elisp sections contain the material, but are
> > not suitable for readers who don't already understand generalised
> > variables.

> > In that CL page, a generalised variable is effectively defined as
> > something you can use `setf' on within the first three paragraphs.  In
> > the elisp sections, that identification is not present in the opening
> > paragraphs - there is no definition on that opening page.  The first
> > sub-page does not define a generalised variable as something you can use
> > setf on - it merely says setf is a way to access one.  But that
> > definition needs to be in the top level page, and it needs to be clear
> > that it _is_ a definition.

> It seems strange to me to define something in terms of its uses, but
> if people think this is better than the other way around, I don't
> mind.

> Can you propose how to change what we have now to include what you
> think should be there?

A proposed patch is below.

> > The CL page gives a complete list of forms setf will work with.  The
> > elisp page merely gives a list, without it being clear whether that list
> > is complete or not.

> It's supposed to be a complete list, but if someone knows of other
> forms, let them speak up.

OK, I've made this clear in the patch.

A quick summary of what I've changed:
(i) a GV is defined as something `setf' can be used to store into.
(ii) I've removed the insinuation that hackers would have difficulty
remembering two functions (a get and a set) rather than just one.
(iii) I've made clear that the list of GVs in the page is complete.
(iv) I've removed the suggestion that setf has superseded, or is in
the process of superseding, setq.
(v) I've made minor corrections to the English.



diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index e025d3fd10..355cf5a09a 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -2316,11 +2316,12 @@ Generalized Variables
 
 @cindex generalized variable
 @cindex place form
-A @dfn{generalized variable} or @dfn{place form} is one of the many places
-in Lisp memory where values can be stored.  The simplest place form is
-a regular Lisp variable.  But the @sc{car}s and @sc{cdr}s of lists, elements
-of arrays, properties of symbols, and many other locations are also
-places where Lisp values are stored.
+A @dfn{generalized variable} or @dfn{place form} is one of the many
+places in Lisp memory where values can be stored using the @code{setf}
+macro (@pxref{Setting Generalized Variables}).  The simplest place
+form is a regular Lisp variable.  But the @sc{car}s and @sc{cdr}s of
+lists, elements of arrays, properties of symbols, and many other
+locations are also places where Lisp values get stored.
 
 Generalized variables are analogous to lvalues in the C
 language, where @samp{x = a[i]} gets an element from an array
@@ -2341,8 +2342,8 @@ Setting Generalized Variables
 accepts arbitrary place forms on the left side rather than just
 symbols.  For example, @code{(setf (car a) b)} sets the car of
 @code{a} to @code{b}, doing the same operation as @code{(setcar a b)},
-but without having to remember two separate functions for setting and
-accessing every type of place.
+but without you having to use two separate functions for setting and
+accessing this type of place.
 
 @defmac setf [place form]@dots{}
 This macro evaluates @var{form} and stores it in @var{place}, which
@@ -2352,18 +2353,19 @@ Setting Generalized Variables
 @var{form}.
 @end defmac
 
-The following Lisp forms will work as generalized variables, and
-so may appear in the @var{place} argument of @code{setf}:
+The following Lisp forms are the forms in Emacs that will work as
+generalized variables, and so may appear in the @var{place} argument
+of @code{setf}:
 
 @itemize
 @item
-A symbol naming a variable.  In other words, @code{(setf x y)} is
-exactly equivalent to @code{(setq x y)}, and @code{setq} itself is
-strictly speaking redundant given that @code{setf} exists.  Many
-programmers continue to prefer @code{setq} for setting simple
-variables, though, purely for stylistic or historical reasons.
-The macro @code{(setf x y)} actually expands to @code{(setq x y)},
-so there is no performance penalty for using it in compiled code.
+A symbol.  In other words, @code{(setf x y)} is exactly equivalent to
+@code{(setq x y)}, and @code{setq} itself is strictly speaking
+redundant given that @code{setf} exists.  Most programmers will
+continue to prefer @code{setq} for setting simple variables, though,
+for stylistic and historical reasons.  The macro @code{(setf x y)}
+actually expands to @code{(setq x y)}, so there is no performance
+penalty for using it in compiled code.
 
 @item
 A call to any of the following standard Lisp functions:


> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#30241: Emacs 26.0.91: "Generalized variables" are not defined.

[Eli Zaretskii]

> Date: Tue, 20 Mar 2018 20:51:31 +0000
> Cc: drew.adams@oracle.com, 30241@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Maybe there's still a chance to get this doc fix into Emacs 26.  What do
> you say?

Documentation changes are always welcome on the release branch, as
they cannot possibly break anything.

Thanks.

bug#30241: (Emacs 26.0.91: "Generalized variables" are not defined.)

[Alan Mackenzie]

On Wed, Jan 24, 2018 at 20:15:02 +0000, GNU bug Tracking System wrote:

[ .... ]

Bug fixed, by enhancing the documentation, in commit
4ab455147069d4b7247ba3aff0da3dba3a671df6.

-- 
Alan Mackenzie (Nuremberg, Germany).