bug#52290: 28.0.90; Undocumented generalized variables

bug#52290: 28.0.90; Undocumented generalized variables

[Phil Sainty]

Many standard generalized variables are not listed in the manual at
(info "(elisp)Setting Generalized Variables") or otherwise documented
as generalized vars so far as I can see.

The omission of `buffer-local-value' was what led me to check these,
as it's utilised by the likes of `electric-indent-local-mode' for
the :variable declaration, which I found confusing until I'd looked
at the code in detail, seen that `setf' was being used on that form,
and found that gv.el did indeed define this as a generalized var
even though the docs didn't mention it.

Comparing the manual node with the contents of gv.el...


In 27.2 there are fairly few omissions from the manual:

- buffer-local-value
- char-table-range
- cond
- cons
- edebug-after
- if
- let
- let*
- logand
- progn


In 28.0.90 the list is huge:

- buffer-file-name
- buffer-local-value
- buffer-modified-p
- buffer-name
- buffer-string
- buffer-substring
- char-table-range
- cond
- cons
- current-buffer
- current-column
- current-global-map
- current-input-mode
- current-local-map
- current-window-configuration
- default-file-modes
- documentation-property
- edebug-after
- eq
- error
- face-background
- face-background-pixmap
- face-font
- face-foreground
- face-underline-p
- file-modes
- frame-height
- frame-parameters
- frame-visible-p
- frame-width
- get-register
- getenv
- global-key-binding
- gv-deref
- if
- let
- let*
- local-key-binding
- logand
- mark
- mark-marker
- marker-position
- mouse-position
- plist-get
- point
- point-marker
- point-max
- point-min
- progn
- read-mouse-position
- screen-height
- screen-width
- selected-frame
- selected-screen
- selected-window
- standard-case-table
- substring
- syntax-table
- visited-file-modtime
- window-height
- window-width
- x-get-secondary-selection

This list of omissions is close to twice the length of the *documented*
cases.

I think the manual should list these, but also at this point I think the
help for any generalized variable should *automatically* state that 
fact,
so that any future omissions are still covered to some extent, and also
so that non-standard generalized vars defined by other libraries will
have some documentation.


-Phil





In GNU Emacs 28.0.90 (build 3, x86_64-pc-linux-gnu, X toolkit, cairo 
version 1.15.10, Xaw scroll bars)
  of 2021-12-03 built on phil-lp
Repository revision: 292ae07e7150a9515b587bd98f9e93ab42c3fb29
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 
11.0.12008000
System Description: Ubuntu 18.04.6 LTS

Configured using:
  'configure --prefix=/home/phil/emacs/28.0/usr/local
  --with-x-toolkit=lucid --without-sound --with-native-compilation'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM LUCID ZLIB

Important settings:
   value of $LC_MONETARY: en_NZ.UTF-8
   value of $LC_NUMERIC: en_NZ.UTF-8
   value of $LC_TIME: en_NZ.UTF-8
   value of $LANG: en_NZ.UTF-8
   value of $XMODIFIERS: @im=ibus
   locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
   tooltip-mode: t
   global-eldoc-mode: t
   eldoc-mode: t
   show-paren-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   indent-tabs-mode: t
   transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow mail-extr emacsbug message rmc puny dired dired-loaddefs rfc822
mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail
rmail-loaddefs auth-source eieio eieio-core eieio-loaddefs
password-cache json map text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils sort
comp comp-cstr warnings subr-x rx cl-seq cl-macs cl-extra help-mode seq
byte-opt gv cl-loaddefs cl-lib bytecomp byte-compile cconv rect
mule-util jka-compr info iso-transl tooltip eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
x multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 105925 9366)
  (symbols 48 8419 1)
  (strings 32 29142 2540)
  (string-bytes 1 935587)
  (vectors 16 18250)
  (vector-slots 8 355739 12899)
  (floats 8 34 34)
  (intervals 56 826 11)
  (buffers 992 13))

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Phil Sainty <psainty@orcon.net.nz> writes:

> In 28.0.90 the list is huge:

I don't think many of those are new -- they were just moved from
cl-lib.el to gv.el, because it seemed weird to have them in cl-lib.el.

And we probably want to deprecate a whole bunch of them, because many of
them are just nonsensical.

But we don't really have a deprecation mechanisme for generalised
variables, so it's just hard.

> I think the manual should list these, but also at this point I think
> the help for any generalized variable should *automatically* state
> that fact, so that any future omissions are still covered to some
> extent, and also so that non-standard generalized vars defined by
> other libraries will have some documentation.

The manual should absolutely not list most of these, because we don't
want anybody to use them, and we want to remove many (most?) of the
undocumented ones.

We just have to figure out how to do that in an orderly fashion.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52290: 28.0.90; Undocumented generalized variables

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> And we probably want to deprecate a whole bunch of them, because many of
> them are just nonsensical.

Which are nonsensical?

Michael.

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>> And we probably want to deprecate a whole bunch of them, because many of
>> them are just nonsensical.
>
> Which are nonsensical?

I think Stefan had a list of the worst ones...  `point-max', for
instance, is pretty egregious.  Well, looking at that list -- most of
are just the worst.  🤐

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52290: 28.0.90; Undocumented generalized variables

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think Stefan had a list of the worst ones...  `point-max', for
> instance, is pretty egregious.  Well, looking at that list -- most of
> are just the worst.  🤐

Ok, `point-max' is not so useful.

I only wanted to note that some generalized vars might be more useful
with `cl-letf' than with plain setf, you get some extra kinds of
excursions gratis (sometimes doesn't work as expected though).  Example:

#+begin_src emacs-lisp
(cl-letf (((cons (point-min) (point-max))
           (cons (line-beginning-position) (line-end-position))))
        (recursive-edit))
#+end_src

Michael.

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I only wanted to note that some generalized vars might be more useful
> with `cl-letf' than with plain setf, you get some extra kinds of
> excursions gratis (sometimes doesn't work as expected though).  Example:
>
> #+begin_src emacs-lisp
> (cl-letf (((cons (point-min) (point-max))
>            (cons (line-beginning-position) (line-end-position))))
>         (recursive-edit))
> #+end_src

That's...  quote obscure.  😀

Being able to say `(decf (point))' seems kinda nice, but I think even
that is a bit on the obscure side.  I.e., all the generalised variables
that are used for side effect (as opposed to mutating an explicit
object) are liable to cause confusion.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52290: 28.0.90; Undocumented generalized variables

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> [...]  I.e., all the generalised variables that are used for side
> effect (as opposed to mutating an explicit object) are liable to cause
> confusion.

I would want to limit this to the kind of side effect: is it undoubtedly
clear what it is, is there a "canonical" side effect?  This is the case
e.g. for `buffer-modified-p' to a high degree, but not for
e.g. `point-max': there are several ways to achieve that `point-max'
will return a certain value - killing a certain amount of text, for
example.  Or narrowing.  Narrowing was not the thing that came to my
mind first.  A setter for it might cause confusion because the semantics
are not clear, in contrast to `buffer-modified-p', I think, where it is
quite clear.

I mean, Emacs is an editor, so we have more aspects of state than
variable bindings.  Setting variables can also have other side effects
than simply changing the variable's binding.  Per se I don't see a
problem in considering more kinds of state (more than variables) as
places.  OTOH, `point-max' for example is not really a self-contained
part of state, it's a value of a computation, a derived value.

The classification result can be a bit subjective and depend on the
viewing point, of course.


Michael.

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I would want to limit this to the kind of side effect: is it undoubtedly
> clear what it is, is there a "canonical" side effect?

Nope, I don't think so.

Some time back, somebody wrote a library to make Emacs more
functional-ish, and the main bugaboo was Emacs' buffer concept.  I get
the feeling that many (some) people that haven't encountered it before
are just horrified by it.  "You mean...  you put...  text!!!...  into a
... buffer!!!!...  and then you operate on it!?  WHERE"S MY SMELLING
SALTS"

So the library was like (remove-empty-lines BUF) which returned a new
buffer with the empty lines removed.  (I think.  My brain may be making
that bit up.)

> This is the case e.g. for `buffer-modified-p' to a high degree, but
> not for e.g. `point-max': there are several ways to achieve that
> `point-max' will return a certain value - killing a certain amount of
> text, for example.  Or narrowing.  Narrowing was not the thing that
> came to my mind first.  A setter for it might cause confusion because
> the semantics are not clear, in contrast to `buffer-modified-p', I
> think, where it is quite clear.

Yeah, I think so to.  "Setting" `point-max' could mean so many different
things, but setting `buffer-modified-p' can only mean one thing, I
think.  (And note that `buffer-modified-p' has an (optional) buffer
parameter), so if we go by "a setter should always mention the object
it's setting", we're kinda covered.)

> I mean, Emacs is an editor, so we have more aspects of state than
> variable bindings.  Setting variables can also have other side effects
> than simply changing the variable's binding.  Per se I don't see a
> problem in considering more kinds of state (more than variables) as
> places.  OTOH, `point-max' for example is not really a self-contained
> part of state, it's a value of a computation, a derived value.
>
> The classification result can be a bit subjective and depend on the
> viewing point, of course.

Indeed.  I think `point' is perhaps the debatable tipping, er, point.
`(decf (point))' is pretty hard to misunderstand (as a synonym for
`(backward-char 1)'), but I think even that's too ... obscure.
Probably.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52290: 28.0.90; Undocumented generalized variables

[Phil Sainty]

On 2021-12-05 14:35, Lars Ingebrigtsen wrote:
> I don't think many of those are new -- they were just moved from
> cl-lib.el to gv.el, because it seemed weird to have them in cl-lib.el.

Aha. I'd seen that the cl docs had a bunch of extensions, but didn't
think to re-check those docs in 28.0.90.


> But we don't really have a deprecation mechanisme for generalised
> variables, so it's just hard.

I guess we'd want a new define-obsolete-* function, and for the setf
macro to be flagging uses of obsolete PLACE forms at compile time?

It looks to me as if `gv-get' might be the right place to be checking
this (but I'm only looking at the internals for the first time and
don't have a good handle on this stuff).


> The manual should absolutely not list most of these, because we don't
> want anybody to use them, and we want to remove many (most?) of the
> undocumented ones.
> 
> We just have to figure out how to do that in an orderly fashion.

Fair enough.  Should we start by deciding which ones we *should*
document, and at least get that much added for 28.1?

`buffer-local-value' is a clear "yes" vote from me (and I don't
currently have an opinion on anything else).


-Phil

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Phil Sainty <psainty@orcon.net.nz> writes:

> I guess we'd want a new define-obsolete-* function, and for the setf
> macro to be flagging uses of obsolete PLACE forms at compile time?

Yup.

> It looks to me as if `gv-get' might be the right place to be checking
> this (but I'm only looking at the internals for the first time and
> don't have a good handle on this stuff).

Perhaps Stefan has some opinions here.

> Fair enough.  Should we start by deciding which ones we *should*
> document, and at least get that much added for 28.1?
>
> `buffer-local-value' is a clear "yes" vote from me (and I don't
> currently have an opinion on anything else).

Yes, `buffer-local-value' seems useful.

I think we should implement the obsoletion mechanism and then just go
through that list and obsolete all the stuff that doesn't seem useful
(and isn't used in-tree).  And then document the rest, as well as make
the *Help* buffer mention that they're generalised variables.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think we should implement the obsoletion mechanism and then just go
> through that list and obsolete all the stuff that doesn't seem useful
> (and isn't used in-tree).  And then document the rest, as well as make
> the *Help* buffer mention that they're generalised variables.

The obsoletion mechanism is now in place in Emacs 29, and I've obsoleted
a whole bunch of nonsensical gv-setters.  (Feel free to obsolete more of
them, everybody.)

I've now made *Help* mention whether a function is a generalised
variable.

I'm not sure there's need to document all the generalised variables in
the manual, though.  The *Help* buffer should be sufficient, and if
they're not obvious (i.e., need documentation), then they should
probably be obsoleted.

But it might be nice to document, say, `if'.

bug#52290: 28.0.90; Undocumented generalized variables

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I've now made *Help* mention whether a function is a generalised
> variable.

+(defun help-fns--generalized-variable (function)
+  (when (get function 'gv-expander)
+    (insert (format-message "  `%s' is also a " function)
+            (buttonize "generalized variable"
+                       (lambda (_) (info "(elisp)Generalized Variables")))
+            ".\n")))

Can we try to find a better wording?  Not the function is a generalized
variable, a form that is a call of the function is.  This is also a
wording that the manual uses.  Generalized variables are an abstract
concept that not everybody is used to; I think we should be more
accurate to avoid to confuse people.

Michael.

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Can we try to find a better wording?  Not the function is a generalized
> variable, a form that is a call of the function is.  This is also a
> wording that the manual uses.  Generalized variables are an abstract
> concept that not everybody is used to; I think we should be more
> accurate to avoid to confuse people.

Yes, please do tweak the wording.  I couldn't find any concise way of
saying something here accurately, so I went for vague instead (note that
I don't mention "function" anywhere in the text), and just punt to the
manual.

bug#52290: 28.0.90; Undocumented generalized variables

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> > Can we try to find a better wording?  Not the function is a generalized
> > variable, a form that is a call of the function is. [...]

> Yes, please do tweak the wording.  I couldn't find any concise way of
> saying something here accurately, so I went for vague instead (note that
> I don't mention "function" anywhere in the text), and just punt to the
> manual.

Ok - not that easy!  Alternatives I see are "... `setf'able'" or
"... can be used in generalized variables" or "...in place expressions"
or "can be used in gv forms - see [Generalized Variables]."

Does something like that sound better than what we have?

BTW, another problem is that `alist-get' tells how it works as place
(because it's not totally trivial in this case), and at the end "... is
also a generalized variable" is added - kind of misplaced because the
reader already knows at this point.  Personally I would prefer that hint
near the beginning (without "also") - in all cases I think.

WDYT?

TIA,

Michael.

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Ok - not that easy!  Alternatives I see are "... `setf'able'" or
> "... can be used in generalized variables" or "...in place expressions"
> or "can be used in gv forms - see [Generalized Variables]."

"`setf'-able" is perhaps an improvement -- I think more people know what
that means than "generalized variable".

> BTW, another problem is that `alist-get' tells how it works as place
> (because it's not totally trivial in this case), and at the end "... is
> also a generalized variable" is added - kind of misplaced because the
> reader already knows at this point.  Personally I would prefer that hint
> near the beginning (without "also") - in all cases I think.

The user has asked for the documentation for the function, so "also" is
appropriate.  And the info (that's it's setf-able) is unlikely to be
what most people are interested in, so keeping it at the bottom is fine.

bug#52290: 28.0.90; Undocumented generalized variables

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I've now made *Help* mention whether a function is a generalised
> variable.
>
> I'm not sure there's need to document all the generalised variables in
> the manual, though.  The *Help* buffer should be sufficient, and if
> they're not obvious (i.e., need documentation), then they should
> probably be obsoleted.
>
> But it might be nice to document, say, `if'.

I've now mentioned if and cond in the manual here.

bug#52290: 28.0.90; Undocumented generalized variables

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I think people have extended setf in questionable ways,
allowing arithmetic and comparisons in the first argument.

I think I understand what (setq (logand x 7) NNN) does, but I think it
pushes the meaning of setf in a way that we should refrain from.

You could equally well define (setq (+ x 5) NNN), but why support
either of them?

What about (setq (+ x y) NNN) -- should that set x, or y?
Or both?  It could do (setq x 0 y NNN), or (setq x NNN y 0),
or various other things.

I don't think that eq in the first argument is coherent at all.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)