Proposal to change cursor appearance to indicate region activation

Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

By default in Emacs, when the region is active but empty, there's no visual indication of this status. Neither is there indication of the active region if point is one less than mark and blink-cursor-mode is off. Also, if point is greater than mark, the active region is highlighted, but the block cursor does an inverse-video highlight of the character following the region, which (speaking from experience) an Emacs newbie finds distracting, since it seems to indicate that that character is also part of the region, even though it actually isn't. Adding to the confusion, the inverse-video highlighted character actually is part of the region in the case that point is less than mark. Setting the cursor type to bar when the region is active solves all those problems, and I recommend it as the de
 fault for Emacs.

(add-hook 'deactivate-mark-hook (lambda () (setq cursor-type t)))
(add-hook 'activate-mark-hook (lambda () (setq cursor-type 'bar)))

Simply setting the cursor type permanently to bar would solve most of the problems, but still wouldn't indicate an active empty region. And a permanent bar cursor makes it hard to find the cursor on screen if blink mode is off. So I recommend keeping the current default of a block cursor when the region isn't active.

Stephan said (in comment about bug 14225) that if a dynamic cursor is the default, then people will probably start experiencing bugs because of it. For me it's only triggered bugs 13027, 14225, and maybe 13169, which are all fixed, and when I restart Emacs I get a bar cursor in each buffer (reloaded using desktop mode) until I do keyboard-quit, but it doesn't bother me much since I don't restart Emacs very often.

RE: Proposal to change cursor appearance to indicate region activation

[Drew Adams]

> By default in Emacs, when the region is active but empty, 
> there's no visual indication of this status.

See below.

> Neither is there indication of the active region if
> point is one less than mark and blink-cursor-mode is off.

Hm.  That's not what I see.

> Also, if point is greater than mark, the active region
> is highlighted, but the block cursor does an inverse-video
> highlight of the character following the region, which
> (speaking from experience) an Emacs newbie finds
> distracting, since it seems to indicate that that character
> is also part of the region

It's been a long since I was a newbie wrt transient-mark mode.  But I have a
hard time believing that inverse video has much, if anything, to do with it.

A block cursor itself is the cause of any such confusion, I think, regardless of
inverse video.  It sits on top of a char, so a priori it is ambiguous whether
the region includes that character or not.

Any editor would choose to be consistent here, whichever choice were made
(include it or not).  A newbie just has to discover which choice has been made.
;-)

And a moment's thought would anyway reach the conclusion that choosing to
include the char under the cursor would mean that the active region always has
at least one char (the one under the cursor).

> Adding to the confusion, the inverse-video 
> highlighted character actually is part of the region in the 
> case that point is less than mark.  Setting the cursor type to 
> bar when the region is active solves all those problems, and 
> I recommend it as the default for Emacs.

I don't see those as great problems.  But you could argue that I am already used
to the current way.

> (add-hook 'deactivate-mark-hook (lambda () (setq cursor-type t)))
> (add-hook 'activate-mark-hook (lambda () (setq cursor-type 'bar)))

I wouldn't have a problem with that being the default behavior, but I don't see
a crying need for it either.  You might also want to conditionalize the
activation part with (when transient-mark-mode (setq...)): there are still some
who are not transient-mark modists.

Making default behavior depend on such hooks makes it more difficult for users
(especially newbies) to change the behavior.  There could be ways around that,
wrapping the complexity in defcustoms or some such.  But as you present it, it
requires users to fiddle with Lisp a bit to make changes.

[I think you will find, BTW, that with recent Emacs 24 builds a `mouse-1' click
sometimes leaves the cursor as a bar and sometimes as a block.  A bug apparently
- it does not seem to happen with 24.3 or earlier.  Seems to depend on the line
clicked, for some reason.  At least that's what I see on MS Windows.]

> Simply setting the cursor type permanently to bar would solve 
> most of the problems, but still wouldn't indicate an active 
> empty region.  And a permanent bar cursor makes it hard to 
> find the cursor on screen if blink mode is off.  So I 
> recommend keeping the current default of a block cursor when 
> the region isn't active.

Yes.

A bar cursor does not particularly stand out as an indication of an empty active
region, however.  It might be better than no indication, but it is not really
going to prevent people from getting into the typical troubles from not
realizing that the region is active and empty.

FWIW, I use code[*] that indicates the size of the active region in the mode
line (using face `region').  It is an option (choice) whether it indicates an
empty active region also (useful or distracting, depending on your point of
view).

I also use a dynamically changing cursor[**] for other purposes than indicating
the active region:

* change color when using an input method
* change shape (to block, by default) when Emacs is idle
* change shape (to block, by default) when in overwrite mode or read-only

So in my case the cursor is a bar when I'm actually editing, regardless of
whether the region is active.  It is a block when Emacs is idle or the buffer is
read-only or in overwrite mode.

And I use `blink-cursor-mode', so the block cursor that appears when Emacs is
idle does not confuse wrt the active region.

In sum, your arguments are sound and your proposal is reasonable.  This is the
kind of simple observation and thinking that we need more of, IMHO.

But there are multiple ways to skin the cat.  I'm neutral on your proposal wrt
the resulting behavior, and slightly negative wrt the mechanism (hooks).

If we do what you suggest by default then it might be good to wrap the hook
stuff in defcustoms (e.g. :set sexps) or minor modes, so users don't need to
fiddle with Lisp to make simple behavior changes.  (No, it's not a big deal, but
why not make it easier?)

[*]
code: http://www.emacswiki.org/emacs-en/download/modeline-posn.el
description: http://www.emacswiki.org/ModeLinePosition

[**]
code: http://www.emacswiki.org/emacs-en/download/cursor-chg.el
description: http://www.emacswiki.org/ChangingCursorDynamically

[PATCH] Re: Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

Stefan Monnier wrote on 23 Nov 2013:
>> (add-hook 'deactivate-mark-hook (lambda () (setq cursor-type t)))
>> (add-hook 'activate-mark-hook (lambda () (setq cursor-type 'bar)))
>
> I like the idea, but this collides with other uses of cursor-type.
> Could someone cook up a patch which only does the above if cursor-type
> has not been modified (and which lets users opt-out if they prefer).
> Also, the patch should directly modify deactivate-mark and activate-mark
> rather than using the hooks.

Ok. Patch attached below.

With dynamic-cursor-mode turned off, it won't interfere with other uses of cursor-type.

And if you enable/disable dynamic-cursor-mode buffer-locally, it won't interfere with other uses of cursor-type in other buffers. In fact I have a couple of minor modes in my own packages that rely on this separation.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dynamic-cursor-mode.patch --]
[-- Type: text/x-diff, Size: 991 bytes --]

Third hunk adjusted to avoid conflict with current Emacs trunk.
--- emacs-24.4/lisp/simple.el
+++ emacs-24.4/lisp/simple.el
@@ -4391,6 +4391,12 @@
 (declare-function x-selection-exists-p "xselect.c"
                   (&optional selection terminal))
 
+(defcustom dynamic-cursor-mode 't
+  "If non-nil, `cursor-type' is set dynamically to reflect `mark-active'."
+  :type 'boolean
+  :version "25.1"
+  :group 'editing-basics)
+
 (defun deactivate-mark (&optional force)
   "Deactivate the mark.
 If Transient Mark mode is disabled, this function normally does
@@ -4430,6 +4436,7 @@
      ((eq transient-mark-mode 'lambda)
       (setq transient-mark-mode nil)))
     (setq mark-active nil)
+    (if dynamic-cursor-mode (setq cursor-type 't))
     (run-hooks 'deactivate-mark-hook)
     (redisplay--update-region-highlight (selected-window))))
 
@@ -4445,3 +4452,4 @@
+      (if dynamic-cursor-mode (setq cursor-type 'bar))
       (run-hooks 'activate-mark-hook))))
 
 (defun set-mark (pos)

Re: [PATCH] Re: Proposal to change cursor appearance to indicate region activation

[Stefan Monnier]

> +(defcustom dynamic-cursor-mode 't
> +  "If non-nil, `cursor-type' is set dynamically to reflect `mark-active'."
> +  :type 'boolean
> +  :version "25.1"
> +  :group 'editing-basics)

I'd rather see a `define-minor-mode' here, especially since you already
chose a "...-mode" name.  Nitpick: no need to quote t.

> @@ -4430,6 +4436,7 @@
>       ((eq transient-mark-mode 'lambda)
>        (setq transient-mark-mode nil)))
>      (setq mark-active nil)
> +    (if dynamic-cursor-mode (setq cursor-type 't))
>      (run-hooks 'deactivate-mark-hook)
>      (redisplay--update-region-highlight (selected-window))))
>
> @@ -4445,3 +4452,4 @@
> +      (if dynamic-cursor-mode (setq cursor-type 'bar))
>        (run-hooks 'activate-mark-hook))))
>
>  (defun set-mark (pos)

But if the user has set cursor-type in his .emacs, we'll now overwrite
his choice.  So we can't enable this code by default.  Two options:
- keep the default as nil.
- change the code to only modify the cursor-type if it hasn't been changed.


        Stefan

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

Stefan Monnier wrote:
> I'd rather see a `define-minor-mode' here, especially since you already
> chose a "...-mode" name.

Ok, the attached dynamic-cursor-mode-gross.patch does that.

Can't just add ⌜:global t⌝ to dynamic-cursor-mode, since the mode needs to be en/dis-ableable buffer-locally. That's why a pair of minor modes is necessary.

But I recommend against implementing the feature that way, since it's gross.

shift-select-mode is a defcustom, not an actual minor mode, so that's precedence. I guess it's bad precedence.

But assuming things with a ⌜-mode⌝ suffix in their names are supposed to be actual modes, would it be ok to just drop the suffix?
The attached dynamic-cursor-mode-1.patch does that.

Maybe add a ⌜use-⌝ prefix, like for use-empty-active-region (which is a defcustom)?

Or ⌜enable-⌝ prefix, like for enable-dir-local-variables, enable-recursive-minibuffers, and enable-multibyte-characters (which are all variables)?

> But if the user has set cursor-type in his .emacs, we'll now overwrite
> his choice.  So we can't enable this code by default.  Two options:
> - keep the default as nil.
> - change the code to only modify the cursor-type if it hasn't been changed.

Oops. Fixed in both of the attached patches.

Or did you mean _ever_ changed, rather than just at startup? In that case, what happens if the user first tries ⌜(setq-default cursor-type 'bar)⌝ while looking for nice features in Emacs, then later decides to try the dynamic cursor? It won't work, and that failure might seem like a bug, because by manually turning on the feature, he surely expects it to control the cursor, even though he set it to something nonstandard before.

It seems the right way would be for set-default and setq-default on cursor-type to automatically trigger a call of (setq-default dynamic-cursor nil), but I'm not aware of any feature in Emacs to trigger functions when variables are set. This is the same problem as for bug #19068, where there's no way for writes to the message-directory variable to automatically trigger updates to its dependent variables.

So I guess just keep the default as nil. :-(


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dynamic-cursor-mode-gross.patch --]
[-- Type: text/x-diff, Size: 1830 bytes --]

Third hunk adjusted to avoid conflict with current Emacs trunk.
--- emacs-24.4/lisp/simple.el
+++ emacs-24.4/lisp/simple.el
@@ -4391,6 +4391,31 @@
 (declare-function x-selection-exists-p "xselect.c"
                   (&optional selection terminal))
 
+(define-minor-mode dynamic-cursor-mode
+  "Toggle Dynamic Cursor mode.
+With a prefix argument ARG, enable Dynamic Cursor mode if ARG is
+positive, and disable it otherwise.  If called from Lisp, enable
+Dynamic Cursor mode if ARG is omitted or nil.
+
+Dynamic Cursor mode is a buffer-local minor mode.  When enabled,
+`cursor-type' is set dynamically to reflect `mark-active'.")
+
+(define-globalized-minor-mode global-dynamic-cursor-mode
+  dynamic-cursor-mode dynamic-cursor-mode)
+
+;; Enabling before init lets user disable in init file
+(global-dynamic-cursor-mode)
+
+;; Auto-disable to avoid conflict if cursor-type set in init file.
+;; Use emacs-startup-hook instead of after-init-hook in case
+;; cursor-type set in some file loaded as command line option.
+(defun maybe-disable--dynamic-cursor ()
+  (unless (eq (default-value 'cursor-type)
+	      (eval (car (get 'cursor-type 'standard-value))))
+    (global-dynamic-cursor-mode 0)))
+
+(add-hook 'emacs-startup-hook #'maybe-disable--dynamic-cursor)
+
 (defun deactivate-mark (&optional force)
   "Deactivate the mark.
 If Transient Mark mode is disabled, this function normally does
@@ -4430,6 +4455,7 @@
      ((eq transient-mark-mode 'lambda)
       (setq transient-mark-mode nil)))
     (setq mark-active nil)
+    (if dynamic-cursor-mode (setq cursor-type t))
     (run-hooks 'deactivate-mark-hook)
     (redisplay--update-region-highlight (selected-window))))
 
@@ -4445,3 +4471,4 @@
+      (if dynamic-cursor-mode (setq cursor-type 'bar))
       (run-hooks 'activate-mark-hook))))
 
 (defun set-mark (pos)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: dynamic-cursor-mode-1.patch --]
[-- Type: text/x-diff, Size: 1411 bytes --]

Third hunk adjusted to avoid conflict with current Emacs trunk.
--- emacs-24.4/lisp/simple.el
+++ emacs-24.4/lisp/simple.el
@@ -4391,6 +4391,22 @@
 (declare-function x-selection-exists-p "xselect.c"
                   (&optional selection terminal))
 
+(defcustom dynamic-cursor t
+  "If non-nil, `cursor-type' is set dynamically to reflect `mark-active'."
+  :type 'boolean
+  :version "25.1"
+  :group 'editing-basics)
+
+;; Auto-disable to avoid conflict if cursor-type set in init file.
+;; Use emacs-startup-hook instead of after-init-hook in case
+;; cursor-type set in some file loaded as command line option.
+(defun maybe-disable--dynamic-cursor ()
+  (unless (eq (default-value 'cursor-type)
+	      (eval (car (get 'cursor-type 'standard-value))))
+    (setq dynamic-cursor nil)))
+
+(add-hook 'emacs-startup-hook #'maybe-disable--dynamic-cursor)
+
 (defun deactivate-mark (&optional force)
   "Deactivate the mark.
 If Transient Mark mode is disabled, this function normally does
@@ -4430,6 +4446,7 @@
      ((eq transient-mark-mode 'lambda)
       (setq transient-mark-mode nil)))
     (setq mark-active nil)
+    (if dynamic-cursor (setq cursor-type t))
     (run-hooks 'deactivate-mark-hook)
     (redisplay--update-region-highlight (selected-window))))
 
@@ -4445,3 +4462,4 @@
+      (if dynamic-cursor (setq cursor-type 'bar))
       (run-hooks 'activate-mark-hook))))
 
 (defun set-mark (pos)

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Stefan Monnier]

> +(defun maybe-disable--dynamic-cursor ()
> +  (unless (eq (default-value 'cursor-type)
> +	      (eval (car (get 'cursor-type 'standard-value))))
> +    (global-dynamic-cursor-mode 0)))

Hmm... that's rather ugly.

I was thinking of rather doing something like:

(defvar-local dynamic-cursor-mode--set nil
  "If non-nil, we've temporarily modified the cursor.")

> +    (if dynamic-cursor-mode (setq cursor-type t))
       (when dynamic-cursor-mode
         (if (and dynamic-cursor-mode--set (eq cursor-type 'bar))
             (setq cursor-type t))
         (setq dynamic-cursor-mode--set nil))

and

> +      (if dynamic-cursor-mode (setq cursor-type 'bar))
         (if (and dynamic-cursor-mode (eq cursor-type t))
           (setq cursor-type 'bar dynamic-cursor-mode--set t))

So you don't need a buffer-local activation of dynamic-cursor-mode.


        Stefan

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

Stefan Monnier wrote:
>> +(defun maybe-disable--dynamic-cursor ()
>> +  (unless (eq (default-value 'cursor-type)
>> +	      (eval (car (get 'cursor-type 'standard-value))))
>> +    (global-dynamic-cursor-mode 0)))
>
> Hmm... that's rather ugly.

I agree.

> I was thinking of rather doing something like:
>
> (defvar-local dynamic-cursor-mode--set nil
>   "If non-nil, we've temporarily modified the cursor.")
>
>> +    (if dynamic-cursor-mode (setq cursor-type t))
>        (when dynamic-cursor-mode
>          (if (and dynamic-cursor-mode--set (eq cursor-type 'bar))
>              (setq cursor-type t))
>          (setq dynamic-cursor-mode--set nil))
>
> and
>
>> +      (if dynamic-cursor-mode (setq cursor-type 'bar))
>          (if (and dynamic-cursor-mode (eq cursor-type t))
>            (setq cursor-type 'bar dynamic-cursor-mode--set t))

That's even uglier! ;-) You still have a global minor mode (and its variable), and a separately-named buffer-local variable, so it's no simpler. And the logic is more complex.

And it appears it would fail in the example case I gave in my previous message:
(setq-default cursor-type 'bar)
A few days later in the same Emacs session...
(dynamic-cursor-mode)

The user would reasonably think the failure is a bug in dynamic-cursor-mode; after all, the point of the mode is to control the cursor.

I guess you could fix that failure, but make sure it doesn't then fail if the user set some other cursor type (e.g. 'hbar) instead of 'bar.

I agree that there shouldn't be a buffer-local minor mode for dynamic-cursor; it's overkill, since all it's used for is the variable. But the global minor mode that you want is also overkill for the same reason. I suggest using a defcustom and just dropping the potentially-misleading ⌜-mode⌝ suffix, and maybe adding a ⌜use-⌝ or ⌜enable-⌝ prefix, analogous to the other variables I cited.

> So you don't need a buffer-local activation of dynamic-cursor-mode.

I still would need it. If (global) dynamic-cursor-mode is enabled, your code provides no way to prevent it from operating in a buffer in which cursor-type happens to be t.

I agree that having a pair of minor modes is gross. The cleanest way to do it is with a defcustom (for which I can do (setq-local dynamic-cursor nil) when necessary) with no ⌜-mode⌝ suffix, and since Emacs lacks triggers on variable writes, simply have the default be nil in order to avoid the ugliness that would be needed to avoid conflict with other uses of cursor-type. Then dynamic-cursor's control of cursor-type won't surprise or annoy users, because they only get it if they choose to enable it.

Modifying the «set» family of functions to provide triggers would be the ideal solution, as I described in my previous message. E.g. have a 'triggers property for each symbol, storing a list of functions to run when the symbol (i.e. dynamic variable) is set, analogous to a hook. Then dynamic-cursor could default to t, with no ugliness needed to avoid conflict.

RE: [PATCH] Proposal to change cursor appearance to indicate region activation

[Drew Adams]

> > I was thinking of rather doing something like:...
> 
> That's even uglier! ;-) You still have a global minor mode (and its
> variable), and a separately-named buffer-local variable, so it's no
> simpler. And the logic is more complex.
> 
> And it appears it would fail in the example case I gave in my previous
> message: (setq-default cursor-type 'bar)
> A few days later in the same Emacs session... (dynamic-cursor-mode)
> 
> The user would reasonably think the failure is a bug in dynamic-cursor-mode;
> after all, the point of the mode is to control the cursor.
> 
> I guess you could fix that failure, but make sure it doesn't then fail if
> the user set some other cursor type (e.g. 'hbar) instead of 'bar.
> 
> I agree that there shouldn't be a buffer-local minor mode for dynamic-
> cursor; it's overkill, since all it's used for is the variable. But the
> global minor mode that you want is also overkill for the same reason. I
> suggest using a defcustom and just dropping the potentially-misleading ⌜-
> mode⌝ suffix, and maybe adding a ⌜use-⌝ or ⌜enable-⌝ prefix, analogous to
> the other variables I cited.
> 
> > So you don't need a buffer-local activation of dynamic-cursor-mode.
> 
> I still would need it. If (global) dynamic-cursor-mode is enabled, your code
> provides no way to prevent it from operating in a buffer in which cursor-
> type happens to be t.
> 
> I agree that having a pair of minor modes is gross. The cleanest way to do
> it is with a defcustom (for which I can do (setq-local dynamic-cursor nil)
> when necessary) with no ⌜-mode⌝ suffix, and since Emacs lacks triggers on
> variable writes, simply have the default be nil in order to avoid the
> ugliness that would be needed to avoid conflict with other uses of cursor-
> type. Then dynamic-cursor's control of cursor-type won't surprise or annoy
> users, because they only get it if they choose to enable it.
> 
> Modifying the «set» family of functions to provide triggers would be the
> ideal solution, as I described in my previous message. E.g. have a 'triggers
> property for each symbol, storing a list of functions to run when the symbol
> (i.e. dynamic variable) is set, analogous to a hook. Then dynamic-cursor
> could default to t, with no ugliness needed to avoid conflict.

I have not followed this in detail.  You've seen the approach I use in
`cursor-chg.el': just a global minor mode.  I'm wondering what else is
needed - what the use case is.

Are you trying to find a good way to let users opt out of automatic cursor
changing for particular buffers, i.e., disable the global nature of the
mode for some buffers?  If so, is that really an important feature?

I would think that needing to turn off the mode for a particular buffer
would be pretty rare, and could be accommodated just by toggling the
(global) mode temporarily when in such a buffer.  That toggling could be
manual (probably sufficient, I'm thinking) or automatic (e.g. on a mode
hook for the buffer or on `change-major-mode-hook').

RE: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

Drew Adams wrote:
> Are you trying to find a good way to let users opt out of automatic cursor
> changing for particular buffers, i.e., disable the global nature of the
> mode for some buffers?

I already found the right way before I even sent my patch. It's setq-local on a defcustom, and a pair of one-line additions to de/activate-mark that are simply predicated on that variable. It couldn't possibly be any simpler.

I just unwisely added a ⌜-mode⌝ suffix to the variable's name, and made the functional mistake of setting it to t by default. I should have followed your advice about the default. ;-)

> If so, is that really an important feature?
>
> I would think that needing to turn off the mode for a particular buffer
> would be pretty rare,

I have a couple buffer-local minor modes that need to do it, and I routinely have those modes active in some buffers but not in others.

> and could be accommodated just by toggling the
> (global) mode temporarily when in such a buffer.

That won't work, since my modes that need to turn off dynamic-cursor need to remain active (buffer-locally) even when I switch to another buffer, and dynamic-cursor should remain active in that other buffer.

Even if there were no need to enable/disable it buffer-locally, my original patch is still the simplest implementation. Just drop the ⌜-mode⌝ suffix and change the default to nil.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Stefan Monnier]

> functional mistake of setting it to t by default. I should have followed
> your advice about the default. ;-)

Having it nil by default makes it basically useless: there are already
packages (such as cursor-chg.el) which provided that kind
of functionality.


        Stefan

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

Stefan Monnier wrote:
> Having it nil by default makes it basically useless: there are already
> packages (such as cursor-chg.el) which provided that kind
> of functionality.

There are already other modes that are part of Emacs but nil by default. Some that I use include electric-pair, show-paren, winner, delete-selection, whitespace, savehist, and desktop-save.

And a couple days ago you wrote:
> Two options:
> - keep the default as nil.
> - change the code to only modify the cursor-type if it hasn't been changed.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Stefan Monnier]

>> Having it nil by default makes it basically useless: there are already
>> packages (such as cursor-chg.el) which provided that kind
>> of functionality.
> There are already other modes that are part of Emacs but nil by
> default.  Some that I use include electric-pair, show-paren, winner,
> delete-selection, whitespace, savehist, and desktop-save.

Of course.  But to me the whole point of this exercise is to change
the default.


        Stefan

[PATCH] Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 3796 bytes --]

Emacs lets you hook a mode, so you can run some function(s) whenever the mode is turned on or off.

The patch attached below lets you hook a symbol too, so you can run some function(s) whenever the global or buffer-local variable by that name is set, or a dynamic variable by that name is bound, set, or unbound.

It works for the entire family of Elisp «set» functions (and makunbound), in both Elisp and C. It also works for dynamic-let bindings. It is not implemented for lexical-let bindings.

The purpose of this new feature is to enable a proper fix for bug #19068, and to enable a solution to Stefan's requirement that dynamic-cursor-mode be enabled by default as a condition for adding the latter to Emacs.

Use it like this:
(defun tattle (sym env)
  (message "Symbol %S modified in env %S. New value: %S"
	   sym env (if (boundp sym) (symbol-value sym) 'none)))
(add-hook 'foo-varhook #'tattle)
(put 'foo 'varhook 'foo-varhook) ; The varhook property is the trigger

(setq foo 'bar) → ⌜Symbol foo modified in env global. New value: bar⌝

(setq lexical-binding nil)
(let ((foo 'bar1)) (setq foo 'bar2)) →
⌜Symbol foo modified in env dyn-bind. New value: bar1
Symbol foo modified in env dyn-local. New value: bar2
Symbol foo modified in env dyn-unbind. New value: bar⌝

(makunbound 'foo) → ⌜Symbol foo modified in env global. New value: none⌝
(setq-local foo 'bar) → ⌜Symbol foo modified in env buffer-local. New value: bar⌝
(makunbound 'foo) → ⌜Symbol foo modified in env buffer-local. New value: none⌝

The varhook property must be a hook. To turn off the varhook, set the property to nil.

The indirection through a hook, rather than putting the list of functions directly in the varhook property, lets you turn the varhook on/off without having to add/remove all your functions on the hook.

After you turn it off, if there are no other properties on the symbol, use
(setf (symbol-plist 'foo) nil)
to get rid of the superfluous property list that just records nil for varhook. If you leave the list there, it causes a minor slowdown (the time required to check whether the property is nil) when setting the symbol. The varhook feature is optimized to immediately skip a symbol if the property list is nil.

Each function on the hook must take two arguments:
0. The symbol S that was set (or bound or unbound), which is passed so you can have one function deal with multiple symbols rather than needing a separate function for each symbol.
1. The environment or event in which S was set/bound/unbound. This is one of the following symbols:
global: S was set in the global env.
buffer-local: S was set in the env of the current buffer.
dyn-local: S was set in the innermost dynamic env in which S is bound.
dyn-bind: S was bound in a new dynamic env (created by dynamic «let»).
dyn-unbind: The innermost dynamic env in which S was bound was destroyed.

For lexical bindings, varhook isn't triggered.

The names ⌜dyn-⌝ are used instead of ⌜let-⌝ for clarity, since ⌜let⌝ is also used for lexical bindings in Lisp.

If your function receives dyn-unbind and tries to read S, it will get the value bound to in an outer env, i.e. in the innermost dynamic env in which S is still bound, or in the buffer-local or global env.

If you're only interested in global settings, just wrap your hook function's body in
(when (eq env 'global) ...)

You get recursion if your function sets the symbol in any env (except lexical). Make sure you have a terminating condition.

The varhook is run not only when the symbol is set, but also when it's made unbound, either globally or buffer-locally. Make sure your function checks for this before trying to read the variable.

Patch applies to trunk.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: varhook.patch --]
[-- Type: text/x-diff, Size: 10243 bytes --]

--- src/lisp.h
+++ src/lisp.h
@@ -3391,6 +3391,14 @@
 EXFUN (Fbyteorder, 0) ATTRIBUTE_CONST;
 
 /* Defined in data.c.  */
+typedef enum
+  {
+    Dyn_Unbind = -1,
+    Dyn_Current = 0,
+    Dyn_Bind = 1,
+    Dyn_Skip = 2,
+    Dyn_Global = 3
+  } Dyn_Bind_Direction;
 extern Lisp_Object indirect_function (Lisp_Object);
 extern Lisp_Object find_symbol_value (Lisp_Object);
 enum Arith_Comparison {
@@ -3438,10 +3446,23 @@
 					   Lisp_Object);
 extern _Noreturn Lisp_Object wrong_type_argument (Lisp_Object, Lisp_Object);
 extern Lisp_Object do_symval_forwarding (union Lisp_Fwd *);
+extern void run_varhook (struct Lisp_Symbol*, bool, Dyn_Bind_Direction);
 extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool);
+extern void set_internal_1 (Lisp_Object, Lisp_Object, Lisp_Object, bool,
+			    Dyn_Bind_Direction);
+extern void set_default_internal (Lisp_Object, Lisp_Object,
+				  Dyn_Bind_Direction);
 extern void syms_of_data (void);
 extern void swap_in_global_binding (struct Lisp_Symbol *);
 
+INLINE void
+try_run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Direction dir)
+{
+  /* Avoid the call in the usual case of nil property list just to save time. */
+  if (!NILP (sym->plist))
+    run_varhook (sym, buf_local, dir);
+}
+
 /* Defined in cmds.c */
 extern void syms_of_cmds (void);
 extern void keys_of_cmds (void);
@@ -3905,9 +3926,9 @@
    should no longer be used.  */
 extern Lisp_Object Vrun_hooks;
 extern void run_hook_with_args_2 (Lisp_Object, Lisp_Object, Lisp_Object);
-extern Lisp_Object run_hook_with_args (ptrdiff_t nargs, Lisp_Object *args,
-				       Lisp_Object (*funcall)
-				       (ptrdiff_t nargs, Lisp_Object *args));
+extern Lisp_Object run_hook_with_args (ptrdiff_t, Lisp_Object *,
+				       Lisp_Object (*) (ptrdiff_t, Lisp_Object *));
+extern Lisp_Object funcall_nil (ptrdiff_t, Lisp_Object *);
 extern _Noreturn void xsignal (Lisp_Object, Lisp_Object);
 extern _Noreturn void xsignal0 (Lisp_Object);
 extern _Noreturn void xsignal1 (Lisp_Object, Lisp_Object);
--- src/eval.c
+++ src/eval.c
@@ -2357,7 +2357,7 @@
 \f
 /* Run hook variables in various ways.  */
 
-static Lisp_Object
+Lisp_Object
 funcall_nil (ptrdiff_t nargs, Lisp_Object *args)
 {
   Ffuncall (nargs, args);
@@ -3142,9 +3142,12 @@
       specpdl_ptr->let.old_value = SYMBOL_VAL (sym);
       grow_specpdl ();
       if (!sym->constant)
-	SET_SYMBOL_VAL (sym, value);
+	{
+	  SET_SYMBOL_VAL (sym, value);
+	  try_run_varhook (sym, false, Dyn_Bind);
+	}
       else
-	set_internal (symbol, value, Qnil, 1);
+	set_internal_1 (symbol, value, Qnil, 1, Dyn_Bind);
       break;
     case SYMBOL_LOCALIZED:
       if (SYMBOL_BLV (sym)->frame_local)
@@ -3176,7 +3179,7 @@
 	      {
 		specpdl_ptr->let.kind = SPECPDL_LET_DEFAULT;
 		grow_specpdl ();
-		Fset_default (symbol, value);
+		set_default_internal (symbol, value, Dyn_Bind);
 		return;
 	      }
 	  }
@@ -3184,7 +3187,7 @@
 	  specpdl_ptr->let.kind = SPECPDL_LET;
 
 	grow_specpdl ();
-	set_internal (symbol, value, Qnil, 1);
+	set_internal_1 (symbol, value, Qnil, 1, Dyn_Bind);
 	break;
       }
     default: emacs_abort ();
@@ -3320,6 +3323,7 @@
 	    if (sym->redirect == SYMBOL_PLAINVAL)
 	      {
 		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
+		try_run_varhook (sym, false, Dyn_Unbind);
 		break;
 	      }
 	    else
@@ -3329,8 +3333,8 @@
 	      }
 	  }
 	case SPECPDL_LET_DEFAULT:
-	  Fset_default (specpdl_symbol (specpdl_ptr),
-			specpdl_old_value (specpdl_ptr));
+	  set_default_internal (specpdl_symbol (specpdl_ptr),
+				specpdl_old_value (specpdl_ptr), Dyn_Unbind);
 	  break;
 	case SPECPDL_LET_LOCAL:
 	  {
@@ -3342,7 +3346,7 @@
 	    /* If this was a local binding, reset the value in the appropriate
 	       buffer, but only if that buffer's binding still exists.  */
 	    if (!NILP (Flocal_variable_p (symbol, where)))
-	      set_internal (symbol, old_value, where, 1);
+	      set_internal_1 (symbol, old_value, where, 1, Dyn_Unbind);
 	  }
 	  break;
 	}
@@ -3537,7 +3541,7 @@
 	    Lisp_Object sym = specpdl_symbol (tmp);
 	    Lisp_Object old_value = specpdl_old_value (tmp);
 	    set_specpdl_old_value (tmp, Fdefault_value (sym));
-	    Fset_default (sym, old_value);
+	    set_default_internal (sym, old_value, Dyn_Skip);
 	  }
 	  break;
 	case SPECPDL_LET_LOCAL:
@@ -3553,7 +3557,7 @@
 	      {
 		set_specpdl_old_value
 		  (tmp, Fbuffer_local_value (symbol, where));
-		set_internal (symbol, old_value, where, 1);
+		set_internal_1 (symbol, old_value, where, 1, Dyn_Skip);
 	      }
 	  }
 	  break;
--- src/data.c
+++ src/data.c
@@ -1167,6 +1168,42 @@
   xsignal1 (Qvoid_variable, symbol);
 }
 
+/* For the symbol S that was just set, if the varhook property is set to
+   a hook, run the functions on that hook. To those functions, pass S
+   as the first argument, and as the second argument, pass a symbol
+   indicating the environment in which S was set. */
+
+void
+run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Direction dir)
+{
+  Lisp_Object hook_and_args[3];
+  if (dir == Dyn_Skip) /* From backtrace_eval_unrewind */
+    return;
+  hook_and_args[0] = Fplist_get (sym->plist, Qvarhook);
+  if (NILP (hook_and_args[0]))
+    return;
+  XSETSYMBOL (hook_and_args[1], sym);
+  switch (dir)
+    {
+    case Dyn_Current:
+      {
+	bool shadowed;
+	if (buf_local)
+	  shadowed = let_shadows_buffer_binding_p (sym);
+	else shadowed = let_shadows_global_binding_p (hook_and_args[1]);
+	if (shadowed) hook_and_args[2] = Qdyn_local;
+	else if (buf_local) hook_and_args[2] = Qbuffer_local;
+	else hook_and_args[2] = Qglobal;
+	break;
+      }
+    case Dyn_Global: hook_and_args[2] = Qglobal; break;
+    case Dyn_Bind: hook_and_args[2] = Qdyn_bind; break;
+    case Dyn_Unbind: hook_and_args[2] = Qdyn_unbind; break;
+    default: emacs_abort ();
+    }
+  run_hook_with_args (3, hook_and_args, funcall_nil);
+}
+
 DEFUN ("set", Fset, Sset, 2, 2, 0,
        doc: /* Set SYMBOL's value to NEWVAL, and return NEWVAL.  */)
   (register Lisp_Object symbol, Lisp_Object newval)
@@ -1187,6 +1224,21 @@
 set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
 	      bool bindflag)
 {
+  set_internal_1 (symbol, newval, where, bindflag, Dyn_Current);
+}
+
+/* Like set_internal but with direction argument to indicate whether this
+   function call is due to a binding (1), an unbinding (-1), or neither (0).
+   As special cases, a value of 2 is a flag to disable run_varhook so that
+   varhooks aren't run during backtraces, and a value of 3 is a flag
+   indicating that this function call is due to set_default, which allows
+   run_varhook to distinguish beween the global and the dyn-local binding.
+*/
+
+void
+set_internal_1 (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+		bool bindflag, Dyn_Bind_Direction dir)
+{
   bool voide = EQ (newval, Qunbound);
   struct Lisp_Symbol *sym;
   Lisp_Object tem1;
@@ -1212,9 +1264,15 @@
   switch (sym->redirect)
     {
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
+    case SYMBOL_PLAINVAL:
+      {
+	SET_SYMBOL_VAL (sym, newval);
+	try_run_varhook (sym, false, dir);
+	return;
+      }
     case SYMBOL_LOCALIZED:
       {
+	bool buf_local = true;
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 	if (NILP (where))
 	  {
@@ -1258,6 +1316,7 @@
 		   indicating that we're seeing the default value.
 		   Likewise if the variable has been let-bound
 		   in the current buffer.  */
+		buf_local = false;
 		if (bindflag || !blv->local_if_set
 		    || let_shadows_buffer_binding_p (sym))
 		  {
@@ -1299,6 +1358,7 @@
 				       BUFFERP (where)
 				       ? XBUFFER (where) : current_buffer);
 	  }
+	try_run_varhook (sym, buf_local, dir);
 	break;
       }
     case SYMBOL_FORWARDED:
@@ -1324,6 +1384,9 @@
 	  }
 	else
 	  store_symval_forwarding (/* sym, */ innercontents, newval, buf);
+	try_run_varhook (sym,
+			 (XFWDTYPE (innercontents))==Lisp_Fwd_Buffer_Obj,
+			 dir);
 	break;
       }
     default: emacs_abort ();
@@ -1413,6 +1476,17 @@
 for this variable.  */)
   (Lisp_Object symbol, Lisp_Object value)
 {
+  set_default_internal (symbol, value, Dyn_Global);
+  return value;
+}
+
+/* Like Fset_default, but with direction argument. See set_internal_1 for
+   a description of this argument. */
+
+void
+set_default_internal (Lisp_Object symbol, Lisp_Object value,
+		      Dyn_Bind_Direction dir)
+{
   struct Lisp_Symbol *sym;
 
   CHECK_SYMBOL (symbol);
@@ -1423,7 +1497,7 @@
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
-	return value;
+	return;
     }
   sym = XSYMBOL (symbol);
 
@@ -1431,7 +1505,11 @@
   switch (sym->redirect)
     {
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: return Fset (symbol, value);
+    case SYMBOL_PLAINVAL:
+      {
+	set_internal_1 (symbol, value, Qnil, false, dir);
+	return;
+      }
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
@@ -1442,7 +1520,8 @@
 	/* If the default binding is now loaded, set the REALVALUE slot too.  */
 	if (blv->fwd && EQ (blv->defcell, blv->valcell))
 	  store_symval_forwarding (blv->fwd, value, NULL);
-	return value;
+	try_run_varhook (sym, false, dir);
+	return;
       }
     case SYMBOL_FORWARDED:
       {
@@ -1468,10 +1547,14 @@
 		  if (!PER_BUFFER_VALUE_P (b, idx))
 		    set_per_buffer_value (b, offset, value);
 	      }
-	    return value;
+	    try_run_varhook (sym, false, dir);
+	    return;
 	  }
 	else
-	  return Fset (symbol, value);
+	  {
+	    set_internal_1 (symbol, value, Qnil, false, dir);
+	    return;
+	  }
       }
     default: emacs_abort ();
     }
@@ -3470,6 +3553,13 @@
   DEFSYM (Qad_advice_info, "ad-advice-info");
   DEFSYM (Qad_activate_internal, "ad-activate-internal");
 
+  DEFSYM (Qvarhook, "varhook");
+  DEFSYM (Qglobal, "global");
+  DEFSYM (Qbuffer_local, "buffer-local");
+  DEFSYM (Qdyn_local, "dyn-local");
+  DEFSYM (Qdyn_bind, "dyn-bind");
+  DEFSYM (Qdyn_unbind, "dyn-unbind");
+
   error_tail = pure_cons (Qerror, Qnil);
 
   /* ERROR is used as a signaler for random errors for which nothing else is

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 307 bytes --]

Stefan Monnier wrote:
> Of course.  But to me the whole point of this exercise is to change
> the default.

Ok. The attached patch relies on the varhook feature, which I implemented so that it's possible to enable dynamic-cursor-mode by default without causing any conflict with other uses of cursor-type.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dynamic-cursor-mode-2.patch --]
[-- Type: text/x-diff, Size: 1902 bytes --]

Third hunk adjusted to avoid conflict with current Emacs trunk.
--- emacs-24.4/lisp/simple.el
+++ emacs-24.4/lisp/simple.el
@@ -4391,6 +4391,34 @@
 (declare-function x-selection-exists-p "xselect.c"
                   (&optional selection terminal))
 
+(define-minor-mode dynamic-cursor-mode
+  "Toggle Dynamic Cursor mode.
+With a prefix argument ARG, enable Dynamic Cursor mode if ARG is
+positive, and disable it otherwise.  If called from Lisp, enable
+Dynamic Cursor mode if ARG is omitted or nil.
+
+Dynamic Cursor mode is a global minor mode.  When enabled,
+`cursor-type' is set dynamically to reflect `mark-active'.
+
+Dynamic Cursor mode can be enabled or disabled buffer-locally
+using (setq-local dynamic-cursor-mode t)
+or (setq-local dynamic-cursor-mode nil).
+This will override the global setting.
+
+Setting `cursor-type' globally or buffer-locally will automatically
+disable Dynamic Cursor mode in the same environment."
+  :global t
+  :init-value t)
+
+(defvar cursor-type-varhook nil)
+(add-hook 'cursor-type-varhook
+	  (lambda (_sym env)
+	    (if (eq env 'global)
+		(setq-default dynamic-cursor-mode nil)
+	      (if (eq env 'buffer-local)
+		  (setq-local dynamic-cursor-mode nil)))))
+(put 'cursor-type 'varhook 'cursor-type-varhook)
+
 (defun deactivate-mark (&optional force)
   "Deactivate the mark.
 If Transient Mark mode is disabled, this function normally does
@@ -4430,6 +4458,8 @@
      ((eq transient-mark-mode 'lambda)
       (setq transient-mark-mode nil)))
     (setq mark-active nil)
+    (let ((cursor-type-varhook nil))
+      (if dynamic-cursor-mode (setq cursor-type t)))
     (run-hooks 'deactivate-mark-hook)
     (redisplay--update-region-highlight (selected-window))))
 
@@ -4445,3 +4475,5 @@
+      (let ((cursor-type-varhook nil))
+	(if dynamic-cursor-mode (setq cursor-type 'bar)))
       (run-hooks 'activate-mark-hook))))
 
 (defun set-mark (pos)

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[David Kastrup]

Kelly Dean <kelly@prtime.org> writes:

> +(define-minor-mode dynamic-cursor-mode
> +  "Toggle Dynamic Cursor mode.
> +With a prefix argument ARG, enable Dynamic Cursor mode if ARG is
> +positive, and disable it otherwise.  If called from Lisp, enable
> +Dynamic Cursor mode if ARG is omitted or nil.
> +
> +Dynamic Cursor mode is a global minor mode.  When enabled,
> +`cursor-type' is set dynamically to reflect `mark-active'.
> +
> +Dynamic Cursor mode can be enabled or disabled buffer-locally
> +using (setq-local dynamic-cursor-mode t)
> +or (setq-local dynamic-cursor-mode nil).
> +This will override the global setting.
> +
> +Setting `cursor-type' globally or buffer-locally will automatically
> +disable Dynamic Cursor mode in the same environment."
> +  :global t
> +  :init-value t)
> +
> +(defvar cursor-type-varhook nil)
> +(add-hook 'cursor-type-varhook
> +	  (lambda (_sym env)
> +	    (if (eq env 'global)
> +		(setq-default dynamic-cursor-mode nil)
> +	      (if (eq env 'buffer-local)
> +		  (setq-local dynamic-cursor-mode nil)))))
> +(put 'cursor-type 'varhook 'cursor-type-varhook)

Ugh.  That's implementing and using a sledgehammer (and one which slows
down any variable access on a symbol with properties) on a comparatively
straightforward problem, resulting in pretty inscrutable code.  I don't
think that this approach is worth the complexity.

As to the varhook feature itself: apart from the performance impact, it
also has the problem that one cannot usefully manipulate such a varhook
using add-hook or remove-hook.  That makes it a feature that does not
scale to multiple applications (like variable profiling).

-- 
David Kastrup

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[David Kastrup]

David Kastrup <dak@gnu.org> writes:

> Kelly Dean <kelly@prtime.org> writes:
>
>> +(define-minor-mode dynamic-cursor-mode
>> +  "Toggle Dynamic Cursor mode.
>> +With a prefix argument ARG, enable Dynamic Cursor mode if ARG is
>> +positive, and disable it otherwise.  If called from Lisp, enable
>> +Dynamic Cursor mode if ARG is omitted or nil.
>> +
>> +Dynamic Cursor mode is a global minor mode.  When enabled,
>> +`cursor-type' is set dynamically to reflect `mark-active'.
>> +
>> +Dynamic Cursor mode can be enabled or disabled buffer-locally
>> +using (setq-local dynamic-cursor-mode t)
>> +or (setq-local dynamic-cursor-mode nil).
>> +This will override the global setting.
>> +
>> +Setting `cursor-type' globally or buffer-locally will automatically
>> +disable Dynamic Cursor mode in the same environment."
>> +  :global t
>> +  :init-value t)
>> +
>> +(defvar cursor-type-varhook nil)
>> +(add-hook 'cursor-type-varhook
>> +	  (lambda (_sym env)
>> +	    (if (eq env 'global)
>> +		(setq-default dynamic-cursor-mode nil)
>> +	      (if (eq env 'buffer-local)
>> +		  (setq-local dynamic-cursor-mode nil)))))
>> +(put 'cursor-type 'varhook 'cursor-type-varhook)
>
> Ugh.  That's implementing and using a sledgehammer (and one which slows
> down any variable access on a symbol with properties) on a comparatively
> straightforward problem, resulting in pretty inscrutable code.  I don't
> think that this approach is worth the complexity.
>
> As to the varhook feature itself: apart from the performance impact, it
> also has the problem that one cannot usefully manipulate such a varhook
> using add-hook or remove-hook.  That makes it a feature that does not
> scale to multiple applications (like variable profiling).

Aaaand another data point.  You put in a fast access path for the case
where the symbol has no properties.  However, if I do

(cl-loop for s being the symbols
         with x = 0 with p = 0
         finally return (/ (* p 100) x)
         do (when (boundp s) (setq x (1+ x))
                             (if (symbol-plist s)
                                 (setq p (1+ p)))))

which calculates the percentage of bound symbols (those are the ones
likely to be relevant for variable access, as opposed to symbols like f1
which are mostly used as interned strings and property list containers)
with a non-nil property list, I get 76 in my current Emacs session.

So 76% of all variable accesses will be slowed down searching for the
varhook property.  That seems expensive.

-- 
David Kastrup

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

David Kastrup wrote:
> As to the varhook feature itself: apart from the performance impact, it
> also has the problem that one cannot usefully manipulate such a varhook
> using add-hook or remove-hook.  That makes it a feature that does not
> scale to multiple applications (like variable profiling).

You can have multiple functions watch a symbol, and a function that watches multiple symbols. I don't understand the use case you're talking about that the varhook feature won't work for. Can you explain?

> Aaaand another data point.  You put in a fast access path for the case
> where the symbol has no properties.  However, if I do
[snip]
> So 76% of all variable accesses will be slowed down searching for the
> varhook property.  That seems expensive.

You're right. I've fixed it so property lists will no longer impact the speed, but I guess that doesn't matter if Stefan vetoes the feature.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[David Kastrup]

Kelly Dean <kelly@prtime.org> writes:

> David Kastrup wrote:
>> As to the varhook feature itself: apart from the performance impact, it
>> also has the problem that one cannot usefully manipulate such a varhook
>> using add-hook or remove-hook.  That makes it a feature that does not
>> scale to multiple applications (like variable profiling).
>
> You can have multiple functions watch a symbol, and a function that
> watches multiple symbols. I don't understand the use case you're
> talking about that the varhook feature won't work for. Can you
> explain?

Multiple independent use cases.  add-hook/remove-hook is a mechanism for
organizing independent use cases for one feature, but there is no such
mechanism for organizing independent use cases for the varhook feature
in your implementation even though you actually use add-hook.  But it
requires first individually allocating, naming, and using a hook for any
variable you might want to varhook into.  So you manage to combine two
mechanisms intended for generality in a manner which no longer works in
general.

-- 
David Kastrup

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

David Kastrup wrote:
> Multiple independent use cases.  add-hook/remove-hook is a mechanism for
> organizing independent use cases for one feature, but there is no such
> mechanism for organizing independent use cases for the varhook feature
> in your implementation even though you actually use add-hook.  But it
> requires first individually allocating, naming, and using a hook for any
> variable you might want to varhook into.

IIUC, you mean independent uses of varhook might choose different symbols for the hook. It seems that would be solved by the convention of using the symbol ⌜foo-varhook⌝ as the hook for ⌜foo⌝; all independent uses would add/remove their functions on that same hook.

I guess another solution would be to put the car of the list of functions directly in a dedicated varhook slot for the target symbol, rather than indirecting through a regular hook. That would increase the size of each symbol from 24 to 28 bytes (on 32-bit platforms). But even in my main Emacs session, which has been up for 54 days (with 300MB reserved memory), I only have 10k symbols, so an extra 40kB of memory usage isn't much overhead.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[David Kastrup]

Kelly Dean <kelly@prtime.org> writes:

> David Kastrup wrote:
>> Multiple independent use cases.  add-hook/remove-hook is a mechanism for
>> organizing independent use cases for one feature, but there is no such
>> mechanism for organizing independent use cases for the varhook feature
>> in your implementation even though you actually use add-hook.  But it
>> requires first individually allocating, naming, and using a hook for any
>> variable you might want to varhook into.
>
> IIUC, you mean independent uses of varhook might choose different
> symbols for the hook. It seems that would be solved by the convention
> of using the symbol ⌜foo-varhook⌝ as the hook for ⌜foo⌝; all
> independent uses would add/remove their functions on that same hook.

A convention is not an interface.

> I guess another solution would be to put the car of the list of
> functions directly in a dedicated varhook slot for the target symbol,
> rather than indirecting through a regular hook. That would increase
> the size of each symbol from 24 to 28 bytes (on 32-bit platforms). But
> even in my main Emacs session, which has been up for 54 days (with
> 300MB reserved memory), I only have 10k symbols, so an extra 40kB of
> memory usage isn't much overhead.

I have 40k right now, and this session has been up for few minutes
(admittedly, using desktop-load).  But at any rate, this does not appear
like a feature that should be a central part of the data/operation in a
production Emacs, and consequently it should not be a required part of
the implementation of a production Emacs feature either.

I also have to warn you that there is little point in trying to convince
or coax me since I more or less have the authority of a court jester.
So it is pointless to argue away problems with me since it does not help
getting the code accepted.

-- 
David Kastrup

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

> I also have to warn you that there is little point in trying to convince
> or coax me since I more or less have the authority of a court jester.
> So it is pointless to argue away problems with me since it does not help
> getting the code accepted.

But when anybody raises reasonable concerns about a proposed feature, it's appropriate to try to address the concerns. In this case, I didn't notice that the optimization in my first implementation (now fixed) was rather useless due to the majority of symbols having property lists until you pointed it out. Nor would I have realized that I only have a fraction of the symbols that other people have. And if the interface should be changed, it's good for somebody to point it out.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[David Kastrup]

Kelly Dean <kelly@prtime.org> writes:

>> I also have to warn you that there is little point in trying to
>> convince or coax me since I more or less have the authority of a
>> court jester.  So it is pointless to argue away problems with me
>> since it does not help getting the code accepted.
>
> But when anybody raises reasonable concerns about a proposed feature,
> it's appropriate to try to address the concerns. In this case, I
> didn't notice that the optimization in my first implementation (now
> fixed) was rather useless due to the majority of symbols having
> property lists until you pointed it out. Nor would I have realized
> that I only have a fraction of the symbols that other people have. And
> if the interface should be changed, it's good for somebody to point it
> out.

But in that case you are convincing Emacs rather than me.  Which
obviously is making a difference.

-- 
David Kastrup

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

I wrote:
> another solution would be to put the car of the list of functions directly in a dedicated varhook slot

I meant the first cons cell of the list of functions, of course.

Oops. ;-)

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

> The patch attached below lets you hook a symbol too, so you can run
> some function(s) whenever the global or buffer-local variable by that
> name is set, or a dynamic variable by that name is bound, set,
> or unbound.

NACK from me.

> The purpose of this new feature is to enable a proper fix for bug
> #19068, and to enable a solution to Stefan's requirement that
> dynamic-cursor-mode be enabled by default as a condition for adding
> the latter to Emacs.

All that just to avoid using the minor mode?  I already posted the code
that I think should be used for that minor mode.


        Stefan

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> NACK from me.

Why? Just because of the slowdown for symbols that have property lists? Or some additional reason?

I.e. if I eliminate the slowdown, then will you still have an objection to the feature?

> All that just to avoid using the minor mode?

The updated patch I sent you does make dynamic-cursor-mode be a minor mode rather than a defcustom, because you said you want it to be a minor mode. I didn't implement varhook in order to avoid that.

> I already posted the code
> that I think should be used for that minor mode.

And I explained why your code doesn't provide a complete solution. You didn't respond.

I implemented varhook to completely solve the problem of dynamic-cursor-mode interfering with other uses of cursor-type in all cases, while ensuring that dynamic-cursor-mode itself can reliably be (re)enabled in all cases.

Obviously, there are additional uses for this feature too. For just one example, without varhook, what solution would you propose for bug #19068?

Re: [PATCH] Run hook when variable is set

[Lars Ingebrigtsen]

Kelly Dean <kelly@prtime.org> writes:

> Obviously, there are additional uses for this feature too. For just
> one example, without varhook, what solution would you propose for bug
> #19068?

I don't think #19068 is a bug, so there's nothing to be fixed.

These are independent variables for a reason, but are initialised from
other base variables.  They should not change just because the base
variables change.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

>> NACK from me.
> Why?  Just because of the slowdown for symbols that have property lists?

For the same reason I want to kill the `intangible' text-property: this
operates at too-low a level, so it's bound to introduce buggy
interactions between unsuspecting packages.

I like the idea of such hooks (which I've always thought of as
"watchers" rather than hooks), actually, but only for purposes such
as debugging.

If we want to use such hooks for purposes such as "automatically
recompute values of dependent vars", then I think the right way is to
introduce a new layer which checks&runs these hooks, using the "raw"
`setq' underneath.

That's what I've done to add an "fset hook" (where the above layer which
checks&runs the hook is `defalias'), but that was easy because the
layering was pretty much already there.

> I.e. if I eliminate the slowdown, then will you still have an
> objection to the feature?

Eliminating the slowdown is easy: extend the `constant' bit of
variables to have 3 values: normal, read-only, and "hooked".
This way normal vars are just as fast as before.

But yes, I would still object to the use of such a low-level mechanism
to solve your problem.

> And I explained why your code doesn't provide a complete solution.
> You didn't respond.

I must have misunderstood or overlooked it.  Can you retry?

> Obviously, there are additional uses for this feature too.  For just one
> example, without varhook, what solution would you propose for bug #19068?

For "a good solution" to bug#19068, we need to extend defcustom with
dependency management, and then we need to tell people to use something
else than `setq' (this something else will then be in charge of
triggering the defcustom dependency management).


        Stefan

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

> If we want to use such hooks for purposes such as "automatically
> recompute values of dependent vars", then I think the right way is to
> introduce a new layer which checks&runs these hooks, using the "raw"
> `setq' underneath.

You mean something like setq-with-hook, which you can use in place of setq? Then in general, you also need set-with-hook, setq-default-with-hook, set-default-with-hook, setq-local-with-hook, and let-with-hook. That seems cumbersome.

Varhook handles even that last case. E.g. with my fix for bug #19068, you can do:

message-directory ; → "~/mail/"
message-auto-save-directory ; → "/root/mail/drafts/"

(let ((message-directory "foo"))
  message-auto-save-directory ; → "/root/foo/drafts/"
  (setq message-directory "bar")
  message-auto-save-directory) ; → "/root/bar/drafts/"

message-directory ; → "~/mail/"
message-auto-save-directory ; → "/root/mail/drafts/" (restored because the «let» exited)

I didn't do anything special to enable synchronized let-binding for #19068; that ability comes automatically from varhook. Of course, in the particular case of message-directory, you're unlikely to want to let-bind it. But in general, varhook lets you easily keep your variables synchronized.

> For the same reason I want to kill the `intangible' text-property: this
> operates at too-low a level, so it's bound to introduce buggy
> interactions between unsuspecting packages.

But for variables that are supposed to be linked, requiring use of higher-level «-with-hook» functions results in bugs when you use the regular «set» functions.

Having documentation saying, ‟for this variable, use the «-with-hook» functions rather than the regular «set» functions”, and expecting everybody to remember to do that, doesn't seem like a good solution.

If you do that, then in case a user is prone to forgetting which variables need setq-with-hook and which ones only need setq, he can just always use the former (which works like setq if there's no hook). But then, why use the longer name for the more common function?

It makes more sense to have the default behavior (and short name) be to run the hook. You can still disable the hook (or remove the relevant function from the hook) when you need to intentionally unsynchronize your variables, which you'll normally only need to do during debugging.

>> And I explained why your code doesn't provide a complete solution.
>> You didn't respond.
>
> I must have misunderstood or overlooked it.  Can you retry?

The message is here:
https://lists.gnu.org/archive/html/emacs-devel/2015-01/msg00786.html

In summary:
cursor-type can be changed buffer-locally, which doesn't affect other buffers or the global setting. Therefore dynamic-cursor-mode must be disabled in the same buffer, but not in other buffers or globally.

CT can be changed globally, which doesn't affect CT in buffers that have it set locally. Therefore DCM must be disabled globally, but not in buffers that have it enabled locally.

Turning on DCM globally must work, regardless of whether the user previously globally set CT to a nonstandard value.

Turning on DCM locally must not affect CT globally.

Turning off DCM locally must affect neither CT nor DCM in other buffers.

It doesn't appear to be possible to satisfy all of those conditions without varhook. With varhook, it's easy: the hook function disables DCM globally when CT is set globally, and disables DCM locally when CT is set locally.


BTW, define-minor-mode (with the fix for bug #19685 applied) does produce documentation implying that it's appropriate to set a mode's variable directly; you were right about that. So the updated patch I sent you a couple days ago implements DCM as a mode.

And I implemented a minor enhancement (bug #19690) to help prevent users from being confused about setting local vs. global minor modes in general.

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

> You mean something like setq-with-hook, which you can use in place of setq?
> Then in general, you also need set-with-hook, setq-default-with-hook,
> set-default-with-hook, setq-local-with-hook, and let-with-hook. That
> seems cumbersome.

Yes, you can make it cumbersome if you want.  Or you can drop the
let-with-hook since it would be a misfeature anyway, and then
consolidate the remaining 4 into a single function.

You'd probably want a second function which does the "read value with
hook", OTOH.

> In summary:
> cursor-type can be changed buffer-locally, which doesn't affect other
> buffers or the global setting. Therefore dynamic-cursor-mode must be
> disabled in the same buffer, but not in other buffers or globally.

The code I provided disables the effect of dynamic-cursor-mode as soon
as the value of cursor-type is not the expected one (without
needing to disable dynamic-cursor-mode).  So AFAICT the problem
doesn't exist.


        Stefan

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> you can drop the let-with-hook

Without it, code which would be simply:
(let-with-hook ((hooked-var foo))
	       (body))

would have to become:
(setq tmp hooked-var)
(setq-with-hook hooked-var foo)
(body)
(setq-with-hook hooked-var tmp)

That's cumbersome.

> since it would be a misfeature anyway,

If it's missing (and you don't manually do it as above), then the hooked variable and the dependent variable become unsynchronized when the former is let-bound. That's bound to introduce bugs.

> and then
> consolidate the remaining 4 into a single function.

How do you combine setq-with-hook, set-with-hook, setq-default-with-hook, set-default-with-hook, and setq-local-with-hook into a single function?

However you do it, if it's really a reasonable thing to do, then why aren't setq, set, setq-default, set-default, and setq-local also combined into a single function in the same way?

Anyway, since you said varhook, or something like it, would be useful for debugging even though you don't want to use it to implement regular features, is my implementation reasonably close to what you had in mind?


WRT DCM, in deactivate-mark, you put:
(when dynamic-cursor-mode
         (if (and dynamic-cursor-mode--set (eq cursor-type 'bar))
             (setq cursor-type t))
         (setq dynamic-cursor-mode--set nil))

and in activate-mark:
(if (and dynamic-cursor-mode (eq cursor-type t))
           (setq cursor-type 'bar dynamic-cursor-mode--set t))

That would fail in this case:
(setq-default cursor-type 'bar)
Later in the same Emacs session...
(dynamic-cursor-mode)

The user would reasonably think that DCM's failure to work in that case is a bug.

To solve that, you wrote, ⌜dynamic-cursor-mode can do (setq-default cursor-type t).⌝ IIUC, you meant:
(define-minor-mode dynamic-cursor-mode
  "docs..." :global t :init-value t
  (if dynamic-cursor-mode
      (setq-default cursor-type t)))

In that case, how do you turn it on buffer-locally? If cursor-type isn't set to t, and you do:
(setq-local dynamic-cursor-mode t)

then it won't work. If you instead do:
(make-local-variable 'dynamic-cursor-mode)
(dynamic-cursor-mode)

then DCM is enabled only locally, which is the correct thing to do, but cursor-type is globally set to t, which is a bug. Turning on DCM locally must not affect CT globally.

I guess you could fix that bug with:
(define-minor-mode dynamic-cursor-mode
  "docs..." :global t :init-value t
  (if dynamic-cursor-mode
      (if (assq 'dynamic-cursor-mode (buffer-local-variables))
	  (setq cursor-type t)
	(setq-default cursor-type t))))

> The code I provided disables the effect of dynamic-cursor-mode as soon
> as the value of cursor-type is not the expected one (without
> needing to disable dynamic-cursor-mode).

This has the edge case that if the mark happens to be active when you set cursor-type to 'bar, then the setting doesn't stick; DCM doesn't realize that CT was set outside of DCM's control, so DCM overrides the setting. Similarly, if you previously always used a non-standard cursor type, and were therefore mislead into thinking DCM was disabled, then if you set cursor-type to t, DCM unexpectedly starts operating. That isn't user-friendly.

Another poor result of your design is that if you set a non-standard cursor type in your init file, then start Emacs and do C-h m, DCM is listed as enabled, even though it's having no effect, which is a great way to confuse the user, or convince him that there's a bug.

Using varhook for DCM avoids those problems.

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

>> you can drop the let-with-hook

> Without it, code which would be simply:
> (let-with-hook ((hooked-var foo))
> 	       (body))

> would have to become:
> (setq tmp hooked-var)
> (setq-with-hook hooked-var foo)
> (body)
> (setq-with-hook hooked-var tmp)

> That's cumbersome.

No, if you really need it, it's a simple (cl-letf (((get-var) foo)) ...).

>> since it would be a misfeature anyway,
> If it's missing (and you don't manually do it as above), then the hooked
> variable and the dependent variable become unsynchronized when the former is
> let-bound.  That's bound to introduce bugs.

Right, and I think the bug is in let-binding such vars.

> How do you combine setq-with-hook, set-with-hook,
> setq-default-with-hook, set-default-with-hook, and
> setq-local-with-hook into a single function?

The obvious way: (set-with-hook VAR VALUE LOCAL)

> Anyway, since you said varhook, or something like it, would be useful
> for debugging even though you don't want to use it to implement
> regular features, is my implementation reasonably close to what you
> had in mind?

I think I'd rather have something yet a bit simpler:
- extend the `constant' bit to allow a "hooked" value, for efficiency.
  Provide a function to set/unset this "hooked" annotation.
- when setting a hooked var, call a global Elisp function with args VAR and VAL.
Then you can provide debugging functions to "watch/unwatch" variables.

> That would fail in this case:
> (setq-default cursor-type 'bar)
> Later in the same Emacs session...
> (dynamic-cursor-mode)

As I already said, enabling dynamic-cursor-mode should set cursor-type to t.

> The user would reasonably think that DCM's failure to work in that case is a bug.

> In that case, how do you turn it on buffer-locally?

I don't see a need for it.

> then it won't work. If you instead do:
> (make-local-variable 'dynamic-cursor-mode)
> (dynamic-cursor-mode)
> then DCM is enabled only locally,

Not really.  Such usage is unsupported.  Users who do that get what they
deserve.

> This has the edge case that if the mark happens to be active when you set
> cursor-type to 'bar, then the setting doesn't stick;

That's right.  I'm not worried about that either.


        Stefan

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
>> Without it, code which would be simply:
>> (let-with-hook ((hooked-var foo))
>> 	       (body))
>
>> would have to become:
>> (setq tmp hooked-var)
>> (setq-with-hook hooked-var foo)
>> (body)
>> (setq-with-hook hooked-var tmp)
>
>> That's cumbersome.
>
> No, if you really need it, it's a simple (cl-letf (((get-var) foo)) ...).

I'm lost.

> I think I'd rather have something yet a bit simpler:
> - extend the `constant' bit to allow a "hooked" value, for efficiency.
> Provide a function to set/unset this "hooked" annotation.
> - when setting a hooked var, call a global Elisp function

IIUC, you mean you want a centralized handler for all hooked vars, rather than enabling use of separate functions for different symbols like varhook does.

>> That would fail in this case:
>> (setq-default cursor-type 'bar)
>> Later in the same Emacs session...
>> (dynamic-cursor-mode)
>
> As I already said, enabling dynamic-cursor-mode should set cursor-type to t.

Come on. I acknowledged that just two lines later in my message. Please don't imply that I ignored it.

>> then it won't work. If you instead do:
>> (make-local-variable 'dynamic-cursor-mode)
>> (dynamic-cursor-mode)
>> then DCM is enabled only locally,
>
> Not really.  Such usage is unsupported.  Users who do that get what they
> deserve.

Even though in the very next paragraph I showed a simple way to handle it properly even using your preferred way of implementing DCM?

IOW, you would intentionally omit that proper handling just for the sake of aggravating users who want to control DCM buffer-locally?

>> This has the edge case that if the mark happens to be active when you set
>> cursor-type to 'bar, then the setting doesn't stick;
>
> That's right.  I'm not worried about that either.

Well yeah, I guess you can just declare it to not be a bug.

This reminds me of the joke: How many Microsoft programmers does it take to change a lightbulb?
Answer: none. They just change the standard to darkness.


And what about the fact that your way of implementing DCM shows the mode as enabled, even though it effectively isn't, if the user set a non-standard cursor-type in his init file? I guess you think that's user-friendly too.

Splitting «technically enabled» from «actually in effect» results in a bad API too. For any other mode foo, if you want to know whether the mode is in effect, there's a variable to tell you, so the only expression you need is simply:
foo

But for your version of DCM, there's no such variable, so you would require the special-case expression:
(and dynamic-cursor-mode
     (or (eq cursor-type t)
	 (and dynamic-cursor-mode--set (eq cursor-type 'bar))))

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

OK, let's drop this whole idea.  Too much arguing for a negligible
feature already available via other packages.


        Stefan


>>>>> "Kelly" == Kelly Dean <kelly@prtime.org> writes:

> Stefan Monnier wrote:
>>> Without it, code which would be simply:
>>> (let-with-hook ((hooked-var foo))
>>> (body))
>> 
>>> would have to become:
>>> (setq tmp hooked-var)
>>> (setq-with-hook hooked-var foo)
>>> (body)
>>> (setq-with-hook hooked-var tmp)
>> 
>>> That's cumbersome.
>> 
>> No, if you really need it, it's a simple (cl-letf (((get-var) foo)) ...).

> I'm lost.

>> I think I'd rather have something yet a bit simpler:
>> - extend the `constant' bit to allow a "hooked" value, for efficiency.
>> Provide a function to set/unset this "hooked" annotation.
>> - when setting a hooked var, call a global Elisp function

> IIUC, you mean you want a centralized handler for all hooked vars, rather
> than enabling use of separate functions for different symbols like
> varhook does.

>>> That would fail in this case:
>>> (setq-default cursor-type 'bar)
>>> Later in the same Emacs session...
>>> (dynamic-cursor-mode)
>> 
>> As I already said, enabling dynamic-cursor-mode should set cursor-type to t.

> Come on. I acknowledged that just two lines later in my message. Please
> don't imply that I ignored it.

>>> then it won't work. If you instead do:
>>> (make-local-variable 'dynamic-cursor-mode)
>>> (dynamic-cursor-mode)
>>> then DCM is enabled only locally,
>> 
>> Not really.  Such usage is unsupported.  Users who do that get what they
>> deserve.

> Even though in the very next paragraph I showed a simple way to handle it
> properly even using your preferred way of implementing DCM?

> IOW, you would intentionally omit that proper handling just for the sake of
> aggravating users who want to control DCM buffer-locally?

>>> This has the edge case that if the mark happens to be active when you set
>>> cursor-type to 'bar, then the setting doesn't stick;
>> 
>> That's right.  I'm not worried about that either.

> Well yeah, I guess you can just declare it to not be a bug.

> This reminds me of the joke: How many Microsoft programmers does it take to change a lightbulb?
> Answer: none. They just change the standard to darkness.


> And what about the fact that your way of implementing DCM shows the mode as enabled, even though it effectively isn't, if the user set a non-standard cursor-type in his init file? I guess you think that's user-friendly too.

> Splitting «technically enabled» from «actually in effect» results in a bad API too. For any other mode foo, if you want to know whether the mode is in effect, there's a variable to tell you, so the only expression you need is simply:
> foo

> But for your version of DCM, there's no such variable, so you would require the special-case expression:
> (and dynamic-cursor-mode
>      (or (eq cursor-type t)
> 	 (and dynamic-cursor-mode--set (eq cursor-type 'bar))))

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> OK, let's drop this whole idea.

Ok. Drop debugging hooks too, or just DCM? I can modify varhook to work the way you described. Useful even if it won't be used for non-debug features.

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

>> OK, let's drop this whole idea.
> Ok. Drop debugging hooks too, or just DCM?

Just DCM.


        Stefan

[PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 2747 bytes --]

Updated patch attached. Now designed only for debugging and profiling. Applies to trunk.

Changes:
It now uses a single centralized hook, rather than one per symbol.
It passes the new value as an argument, rather than passing only the symbol and environment.
The speed (for both hooked and unhooked symbols) is unaffected by property lists.
Unhooked symbols are immediately skipped.

It still passes the environment, since it's useful during debugging to notice when your setq is accidentally setting a buffer-local variable that you thought didn't exist, or setting a global that you thought had been made buffer-local, or setting a dynamic local because some other code (that called your code) let-bound a symbol that you thought you were setting globally.

Also, if you do:
(let ((foo 'bar)) (setq-default foo 'baz))
then varhook now intentionally reports env as ⌜invalid⌝ (but the behavior of the code is not changed). If that's actually a valid thing to do, then I'll change how it's reported.

I'm unclear on your reason for extending the «constant» field to include the «hooked» bit, rather than giving the latter its own name. Either way, a new bit is needed (I can't fit the meaning of «hooked» into «constant»'s current two bits), and either way, the size of a symbol remains unchanged: 24 bytes on 32-bit platforms, and 48 bytes on 64-bits. The bit field now has 21 and 53 remaining unused bits, respectively, after «hooked» is added.

Example usage:
(setq syms-to-watch '(foo bar poo par goo gar))
(setq syms-to-pause-on '(poo par))
(setq sym-profiles '((foo 0) (bar 0) (goo 0) (gar 0)))
(setq nonglobals-to-barf-on '(goo gar)) ; Supposed to be set only globally

(defun tattle (sym env val)
  (if (boundp sym)
      (message "Symbol %S modified in env %S. New value: %S"
	       sym env val)
    (message "Symbol %S unbound in env %S" sym env)))

(defun pause (sym _env _val)
  (if (memq sym syms-to-pause-on)
      (unless (y-or-n-p "Continue? ")
	(keyboard-quit))))

(defun profile (sym _env _val)
  (let ((p (assq sym sym-profiles)))
    (if p (incf (cadr p)))))

(defun barf-nonglobal (sym env _val)
  (and (not (eq env 'global))
       (memq sym nonglobals-to-barf-on)
       (debug)))

(add-hook 'varhook #'tattle) ; There's only one hook, used for all symbols
(add-hook 'varhook #'pause t)
(add-hook 'varhook #'profile)
(add-hook 'varhook #'barf-nonglobal t)
(mapc #'hook syms-to-watch)

;; Open *Messages* in another window, then do your debugging...

(mapc #'unhook syms-to-watch) ; When you're done.

(cl-loop for s being the symbols ; Check if anything is still hooked
	 with hooked = nil finally return hooked
	 do (if (hookedp s) (push s hooked)))



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: varhook-single.patch --]
[-- Type: text/x-diff, Size: 13081 bytes --]

--- src/lisp.h
+++ src/lisp.h
@@ -1613,6 +1613,9 @@
   /* True if pointed to from purespace and hence can't be GC'd.  */
   bool_bf pinned : 1;
 
+  /* True means that setting this symbol will run varhook.  */
+  bool_bf hooked : 1;
+
   /* The symbol's name, as a Lisp string.  */
   Lisp_Object name;
 
@@ -3391,6 +3394,14 @@
 EXFUN (Fbyteorder, 0) ATTRIBUTE_CONST;
 
 /* Defined in data.c.  */
+typedef enum
+  {
+    Dyn_Unbind = -1,
+    Dyn_Current = 0,
+    Dyn_Bind = 1,
+    Dyn_Skip = 2,
+    Dyn_Global = 3
+  } Dyn_Bind_Direction;
 extern Lisp_Object indirect_function (Lisp_Object);
 extern Lisp_Object find_symbol_value (Lisp_Object);
 enum Arith_Comparison {
@@ -3438,10 +3449,23 @@
 					   Lisp_Object);
 extern _Noreturn Lisp_Object wrong_type_argument (Lisp_Object, Lisp_Object);
 extern Lisp_Object do_symval_forwarding (union Lisp_Fwd *);
+extern void run_varhook (struct Lisp_Symbol*, bool, Dyn_Bind_Direction, Lisp_Object);
 extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool);
+extern void set_internal_1 (Lisp_Object, Lisp_Object, Lisp_Object, bool,
+			    Dyn_Bind_Direction);
+extern void set_default_internal (Lisp_Object, Lisp_Object,
+				  Dyn_Bind_Direction);
 extern void syms_of_data (void);
 extern void swap_in_global_binding (struct Lisp_Symbol *);
 
+INLINE void
+try_run_varhook (struct Lisp_Symbol* sym, bool buf_local,
+		 Dyn_Bind_Direction dir, Lisp_Object value)
+{
+  if (sym->hooked)
+    run_varhook (sym, buf_local, dir, value);
+}
+
 /* Defined in cmds.c */
 extern void syms_of_cmds (void);
 extern void keys_of_cmds (void);
@@ -3905,9 +3929,9 @@
    should no longer be used.  */
 extern Lisp_Object Vrun_hooks;
 extern void run_hook_with_args_2 (Lisp_Object, Lisp_Object, Lisp_Object);
-extern Lisp_Object run_hook_with_args (ptrdiff_t nargs, Lisp_Object *args,
-				       Lisp_Object (*funcall)
-				       (ptrdiff_t nargs, Lisp_Object *args));
+extern Lisp_Object run_hook_with_args (ptrdiff_t, Lisp_Object *,
+				       Lisp_Object (*) (ptrdiff_t, Lisp_Object *));
+extern Lisp_Object funcall_nil (ptrdiff_t, Lisp_Object *);
 extern _Noreturn void xsignal (Lisp_Object, Lisp_Object);
 extern _Noreturn void xsignal0 (Lisp_Object);
 extern _Noreturn void xsignal1 (Lisp_Object, Lisp_Object);
--- src/eval.c
+++ src/eval.c
@@ -2357,7 +2357,7 @@
 \f
 /* Run hook variables in various ways.  */
 
-static Lisp_Object
+Lisp_Object
 funcall_nil (ptrdiff_t nargs, Lisp_Object *args)
 {
   Ffuncall (nargs, args);
@@ -3142,9 +3142,12 @@
       specpdl_ptr->let.old_value = SYMBOL_VAL (sym);
       grow_specpdl ();
       if (!sym->constant)
-	SET_SYMBOL_VAL (sym, value);
+	{
+	  SET_SYMBOL_VAL (sym, value);
+	  try_run_varhook (sym, false, Dyn_Bind, value);
+	}
       else
-	set_internal (symbol, value, Qnil, 1);
+	set_internal_1 (symbol, value, Qnil, 1, Dyn_Bind);
       break;
     case SYMBOL_LOCALIZED:
       if (SYMBOL_BLV (sym)->frame_local)
@@ -3176,7 +3179,7 @@
 	      {
 		specpdl_ptr->let.kind = SPECPDL_LET_DEFAULT;
 		grow_specpdl ();
-		Fset_default (symbol, value);
+		set_default_internal (symbol, value, Dyn_Bind);
 		return;
 	      }
 	  }
@@ -3184,7 +3187,7 @@
 	  specpdl_ptr->let.kind = SPECPDL_LET;
 
 	grow_specpdl ();
-	set_internal (symbol, value, Qnil, 1);
+	set_internal_1 (symbol, value, Qnil, 1, Dyn_Bind);
 	break;
       }
     default: emacs_abort ();
@@ -3319,7 +3322,9 @@
 	    struct Lisp_Symbol *sym = XSYMBOL (specpdl_symbol (specpdl_ptr));
 	    if (sym->redirect == SYMBOL_PLAINVAL)
 	      {
-		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
+		Lisp_Object oldval = specpdl_old_value (specpdl_ptr);
+		SET_SYMBOL_VAL (sym, oldval);
+		try_run_varhook (sym, false, Dyn_Unbind, oldval);
 		break;
 	      }
 	    else
@@ -3329,8 +3334,8 @@
 	      }
 	  }
 	case SPECPDL_LET_DEFAULT:
-	  Fset_default (specpdl_symbol (specpdl_ptr),
-			specpdl_old_value (specpdl_ptr));
+	  set_default_internal (specpdl_symbol (specpdl_ptr),
+				specpdl_old_value (specpdl_ptr), Dyn_Unbind);
 	  break;
 	case SPECPDL_LET_LOCAL:
 	  {
@@ -3342,7 +3347,7 @@
 	    /* If this was a local binding, reset the value in the appropriate
 	       buffer, but only if that buffer's binding still exists.  */
 	    if (!NILP (Flocal_variable_p (symbol, where)))
-	      set_internal (symbol, old_value, where, 1);
+	      set_internal_1 (symbol, old_value, where, 1, Dyn_Unbind);
 	  }
 	  break;
 	}
@@ -3537,7 +3542,7 @@
 	    Lisp_Object sym = specpdl_symbol (tmp);
 	    Lisp_Object old_value = specpdl_old_value (tmp);
 	    set_specpdl_old_value (tmp, Fdefault_value (sym));
-	    Fset_default (sym, old_value);
+	    set_default_internal (sym, old_value, Dyn_Skip);
 	  }
 	  break;
 	case SPECPDL_LET_LOCAL:
@@ -3553,7 +3558,7 @@
 	      {
 		set_specpdl_old_value
 		  (tmp, Fbuffer_local_value (symbol, where));
-		set_internal (symbol, old_value, where, 1);
+		set_internal_1 (symbol, old_value, where, 1, Dyn_Skip);
 	      }
 	  }
 	  break;
@@ -3828,6 +3833,14 @@
 still determine whether to handle the particular condition.  */);
   Vdebug_on_signal = Qnil;
 
+  DEFVAR_LISP ("varhook", Vvarhook,
+	       doc: /* This is the hook run when hooked symbols are set.
+The following arguments are passed:
+The symbol that was set.
+The environment in which it was set.
+The new value.  */);
+  Vvarhook = Qnil;
+
   /* When lexical binding is being used,
    Vinternal_interpreter_environment is non-nil, and contains an alist
    of lexically-bound variable, or (t), indicating an empty
--- src/data.c
+++ src/data.c
@@ -612,6 +613,15 @@
 \f
 /* Extract and set components of symbols.  */
 
+DEFUN ("hookedp", Fhookedp, Shookedp, 1, 1, 0,
+       doc: /* Return t if SYMBOL is hooked.
+When hooked, setting SYMBOL will run `varhook'.  */)
+  (register Lisp_Object symbol)
+{
+  CHECK_SYMBOL (symbol);
+  return XSYMBOL (symbol)->hooked ? Qt : Qnil;
+}
+
 DEFUN ("boundp", Fboundp, Sboundp, 1, 1, 0,
        doc: /* Return t if SYMBOL's value is not void.
 Note that if `lexical-binding' is in effect, this refers to the
@@ -661,6 +671,26 @@
   return NILP (XSYMBOL (symbol)->function) ? Qnil : Qt;
 }
 
+DEFUN ("hook", Fhook, Shook, 1, 1, 0,
+       doc: /* Hook SYMBOL. When hooked, setting it will run `varhook'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  CHECK_SYMBOL (symbol);
+  XSYMBOL (symbol)->hooked = true;
+  return symbol;
+}
+
+DEFUN ("unhook", Funhook, Sunhook, 1, 1, 0,
+       doc: /* Unhook SYMBOL. When unhooked, setting it will not run `varhook'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  CHECK_SYMBOL (symbol);
+  XSYMBOL (symbol)->hooked = false;
+  return symbol;
+}
+
 DEFUN ("makunbound", Fmakunbound, Smakunbound, 1, 1, 0,
        doc: /* Make SYMBOL's value be void.
 Return SYMBOL.  */)
@@ -1167,6 +1197,48 @@
   xsignal1 (Qvoid_variable, symbol);
 }
 
+/* For the symbol S that was just set, run varhook.
+   To the functions on the hook, pass S as the first argument. As the second
+   argument, pass a symbol indicating the environment in which S was set.
+   As the third argument, pass the value to which S was set.  */
+
+void
+run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Direction dir,
+	     Lisp_Object value)
+{
+  Lisp_Object hook_and_args[4];
+  if (dir == Dyn_Skip) /* From backtrace_eval_unrewind */
+    return;
+  hook_and_args[0] = Qvarhook;
+  XSETSYMBOL (hook_and_args[1], sym);
+  switch (dir)
+    {
+    case Dyn_Current:
+      {
+	bool shadowed;
+	if (buf_local)
+	  shadowed = let_shadows_buffer_binding_p (sym);
+	else shadowed = let_shadows_global_binding_p (hook_and_args[1]);
+	if (shadowed) hook_and_args[2] = Qdyn_local;
+	else if (buf_local) hook_and_args[2] = Qbuf_local;
+	else hook_and_args[2] = Qglobal;
+	break;
+      }
+    case Dyn_Global:
+      {
+	if (let_shadows_global_binding_p (hook_and_args[1]))
+	  hook_and_args[2] = Qinvalid;
+	else hook_and_args[2] = Qglobal;
+	break;
+      }
+    case Dyn_Bind: hook_and_args[2] = Qdyn_bind; break;
+    case Dyn_Unbind: hook_and_args[2] = Qdyn_unbind; break;
+    default: emacs_abort ();
+    }
+  hook_and_args[3] = value;
+  run_hook_with_args (4, hook_and_args, funcall_nil);
+}
+
 DEFUN ("set", Fset, Sset, 2, 2, 0,
        doc: /* Set SYMBOL's value to NEWVAL, and return NEWVAL.  */)
   (register Lisp_Object symbol, Lisp_Object newval)
@@ -1187,6 +1259,21 @@
 set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
 	      bool bindflag)
 {
+  set_internal_1 (symbol, newval, where, bindflag, Dyn_Current);
+}
+
+/* Like set_internal but with direction argument to indicate whether this
+   function call is due to a binding (1), an unbinding (-1), or neither (0).
+   As special cases, a value of 2 is a flag to disable run_varhook so that
+   varhooks aren't run during backtraces, and a value of 3 is a flag
+   indicating that this function call is due to set_default, which allows
+   run_varhook to distinguish beween the global and the dyn-local binding.
+*/
+
+void
+set_internal_1 (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+		bool bindflag, Dyn_Bind_Direction dir)
+{
   bool voide = EQ (newval, Qunbound);
   struct Lisp_Symbol *sym;
   Lisp_Object tem1;
@@ -1212,9 +1299,15 @@
   switch (sym->redirect)
     {
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
+    case SYMBOL_PLAINVAL:
+      {
+	SET_SYMBOL_VAL (sym, newval);
+	try_run_varhook (sym, false, dir, newval);
+	return;
+      }
     case SYMBOL_LOCALIZED:
       {
+	bool buf_local = true;
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 	if (NILP (where))
 	  {
@@ -1258,6 +1351,7 @@
 		   indicating that we're seeing the default value.
 		   Likewise if the variable has been let-bound
 		   in the current buffer.  */
+		buf_local = false;
 		if (bindflag || !blv->local_if_set
 		    || let_shadows_buffer_binding_p (sym))
 		  {
@@ -1299,6 +1393,7 @@
 				       BUFFERP (where)
 				       ? XBUFFER (where) : current_buffer);
 	  }
+	try_run_varhook (sym, buf_local, dir, newval);
 	break;
       }
     case SYMBOL_FORWARDED:
@@ -1324,6 +1419,9 @@
 	  }
 	else
 	  store_symval_forwarding (/* sym, */ innercontents, newval, buf);
+	try_run_varhook (sym,
+			 (XFWDTYPE (innercontents))==Lisp_Fwd_Buffer_Obj,
+			 dir, newval);
 	break;
       }
     default: emacs_abort ();
@@ -1413,6 +1511,17 @@
 for this variable.  */)
   (Lisp_Object symbol, Lisp_Object value)
 {
+  set_default_internal (symbol, value, Dyn_Global);
+  return value;
+}
+
+/* Like Fset_default, but with direction argument. See set_internal_1 for
+   a description of this argument. */
+
+void
+set_default_internal (Lisp_Object symbol, Lisp_Object value,
+		      Dyn_Bind_Direction dir)
+{
   struct Lisp_Symbol *sym;
 
   CHECK_SYMBOL (symbol);
@@ -1423,7 +1532,7 @@
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
-	return value;
+	return;
     }
   sym = XSYMBOL (symbol);
 
@@ -1431,7 +1540,11 @@
   switch (sym->redirect)
     {
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: return Fset (symbol, value);
+    case SYMBOL_PLAINVAL:
+      {
+	set_internal_1 (symbol, value, Qnil, false, dir);
+	return;
+      }
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
@@ -1442,7 +1555,8 @@
 	/* If the default binding is now loaded, set the REALVALUE slot too.  */
 	if (blv->fwd && EQ (blv->defcell, blv->valcell))
 	  store_symval_forwarding (blv->fwd, value, NULL);
-	return value;
+	try_run_varhook (sym, false, dir, value);
+	return;
       }
     case SYMBOL_FORWARDED:
       {
@@ -1468,10 +1582,14 @@
 		  if (!PER_BUFFER_VALUE_P (b, idx))
 		    set_per_buffer_value (b, offset, value);
 	      }
-	    return value;
+	    try_run_varhook (sym, false, dir, value);
+	    return;
 	  }
 	else
-	  return Fset (symbol, value);
+	  {
+	    set_internal_1 (symbol, value, Qnil, false, dir);
+	    return;
+	  }
       }
     default: emacs_abort ();
     }
@@ -3470,6 +3588,14 @@
   DEFSYM (Qad_advice_info, "ad-advice-info");
   DEFSYM (Qad_activate_internal, "ad-activate-internal");
 
+  DEFSYM (Qvarhook, "varhook");
+  DEFSYM (Qglobal, "global");
+  DEFSYM (Qbuf_local, "buf-local");
+  DEFSYM (Qdyn_local, "dyn-local");
+  DEFSYM (Qdyn_bind, "dyn-bind");
+  DEFSYM (Qdyn_unbind, "dyn-unbind");
+  DEFSYM (Qinvalid, "invalid");
+
   error_tail = pure_cons (Qerror, Qnil);
 
   /* ERROR is used as a signaler for random errors for which nothing else is
@@ -3609,8 +3735,11 @@
   defsubr (&Sindirect_function);
   defsubr (&Ssymbol_plist);
   defsubr (&Ssymbol_name);
+  defsubr (&Shook);
+  defsubr (&Sunhook);
   defsubr (&Smakunbound);
   defsubr (&Sfmakunbound);
+  defsubr (&Shookedp);
   defsubr (&Sboundp);
   defsubr (&Sfboundp);
   defsubr (&Sfset);
--- src/alloc.c
+++ src/alloc.c
@@ -3392,4 +3392,5 @@
   p->interned = SYMBOL_UNINTERNED;
   p->constant = 0;
+  p->hooked = false;
   p->declared_special = false;
   p->pinned = false;

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> I'm unclear on your reason for extending the «constant» field to
> include the «hooked» bit, rather than giving the latter its own name.

I thought it was clear: so that the new feature does not cost anything
as long as you don't use it.  I.e. that extends existing tests of
"constant is 0" to mean both "not constant" and "not hooked either".

> Either way, a new bit is needed (I can't fit the meaning of «hooked»
> into «constant»'s current two bits)

Why not?  Currently `constant' occupies two bits but is used as
a boolean value.  So there's a bit available right there.  And if not,
you can add a bit to that field.

> Example usage:

I don't understand your example: what's `varhook'?  What's `hook'?
What's `unhook'?

> +INLINE void
> +try_run_varhook (struct Lisp_Symbol* sym, bool buf_local,
> +		 Dyn_Bind_Direction dir, Lisp_Object value)
> +{
> +  if (sym->hooked)
> +    run_varhook (sym, buf_local, dir, value);
> +}

The hook should be called *to perform the assignment*, rather than
calling it after performing the assignment ourselves.
And it should not use run_hook* but funcall (i.e. the hook should be
manipulated in Elisp via `add-function' rather than via `add-hook').

> -extern Lisp_Object run_hook_with_args (ptrdiff_t nargs, Lisp_Object *args,
> -				       Lisp_Object (*funcall)
> -				       (ptrdiff_t nargs, Lisp_Object *args));
> +extern Lisp_Object run_hook_with_args (ptrdiff_t, Lisp_Object *,
> +				       Lisp_Object (*) (ptrdiff_t, Lisp_Object *));

What for?

>       if (!sym->constant)
> -	SET_SYMBOL_VAL (sym, value);
> +	{
> +	  SET_SYMBOL_VAL (sym, value);
> +	  try_run_varhook (sym, false, Dyn_Bind, value);
> +	}

You just slowed down all the code even if the feature is not used.
I think we can do better.

>  	    if (sym->redirect == SYMBOL_PLAINVAL)
>  	      {
> -		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
> +		Lisp_Object oldval = specpdl_old_value (specpdl_ptr);
> +		SET_SYMBOL_VAL (sym, oldval);
> +		try_run_varhook (sym, false, Dyn_Unbind, oldval);

Same here.

> +run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Direction dir,
> +	     Lisp_Object value)
> +{
> +  Lisp_Object hook_and_args[4];
> +  if (dir == Dyn_Skip) /* From backtrace_eval_unrewind */
> +    return;

Hmm... didn't think of backtrace_eval_unrewind.  Indeed, I think we
don't want this to trigger a hook (well, maybe there are cases where
we'd want this, but I don't think the current code can handle it
correctly/safely).

> +  hook_and_args[0] = Qvarhook;
> +  XSETSYMBOL (hook_and_args[1], sym);
> +  switch (dir)
> +    {
> +    case Dyn_Current:
> +      {
> +	bool shadowed;
> +	if (buf_local)
> +	  shadowed = let_shadows_buffer_binding_p (sym);
> +	else shadowed = let_shadows_global_binding_p (hook_and_args[1]);
> +	if (shadowed) hook_and_args[2] = Qdyn_local;
> +	else if (buf_local) hook_and_args[2] = Qbuf_local;
> +	else hook_and_args[2] = Qglobal;
> +	break;
> +      }
> +    case Dyn_Global:
> +      {
> +	if (let_shadows_global_binding_p (hook_and_args[1]))
> +	  hook_and_args[2] = Qinvalid;
> +	else hook_and_args[2] = Qglobal;
> +	break;
> +      }
> +    case Dyn_Bind: hook_and_args[2] = Qdyn_bind; break;
> +    case Dyn_Unbind: hook_and_args[2] = Qdyn_unbind; break;
> +    default: emacs_abort ();
> +    }
> +  hook_and_args[3] = value;
> +  run_hook_with_args (4, hook_and_args, funcall_nil);

I think I'd rather turn `dir' into a Lisp_Object (a symbol) and send
this directly to the hook.  And then, if/when needed we can try and
make available to Elisp the functionality of things like
let_shadows_*_binding_p.

BTW in order to keep the performance cost of this patch to a minimum, we
may end up reducing its precision.

> +  DEFSYM (Qvarhook, "varhook");

Ah, here it is.  Please use a name more along the lines of Elisp style.
E.g. `variable-watch-function'.

> +  defsubr (&Shook);
> +  defsubr (&Sunhook);
> +  defsubr (&Shookedp);

Use less generic names.  E.g. `variable-watch-set',
`variable-watch-unset', and `variable-watch-p'.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> I thought it was clear: so that the new feature does not cost anything
> as long as you don't use it.  I.e. that extends existing tests of
> "constant is 0" to mean both "not constant" and "not hooked either".

Oh, I see. Yes, that would avoid the extra conditional branch in specbind (but not in unbind_to).

But avoiding the conditional branch that way for the set_internal function (or set_internal_1 in my patch) would require duplicating the entire function, in both the source code and the compiled code. The first version would be the same as now (without my patch), except it would call the second version if «constant» indicates «hooked». The second version would be the same as now (_with_ my patch), except with the «constant» check omitted and the three calls to try_run_varhook replaced by run_varhook.

> Currently `constant' occupies two bits but is used as
> a boolean value.

The definition of Lisp_Symbol says about «constant», ‟If the value is 3, then the var can be changed, but only by `defconst'.” I assumed that comment was accurate.

But I grepped for ⌜->constant⌝ and ⌜SYMBOL_CONSTANT_P⌝ just now, and it looks like the comment is wrong. I guess it is/was a planned feature.

> So there's a bit available right there.  And if not,
> you can add a bit to that field.

Ok.

> I don't understand your example: what's `varhook'?  What's `hook'?
> What's `unhook'?

The ⌜varhook⌝ symbol is the hook (an ordinary hook) that's run when hooked symbols are set. The «hook» function sets the «hooked» bit, «unhook» clears that bit, and «hookedp» returns it. All documented in the docstrings in my patch, but I guess my example usage should mention it at the beginning.

> The hook should be called *to perform the assignment*, rather than
> calling it after performing the assignment ourselves.

I'm confused. You mean if a symbol is hooked, then when you attempt to set it (using the usual setq, etc), the hook should be run _instead_ of actually setting the symbol?

That would give the hook the option to perform the requested assignment, or instead perform a different assignment, or none at all. I don't see the reason for doing this.

Or do you mean the higher-level feature you talked about a few days ago, where hooking has no effect on setq, etc, and you have to use setq-with-hook instead of setq if you want the hook to run? That wouldn't be useful for debugging and profiling, which are the current goals.

> And it should not use run_hook* but funcall (i.e. the hook should be
> manipulated in Elisp via `add-function' rather than via `add-hook').

That would make sense if the hook replaces the actual assignment, like «around» advice can replace a function. But I don't see why to do this.

I'm sure I must have misunderstood what you meant.

Or did you mean just to enable writing a handler function that lets you pause on a variable, manually set some value (different from what the original program set) as an experiment while debugging, then continue? You can already do that (it doesn't matter that varhook lets the original program set the value, because the handler runs afterward, so you can override what the program set, before resuming the program); just make sure your handler temporarily unhooks the variable before setting it, to avoid infinite recursion.

>> -extern Lisp_Object run_hook_with_args (ptrdiff_t nargs, Lisp_Object *args,
>> -				       Lisp_Object (*funcall)
>> -				       (ptrdiff_t nargs, Lisp_Object *args));
>> +extern Lisp_Object run_hook_with_args (ptrdiff_t, Lisp_Object *,
>> +				       Lisp_Object (*) (ptrdiff_t, Lisp_Object *));
>
> What for?

A code cleanup that I included in this patch by accident. (I forgot that I only added the next function signature, not this one, then I edited both to be consistent with the surrounding code because I thought I added both.) I missed it when reviewing the patch because, as is well-known, I can't read. ;-)

I'll remove it.

>>       if (!sym->constant)
>> -	SET_SYMBOL_VAL (sym, value);
>> +	{
>> +	  SET_SYMBOL_VAL (sym, value);
>> +	  try_run_varhook (sym, false, Dyn_Bind, value);
>> +	}
>
> You just slowed down all the code even if the feature is not used.
> I think we can do better.

This is in specbind. Yes, putting the «hooked» bit into the «constant» field could avoid this extra conditional branch.

>>  	    if (sym->redirect =3D=3D SYMBOL_PLAINVAL)
>>  	      {
>> -		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
>> +		Lisp_Object oldval =3D specpdl_old_value (specpdl_ptr);
>> +		SET_SYMBOL_VAL (sym, oldval);
>> +		try_run_varhook (sym, false, Dyn_Unbind, oldval);
>
> Same here.

This is in unbind_to, which doesn't already check the «constant» field, so even if «hooked» were put into «constant», it would save nothing here; it would just result in adding a check of «constant» rather than adding a check of the separate «hooked» bit.

But remember that try_run_varhook is inlined, and the target of the «sym» pointer is already in L1 cache at this point, so if the «hooked» bit isn't set, all that's executed here is a conditional branch on that bit. Well and I guess an «and» mask to extract it from the bit field (no bit shift needed, since it's just used for comparison to zero). There's no function call or main memory access.

I'm not very familiar with CPU branch prediction, but IIUC, it will result in almost all executions of try_run_varhook (in specbind, unbind_to, set_internal_1, and set_default_internal) being as fast as a «jump» instruction--one CPU cycle, or maybe less on superscalar CPUs (not very familiar with this either).

So I think even if «hooked» is left as a separate bit, it will cost practically nothing extra compared to making it be part of «constant» (and the latter would increase complexity and require code duplication). Is there someone on the mailing list who understands branch prediction well and can comment?

> I think I'd rather turn `dir' into a Lisp_Object (a symbol) and send
> this directly to the hook.  And then, if/when needed we can try and
> make available to Elisp the functionality of things like
> let_shadows_*_binding_p.

Ok. I'd also have to pass buf_local, so that the hook would have the necessary information to replicate what run_varhook currently does. But I don't understand what the advantage would be, compared to how run_varhook currently works.

> BTW in order to keep the performance cost of this patch to a minimum, we
> may end up reducing its precision.

I think even the current implementation might have close to zero performance cost. But I haven't tested to confirm this.

>> +  DEFSYM (Qvarhook, "varhook");
>
> Ah, here it is.  Please use a name more along the lines of Elisp style.
> E.g. `variable-watch-function'.
>
>> +  defsubr (&Shook);
>> +  defsubr (&Sunhook);
>> +  defsubr (&Shookedp);
>
> Use less generic names.  E.g. `variable-watch-set',
> `variable-watch-unset', and `variable-watch-p'.

Ok. However (just a nitpick), using ⌜symbol-watch-set⌝, ⌜symbol-watch-unset⌝ and ⌜symbol-watch-p⌝ would be more accurate, because those functions set/check the «hooked» bit not just for a particular variable, but for all non-lexical variables of the specified symbol (i.e. all occurrences of the symbol in all non-lexical run-time environments: global, buffer-local, and dynamic). Using ⌜variable⌝ in those functions' names would imply that individual variables of a symbol can be hooked/unhooked, when in fact varhook doesn't support that. It only supports all-or-none.

But ⌜variable-watch-function⌝ is not misleading, since it doesn't suggest that it applies to just one particular variable. It applies to all hooked variables.

It's unfortunate that in Emacs, an instance of the Lisp_Symbol structure, which records a representation of (and information associated with) a symbol, is itself called a ‟symbol”. This is probably why people say symbols don't occur in multiple environments.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> Oh, I see. Yes, that would avoid the extra conditional branch in
> specbind

Exactly.

> (but not in unbind_to).

Actually, we could also avoid this extra cost if we don't push a normal
specbind record on the pdl, but instead we'd push a record_unwind that
will call the hook function again on the way back.
I haven't looked at whether that could be done without too much
gymnastics, tho.  So maybe the extra check in unbind_to is OK.

> But avoiding the conditional branch that way for the set_internal function
> (or set_internal_1 in my patch) would require duplicating the entire
> function, in both the source code and the compiled code.

I don't understand why.

> The definition of Lisp_Symbol says about «constant», ‟If the value is 3,
> then the var can be changed, but only by `defconst'.”  I assumed that comment
> was accurate.

Oh, sorry, that must have been a messed up commit of mine (I've had such
a tentative change in my local tree, but I ended up not installing it,
since defconst vars are modified much too often).

> The ⌜varhook⌝ symbol is the hook (an ordinary hook) that's run when hooked
> symbols are set.

Yes, I saw it afterwards.  But the name was too generic, making it sound
like a local var or local function.

>> The hook should be called *to perform the assignment*, rather than
>> calling it after performing the assignment ourselves.
> I'm confused. You mean if a symbol is hooked, then when you attempt to set
> it (using the usual setq, etc), the hook should be run _instead_ of actually
> setting the symbol?

That's right.  This way the hook can see the value before the assignment
(very useful for watchpoint-style debugging) and can decide to block the
assignment altogether (e.g. let's say you want to prevent assignment of
values of the wrong type to a particular variable; or you want to
implement the "defconst makes the variable read-only", ...).

> I don't see the reason for doing this.

Because it's more general.  The real question is: why do it otherwise?

I happen to have an answer for that question as well: because, if
the assignment calls the hook, how does the hook perform the assignment?
(I.e. we need some way to perform the assignment without triggering the
hook)

Ideally this is done by having two layers: a top-layer that triggers the
hook(s) and a bottom layer that performs a "raw" access.  That's how
I did it for defalias-vs-fset, but for setq we don't have 2 layers yet,
so either we need to add a new layer (and since we want the hook to
trigger on plain old `setq' it means the new layer has to be the "raw"
assignment).

> to be consistent with the surrounding code because I thought I added both.)

Indeed, we're not consistent in this respect, but I don't think
consistency is that important here, and if I get to choose I prefer the
prototypes with variable names than without.

> So I think even if «hooked» is left as a separate bit, it will cost
> practically nothing extra compared to making it be part of «constant» (and
> the latter would increase complexity and require code duplication). Is there
> someone on the mailing list who understands branch prediction well and
> can comment?

Branch prediction should work well, indeed, but you still have extra
costs (e.g. because of the density of branches, for example, and CPUs
tend to be unable to predict more than one branch per cycle, because of
the need to extract the bit, ...).

So the slowdown might not be terrible, but remember this is a very core
part of Elisp execution (slightly less so with lexical-scoping, but
still), so the impact could be measurable.  This downside has to be
weighted against the fact that noone is using this feature right now,
and there hasn't been many requests for it (I've been toying with the
idea for many years, but never bumped into a strong enough need for it
to get me coding).

> Ok.  However (just a nitpick), using ⌜symbol-watch-set⌝, ⌜symbol-watch-unset⌝
> and ⌜symbol-watch-p⌝ would be more accurate,

Fine by me.

> because those functions set/check the «hooked» bit not just for
> a particular variable, but for all non-lexical variables of the
> specified symbol (i.e. all occurrences of the symbol in all
> non-lexical run-time environments: global, buffer-local, and
> dynamic).

[ The difference between symbol and variables in Elisp's dynamic binding
  is quite messy indeed.  `symbol-watch' has the downside that it might
  make it sound like `fset' and `set-symbol-plist' would also
  be hooked.  ]

> But ⌜variable-watch-function⌝ is not misleading, since it doesn't suggest
> that it applies to just one particular variable.  It applies to all
> hooked variables.

But please use `symbol-watch-function' for consistency.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
>> But avoiding the conditional branch that way for the set_internal function
>> (or set_internal_1 in my patch) would require duplicating the entire
>> function, in both the source code and the compiled code.
>
> I don't understand why.

set_internal_1 (which is just set_internal with an additional argument; the new name is just to avoid having to change other code) has try_run_varhook in three places: one for each of the cases of
⌜switch (sym->redirect)⌝ except SYMBOL_VARALIAS. The first three things that set_internal_1 does are:
set «voide»
check «symbol»
check the «constant» bit, using ⌜if (SYMBOL_CONSTANT_P (symbol))⌝

If «hooked» is put into «constant», and checked only within that «if» block at the beginning of set_internal_1, then the entire remainder of the function must be duplicated, with one version to handle the case that «hooked» is false, and one version to handle the case that «hooked» is true.

For example, if you have:
(defun set-internal-1 (args)
  (block nil ; Because Elisp isn't CL
    (if (constant-p) ; This is the «if» near the start of set_internal_1
	(if (forbidden-p)
	    (error "setting constant")
	  (return)))
    (do-some-stuff)
    (and-many-more-lines-of-stuff)
    (set-some-variable)
    (if hooked (run-varhook)))) ; This is what try_run_varhook does

Your goal is to avoid the «if» at the end. In order to do that, you'd need two functions:
(defun set-internal-1 (args)
  (block nil
    (if (constant-p)
	(if hooked ; Mutually exclusive with actual constant
	    (progn
	      (set-internal-1-and-run-varhook args)
	      (return))
	  (if (forbidden-p)
	      (error "setting constant")
	    (return))))
    (do-some-stuff)
    (and-many-more-lines-of-stuff)
    (set-some-variable)))

(defun set-internal-1-and-run-varhook (args)
  (do-some-stuff)
  (and-many-more-lines-of-stuff)
  (set-some-variable)
  (run-varhook))

You can't factor the common parts out to a separate function, because function call overhead would be even greater than the conditional-branch overhead of try_run_varhook, which would defeat the purpose of putting «hooked» into «constant». I guess you could factor the common parts out to an _inline_ function, assuming the compiler would follow your advice to inline such a big function, but it would still be duplicated in the compiled code.

If the performance of Emacs would really be measurably affected by that «if» (but as I've shown, it isn't, even in the worst case), I see three other ways to optimize set_internal that together would be more effective than eliminating the «if»:
0. Untag «symbol» once, rather than twice (currently it's done in SYMBOL_CONSTANT_P and again to set «sym»).
1. Delay the setting of «voide», since it's not needed in the case of SYMBOL_PLAINVAL.
2. Swap the order of the SYMBOL_PLAINVAL and SYMBOL_VARALIAS cases of the «switch» statement, which would avoid an unnecessary conditional branch before the SYMBOL_PLAINVAL case (which is the common case). IIUC, putting SYMBOL_VARALIAS before SYMBOL_PLAINVAL (which is how the code is now) is just as expensive as my branch that you're objecting to.

But if you still want me to eliminate my branch, I'll do it.

> This way the hook can see the value before the assignment
> (very useful for watchpoint-style debugging) and can decide to block the
> assignment altogether (e.g. let's say you want to prevent assignment of
> values of the wrong type to a particular variable; or you want to
> implement the "defconst makes the variable read-only", ...).

I see. Yes, that would make varhook more useful. I'll implement it.

> The real question is: why do it otherwise?
>
> I happen to have an answer for that question as well: because, if
> the assignment calls the hook, how does the hook perform the assignment?
> (I.e. we need some way to perform the assignment without triggering the
> hook)

That's easy even in the current version of varhook: in your handler function, just unhook the symbol before performing the assignment, then re-hook it before the handler exits.

But I'll change it so even that isn't necessary.

> Ideally this is done by having two layers: a top-layer that triggers the
> hook(s) and a bottom layer that performs a "raw" access.  That's how
> I did it for defalias-vs-fset, but for setq we don't have 2 layers yet,
> so either we need to add a new layer (and since we want the hook to
> trigger on plain old `setq' it means the new layer has to be the "raw"
> assignment).

I think I have an ideal solution in mind. But I'll try to implement it tomorrow before posting a description, so I can deny I ever had this idea if it turns out to be stupid.

>> Ok.  However (just a nitpick), using ⌜symbol-watch-set⌝, ⌜symbol-watch-unset⌝
>> and ⌜symbol-watch-p⌝ would be more accurate,
>
> Fine by me.

I'd like to bikeshed the function names a bit more. Handler functions for varhook are already capable of not only reading variables, but also writing them. And now, writing (or blocking of writing) is actually going to be one of the intended use cases. That means ‟watch” is a misleading way of describing it; watching (which implies only reading) is only one of the things that a handler might do.

So now, I recommend ⌜symbol-hook-set⌝, ⌜symbol-hook-unset⌝, and ⌜symbol-hooked-p⌝. (Actually, I prefer ⌜symbol-hook⌝ and ⌜symbol-unhook⌝, but if these are too short then the others are ok.)

>> But ⌜variable-watch-function⌝ is not misleading, since it doesn't suggest
>> that it applies to just one particular variable.  It applies to all
>> hooked variables.
>
> But please use `symbol-watch-function' for consistency.

And now, for consistency, it would be ⌜symbol-hook-function⌝, but with the planned interface change to use add-function instead of add-hook, it would be more descriptive to call it ⌜symbol-mutate-function⌝ (from the sense of ‟mutable variable”, since it's run when a hooked variable is mutated). This name also has the advantage of eliciting visions of your program creating monsters.

BTW, why is the order of the first two arguments to advice-add reversed from add-function? And the words in their names too.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> Your goal is to avoid the «if» at the end. In order to do that, you'd need two functions:
> (defun set-internal-1 (args)
>   (block nil
>     (if (constant-p)
> 	(if hooked ; Mutually exclusive with actual constant
> 	    (progn
> 	      (set-internal-1-and-run-varhook args)
> 	      (return))
> 	  (if (forbidden-p)
> 	      (error "setting constant")
> 	    (return))))
>     (do-some-stuff)
>     (and-many-more-lines-of-stuff)
>     (set-some-variable)))

> (defun set-internal-1-and-run-varhook (args)
>   (do-some-stuff)
>   (and-many-more-lines-of-stuff)
>   (set-some-variable)
>   (run-varhook))

No, the idea was rather to do:

(defun set-internal-1 (args)
  (block nil ; Because Elisp isn't CL
    (if (constant-or-hooked-p)
        (if (constant-p)
            (if (forbidden-p)
                (error "setting constant")
              (return))
          (funcall symbol-watch-function ..args..))
      (do-some-stuff)
      (and-many-more-lines-of-stuff)
      (set-some-variable))))

> I'd like to bikeshed the function names a bit more. Handler functions for
> varhook are already capable of not only reading variables, but also writing
> them. And now, writing (or blocking of writing) is actually going to be one
> of the intended use cases. That means ‟watch” is a misleading way of
> describing it; watching (which implies only reading) is only one of the
> things that a handler might do.

I think watchpoints usually have the functionality of catching the
modifications, being able to see the "before-change value" and being
able to replace the assignment with something else.  So "watch" makes
a lot of sense to me.

> BTW, why is the order of the first two arguments to advice-add reversed from
> add-function? And the words in their names too.

Don't ask,


        Stefan

[PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 4621 bytes --]

Stefan Monnier wrote:
> No, the idea was rather to do:
>
> (defun set-internal-1 (args)
>   (block nil ; Because Elisp isn't CL
>     (if (constant-or-hooked-p)
>         (if (constant-p)
>             (if (forbidden-p)
>                 (error "setting constant")
>               (return))
>           (funcall symbol-watch-function ..args..))
>       (do-some-stuff)
>       (and-many-more-lines-of-stuff)
>       (set-some-variable))))

Then the many more lines of stuff have to be copied into symbol-watch-function. Those lines can't just be skipped; they're necessary for correctly setting localized and forwarded symbols. By default, hooking a symbol must not change the behavior of Emacs, except for slowing it down. A hook handler can optionally change the behavior, but just hooking the symbol must cause no change if the handler doesn't.

I decided to solve the problem by factoring out the many more lines of stuff to a separate function, so they don't have to be duplicated. This introduces an extra function call for them, but that's ok, since setting localized and forwarded symbols isn't the hot path (and they're already slow anyway). This change has no effect on the hot path.

> I think watchpoints usually have the functionality of catching the
> modifications, being able to see the "before-change value" and being
> able to replace the assignment with something else.  So "watch" makes
> a lot of sense to me.

Um, ok. I already improved the names, but I can change them again if necessary.


Updated patches attached. The first applies to 24.4. The second applies to trunk (using «patch», not «git apply»).


Feature improvements:

It reports the environment as «dyn-local» rather than «invalid» for set-default within a dynamic let-binding, since you said that's a valid thing to do.

It passes not only the new value being set, but also the current value of the variable.

The function names are more informative.

The API now uses function advice instead of a standard hook. Whenever a hooked variable is set, symbol-setter-function is called. Since it's the setter function, it's natural that overriding it would override the setting that occurs, so this is how I implemented the API. As a result, it's not necessary to explicitly set the variable in your handler, which means it's not necessary to temporarily unhook it either; all you have to do is return the value you want to set it to. See the docstring for symbol-setter-function for details.


Performance improvements:

I put «hooked» into «constant», and removed my branches from the hot paths of specbind and set_internal.

I did the minor optimizations of set_internal I mentioned previously, plus the relevant one in specbind and in find_symbol_value. Also in a few others that aren't on hot paths, but they were trivial so I did them anyway.

I inlined grow_specpdl, which speeds up specbind, which more than compensates for the conditional branch that varhook requires in unbind_to. I also inlined the first part (the hot path) of set_internal (i.e. of set_internal_1 in my patch). Both of those are fairly small and not used in very many places. The cost is an increase in the size of the Emacs executable by about 20kB (out of 18.9MB on my system, so that's about 0.1%).

The result of these changes is a measurable performance improvement compared to unmodified Emacs for the common cases, i.e. for setting and dynamically binding plainval (non-buffer-local, non-forwarded) symbols. So now, my patch actually _does_ make Emacs faster. ;-) Here are the benchmarks:

(require 'cl)

(defun test ()
  (setq x 0)
  (benchmark-run-compiled 100000 (incf x)))

(defun test1 ()
  (setq x 0)
  (benchmark-run-compiled
      (do () ((> x 100000)) (incf x))))

(defun test2 ()
  (benchmark-run-compiled 1000
    (letrec
	((foo
	  (lambda (x)
	    (let ((x (1+ x)))
	      (if (> x 100)
		  x
		(1- (funcall foo x)))))))
      (funcall foo 0))))

Execution time results to 4 significant digits:

test
           minimum  average  maximum
Emacs 24.4 0.006253 0.007259 0.009721
with patch 0.001816 0.002148 0.002425
Average improvement: 70%

test1
           minimum  average  maximum
Emacs 24.4 0.01074  0.01075  0.01078
with patch 0.009081 0.009301 0.009671
Average improvement: 13%

test2
           minimum  average  maximum
Emacs 24.4 0.04401  0.04409  0.04417
with patch 0.04196  0.04202  0.04207
Average improvement: 4.7%

For all three tests, the slowest run with the patch was faster than the fastest run without the patch.



[-- Attachment #2: varhook-advice-24.4.patch.gz --]
[-- Type: application/octet-stream, Size: 8998 bytes --]

[-- Attachment #3: varhook-advice.patch.gz --]
[-- Type: application/octet-stream, Size: 8632 bytes --]

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

>> No, the idea was rather to do:
>> 
>> (defun set-internal-1 (args)
>> (block nil ; Because Elisp isn't CL
>> (if (constant-or-hooked-p)
>> (if (constant-p)
>> (if (forbidden-p)
>> (error "setting constant")
>> (return))
>> (funcall symbol-watch-function ..args..))
>> (do-some-stuff)
>> (and-many-more-lines-of-stuff)
>> (set-some-variable))))

> Then the many more lines of stuff have to be copied into
> symbol-watch-function. Those lines can't just be skipped;

We don't really need to copy those lines, instead symbol-watch-function
should somehow (in the "normal" case) call us back, and in way that
doesn't trigger the (constant-or-hooked-p) check.

Of course, for that "...args..." should have enough info that we can
indeed "call it back" properly.

> By default, hooking a symbol must not change the behavior of Emacs,
> except for slowing it down.  A hook handler can optionally change the
> behavior, but just hooking the symbol must cause no change if the
> handler doesn't.

Agreed.

> Updated patches attached. The first applies to 24.4.

We don't want to install this in emacs-24, so only the trunk (aka
"master") code is important in this respect.
[ BTW, even for 30KB patches, I prefer them uncompressed, because
  I typically read them directly in Gnus.  ]

> +  /* When masked with SYMBOL_CONSTANT_MASK, non-zero means symbol is
> +     constant, i.e. changing its value should signal an error.  If the
> +     value is 3, then the var can be changed, but only by `defconst'.
> +     When masked with SYMBOL_HOOKED_MASK, non-zero means setting
> +     symbol will run varhook.  These two fields are combined into one
> +     in order to optimize the fast path of unhooked non-constants by
> +     having only one conditional branch for that case.  */

We don't need this special value 3 for defconst, not only because the
code doesn't use it, but also since if we want that unimplemented
functionality, we can now implement it using your new
symbol-setter system.

> +typedef enum
> +  {
> +    Dyn_Unbind = -1,
> +    Dyn_Current = 0,
> +    Dyn_Bind = 1,
> +    Dyn_Skip = 2,
> +    Dyn_Global = 3
> +  } Dyn_Bind_Direction;

In which sense is this a "direction"?

> +/* Like set_internal but with direction argument to indicate whether this
> +   function call is due to a binding (1), an unbinding (-1), or neither (0).
> +   As special cases, a value of 2 is a flag to disable run_varhook so that
> +   varhooks aren't run during backtraces, and a value of 3 is a flag
> +   indicating that this function call is due to set_default, which allows
> +   run_varhook to distinguish beween the global and the dyn-local binding.  */

Please use the Dyn_* names rather than the numerical constants in
the comment.

> +   Return the result of symbol-setter-function. The variable will be set
> +   (by code that calls run_varhook) to that result, overriding the value to
> +   which the setter is attempting to set the variable.  */

That's a good idea, to circumvent the question of how to not-trigger the
hooked-p check recursively when the hook function calls the setter (tho
the question partly remains, in case the hook function *accidentally*
sets one of the hooked variables).

It does mean that the hooks can't redirect the assignment elsewhere, but
maybe it's a good thing anyway.

> +	bool shadowed;
> +	if (buf_local)
> +	  shadowed = let_shadows_buffer_binding_p (sym);
> +	else shadowed = let_shadows_global_binding_p (symbol);

Aka
        bool shadowed = (buf_local ? let_shadows_buffer_binding_p (sym)
                         : let_shadows_global_binding_p (symbol));

> +	if (shadowed) env = Qdyn_local;
> +	else if (buf_local) env = Qbuf_local;
> +	else env = Qglobal;

Why does the hook need to know about those different cases?

> + start: /* Just to avoid reindentation in the varhook patch */

"diff -bw" works just as well for me.

> +			 (XFWDTYPE (innercontents))==Lisp_Fwd_Buffer_Obj,
                                                   ^^
We like to put spaces around infix operators.

> +DEFUN ("symbol-setter-function", Fsymbol_setter_function, Ssymbol_setter_function, 4, 4, 0,

Hmm, no symbol-setter-function should be a variable (holding
a function), modified via add-function/remove-function.

Also the docstring should not recommend :override (which should be
a rather rare case, the more useful cases are probably :before
and :around), and in general it should just say how the function is
called and what it should return, without giving a primer about how to
modify such *-function variables, since we don't want to do that for
every *-function variable.

OTOH the docstring should document the different possible values of ENV
and their respective meaning.


        Stefan

[PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 4470 bytes --]

Stefan Monnier wrote:
> We don't want to install this in emacs-24, so only the trunk (aka
> "master") code is important in this respect.

Understood. Last time I included the 24.4 patch just to ensure you could reproduce my benchmark results. (Due to some logistical issues, I can't run trunk on a system that has stable performance, so I can't reliably benchmark it.)

>> +typedef enum
>> +  {
>> +    Dyn_Unbind = -1,
>> +    Dyn_Current = 0,
>> +    Dyn_Bind = 1,
>> +    Dyn_Skip = 2,
>> +    Dyn_Global = 3
>> +  } Dyn_Bind_Direction;
>
> In which sense is this a "direction"?

Originally I had just the first three values, and it was a direction in the sense of movement up or down the dynamic-binding stack. Later I discovered that I needed the last two values too. I've changed it to a more appropriate name.

> That's a good idea, to circumvent the question of how to not-trigger the
> hooked-p check recursively when the hook function calls the setter (tho
> the question partly remains, in case the hook function *accidentally*
> sets one of the hooked variables).

The docstring for symbol-setter-function says lexical binding is required for the hook function, which means its local variables won't trigger varhook. If the hook function does set a dynamic variable that's hooked, and has no terminating condition for the recursion, you'll immediately find out when it exceeds max-lisp-eval-depth. So, don't do that. ;-)

> It does mean that the hooks can't redirect the assignment elsewhere, but
> maybe it's a good thing anyway.

They still can. Just temporarily unhook the destination variable before setting it. But yes, it would be easy to introduce bugs by doing that, so I guess the documentation should discourage it.

>> +	if (shadowed) env = Qdyn_local;
>> +	else if (buf_local) env = Qbuf_local;
>> +	else env = Qglobal;
>
> Why does the hook need to know about those different cases?

So the user can notice, during debugging, if setq is setting the symbol in a different environment than the one he intended, e.g. due to an unexpected buffer-local variable or due to a missing buffer-local variable, or due to an unexpected dynamic binding of a global variable in code that calls code that uses setq with the assumption that the global (not a dynamic local) variable will be set. Also, this enables detailed profiling of globals vs. buffer-locals vs. dynamic bindings. And it lets your hook function filter out the cases you want to ignore, e.g. if you only want to watch global settings, not buffer-local or let-local.

>> +DEFUN ("symbol-setter-function", Fsymbol_setter_function, Ssymbol_setter_function, 4, 4, 0,
>
> Hmm, no symbol-setter-function should be a variable (holding
> a function), modified via add-function/remove-function.

I don't see why; the only difference is using add-function with an unquoted variable name vs. using advice-add with a quoted function name. It also makes the hook run a bit slower. And it results in the help page for the function exposing the advice mechanism's internal representation of the advice, rather than cleanly showing e.g. ⌜:before advice: `mywatcher'⌝. That seems like a bad idea.

But anyway I changed it to what you want, IIUC. I hope I misunderstood.

> Also the docstring should not recommend :override (which should be
> a rather rare case, the more useful cases are probably :before
> and :around)

The docstring already says to use :before if you just need to watch variables, but not block or override attempts to set them. And :around is the same as :override in this case, since all the standard function does is return newval, which is overridden by either :around or :override; the standard function doesn't do any processing. Unless maybe you want to have multiple pieces of advice wrapped around each other, all blocking/overriding attempts to set variables?

Anyway I changed the docstring to recommend :around instead of :override.

I also made the other changes you wanted. And I cleaned up the patch a bit, consolidating set_internal and set_internal_1, so the extra name isn't needed anymore. (This is just a cosmetic change; it doesn't affect the compiled code, since one was just a wrapper for the other and both were inlined.) And I fixed a bug: it was reporting the wrong environment for setq-default if you do:
(setq-local foo 'bar)
(let ((foo 'baz)) (setq-default foo 'biz))

Updated patch attached.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: varhook-advice-1.patch --]
[-- Type: text/x-diff, Size: 31656 bytes --]

--- src/lisp.h
+++ src/lisp.h
@@ -305,6 +305,17 @@
 #endif
 
 
+/* These are the masks for the constant_or_hooked field of Lisp_Symbol.
+   Bit 0 stores the constant field.  Bit 1 stores the hooked field.  */
+#define SYMBOL_CONSTANT_MASK 1
+#define SYMBOL_HOOKED_MASK 2
+
+# define SYM_CONSTANT_P(sym) (((sym)->constant_or_hooked) \
+				       & SYMBOL_CONSTANT_MASK)
+# define SYM_HOOKED_P(sym) (((sym)->constant_or_hooked) \
+				       & SYMBOL_HOOKED_MASK)
+
+
 /* Some operations are so commonly executed that they are implemented
    as macros, not functions, because otherwise runtime performance would
    suffer too much when compiling with GCC without optimization.
@@ -359,7 +370,7 @@
 #define lisp_h_NILP(x) EQ (x, Qnil)
 #define lisp_h_SET_SYMBOL_VAL(sym, v) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value = (v))
-#define lisp_h_SYMBOL_CONSTANT_P(sym) (XSYMBOL (sym)->constant)
+#define lisp_h_SYMBOL_CONSTANT_P(sym) (SYM_CONSTANT_P (XSYMBOL (sym)))
 #define lisp_h_SYMBOL_VAL(sym) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value)
 #define lisp_h_SYMBOLP(x) (XTYPE (x) == Lisp_Symbol)
@@ -1597,10 +1608,13 @@
      3 : it's a forwarding variable, the value is in `forward'.  */
   ENUM_BF (symbol_redirect) redirect : 3;
 
-  /* Non-zero means symbol is constant, i.e. changing its value
-     should signal an error.  If the value is 3, then the var
-     can be changed, but only by `defconst'.  */
-  unsigned constant : 2;
+  /* When masked with SYMBOL_CONSTANT_MASK, non-zero means symbol is
+     constant, i.e. changing its value should signal an error.
+     When masked with SYMBOL_HOOKED_MASK, non-zero means setting
+     symbol will run varhook.  These two fields are combined into one
+     in order to optimize the fast path of unhooked non-constants by
+     having only one conditional branch for that case.  */
+  unsigned constant_or_hooked : 2;
 
   /* Interned state of the symbol.  This is an enumerator from
      enum symbol_interned.  */
@@ -3391,6 +3405,14 @@
 EXFUN (Fbyteorder, 0) ATTRIBUTE_CONST;
 
 /* Defined in data.c.  */
+typedef enum
+  { /* See set_internal for a description of these values */
+    Dyn_Unbind = -1,
+    Dyn_Current = 0,
+    Dyn_Bind = 1,
+    Dyn_Skip = 2,
+    Dyn_Global = 3
+  } Dyn_Bind_Env;
 extern Lisp_Object indirect_function (Lisp_Object);
 extern Lisp_Object find_symbol_value (Lisp_Object);
 enum Arith_Comparison {
@@ -3438,7 +3460,16 @@
 					   Lisp_Object);
 extern _Noreturn Lisp_Object wrong_type_argument (Lisp_Object, Lisp_Object);
 extern Lisp_Object do_symval_forwarding (union Lisp_Fwd *);
-extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool);
+extern Lisp_Object run_varhook (struct Lisp_Symbol*, bool, Dyn_Bind_Env,
+				Lisp_Object, Lisp_Object);
+extern void set_internal_with_varhook (Lisp_Object, Lisp_Object,
+					 Lisp_Object, bool,
+					 Dyn_Bind_Env, struct Lisp_Symbol *);
+extern void set_internal_localized_or_forwarded (Lisp_Object, Lisp_Object,
+						 Lisp_Object, bool,
+						 Dyn_Bind_Env,
+						 struct Lisp_Symbol *);
+extern void set_default_internal (Lisp_Object, Lisp_Object, Dyn_Bind_Env);
 extern void syms_of_data (void);
 extern void swap_in_global_binding (struct Lisp_Symbol *);
 
@@ -4595,6 +4627,65 @@
     return false;
 }
 
+/* Store the value NEWVAL into SYMBOL.
+   If buffer/frame-locality is an issue, WHERE specifies which context to use.
+   (nil stands for the current buffer/frame).
+
+   If BINDFLAG is false, then if this symbol is supposed to become
+   local in every buffer where it is set, then we make it local.
+   If BINDFLAG is true, we don't do that.
+
+   ENV indicates the dynamic environment for this function call, i.e. whether
+   this call is due to a variable binding (Dyn_Bind), an unbinding (Dyn_Unbind),
+   or neither (Dyn_Current). As special cases, a value of Dyn_Skip is a flag
+   to disable run_varhook so that varhooks aren't run during backtraces, and
+   a value of Dyn_Global is a flag indicating that this function call is due
+   to set_default, which allows run_varhook to distinguish beween the global
+   and the dyn-local binding.  */
+
+INLINE void
+set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+	      bool bindflag, Dyn_Bind_Env env)
+{
+  struct Lisp_Symbol *sym;
+
+  /* If restoring in a dead buffer, do nothing.  */
+  /* if (BUFFERP (where) && NILP (XBUFFER (where)->name))
+      return; */
+
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  if (sym->constant_or_hooked)
+    {
+      if (SYM_HOOKED_P (sym))
+	{
+	  set_internal_with_varhook (symbol, newval, where, bindflag, env, sym);
+	  return;
+	}
+      if (NILP (Fkeywordp (symbol))
+	  || !EQ (newval, Fsymbol_value (symbol)))
+	xsignal1 (Qsetting_constant, symbol);
+      else
+	/* Allow setting keywords to their own value.  */
+	return;
+    }
+
+ start:
+  switch (sym->redirect)
+    {
+    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym, newval); return;
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+    default: set_internal_localized_or_forwarded
+	(symbol, newval, where, bindflag, env, sym);
+    }
+}
+
+#define MAYBE_RUN_VARHOOK(result, sym, buf_local, env, oldval, newval)	\
+  {									\
+    if (SYM_HOOKED_P (sym))						\
+      (result) = run_varhook (sym, buf_local, env, oldval, newval);	\
+  }
+
 INLINE_HEADER_END
 
 #endif /* EMACS_LISP_H */
--- src/eval.c
+++ src/eval.c
@@ -267,7 +268,7 @@
   max_lisp_eval_depth = XINT (XCDR (data));
 }
 
-static void grow_specpdl (void);
+static inline void grow_specpdl (void);
 
 /* Call the Lisp debugger, giving it argument ARG.  */
 
@@ -601,6 +602,63 @@
     return quoted;
 }
 
+DEFUN ("void-p", Fvoid_p, Svoid_p, 1, UNEVALLED, 0,
+       doc: /* Return t if ARG has no value.
+If ARG is a non-lexical variable, this is equivalent to
+(not (boundp (quote ARG))).
+
+Unlike `boundp', this function can also test a lexical variable.
+
+See also `void'.
+usage: (void-p ARG)  */)
+  (Lisp_Object args)
+{
+  register Lisp_Object val;
+  struct gcpro gcpro1;
+  GCPRO1 (args);
+  if (CONSP (XCDR (args)))
+    xsignal2 (Qwrong_number_of_arguments, Qvoid_p, Flength (args));
+
+  val = XCAR (args);
+
+  if (SYMBOLP (val))
+    { /* This block is derived from the first block of eval_sub */
+      Lisp_Object lex_binding
+	= !NILP (Vinternal_interpreter_environment)
+	? Fassq (val, Vinternal_interpreter_environment)
+	: Qnil;
+      if (CONSP (lex_binding))
+	val = XCDR (lex_binding);
+      else
+	val = find_symbol_value (val); /* Avoid signaling error if unbound */
+    }
+  else
+    val = eval_sub (val);
+
+  val = EQ (val, Qunbound) ? Qt : Qnil;
+  UNGCPRO;
+  return val;
+}
+
+
+DEFUN ("void", Fvoid, Svoid, 0, 0, 0,
+       doc: /* Return nothing.
+This is the only built-in Elisp function that does not return a value.
+Returning the result of this function enables any other function
+to avoid returning a value.
+
+Setting a variable to the result of this function will unbind the variable.
+For example, (setq foo (void)) is equivalent to (makunbound 'foo), if
+foo is a non-lexical variable.
+
+Unlike `makunbound', this function can also be used to unbind a
+lexical variable.
+
+See also `void-p'.  */)
+  ()
+{
+  return Qunbound;
+}
 
 DEFUN ("defvaralias", Fdefvaralias, Sdefvaralias, 2, 3, 0,
        doc: /* Make NEW-ALIAS a variable alias for symbol BASE-VARIABLE.
@@ -620,7 +678,7 @@
 
   sym = XSYMBOL (new_alias);
 
-  if (sym->constant)
+  if (SYM_CONSTANT_P (sym))
     /* Not sure why, but why not?  */
     error ("Cannot make a constant an alias");
 
@@ -637,7 +695,7 @@
      so that old-code that affects n_a before the aliasing is setup
      still works.  */
   if (NILP (Fboundp (base_variable)))
-    set_internal (base_variable, find_symbol_value (new_alias), Qnil, 1);
+    set_internal (base_variable, find_symbol_value (new_alias), Qnil, 1, Dyn_Current);
 
   {
     union specbinding *p;
@@ -652,7 +710,7 @@
   XSYMBOL (base_variable)->declared_special = 1;
   sym->redirect = SYMBOL_VARALIAS;
   SET_SYMBOL_ALIAS (sym, XSYMBOL (base_variable));
-  sym->constant = SYMBOL_CONSTANT_P (base_variable);
+  sym->constant_or_hooked = SYMBOL_CONSTANT_P (base_variable);
   LOADHIST_ATTACH (new_alias);
   /* Even if docstring is nil: remove old docstring.  */
   Fput (new_alias, Qvariable_documentation, docstring);
@@ -2007,7 +2065,7 @@
    never-used entry just before the bottom of the stack; sometimes its
    address is taken.  */
 
-static void
+static inline void
 grow_specpdl (void)
 {
   specpdl_ptr++;
@@ -3132,8 +3190,6 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS:
-      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
     case SYMBOL_PLAINVAL:
       /* The most common case is that of a non-constant symbol with a
 	 trivial value.  Make that as fast as we can.  */
@@ -3141,11 +3197,15 @@
       specpdl_ptr->let.symbol = symbol;
       specpdl_ptr->let.old_value = SYMBOL_VAL (sym);
       grow_specpdl ();
-      if (!sym->constant)
-	SET_SYMBOL_VAL (sym, value);
+      if (!sym->constant_or_hooked) SET_SYMBOL_VAL (sym, value);
+      else if (SYM_HOOKED_P (sym))
+	SET_SYMBOL_VAL (sym, run_varhook
+			(sym, false, Dyn_Bind, sym->val.value, value));
       else
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, 1, Dyn_Bind);
       break;
+    case SYMBOL_VARALIAS:
+      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
     case SYMBOL_LOCALIZED:
       if (SYMBOL_BLV (sym)->frame_local)
 	error ("Frame-local vars cannot be let-bound");
@@ -3176,7 +3236,7 @@
 	      {
 		specpdl_ptr->let.kind = SPECPDL_LET_DEFAULT;
 		grow_specpdl ();
-		Fset_default (symbol, value);
+		set_default_internal (symbol, value, Dyn_Bind);
 		return;
 	      }
 	  }
@@ -3184,7 +3244,7 @@
 	  specpdl_ptr->let.kind = SPECPDL_LET;
 
 	grow_specpdl ();
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, 1, Dyn_Bind);
 	break;
       }
     default: emacs_abort ();
@@ -3319,7 +3379,9 @@
 	    struct Lisp_Symbol *sym = XSYMBOL (specpdl_symbol (specpdl_ptr));
 	    if (sym->redirect == SYMBOL_PLAINVAL)
 	      {
-		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
+		Lisp_Object oldval = specpdl_old_value (specpdl_ptr);
+		MAYBE_RUN_VARHOOK (oldval, sym, false, Dyn_Unbind, sym->val.value, oldval);
+		SET_SYMBOL_VAL (sym, oldval);
 		break;
 	      }
 	    else
@@ -3329,8 +3391,8 @@
 	      }
 	  }
 	case SPECPDL_LET_DEFAULT:
-	  Fset_default (specpdl_symbol (specpdl_ptr),
-			specpdl_old_value (specpdl_ptr));
+	  set_default_internal (specpdl_symbol (specpdl_ptr),
+				specpdl_old_value (specpdl_ptr), Dyn_Unbind);
 	  break;
 	case SPECPDL_LET_LOCAL:
 	  {
@@ -3342,7 +3404,7 @@
 	    /* If this was a local binding, reset the value in the appropriate
 	       buffer, but only if that buffer's binding still exists.  */
 	    if (!NILP (Flocal_variable_p (symbol, where)))
-	      set_internal (symbol, old_value, where, 1);
+	      set_internal (symbol, old_value, where, 1, Dyn_Unbind);
 	  }
 	  break;
 	}
@@ -3537,7 +3599,7 @@
 	    Lisp_Object sym = specpdl_symbol (tmp);
 	    Lisp_Object old_value = specpdl_old_value (tmp);
 	    set_specpdl_old_value (tmp, Fdefault_value (sym));
-	    Fset_default (sym, old_value);
+	    set_default_internal (sym, old_value, Dyn_Skip);
 	  }
 	  break;
 	case SPECPDL_LET_LOCAL:
@@ -3553,7 +3615,7 @@
 	      {
 		set_specpdl_old_value
 		  (tmp, Fbuffer_local_value (symbol, where));
-		set_internal (symbol, old_value, where, 1);
+		set_internal (symbol, old_value, where, 1, Dyn_Skip);
 	      }
 	  }
 	  break;
@@ -3754,6 +3816,11 @@
   DEFSYM (Qinhibit_debugger, "inhibit-debugger");
   DEFSYM (Qmacro, "macro");
   DEFSYM (Qdeclare, "declare");
+  DEFSYM (Qvoid_p, "void-p");
+  DEFSYM (Qsym, "sym");
+  DEFSYM (Qenv, "env");
+  DEFSYM (Qoldval, "oldval");
+  DEFSYM (Qnewval, "newval");
 
   /* Note that the process handling also uses Qexit, but we don't want
      to staticpro it twice, so we just do it here.  */
@@ -3828,6 +3895,67 @@
 still determine whether to handle the particular condition.  */);
   Vdebug_on_signal = Qnil;
 
+  DEFVAR_LISP ("symbol-setter-function", Vsymbol_setter_function,
+       doc: /* This function is called whenever a hooked variable is set.
+It takes four arguments: SYMBOL, ENV, OLDVAL, NEWVAL. By default, it just
+returns NEWVAL unchanged.
+
+SYMBOL is the symbol being set. ENV is the environment is which it's being
+set. OLDVAL is the current value. NEWVAL is the new value to which the
+setter, i.e. the caller of a function such as `setq', is attempting to set
+the variable. The actual new value to which the variable will be set is the
+return value of this function, which is NEWVAL if this function does not
+have advice that overrides it.
+
+The possible values of ENV are these symbols, with these meanings:
+global: The global environment.
+buf-local: The setter's buffer-local environment.
+dyn-local: The innermost dynamic environment in which SYMBOL is bound.
+dyn-bind: A new dynamic environment, such as creatable using `let'.
+dyn-unbind: The next-outer dynamic environment in which SYMBOL is still bound,
+or the buffer-local environment if SYMBOL is not bound in any dynamic
+environment, or the global environment is SYMBOL is not in the buffer-local
+environment, unshadowed due to destruction of the setter's current
+dynamic environment, such as due to exit of a `let' form.
+
+To watch hooked variables, advise this function using `add-function' with
+:before as the WHERE argument.
+
+To watch hooked variables and optionally override the attempts to set them,
+advise this function with advice that overrides the return value, such as
+by using :override or (preferably) :around as the WHERE argument.
+
+At the time the definition of your advice function is evaluated,
+`lexical-binding' must be t, i.e. your advice must be a closure (even if
+its lexical environment is empty).
+
+If you use overriding advice, your advice must return the value to which to
+set the variable. To avoid overriding the setter's attempt to set the variable
+to NEWVAL, return NEWVAL. To block the attempt, and leave the variable
+unchanged, return OLDVAL. If ENV is dyn-bind or dyn-unbind, you can block
+the change of value, but you can't prevent the corresponding creation or
+destruction of a dynamic environment. Therefore, blocking when ENV is
+dyn-bind will set SYMBOL in the new environment to its value in the outer
+environment, and blocking when ENV is dyn-unbind will set SYMBOL in the
+outer environment to its value in the environment being destroyed.
+
+If the variable is currently void, OLDVAL will be void. If the setter
+is attempting to unbind the variable, NEWVAL will be void. Test for this
+using `void-p'. If you use overriding advice, OLDVAL is void, and you return
+it, the variable will remain void. If NEWVAL is void, and you return it, the
+setter's attempt to unbind the variable succeeds. If neither is void, you
+can still unbind the variable by returning the result of the function `void'.
+
+Don't set the variable in your advice. Instead, if your advice needs
+to set the variable, use `add-function' with overriding advice.
+
+To hook all variables of a symbol, use `symbol-hook'. To unhook them,
+use `symbol-unhook'. If you only want to watch or override some variables
+of a symbol, then filter according to ENV, and if you use overriding advice,
+simply return NEWVAL for the ones you don't want to process.  */);
+  Vsymbol_setter_function =
+    list4 (Qclosure, list1 (Qt), list4 (Qsym, Qenv, Qoldval, Qnewval), Qnewval);
+
   /* When lexical binding is being used,
    Vinternal_interpreter_environment is non-nil, and contains an alist
    of lexically-bound variable, or (t), indicating an empty
@@ -3902,4 +4030,6 @@
   defsubr (&Sbacktrace__locals);
   defsubr (&Sspecial_variable_p);
   defsubr (&Sfunctionp);
+  defsubr (&Svoid);
+  defsubr (&Svoid_p);
 }
--- src/data.c
+++ src/data.c
@@ -612,6 +613,20 @@
 \f
 /* Extract and set components of symbols.  */
 
+DEFUN ("symbol-hooked-p", Fsymbol_hooked_p, Ssymbol_hooked_p, 1, 1, 0,
+       doc: /* Return t if SYMBOL is hooked.
+To hook and unhook it, use `symbol-hook' and `symbol-unhook'.
+When hooked, setting SYMBOL will run `symbol-setter-function'.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    sym = indirect_variable (sym);
+  return SYM_HOOKED_P (sym) ? Qt : Qnil;
+}
+
 DEFUN ("boundp", Fboundp, Sboundp, 1, 1, 0,
        doc: /* Return t if SYMBOL's value is not void.
 Note that if `lexical-binding' is in effect, this refers to the
@@ -661,6 +676,46 @@
   return NILP (XSYMBOL (symbol)->function) ? Qnil : Qt;
 }
 
+DEFUN ("symbol-hook", Fsymbol_hook, Ssymbol_hook, 1, 1, 0,
+       doc: /* Hook SYMBOL.
+When hooked, setting it will run `symbol-setter-function'.
+To unhook it, use `symbol-unhook'.
+To test whether it's hooked, use `symbol-hooked-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  sym->constant_or_hooked |= SYMBOL_HOOKED_MASK;
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->constant_or_hooked |= SYMBOL_HOOKED_MASK;
+    }
+  return symbol;
+}
+
+DEFUN ("symbol-unhook", Fsymbol_unhook, Ssymbol_unhook, 1, 1, 0,
+       doc: /* Unhook SYMBOL.
+When unhooked, setting it will not run `symbol-setter-function'.
+To hook it, use `symbol-hook'.
+To test whether it's hooked, use `symbol-hooked-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  sym->constant_or_hooked &= (SYMBOL_HOOKED_MASK ^ -1);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->constant_or_hooked &= (SYMBOL_HOOKED_MASK ^ -1);
+    }
+  return symbol;
+}
+
 DEFUN ("makunbound", Fmakunbound, Smakunbound, 1, 1, 0,
        doc: /* Make SYMBOL's value be void.
 Return SYMBOL.  */)
@@ -1137,8 +1192,8 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
@@ -1167,54 +1222,97 @@
   xsignal1 (Qvoid_variable, symbol);
 }
 
+/* For the symbol S being set, run symbol-setter-function with these arguments:
+   0. S
+   1. A symbol indicating the environment in which S is being set.
+   2. The current value of S in that environment.
+   3. The value to which the setter is attempting to set the variable.
+
+   Return the result of symbol-setter-function. The variable will be set
+   (by code that calls run_varhook) to that result, overriding the value to
+   which the setter is attempting to set the variable.  */
+
+Lisp_Object
+run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Env rawenv,
+	     Lisp_Object oldval, Lisp_Object newval)
+{
+  Lisp_Object symbol;
+  Lisp_Object env;
+  if (rawenv == Dyn_Skip) /* From backtrace_eval_unrewind */
+    return newval;
+  XSETSYMBOL (symbol, sym);
+  switch (rawenv) /* Resolve Dyn_Current and disambiguate Dyn_Global */
+    {
+    case Dyn_Current:
+      {
+	bool shadowed = (buf_local ? let_shadows_buffer_binding_p (sym)
+                         : let_shadows_global_binding_p (symbol));
+	if (shadowed) env = Qdyn_local;
+	else if (buf_local) env = Qbuf_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Global:
+      {
+	/* let_shadows_buffer_binding_p doesn't disambiguate this case */
+	if (let_shadows_global_binding_p (symbol) &&
+	    NILP (Flocal_variable_p (symbol, Qnil)))
+	  env = Qdyn_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Bind: env = Qdyn_bind; break;
+    case Dyn_Unbind: env = Qdyn_unbind; break;
+    default: emacs_abort ();
+    }
+  return call4 (Vsymbol_setter_function, symbol, env, oldval, newval);
+}
+
 DEFUN ("set", Fset, Sset, 2, 2, 0,
        doc: /* Set SYMBOL's value to NEWVAL, and return NEWVAL.  */)
   (register Lisp_Object symbol, Lisp_Object newval)
 {
-  set_internal (symbol, newval, Qnil, 0);
+  set_internal (symbol, newval, Qnil, 0, Dyn_Current);
   return newval;
 }
 
-/* Store the value NEWVAL into SYMBOL.
-   If buffer/frame-locality is an issue, WHERE specifies which context to use.
-   (nil stands for the current buffer/frame).
-
-   If BINDFLAG is false, then if this symbol is supposed to become
-   local in every buffer where it is set, then we make it local.
-   If BINDFLAG is true, we don't do that.  */
+/* set_internal is in lisp.h due to being inlined.  */
+
+/* Split from set_internal just to avoid an extra conditional branch on the fast
+   path for non-hooked variables.  */
 
 void
-set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
-	      bool bindflag)
+set_internal_with_varhook (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+			     bool bindflag, Dyn_Bind_Env env, struct Lisp_Symbol *sym)
 {
-  bool voide = EQ (newval, Qunbound);
-  struct Lisp_Symbol *sym;
-  Lisp_Object tem1;
-
-  /* If restoring in a dead buffer, do nothing.  */
-  /* if (BUFFERP (where) && NILP (XBUFFER (where)->name))
-      return; */
-
-  CHECK_SYMBOL (symbol);
-  if (SYMBOL_CONSTANT_P (symbol))
+ start:
+  switch (sym->redirect)
     {
-      if (NILP (Fkeywordp (symbol))
-	  || !EQ (newval, Fsymbol_value (symbol)))
-	xsignal1 (Qsetting_constant, symbol);
-      else
-	/* Allow setting keywords to their own value.  */
+    case SYMBOL_PLAINVAL:
+      {
+	SET_SYMBOL_VAL (sym, run_varhook (sym, false, env, sym->val.value, newval));
 	return;
     }
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+    default: set_internal_localized_or_forwarded (symbol, newval, where, bindflag, env, sym);
+    }
+}
 
-  sym = XSYMBOL (symbol);
+/* Split from set_internal to avoid code duplication, because both set_internal and
+   set_internal_with_varhook must call this function.  */
 
- start:
+void
+set_internal_localized_or_forwarded (Lisp_Object symbol, Lisp_Object newval,
+				     Lisp_Object where, bool bindflag,
+				     Dyn_Bind_Env env, struct Lisp_Symbol *sym)
+{
+  bool voide;
+  Lisp_Object tem1;
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
     case SYMBOL_LOCALIZED:
       {
+	bool buf_local = true;
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 	if (NILP (where))
 	  {
@@ -1258,6 +1356,7 @@
 		   indicating that we're seeing the default value.
 		   Likewise if the variable has been let-bound
 		   in the current buffer.  */
+		buf_local = false;
 		if (bindflag || !blv->local_if_set
 		    || let_shadows_buffer_binding_p (sym))
 		  {
@@ -1285,6 +1384,9 @@
 	    set_blv_valcell (blv, tem1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym, buf_local, env, blv_value (blv), newval);
+	voide = EQ (newval, Qunbound);
+
 	/* Store the new value in the cons cell.  */
 	set_blv_value (blv, newval);
 
@@ -1316,6 +1418,11 @@
 	      SET_PER_BUFFER_VALUE_P (buf, idx, 1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym,
+			   (XFWDTYPE (innercontents)) == Lisp_Fwd_Buffer_Obj,
+			   env, do_symval_forwarding (innercontents), newval);
+	voide = EQ (newval, Qunbound);
+
 	if (voide)
 	  { /* If storing void (making the symbol void), forward only through
 	       buffer-local indicator, not through Lisp_Objfwd, etc.  */
@@ -1347,8 +1454,8 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_LOCALIZED:
       {
 	/* If var is set up for a buffer that lacks a local value for it,
@@ -1413,6 +1520,17 @@
 for this variable.  */)
   (Lisp_Object symbol, Lisp_Object value)
 {
+  set_default_internal (symbol, value, Dyn_Global);
+  return value;
+}
+
+/* Like Fset_default, but with ENV argument. See set_internal for
+   a description of this argument. */
+
+void
+set_default_internal (Lisp_Object symbol, Lisp_Object value,
+		      Dyn_Bind_Env env)
+{
   struct Lisp_Symbol *sym;
 
   CHECK_SYMBOL (symbol);
@@ -1423,26 +1541,32 @@
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
-	return value;
+	return;
     }
   sym = XSYMBOL (symbol);
 
  start:
   switch (sym->redirect)
     {
+    case SYMBOL_PLAINVAL:
+      {
+	set_internal (symbol, value, Qnil, false, env);
+	return;
+      }
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: return Fset (symbol, value);
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 
+	MAYBE_RUN_VARHOOK (value, sym, false, env, XCDR (blv->defcell), value);
+
 	/* Store new value into the DEFAULT-VALUE slot.  */
 	XSETCDR (blv->defcell, value);
 
 	/* If the default binding is now loaded, set the REALVALUE slot too.  */
 	if (blv->fwd && EQ (blv->defcell, blv->valcell))
 	  store_symval_forwarding (blv->fwd, value, NULL);
-	return value;
+	return;
       }
     case SYMBOL_FORWARDED:
       {
@@ -1456,6 +1580,8 @@
 	    int offset = XBUFFER_OBJFWD (valcontents)->offset;
 	    int idx = PER_BUFFER_IDX (offset);
 
+	    MAYBE_RUN_VARHOOK (value, sym, false, env, per_buffer_default (offset), value);
+
 	    set_per_buffer_default (offset, value);
 
 	    /* If this variable is not always local in all buffers,
@@ -1468,10 +1594,13 @@
 		  if (!PER_BUFFER_VALUE_P (b, idx))
 		    set_per_buffer_value (b, offset, value);
 	      }
-	    return value;
+	    return;
 	  }
 	else
-	  return Fset (symbol, value);
+	  {
+	    set_internal (symbol, value, Qnil, false, env);
+	    return;
+	  }
       }
     default: emacs_abort ();
     }
@@ -1599,7 +1728,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONSTANT_P (sym))
     error ("Symbol %s may not be buffer-local", SDATA (SYMBOL_NAME (variable)));
 
   if (!blv)
@@ -1672,7 +1801,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONSTANT_P (sym))
     error ("Symbol %s may not be buffer-local",
 	   SDATA (SYMBOL_NAME (variable)));
 
@@ -1861,7 +1990,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONSTANT_P (sym))
     error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));
 
   blv = make_blv (sym, forwarded, valcontents);
@@ -3470,6 +3599,12 @@
   DEFSYM (Qad_advice_info, "ad-advice-info");
   DEFSYM (Qad_activate_internal, "ad-activate-internal");
 
+  DEFSYM (Qglobal, "global");
+  DEFSYM (Qbuf_local, "buf-local");
+  DEFSYM (Qdyn_local, "dyn-local");
+  DEFSYM (Qdyn_bind, "dyn-bind");
+  DEFSYM (Qdyn_unbind, "dyn-unbind");
+
   error_tail = pure_cons (Qerror, Qnil);
 
   /* ERROR is used as a signaler for random errors for which nothing else is
@@ -3609,8 +3744,11 @@
   defsubr (&Sindirect_function);
   defsubr (&Ssymbol_plist);
   defsubr (&Ssymbol_name);
+  defsubr (&Ssymbol_hook);
+  defsubr (&Ssymbol_unhook);
   defsubr (&Smakunbound);
   defsubr (&Sfmakunbound);
+  defsubr (&Ssymbol_hooked_p);
   defsubr (&Sboundp);
   defsubr (&Sfboundp);
   defsubr (&Sfset);
@@ -3678,10 +3816,10 @@
   DEFVAR_LISP ("most-positive-fixnum", Vmost_positive_fixnum,
 	       doc: /* The largest value that is representable in a Lisp integer.  */);
   Vmost_positive_fixnum = make_number (MOST_POSITIVE_FIXNUM);
-  XSYMBOL (intern_c_string ("most-positive-fixnum"))->constant = 1;
+  XSYMBOL (intern_c_string ("most-positive-fixnum"))->constant_or_hooked = 1;
 
   DEFVAR_LISP ("most-negative-fixnum", Vmost_negative_fixnum,
 	       doc: /* The smallest value that is representable in a Lisp integer.  */);
   Vmost_negative_fixnum = make_number (MOST_NEGATIVE_FIXNUM);
-  XSYMBOL (intern_c_string ("most-negative-fixnum"))->constant = 1;
+  XSYMBOL (intern_c_string ("most-negative-fixnum"))->constant_or_hooked = 1;
 }
--- src/alloc.c
+++ src/alloc.c
@@ -3390,7 +3390,7 @@
   set_symbol_next (val, NULL);
   p->gcmarkbit = false;
   p->interned = SYMBOL_UNINTERNED;
-  p->constant = 0;
+  p->constant_or_hooked = 0;
   p->declared_special = false;
   p->pinned = false;
   consing_since_gc += sizeof (struct Lisp_Symbol);
--- src/lread.c
+++ src/lread.c
@@ -3821,7 +3821,7 @@
   if ((SREF (string, 0) == ':')
       && EQ (obarray, initial_obarray))
     {
-      XSYMBOL (sym)->constant = 1;
+      XSYMBOL (sym)->constant_or_hooked = 1;
       XSYMBOL (sym)->redirect = SYMBOL_PLAINVAL;
       SET_SYMBOL_VAL (XSYMBOL (sym), sym);
     }
@@ -4042,7 +4042,7 @@
   set_symbol_function (Qunbound, Qnil);
   set_symbol_plist (Qunbound, Qnil);
   SET_SYMBOL_VAL (XSYMBOL (Qnil), Qnil);
-  XSYMBOL (Qnil)->constant = 1;
+  XSYMBOL (Qnil)->constant_or_hooked = 1;
   XSYMBOL (Qnil)->declared_special = true;
   set_symbol_plist (Qnil, Qnil);
   set_symbol_function (Qnil, Qnil);
@@ -4050,7 +4050,7 @@
 
   Qt = intern_c_string ("t");
   SET_SYMBOL_VAL (XSYMBOL (Qt), Qt);
-  XSYMBOL (Qt)->constant = 1;
+  XSYMBOL (Qt)->constant_or_hooked = 1;
   XSYMBOL (Qt)->declared_special = true;
 
   /* Qt is correct even if CANNOT_DUMP.  loadup.el will set to nil at end.  */
--- src/buffer.c
+++ src/buffer.c
@@ -5753,7 +5753,7 @@
 This variable is buffer-local but you cannot set it directly;
 use the function `set-buffer-multibyte' to change a buffer's representation.
 See also Info node `(elisp)Text Representations'.  */);
-  XSYMBOL (intern_c_string ("enable-multibyte-characters"))->constant = 1;
+  XSYMBOL (intern_c_string ("enable-multibyte-characters"))->constant_or_hooked = 1;
 
   DEFVAR_PER_BUFFER ("buffer-file-coding-system",
 		     &BVAR (current_buffer, buffer_file_coding_system), Qnil,
--- src/bytecode.c
+++ src/bytecode.c
@@ -840,7 +840,7 @@
 	    else
 	      {
 		BEFORE_POTENTIAL_GC ();
-		set_internal (sym, val, Qnil, 0);
+		set_internal (sym, val, Qnil, 0, Dyn_Current);
 		AFTER_POTENTIAL_GC ();
 	      }
 	  }
--- src/font.c
+++ src/font.c
@@ -5197,19 +5197,19 @@
     [NUMERIC-VALUE SYMBOLIC-NAME ALIAS-NAME ...]
 NUMERIC-VALUE is an integer, and SYMBOLIC-NAME and ALIAS-NAME are symbols. */);
   Vfont_weight_table = BUILD_STYLE_TABLE (weight_table);
-  XSYMBOL (intern_c_string ("font-weight-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-weight-table"))->constant_or_hooked = 1;
 
   DEFVAR_LISP_NOPRO ("font-slant-table", Vfont_slant_table,
 	       doc: /*  Vector of font slant symbols vs the corresponding numeric values.
 See `font-weight-table' for the format of the vector. */);
   Vfont_slant_table = BUILD_STYLE_TABLE (slant_table);
-  XSYMBOL (intern_c_string ("font-slant-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-slant-table"))->constant_or_hooked = 1;
 
   DEFVAR_LISP_NOPRO ("font-width-table", Vfont_width_table,
 	       doc: /*  Alist of font width symbols vs the corresponding numeric values.
 See `font-weight-table' for the format of the vector. */);
   Vfont_width_table = BUILD_STYLE_TABLE (width_table);
-  XSYMBOL (intern_c_string ("font-width-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-width-table"))->constant_or_hooked = 1;
 
   staticpro (&font_style_table);
   font_style_table = make_uninit_vector (3);

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> The docstring for symbol-setter-function says lexical binding is required
> for the hook function,

I don't think that's true, BTW.  I agree it's recommended (I recommend
using lexical-binding for all new code), but other than that it should
work just fine with dynamically scoped code.

> If the hook function does set a dynamic variable that's hooked, and
> has no terminating condition for the recursion, you'll immediately
> find out when it exceeds max-lisp-eval-depth. So, don't do that. ;-)

That's right.

> So the user can notice, during debugging, if setq is setting the symbol
> in a different environment than the one he intended,

OK, so it's just informative, thanks.

>> Hmm, no symbol-setter-function should be a variable (holding
>> a function), modified via add-function/remove-function.
> I don't see why; the only difference is using add-function with an unquoted
> variable name vs. using advice-add with a quoted function name. It also
> makes the hook run a bit slower.

So far our coding style is to use *-hook, *-functions, and *-function
for places where we expect that it's normal to override/modify existing
behavior, whereas add-advice/defadvice is restricted to "last recourse".

> And it results in the help page for the function exposing the advice
> mechanism's internal representation of the advice, rather than cleanly
> showing e.g. ⌜:before advice: `mywatcher'⌝.

I agree that this is unfortunate, and I hope we can improve the *Help*
for those cases.

In the future we may also use `defgeneric' for such "extension points".

> But anyway I changed it to what you want, IIUC. I hope I misunderstood.

Thanks.

> The docstring already says to use :before if you just need to watch
> variables,

It shouldn't either.  This issue of which kind of advice to use is not
specific to this variable, so there's no need to put it there.

> Unless maybe you want to have multiple pieces of advice wrapped around
> each other, all blocking/overriding attempts to set variables?

That's the intention, yes.

> +  /* When masked with SYMBOL_CONSTANT_MASK, non-zero means symbol is
> +     constant, i.e. changing its value should signal an error.
> +     When masked with SYMBOL_HOOKED_MASK, non-zero means setting
> +     symbol will run varhook.  These two fields are combined into one
> +     in order to optimize the fast path of unhooked non-constants by
> +     having only one conditional branch for that case.  */

Actually I don't see these as two independent elements combined
into one.  I see it as a single value with 3 levels:
- any write you want
- write vetted by Elisp code
- no write at all
We could even drop the last setting and implement it in Elisp via
symbol-setter-function, except that it's currently used for variables
where we'd rather not give Elisp the opportunity to allow writes.

> +DEFUN ("void-p", Fvoid_p, Svoid_p, 1, UNEVALLED, 0,
> +       doc: /* Return t if ARG has no value.
> +If ARG is a non-lexical variable, this is equivalent to
> +(not (boundp (quote ARG))).
> +
> +Unlike `boundp', this function can also test a lexical variable.

I doubt this works on lexical-vars in compiled code.  Why do we need it?

> +This is the only built-in Elisp function that does not return a value.
> +Returning the result of this function enables any other function
> +to avoid returning a value.

So far we've been extra careful to try and make sure Qunbound doesn't
escape into Elisp world.  I don't think we want to reconsider this
design choice, since adding this Qunbound as a "normal" value in Elisp
will just create the need for another special "really unbound" value.
Why do we need that?

> +  DEFVAR_LISP ("symbol-setter-function", Vsymbol_setter_function,
[...]
> +If the variable is currently void, OLDVAL will be void.

Ah, right, I see.  No, we don't want to do that.  I see some
alternatives:
- one is we don't provide OLDVAL at all (let the Elisp code get it
  using `symbol-value' or `symbol-default' according to ENV, instead).
- OLDVAL is either a list of one element containing the old value, or
  nil (when that old value is Qunbound).
- Use a special value to represent "unbound", e.g. a special keyword like
  `::value--unbound::'.
Of course, the same holds for NEWVAL except that the code can't get
NEWVAL via `symbol-value', so for NEWVAL we can't choose option 1.

> +  Vsymbol_setter_function =
> +    list4 (Qclosure, list1 (Qt), list4 (Qsym, Qenv, Qoldval, Qnewval), Qnewval);

You can make it Qnil here and set it to a real value in subr.el, so as
to avoid having to hard-code the internal representation of
a particular function.

Oh, and I think we can keep the name `constant' rather than use the
verbose `constant_or_hooked'.  It's not perfect, but at least it reduces
the patch's size and the number of lines that are too long.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
>> The docstring for symbol-setter-function says lexical binding is required
>> for the hook function,
>
> I don't think that's true, BTW.  I agree it's recommended (I recommend
> using lexical-binding for all new code), but other than that it should
> work just fine with dynamically scoped code.

It was to enable the standard function to always just return newval, even if it's Qunbound. Works for lexical binding (the interpreter doesn't barf), but not for dynamic. But that doesn't matter anymore, since you said I need to use a different way of handling Qunbound.

>> The docstring already says to use :before if you just need to watch
>> variables,
>
> It shouldn't either.  This issue of which kind of advice to use is not
> specific to this variable, so there's no need to put it there.

It's an unusual function. The effect of overriding the return value, and interaction with «let», and how to deal with the case of makunbound, all need to be documented somewhere, and for completeness, it makes sense for the documentation to also cover the case of not overriding the return value. If not in the docstring, then maybe in the manual?

> Actually I don't see these as two independent elements combined
> into one.  I see it as a single value with 3 levels:
> - any write you want
> - write vetted by Elisp code
> - no write at all

If there's just :before advice (e.g. a profiler), then the write is not vetted.

> We could even drop the last setting and implement it in Elisp via
> symbol-setter-function, except that it's currently used for variables
> where we'd rather not give Elisp the opportunity to allow writes.

Yeah, changing the value of t or nil would be quite fun. ;-)

>> +DEFUN ("void-p", Fvoid_p, Svoid_p, 1, UNEVALLED, 0,
[snip]
> I doubt this works on lexical-vars in compiled code.  Why do we need it?

It was to enable checking for Qunbound for lexical newval and oldval in symbol-setter-function.

I've removed it.

> So far we've been extra careful to try and make sure Qunbound doesn't
> escape into Elisp world.  I don't think we want to reconsider this
> design choice, since adding this Qunbound as a "normal" value in Elisp
> will just create the need for another special "really unbound" value.

Qunbound already escapes: you just have to use makunbound instead of «set» to set it, and boundp instead of symbol-value to read it. Lexical variables are second-class because they can't be set to all the values that dynamic variables can be; the purpose of my «void» function was to fix that.

Of course, you'll say makunbound doesn't set a value; it unbinds the variable. But that's not true! For example, consider this:
(defvar foo 0)
(defvar bar 1)
(let ((foo 2)) bar) → 1
(let ((foo 2)) (makunbound 'foo) foo) → Lisp error: (void-variable foo)
foo → 0

In the dynamic environment created by the first «let», foo is bound, but bar is not bound. Therefore an attempt to read the not-bound bar within that environment will look in outer environments until it finds bar bound in one of them. An error would occur if bar weren't bound in any of those outer environments, not even the global one.

In the environment created by the second «let», foo is first bound to 2. If it were unbound, then an attempt to read it would return 0, for the same reason that an attempt to read bar in the first environment returns 1. So what does makunbound do? If it really unbound foo, then the subsequent attempt to read foo would return 0, because foo would be not bound in the dynamic environment. But in fact the attempt to read foo returns Qunbound, which was the value set by makunbound, and the interpreter barfs because you didn't use the special function boundp to read that special value. IOW, Qunbound escapes into the Lisp world.

Then when the «let» exits, the dynamic environment is destroyed, which actually _does_ unbind foo (from Qunbound). So now, the attempt to read foo returns 0, because global foo is no longer shadowed by the dynamic binding of foo to the value Qunbound.

I agree that setting lexicals to Qunbound is absurd. «void» was just to enables lexicals and dynamics to be set to the same values. My point is that letting Qunbound escape into the Lisp world in the first place is also absurd. (CL does it too.) Makunbound should actually make a variable be unbound, not bind it to some special value that triggers an error (and lies about the cause) when you try to read it. Because it makes no sense to actually unbind a dynamic or lexical variable except by destruction of the environment in which the variable occurs, this means makunbound should apply only to global variables.

Because global variables don't shadow anything, binding them to Qunbound is the same as unbinding them so far as Lisp code is concerned, so this implementation detail doesn't escape. And Lisp doesn't let you bind lexicals to Qunbound, so they don't have a problem either. But it _does_ let you bind dynamic variables to Qunbound (and Elisp lets you bind buffer-locals to Qunbound too), which is a problem because they shadow variables in outer environments. I know it's officially been a feature for a long time. But a bug by any other name is still a bug.

I've removed «void». I recommend making makunbound stop working on non-globals too.

> I see some alternatives:
[snip]
> - OLDVAL is either a list of one element containing the old value, or
>   nil (when that old value is Qunbound).

Then run_varhook must cons. That'll generate a lot of garbage if you use it for profiling, or for debugging in a way where you don't just pause to inspect every hooked variable. Is that ok?

> - Use a special value to represent "unbound", e.g. a special keyword like
>   `::value--unbound::'.

Then varhook would be defective. If the hook function returns a special keyword to represent Qunbound (to avoid blocking makunbound), then just hooking unavoidably changes the behavior of Emacs (because it's impossible to set a hooked variable to that keyword), which is the wrong thing to do.

Of course, telling the hook function that a variable is bound to Qunbound without having to cons is easy (could just pass an additional argument, t or nil). The problem is needing to _return_ Qunbound. If run_varhook will cons, then the hook function can just return nil to represent Qunbound, or do (setcar newval ...) and then return newval to set the variable to another value without needing yet another cons.

Or to avoid needing to cons in run_varhook, it could specbind something like Qsymbol_newval_void (to t if the new value is Qunbound), so the hook function could setq symbol-newval-void to t or nil to indicate whether to set the hooked variable to Qunbound. This way, run_varhook could pass the newval argument directly (as it currently does), without wrapping it, except pass nil as a dummy value if the new value is Qunbound. This is what I prefer, but if you don't want me to do it this way, then I'll wrap newval.

> Oh, and I think we can keep the name `constant' rather than use the
> verbose `constant_or_hooked'.  It's not perfect, but at least it reduces
> the patch's size and the number of lines that are too long.

Would ⌜const_hooked⌝ be ok? It's only 4 extra characters. Using ⌜constant⌝ for something that doesn't just mean «constant» is as gross as ⌜set-default⌝ for «set-global». As noted above, hooked doesn't imply vetted, so hooked doesn't mean sort-of-constant. Even if it did, ⌜constant⌝ is a misleading term for something that can be sort-of-constant; ⌜constantness⌝ would be better, since the term allows for the possibility of a range of values. If it's too long, ⌜constness⌝ is only 1 extra character.

The patch is big, but it'll just be a one-time change to the source code. After that, nobody reading the code will care what the patch was, and everybody will be stuck with the misleading name if it isn't changed.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

>> It shouldn't either.  This issue of which kind of advice to use is not
>> specific to this variable, so there's no need to put it there.
> It's an unusual function.

I don't see why it should be special: it take the symbol that's about to
be changed, its old/current value, and the candidate new value and returns
the actual new value that should be used.
Nothing special that I can see about that.

> interaction with «let», and how to deal with the case of makunbound, all
> need to be documented somewhere,

Yes, handling unbound does require special care and needs to be
documented, but it doesn't affect the way add-function is to be used.

> and for completeness, it makes sense for the documentation to also
> cover the case of not overriding the return value.

How to override the value or block the assignment can be inferred from
the above description.

> If not in the docstring, then maybe in the manual?

If you want to give examples, then indeed the manual would be a good
place for it.

>> Actually I don't see these as two independent elements combined
>> into one.  I see it as a single value with 3 levels:
>> - any write you want
>> - write vetted by Elisp code
>> - no write at all
> If there's just :before advice (e.g. a profiler), then the write is
> not vetted.

It is vetted, it's just that the vetting function decides not to always
let things pass through.

>> So far we've been extra careful to try and make sure Qunbound doesn't
>> escape into Elisp world.  I don't think we want to reconsider this
>> design choice, since adding this Qunbound as a "normal" value in Elisp
>> will just create the need for another special "really unbound" value.
> Qunbound already escapes: you just have to use makunbound instead of «set»
> to set it, and boundp instead of symbol-value to read it.

That doesn't let Qunbound escape.  It gives you access to the concept
and lets you control put (and check for) Qunbound in a symbol's
`value' slot.  But you don't actually get access to that value, so you
can't store it in other slots of a symbol, or in a vector, list, ...

> I've removed «void». I recommend making makunbound stop working on
> non-globals too.

`let' on dynamically-scoped variables doesn't make them non-local.

Or rather, whether it does or not is a philosophical question (does
`let' create a new variable which shadows the outer one, or does it just
temporarily change the value of the variable?).

Once you introduce multiple threads with shared memory, those threads
get to test their theory, and usually it's considered that let-binding
in that case should have only a thread-local effect, whereas the current
implementation technique used in Emacs would naturally lead to the
let-bindings having a global effect.

>> - OLDVAL is either a list of one element containing the old value, or
>> nil (when that old value is Qunbound).
> Then run_varhook must cons. That'll generate a lot of garbage if you use it
> for profiling, or for debugging in a way where you don't just pause to
> inspect every hooked variable. Is that ok?

I think it's OK, yes.

>> - Use a special value to represent "unbound", e.g. a special keyword like
>> `::value--unbound::'.
> (because it's impossible to set a hooked variable to that keyword),

Yes, that's a disadvantage.

> Or to avoid needing to cons in run_varhook, it could specbind something like
> Qsymbol_newval_void (to t if the new value is Qunbound), so the hook

Yuck.  That's even worse than using a value that is hopefully never
used elsewhere, like `::value--unbound::'.

> Would ⌜const_hooked⌝ be ok?

How 'bout `writable' with values yes/no/thrufunction?


        Stefan

[PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 3843 bytes --]

Stefan Monnier wrote:
>> I've removed «void». I recommend making makunbound stop working on
>> non-globals too.
>
> `let' on dynamically-scoped variables doesn't make them non-local.
>
> Or rather, whether it does or not is a philosophical question (does
> `let' create a new variable which shadows the outer one, or does it just
> temporarily change the value of the variable?).

Saying there's only a global variable doesn't remove the absurdity of makunbound within the extent of a «let». Suppose foo is currently unbound. Then:
(defvar foo 0) ; Now foo is bound to 0
(let ((foo 1)) ; Now it's bound to 1
  (makunbound 'foo)) ; Now it's bound to unboundedness
;; Now it's unbound from unboundedness, and thereby rebound to 0

You can't just say that makunbound in that case unbinds foo, because then the question arises, from what is foo unbound when the let-binding form exits? If foo were already unbound by makunbound, then there would be nothing left to unbind it from when «let» exits.

Surely you can't say with a straight face that binding to unboundedness is not an absurd concept. Yet that's the alternative to the interpretation that Qunbound escapes as a special second-class value into the Lisp world. Either way, the behavior of makunbound is wrong.

>>> - OLDVAL is either a list of one element containing the old value, or
>>> nil (when that old value is Qunbound).
>> Then run_varhook must cons. That'll generate a lot of garbage if you use it
>> for profiling, or for debugging in a way where you don't just pause to
>> inspect every hooked variable. Is that ok?
>
> I think it's OK, yes.

This seems to be the least-bad option, so I did it this way, even though it makes the API a bit gross. Unfortunately, when you do:
(require 'cl)
(setq x 0)
(symbol-hook 'x)
(benchmark-run-compiled 100000 (incf x))

it now spends half the time doing garbage collection. That's a high price to pay to cater to the brain-dead misbehavior of makunbound.

>> Or to avoid needing to cons in run_varhook, it could specbind something like
>> Qsymbol_newval_void (to t if the new value is Qunbound), so the hook
>
> Yuck.  That's even worse than using a value that is hopefully never
> used elsewhere, like `::value--unbound::'.

Then you'll probably also say ‟yuck” to my xref-push-mark patch that I posted to emacs-devel just now, since it uses that tactic.

>> Would ⌜const_hooked⌝ be ok?
>
> How 'bout `writable' with values yes/no/thrufunction?

That would make sense, but the meaning is inverted from how the code is written, e.g. the code has:
if (constant) a...;
else b...;

so to account for the inversion, it would have to change to:
if (!writeable) a...;
else b...;

which introduces an extra «not» operator. To avoid the extra operator, it would have to change to:
if (writeable) b...;
else a...;

Usually the «a» ends with return, error, or xsignal1, which means no «else» is needed for «b», so changing the order to avoid the extra «not» operator would make the code more awkward.

Since you think of variables as ‟vetted” if they might not be freely writeable, and the meaning of that term isn't inverted, I think ⌜vetted⌝ is the least-bad short name to replace ⌜constant⌝ in the places where it's used to mean «constant or hooked». (Where it means specifically «constant», i.e. the vetting is hardcoded to disallow writing, it still is called ‟constant”, e.g. in the standard macro SYMBOL_CONSTANT_P).

Updated patch attached.

BTW, I accidentally inlined a little too much for set_internal last time. This time I've factored out the part that doesn't need to be inline. This has no speed impact on the hot path, but it reduces the size of the executable by a few kB, and makes the source code a bit clearer too.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: varhook-advice-2.patch --]
[-- Type: text/x-diff, Size: 29019 bytes --]

--- src/lisp.h
+++ src/lisp.h
@@ -305,6 +305,15 @@
 #endif
 
 
+/* These are the masks for the vetted field of Lisp_Symbol.
+   Bit 0 stores the constant field.  Bit 1 stores the hooked field.  */
+#define SYM_CONST 1
+#define SYM_HOOKED 2
+
+# define SYM_CONST_P(sym) (((sym)->vetted) & SYM_CONST)
+# define SYM_HOOKED_P(sym) (((sym)->vetted) & SYM_HOOKED)
+
+
 /* Some operations are so commonly executed that they are implemented
    as macros, not functions, because otherwise runtime performance would
    suffer too much when compiling with GCC without optimization.
@@ -359,7 +370,7 @@
 #define lisp_h_NILP(x) EQ (x, Qnil)
 #define lisp_h_SET_SYMBOL_VAL(sym, v) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value = (v))
-#define lisp_h_SYMBOL_CONSTANT_P(sym) (XSYMBOL (sym)->constant)
+#define lisp_h_SYMBOL_CONSTANT_P(sym) (SYM_CONST_P (XSYMBOL (sym)))
 #define lisp_h_SYMBOL_VAL(sym) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value)
 #define lisp_h_SYMBOLP(x) (XTYPE (x) == Lisp_Symbol)
@@ -1597,10 +1608,13 @@
      3 : it's a forwarding variable, the value is in `forward'.  */
   ENUM_BF (symbol_redirect) redirect : 3;
 
-  /* Non-zero means symbol is constant, i.e. changing its value
-     should signal an error.  If the value is 3, then the var
-     can be changed, but only by `defconst'.  */
-  unsigned constant : 2;
+  /* When masked with SYM_CONST, non-zero means symbol is constant,
+     i.e. changing its value should signal an error.
+     When masked with SYM_HOOKED, non-zero means setting symbol will
+     run varhook.  These two fields are combined into one in order
+     to optimize the fast path of non-hooked non-constants by
+     having only one conditional branch for that case.  */
+  unsigned vetted : 2;
 
   /* Interned state of the symbol.  This is an enumerator from
      enum symbol_interned.  */
@@ -3391,6 +3405,14 @@
 EXFUN (Fbyteorder, 0) ATTRIBUTE_CONST;
 
 /* Defined in data.c.  */
+typedef enum
+  { /* See set_internal for a description of these values.  */
+    Dyn_Unbind = -1,
+    Dyn_Current = 0,
+    Dyn_Bind = 1,
+    Dyn_Skip = 2,
+    Dyn_Global = 3
+  } Dyn_Bind_Env;
 extern Lisp_Object indirect_function (Lisp_Object);
 extern Lisp_Object find_symbol_value (Lisp_Object);
 enum Arith_Comparison {
@@ -3438,7 +3460,15 @@
 					   Lisp_Object);
 extern _Noreturn Lisp_Object wrong_type_argument (Lisp_Object, Lisp_Object);
 extern Lisp_Object do_symval_forwarding (union Lisp_Fwd *);
-extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool);
+extern Lisp_Object run_varhook (struct Lisp_Symbol*, bool, Dyn_Bind_Env,
+				Lisp_Object, Lisp_Object);
+extern void set_internal_vetted (Lisp_Object, Lisp_Object, Lisp_Object, bool,
+					 Dyn_Bind_Env, struct Lisp_Symbol *);
+extern void set_internal_localized_or_forwarded (Lisp_Object, Lisp_Object,
+						 Lisp_Object, bool,
+						 Dyn_Bind_Env,
+						 struct Lisp_Symbol *);
+extern void set_default_internal (Lisp_Object, Lisp_Object, Dyn_Bind_Env);
 extern void syms_of_data (void);
 extern void swap_in_global_binding (struct Lisp_Symbol *);
 
@@ -4595,6 +4626,56 @@
     return false;
 }
 
+/* Store the value NEWVAL into SYMBOL.
+   If buffer/frame-locality is an issue, WHERE specifies which context to use.
+   (nil stands for the current buffer/frame).
+
+   If BINDFLAG is false, then if this symbol is supposed to become
+   local in every buffer where it is set, then we make it local.
+   If BINDFLAG is true, we don't do that.
+
+   ENV indicates the dynamic environment for this function call, i.e. whether
+   this call is due to a variable binding (Dyn_Bind), an unbinding (Dyn_Unbind),
+   or neither (Dyn_Current). As special cases, a value of Dyn_Skip is a flag
+   to disable run_varhook so that varhooks aren't run during backtraces, and
+   a value of Dyn_Global is a flag indicating that this function call is due
+   to set_default, which allows run_varhook to distinguish beween the global
+   and the dyn-local binding.  */
+
+INLINE void
+set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+	      bool bindflag, Dyn_Bind_Env env)
+{
+  struct Lisp_Symbol *sym;
+
+  /* If restoring in a dead buffer, do nothing.  */
+  /* if (BUFFERP (where) && NILP (XBUFFER (where)->name))
+      return; */
+
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  if (sym->vetted)
+    {
+      set_internal_vetted (symbol, newval, where, bindflag, env, sym);
+      return;
+    }
+
+ start:
+  switch (sym->redirect)
+    {
+    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym, newval); return;
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+    default: set_internal_localized_or_forwarded
+	(symbol, newval, where, bindflag, env, sym);
+    }
+}
+
+#define MAYBE_RUN_VARHOOK(result, sym, buf_local, env, oldval, newval)	\
+  {									\
+    if (SYM_HOOKED_P (sym))						\
+      (result) = run_varhook (sym, buf_local, env, oldval, newval);	\
+  }
+
 INLINE_HEADER_END
 
 #endif /* EMACS_LISP_H */
--- src/eval.c
+++ src/eval.c
@@ -267,7 +267,7 @@
   max_lisp_eval_depth = XINT (XCDR (data));
 }
 
-static void grow_specpdl (void);
+static inline void grow_specpdl (void);
 
 /* Call the Lisp debugger, giving it argument ARG.  */
 
@@ -547,7 +547,8 @@
 			= Fassq (sym, Vinternal_interpreter_environment)))
 	    XSETCDR (lex_binding, val); /* SYM is lexically bound.  */
 	  else
-	    Fset (sym, val);	/* SYM is dynamically bound.  */
+	    /* SYM is dynamically bound.  */
+	    set_internal (sym, val, Qnil, false, Dyn_Current);
 
 	  args_left = Fcdr (XCDR (args_left));
 	}
@@ -620,7 +620,7 @@
 
   sym = XSYMBOL (new_alias);
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     /* Not sure why, but why not?  */
     error ("Cannot make a constant an alias");
 
@@ -637,7 +637,7 @@
      so that old-code that affects n_a before the aliasing is setup
      still works.  */
   if (NILP (Fboundp (base_variable)))
-    set_internal (base_variable, find_symbol_value (new_alias), Qnil, 1);
+    set_internal (base_variable, find_symbol_value (new_alias), Qnil, true, Dyn_Current);
 
   {
     union specbinding *p;
@@ -652,7 +652,7 @@
   XSYMBOL (base_variable)->declared_special = 1;
   sym->redirect = SYMBOL_VARALIAS;
   SET_SYMBOL_ALIAS (sym, XSYMBOL (base_variable));
-  sym->constant = SYMBOL_CONSTANT_P (base_variable);
+  sym->vetted = SYMBOL_CONSTANT_P (base_variable);
   LOADHIST_ATTACH (new_alias);
   /* Even if docstring is nil: remove old docstring.  */
   Fput (new_alias, Qvariable_documentation, docstring);
@@ -2007,7 +2007,7 @@
    never-used entry just before the bottom of the stack; sometimes its
    address is taken.  */
 
-static void
+static inline void
 grow_specpdl (void)
 {
   specpdl_ptr++;
@@ -3132,8 +3132,6 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS:
-      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
     case SYMBOL_PLAINVAL:
       /* The most common case is that of a non-constant symbol with a
 	 trivial value.  Make that as fast as we can.  */
@@ -3141,11 +3139,15 @@
       specpdl_ptr->let.symbol = symbol;
       specpdl_ptr->let.old_value = SYMBOL_VAL (sym);
       grow_specpdl ();
-      if (!sym->constant)
-	SET_SYMBOL_VAL (sym, value);
+      if (!sym->vetted) SET_SYMBOL_VAL (sym, value);
+      else if (SYM_HOOKED_P (sym))
+	SET_SYMBOL_VAL (sym, run_varhook
+			(sym, false, Dyn_Bind, sym->val.value, value));
       else
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, true, Dyn_Bind);
       break;
+    case SYMBOL_VARALIAS:
+      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
     case SYMBOL_LOCALIZED:
       if (SYMBOL_BLV (sym)->frame_local)
 	error ("Frame-local vars cannot be let-bound");
@@ -3176,7 +3178,7 @@
 	      {
 		specpdl_ptr->let.kind = SPECPDL_LET_DEFAULT;
 		grow_specpdl ();
-		Fset_default (symbol, value);
+		set_default_internal (symbol, value, Dyn_Bind);
 		return;
 	      }
 	  }
@@ -3184,7 +3186,7 @@
 	  specpdl_ptr->let.kind = SPECPDL_LET;
 
 	grow_specpdl ();
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, true, Dyn_Bind);
 	break;
       }
     default: emacs_abort ();
@@ -3319,7 +3321,9 @@
 	    struct Lisp_Symbol *sym = XSYMBOL (specpdl_symbol (specpdl_ptr));
 	    if (sym->redirect == SYMBOL_PLAINVAL)
 	      {
-		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
+		Lisp_Object oldval = specpdl_old_value (specpdl_ptr);
+		MAYBE_RUN_VARHOOK (oldval, sym, false, Dyn_Unbind, sym->val.value, oldval);
+		SET_SYMBOL_VAL (sym, oldval);
 		break;
 	      }
 	    else
@@ -3329,8 +3333,8 @@
 	      }
 	  }
 	case SPECPDL_LET_DEFAULT:
-	  Fset_default (specpdl_symbol (specpdl_ptr),
-			specpdl_old_value (specpdl_ptr));
+	  set_default_internal (specpdl_symbol (specpdl_ptr),
+				specpdl_old_value (specpdl_ptr), Dyn_Unbind);
 	  break;
 	case SPECPDL_LET_LOCAL:
 	  {
@@ -3342,7 +3346,7 @@
 	    /* If this was a local binding, reset the value in the appropriate
 	       buffer, but only if that buffer's binding still exists.  */
 	    if (!NILP (Flocal_variable_p (symbol, where)))
-	      set_internal (symbol, old_value, where, 1);
+	      set_internal (symbol, old_value, where, true, Dyn_Unbind);
 	  }
 	  break;
 	}
@@ -3537,7 +3541,7 @@
 	    Lisp_Object sym = specpdl_symbol (tmp);
 	    Lisp_Object old_value = specpdl_old_value (tmp);
 	    set_specpdl_old_value (tmp, Fdefault_value (sym));
-	    Fset_default (sym, old_value);
+	    set_default_internal (sym, old_value, Dyn_Skip);
 	  }
 	  break;
 	case SPECPDL_LET_LOCAL:
@@ -3553,7 +3557,7 @@
 	      {
 		set_specpdl_old_value
 		  (tmp, Fbuffer_local_value (symbol, where));
-		set_internal (symbol, old_value, where, 1);
+		set_internal (symbol, old_value, where, true, Dyn_Skip);
 	      }
 	  }
 	  break;
@@ -3828,6 +3832,51 @@
 still determine whether to handle the particular condition.  */);
   Vdebug_on_signal = Qnil;
 
+  DEFVAR_LISP ("symbol-setter-function", Vsymbol_setter_function,
+       doc: /* This function is called whenever a hooked variable is set.
+It takes four arguments: SYMBOL, ENV, OLDVAL, NEWVAL. By default, it just
+returns NEWVAL unchanged.
+
+SYMBOL is the symbol being set. ENV is the environment is which it's being
+set. OLDVAL is a singleton list wrapping the current value, or nil if unbound.
+NEWVAL is a singleton list wrapping the new value to which the setter,
+i.e. the caller of a function such as `setq', is attempting to set the
+variable, or nil if the setter called `makunbound'. The actual new value to
+which the variable will be set is the `car' of the return value of this
+function. The return value is NEWVAL if this function doesn't have advice that
+overrides it. If the return value is nil, then the variable will be unbound.
+
+The possible values of ENV are these symbols, with these meanings:
+global: The global environment.
+buf-local: The setter's buffer-local environment.
+dyn-local: The innermost dynamic environment in which SYMBOL is bound.
+dyn-bind: A new dynamic environment, such as creatable using `let'.
+dyn-unbind: The next-outer dynamic environment in which SYMBOL is still bound,
+or the buffer-local environment if SYMBOL is not bound in any dynamic
+environment, or the global environment is SYMBOL is not in the buffer-local
+environment, unshadowed due to destruction of the setter's current
+dynamic environment, such as due to exit of a `let' form.
+
+If you use overriding advice, your advice must return the value to which to
+set the variable. To avoid overriding the setter's attempt to set the variable
+to NEWVAL, return NEWVAL. To block the attempt, and leave the variable
+unchanged, return OLDVAL. If ENV is dyn-bind or dyn-unbind, you can block
+the change of value, but you can't prevent the corresponding creation or
+destruction of a dynamic environment. Therefore, blocking when ENV is
+dyn-bind will set SYMBOL in the new environment to its value in the outer
+environment, and blocking when ENV is dyn-unbind will set SYMBOL in the
+outer environment to its value in the environment being destroyed.
+
+Don't set the variable in your advice; that would cause a recursive call
+to this function. Instead, if your advice needs to set the variable, use
+`add-function' with overriding advice.
+
+To hook all variables of a symbol, use `symbol-hook'. To unhook them,
+use `symbol-unhook'. If you only want to watch or override some variables
+of a symbol, then filter according to ENV, and if you use overriding advice,
+simply return NEWVAL for the ones you don't want to process.  */);
+  Vsymbol_setter_function = Qnil; /* Set in subr.el */
+
   /* When lexical binding is being used,
    Vinternal_interpreter_environment is non-nil, and contains an alist
    of lexically-bound variable, or (t), indicating an empty
--- src/data.c
+++ src/data.c
@@ -612,6 +613,20 @@
 \f
 /* Extract and set components of symbols.  */
 
+DEFUN ("symbol-hooked-p", Fsymbol_hooked_p, Ssymbol_hooked_p, 1, 1, 0,
+       doc: /* Return t if SYMBOL is hooked.
+To hook and unhook it, use `symbol-hook' and `symbol-unhook'.
+When hooked, setting SYMBOL will run `symbol-setter-function'.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    sym = indirect_variable (sym);
+  return SYM_HOOKED_P (sym) ? Qt : Qnil;
+}
+
 DEFUN ("boundp", Fboundp, Sboundp, 1, 1, 0,
        doc: /* Return t if SYMBOL's value is not void.
 Note that if `lexical-binding' is in effect, this refers to the
@@ -661,6 +676,46 @@
   return NILP (XSYMBOL (symbol)->function) ? Qnil : Qt;
 }
 
+DEFUN ("symbol-hook", Fsymbol_hook, Ssymbol_hook, 1, 1, 0,
+       doc: /* Hook SYMBOL.
+When hooked, setting it will run `symbol-setter-function'.
+To unhook it, use `symbol-unhook'.
+To test whether it's hooked, use `symbol-hooked-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  sym->vetted |= SYM_HOOKED;
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->vetted |= SYM_HOOKED;
+    }
+  return symbol;
+}
+
+DEFUN ("symbol-unhook", Fsymbol_unhook, Ssymbol_unhook, 1, 1, 0,
+       doc: /* Unhook SYMBOL.
+When unhooked, setting it will not run `symbol-setter-function'.
+To hook it, use `symbol-hook'.
+To test whether it's hooked, use `symbol-hooked-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  sym->vetted &= (SYM_HOOKED ^ -1);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->vetted &= (SYM_HOOKED ^ -1);
+    }
+  return symbol;
+}
+
 DEFUN ("makunbound", Fmakunbound, Smakunbound, 1, 1, 0,
        doc: /* Make SYMBOL's value be void.
 Return SYMBOL.  */)
@@ -1137,8 +1192,8 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
@@ -1167,54 +1222,112 @@
   xsignal1 (Qvoid_variable, symbol);
 }
 
+/* For the symbol S being set, run symbol-setter-function with these arguments:
+   0. S
+   1. A symbol indicating the environment in which S is being set.
+   2. The current value of S in that environment.
+   3. The value to which the setter is attempting to set the variable.
+
+   Arguments #2 and #3 are wrapped into singleton lists, unless they're
+   Qunbound, in which case they're replaced by nil.
+
+   Return the unwrapped result of symbol-setter-function. The variable will be
+   set (by code that calls run_varhook) to that result, overriding the value
+   to which the setter is attempting to set the variable.  */
+
+Lisp_Object
+run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Env rawenv,
+	     Lisp_Object oldval, Lisp_Object newval)
+{
+  Lisp_Object symbol;
+  Lisp_Object env;
+  if (rawenv == Dyn_Skip) /* From backtrace_eval_unrewind */
+    return newval;
+  XSETSYMBOL (symbol, sym);
+  switch (rawenv) /* Disambiguate Dyn_Current and Dyn_Global */
+    {
+    case Dyn_Current:
+      {
+	bool shadowed = (buf_local ? let_shadows_buffer_binding_p (sym)
+                         : let_shadows_global_binding_p (symbol));
+	if (shadowed) env = Qdyn_local;
+	else if (buf_local) env = Qbuf_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Global:
+      {
+	/* let_shadows_buffer_binding_p doesn't disambiguate this case */
+	if (let_shadows_global_binding_p (symbol) &&
+	    NILP (Flocal_variable_p (symbol, Qnil)))
+	  env = Qdyn_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Bind: env = Qdyn_bind; break;
+    case Dyn_Unbind: env = Qdyn_unbind; break;
+    default: emacs_abort ();
+    }
+  oldval = EQ (oldval, Qunbound) ? Qnil : list1 (oldval);
+  newval = EQ (newval, Qunbound) ? Qnil : list1 (newval);
+  newval = call4 (Vsymbol_setter_function, symbol, env, oldval, newval);
+  CHECK_LIST (newval);
+  return EQ (newval, Qnil) ? Qunbound : XCAR (newval);
+}
+
 DEFUN ("set", Fset, Sset, 2, 2, 0,
        doc: /* Set SYMBOL's value to NEWVAL, and return NEWVAL.  */)
   (register Lisp_Object symbol, Lisp_Object newval)
 {
-  set_internal (symbol, newval, Qnil, 0);
+  set_internal (symbol, newval, Qnil, false, Dyn_Current);
   return newval;
 }
 
-/* Store the value NEWVAL into SYMBOL.
-   If buffer/frame-locality is an issue, WHERE specifies which context to use.
-   (nil stands for the current buffer/frame).
-
-   If BINDFLAG is false, then if this symbol is supposed to become
-   local in every buffer where it is set, then we make it local.
-   If BINDFLAG is true, we don't do that.  */
+/* set_internal is in lisp.h due to being inlined.  */
+
+/* Factored out from set_internal to avoid inlining the non-hotpath.  */
 
 void
-set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
-	      bool bindflag)
+set_internal_vetted (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+			     bool bindflag, Dyn_Bind_Env env, struct Lisp_Symbol *sym)
 {
-  bool voide = EQ (newval, Qunbound);
-  struct Lisp_Symbol *sym;
-  Lisp_Object tem1;
-
-  /* If restoring in a dead buffer, do nothing.  */
-  /* if (BUFFERP (where) && NILP (XBUFFER (where)->name))
-      return; */
-
-  CHECK_SYMBOL (symbol);
-  if (SYMBOL_CONSTANT_P (symbol))
+  if (SYM_HOOKED_P (sym))
     {
+    start:
+      switch (sym->redirect)
+	{
+	case SYMBOL_PLAINVAL:
+	  SET_SYMBOL_VAL (sym, run_varhook (sym, false, env, sym->val.value, newval));
+	  return;
+	case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+	default:
+	  set_internal_localized_or_forwarded (symbol, newval, where, bindflag, env, sym);
+	  return;
+	}
+    }
       if (NILP (Fkeywordp (symbol))
 	  || !EQ (newval, Fsymbol_value (symbol)))
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
 	return;
-    }
+}
 
-  sym = XSYMBOL (symbol);
+/* Split from set_internal to avoid code duplication, because both set_internal and
+   set_internal_vetted must call this function.  */
 
- start:
+void
+set_internal_localized_or_forwarded (Lisp_Object symbol, Lisp_Object newval,
+				     Lisp_Object where, bool bindflag,
+				     Dyn_Bind_Env env, struct Lisp_Symbol *sym)
+{
+  bool voide;
+  Lisp_Object tem1;
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
     case SYMBOL_LOCALIZED:
       {
+	bool buf_local = true;
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 	if (NILP (where))
 	  {
@@ -1258,6 +1371,7 @@
 		   indicating that we're seeing the default value.
 		   Likewise if the variable has been let-bound
 		   in the current buffer.  */
+		buf_local = false;
 		if (bindflag || !blv->local_if_set
 		    || let_shadows_buffer_binding_p (sym))
 		  {
@@ -1285,6 +1399,9 @@
 	    set_blv_valcell (blv, tem1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym, buf_local, env, blv_value (blv), newval);
+	voide = EQ (newval, Qunbound);
+
 	/* Store the new value in the cons cell.  */
 	set_blv_value (blv, newval);
 
@@ -1316,6 +1433,11 @@
 	      SET_PER_BUFFER_VALUE_P (buf, idx, 1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym,
+			   (XFWDTYPE (innercontents)) == Lisp_Fwd_Buffer_Obj,
+			   env, do_symval_forwarding (innercontents), newval);
+	voide = EQ (newval, Qunbound);
+
 	if (voide)
 	  { /* If storing void (making the symbol void), forward only through
 	       buffer-local indicator, not through Lisp_Objfwd, etc.  */
@@ -1347,8 +1469,8 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_LOCALIZED:
       {
 	/* If var is set up for a buffer that lacks a local value for it,
@@ -1413,6 +1535,17 @@
 for this variable.  */)
   (Lisp_Object symbol, Lisp_Object value)
 {
+  set_default_internal (symbol, value, Dyn_Global);
+  return value;
+}
+
+/* Like Fset_default, but with ENV argument. See set_internal for
+   a description of this argument. */
+
+void
+set_default_internal (Lisp_Object symbol, Lisp_Object value,
+		      Dyn_Bind_Env env)
+{
   struct Lisp_Symbol *sym;
 
   CHECK_SYMBOL (symbol);
@@ -1423,26 +1556,32 @@
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
-	return value;
+	return;
     }
   sym = XSYMBOL (symbol);
 
  start:
   switch (sym->redirect)
     {
+    case SYMBOL_PLAINVAL:
+      {
+	set_internal (symbol, value, Qnil, false, env);
+	return;
+      }
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: return Fset (symbol, value);
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 
+	MAYBE_RUN_VARHOOK (value, sym, false, env, XCDR (blv->defcell), value);
+
 	/* Store new value into the DEFAULT-VALUE slot.  */
 	XSETCDR (blv->defcell, value);
 
 	/* If the default binding is now loaded, set the REALVALUE slot too.  */
 	if (blv->fwd && EQ (blv->defcell, blv->valcell))
 	  store_symval_forwarding (blv->fwd, value, NULL);
-	return value;
+	return;
       }
     case SYMBOL_FORWARDED:
       {
@@ -1456,6 +1595,8 @@
 	    int offset = XBUFFER_OBJFWD (valcontents)->offset;
 	    int idx = PER_BUFFER_IDX (offset);
 
+	    MAYBE_RUN_VARHOOK (value, sym, false, env, per_buffer_default (offset), value);
+
 	    set_per_buffer_default (offset, value);
 
 	    /* If this variable is not always local in all buffers,
@@ -1468,10 +1609,13 @@
 		  if (!PER_BUFFER_VALUE_P (b, idx))
 		    set_per_buffer_value (b, offset, value);
 	      }
-	    return value;
+	    return;
 	  }
 	else
-	  return Fset (symbol, value);
+	  {
+	    set_internal (symbol, value, Qnil, false, env);
+	    return;
+	  }
       }
     default: emacs_abort ();
     }
@@ -1599,7 +1743,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be buffer-local", SDATA (SYMBOL_NAME (variable)));
 
   if (!blv)
@@ -1672,7 +1816,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be buffer-local",
 	   SDATA (SYMBOL_NAME (variable)));
 
@@ -1861,7 +2005,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));
 
   blv = make_blv (sym, forwarded, valcontents);
@@ -3470,6 +3614,12 @@
   DEFSYM (Qad_advice_info, "ad-advice-info");
   DEFSYM (Qad_activate_internal, "ad-activate-internal");
 
+  DEFSYM (Qglobal, "global");
+  DEFSYM (Qbuf_local, "buf-local");
+  DEFSYM (Qdyn_local, "dyn-local");
+  DEFSYM (Qdyn_bind, "dyn-bind");
+  DEFSYM (Qdyn_unbind, "dyn-unbind");
+
   error_tail = pure_cons (Qerror, Qnil);
 
   /* ERROR is used as a signaler for random errors for which nothing else is
@@ -3609,8 +3759,11 @@
   defsubr (&Sindirect_function);
   defsubr (&Ssymbol_plist);
   defsubr (&Ssymbol_name);
+  defsubr (&Ssymbol_hook);
+  defsubr (&Ssymbol_unhook);
   defsubr (&Smakunbound);
   defsubr (&Sfmakunbound);
+  defsubr (&Ssymbol_hooked_p);
   defsubr (&Sboundp);
   defsubr (&Sfboundp);
   defsubr (&Sfset);
@@ -3678,10 +3831,10 @@
   DEFVAR_LISP ("most-positive-fixnum", Vmost_positive_fixnum,
 	       doc: /* The largest value that is representable in a Lisp integer.  */);
   Vmost_positive_fixnum = make_number (MOST_POSITIVE_FIXNUM);
-  XSYMBOL (intern_c_string ("most-positive-fixnum"))->constant = 1;
+  XSYMBOL (intern_c_string ("most-positive-fixnum"))->vetted = SYM_CONST;
 
   DEFVAR_LISP ("most-negative-fixnum", Vmost_negative_fixnum,
 	       doc: /* The smallest value that is representable in a Lisp integer.  */);
   Vmost_negative_fixnum = make_number (MOST_NEGATIVE_FIXNUM);
-  XSYMBOL (intern_c_string ("most-negative-fixnum"))->constant = 1;
+  XSYMBOL (intern_c_string ("most-negative-fixnum"))->vetted = SYM_CONST;
 }
--- src/alloc.c
+++ src/alloc.c
@@ -3390,7 +3390,7 @@
   set_symbol_next (val, NULL);
   p->gcmarkbit = false;
   p->interned = SYMBOL_UNINTERNED;
-  p->constant = 0;
+  p->vetted = 0;
   p->declared_special = false;
   p->pinned = false;
   consing_since_gc += sizeof (struct Lisp_Symbol);
--- src/lread.c
+++ src/lread.c
@@ -3821,7 +3821,7 @@
   if ((SREF (string, 0) == ':')
       && EQ (obarray, initial_obarray))
     {
-      XSYMBOL (sym)->constant = 1;
+      XSYMBOL (sym)->vetted = SYM_CONST;
       XSYMBOL (sym)->redirect = SYMBOL_PLAINVAL;
       SET_SYMBOL_VAL (XSYMBOL (sym), sym);
     }
@@ -4042,7 +4042,7 @@
   set_symbol_function (Qunbound, Qnil);
   set_symbol_plist (Qunbound, Qnil);
   SET_SYMBOL_VAL (XSYMBOL (Qnil), Qnil);
-  XSYMBOL (Qnil)->constant = 1;
+  XSYMBOL (Qnil)->vetted = SYM_CONST;
   XSYMBOL (Qnil)->declared_special = true;
   set_symbol_plist (Qnil, Qnil);
   set_symbol_function (Qnil, Qnil);
@@ -4050,7 +4050,7 @@
 
   Qt = intern_c_string ("t");
   SET_SYMBOL_VAL (XSYMBOL (Qt), Qt);
-  XSYMBOL (Qt)->constant = 1;
+  XSYMBOL (Qt)->vetted = SYM_CONST;
   XSYMBOL (Qt)->declared_special = true;
 
   /* Qt is correct even if CANNOT_DUMP.  loadup.el will set to nil at end.  */
--- src/buffer.c
+++ src/buffer.c
@@ -5753,7 +5753,7 @@
 This variable is buffer-local but you cannot set it directly;
 use the function `set-buffer-multibyte' to change a buffer's representation.
 See also Info node `(elisp)Text Representations'.  */);
-  XSYMBOL (intern_c_string ("enable-multibyte-characters"))->constant = 1;
+  XSYMBOL (intern_c_string ("enable-multibyte-characters"))->vetted = SYM_CONST;
 
   DEFVAR_PER_BUFFER ("buffer-file-coding-system",
 		     &BVAR (current_buffer, buffer_file_coding_system), Qnil,
--- src/bytecode.c
+++ src/bytecode.c
@@ -840,7 +840,7 @@
 	    else
 	      {
 		BEFORE_POTENTIAL_GC ();
-		set_internal (sym, val, Qnil, 0);
+		set_internal (sym, val, Qnil, false, Dyn_Current);
 		AFTER_POTENTIAL_GC ();
 	      }
 	  }
--- src/font.c
+++ src/font.c
@@ -5197,19 +5197,19 @@
     [NUMERIC-VALUE SYMBOLIC-NAME ALIAS-NAME ...]
 NUMERIC-VALUE is an integer, and SYMBOLIC-NAME and ALIAS-NAME are symbols. */);
   Vfont_weight_table = BUILD_STYLE_TABLE (weight_table);
-  XSYMBOL (intern_c_string ("font-weight-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-weight-table"))->vetted = SYM_CONST;
 
   DEFVAR_LISP_NOPRO ("font-slant-table", Vfont_slant_table,
 	       doc: /*  Vector of font slant symbols vs the corresponding numeric values.
 See `font-weight-table' for the format of the vector. */);
   Vfont_slant_table = BUILD_STYLE_TABLE (slant_table);
-  XSYMBOL (intern_c_string ("font-slant-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-slant-table"))->vetted = SYM_CONST;
 
   DEFVAR_LISP_NOPRO ("font-width-table", Vfont_width_table,
 	       doc: /*  Alist of font width symbols vs the corresponding numeric values.
 See `font-weight-table' for the format of the vector. */);
   Vfont_width_table = BUILD_STYLE_TABLE (width_table);
-  XSYMBOL (intern_c_string ("font-width-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-width-table"))->vetted = SYM_CONST;
 
   staticpro (&font_style_table);
   font_style_table = make_uninit_vector (3);
--- lisp/subr.el
+++ lisp/subr.el
@@ -2507,6 +2507,9 @@
 Note that this should end with a directory separator.
 See also `locate-user-emacs-file'.")
 \f
+(setq symbol-setter-function ; Defined in eval.c
+      (lambda (_sym _env _oldval newval) newval))
+\f
 ;;;; Misc. useful functions.
 
 (defsubst buffer-narrowed-p ()

Re: [PATCH] (Updated) Run hook when variable is set

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Saying there's only a global variable doesn't remove the absurdity of makunbound within the extent of a «let». Suppose foo is currently unbound. Then:
  > (defvar foo 0) ; Now foo is bound to 0
  > (let ((foo 1)) ; Now it's bound to 1
  >   (makunbound 'foo)) ; Now it's bound to unboundedness
  > ;; Now it's unbound from unboundedness, and thereby rebound to 0

That seems right to me.

  > You can't just say that makunbound in that case unbinds foo,
  > because then the question arises, from what is foo unbound when
  > the let-binding form exits? If foo were already unbound by
  > makunbound, then there would be nothing left to unbind it from
  > when «let» exits.

The word "unbound" is being used in an ambiguous fashion, but once you
see past that, there is nothing strange about what's really going on.

Maybe we should change terminology to get rid of the ambiguity.
The name 'makunbound' is the cause of it, but that is hard to change now.
We could at least explain it better.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

The purpose of makunbound (Was: Run hook when variable is set)

[Kelly Dean]

> The word "unbound" is being used in an ambiguous fashion, but once you
> see past that, there is nothing strange about what's really going on.

What's the use case of doing makunbound on a dynamically let-bound variable?

Re: The purpose of makunbound

[Stefan Monnier]

>> The word "unbound" is being used in an ambiguous fashion, but once you
>> see past that, there is nothing strange about what's really going on.
> What's the use case of doing makunbound on a dynamically let-bound variable?

What's the use case for disallowing it?


        Stefan

Re: The purpose of makunbound

[Kelly Dean]

Stefan Monnier wrote:
>> What's the use case of doing makunbound on a dynamically let-bound variable?
>
> What's the use case for disallowing it?

Catching errors. If some code F does makunbound on a symbol that it assumes is currently used as the global variable, but F is called from some other code that dynamically let-binds the symbol, then F will accidentally set the dynamic variable to the special unboundedness value rather than unbind the global variable.

If that's a bogus thing to do, then by disallowing it, the interpreter could catch the error. But if there's a legitimate use case for it, then of course it must be allowed. Is there one?

Re: The purpose of makunbound

[Stefan Monnier]

>>> What's the use case of doing makunbound on a dynamically let-bound variable?
>> What's the use case for disallowing it?
> Catching errors.  If some code F does makunbound on a symbol that it assumes
> is currently used as the global variable, but F is called from some other
> code that dynamically let-binds the symbol, then F will accidentally set the
> dynamic variable to the special unboundedness value rather than unbind the
> global variable.

Why would that be an error?  How often have you bumped into that error?

> If that's a bogus thing to do, then by disallowing it, the interpreter could
> catch the error.

I don't know if that would be bogus or not, because I don't know why
makunbound was called in your example.

And remember: a (dynamically scoped) `let' does not necessarily mean to
make a local binding, it can also be understood to mean to temporarily
change the global variable (i.e. it's just a shorthand for

   (let ((old <var>))
     (unwind-protect
         (progn (setq <var> <val>)
                ,@body)
       (setq <var> old)))

with the advantage that it doesn't burp in the first line of <var> is
unbound and it properly sets it back to unbound in the end in that
case).

You talk about "global variable" vs "dynamic variable", but that
difference is only in your head because of the interpretation you chose.


        Stefan

Re: The purpose of makunbound

[Kelly Dean]

Stefan Monnier wrote:
> Why would that be an error?

Because of a classic mistake with dynamic binding:
(defvar bar nil)
(defun foo ()
  (if something-rare...
      (makunbound 'bar)) ; Intent is global
  (do-something))

(let ((bar (something))) ; Intent is lexical
  (do-some-stuff)
  (foo)
  (if bar ; Oops
      (some-other-stuff)))

Well the classic is with «set», not makunbound, but the problem is the same.

There are legitimate cases for using «set» in dynamic «let», so the interpreter must allow it, and you just have to deal with the possibility of that kind of mistake. But for makunbound, if there are no legitimate use cases, then the interpreter could catch it.

> How often have you bumped into that error?

Never.

> And remember: a (dynamically scoped) `let' does not necessarily mean to
> make a local binding, it can also be understood to mean to temporarily
> change the global variable
[snip]
> You talk about "global variable" vs "dynamic variable", but that
> difference is only in your head because of the interpretation you chose.

For single-threaded code, there's no difference, but for multi-threaded, there is, which you already mentioned. Why have inconsistent interpretations for single vs. multi, when you can have a consistent one for both?

And aren't you planning to add multi-threading to Emacs?

Re: The purpose of makunbound

[Stefan Monnier]

>> Why would that be an error?
> Because of a classic mistake with dynamic binding:
> (defvar bar nil)
> (defun foo ()
>   (if something-rare...
>       (makunbound 'bar)) ; Intent is global
>   (do-something))

> (let ((bar (something))) ; Intent is lexical
>   (do-some-stuff)
>   (foo)
>   (if bar ; Oops
>       (some-other-stuff)))

This problematic situation is problematic regardless of the presence of
makunbound.  You simply can't have a (defvar <foo> <val>) for a variable
you need to use lexically elsewhere.

> For single-threaded code, there's no difference, but for multi-threaded,
> there is, which you already mentioned. Why have inconsistent interpretations
> for single vs. multi, when you can have a consistent one for both?

Note that the current implementation technique for "bound/unbound" still
works just fine with multithreading (the implementation of dynamically
scoped "let" is more problematic but it has no particular problematic
interaction with makunbound).

So, adding multi-threading won't make much difference to this "problem".


        Stefan

Re: The purpose of makunbound

[Kelly Dean]

Stefan Monnier wrote:
>>> Why would that be an error?
>> Because of a classic mistake with dynamic binding:
[snip]
> This problematic situation is problematic regardless of the presence of
> makunbound.

True. I was just answering your question about why makunbound would be an error. It's the same as why «set» would be an error.

>> For single-threaded code, there's no difference, but for multi-threaded,
>> there is, which you already mentioned. Why have inconsistent interpretations
>> for single vs. multi, when you can have a consistent one for both?
>
> Note that the current implementation technique for "bound/unbound" still
> works just fine with multithreading (the implementation of dynamically
> scoped "let" is more problematic but it has no particular problematic
> interaction with makunbound).
>
> So, adding multi-threading won't make much difference to this "problem".

You said the difference between global and dynamic variables is just in my head because of the interpretation I chose. So my point was that my interpretation has the advantage of being applicable to both single-threading and multi-threading. It's true that for single-threading, it's just philosophical.

Re: The purpose of makunbound

[Kelly Dean]

Here's a verbatim (including the comments) quote from section 11.4 ((elisp) Void Variables) of the manual in Emacs 24.4:
          (setq x 1)               ; Put a value in the global binding.
               => 1
          (let ((x 2))             ; Locally bind it.
            (makunbound 'x)        ; Void the local binding.
            x)
          error--> Symbol's value as variable is void: x
          x                        ; The global binding is unchanged.
               => 1

Notice the ‟global binding is unchanged”. That matches my interpretation. Your interpretation would be ‟the global binding is changed back”.

And in the same section:
          (boundp 'abracadabra)          ; Starts out void.
               => nil
          (let ((abracadabra 5))         ; Locally bind it.
            (boundp 'abracadabra))
               => t
          (boundp 'abracadabra)          ; Still globally void.
               => nil
          (setq abracadabra 5)           ; Make it globally nonvoid.
               => 5
          (boundp 'abracadabra)
               => t

Notice the ‟still globally void”. That matches my interpretation. Yours would be ‟globally void again”.

I'm pointing this out just because you're saying my interpretation is weird/unusual. It isn't.

Re: The purpose of makunbound

[Kelly Dean]

> The word "unbound" is being used in an ambiguous fashion, but once you
> see past that, there is nothing strange about what's really going on.
>
> Maybe we should change terminology to get rid of the ambiguity.
> The name 'makunbound' is the cause of it, but that is hard to change now.
> We could at least explain it better.

To get rid of the ambiguity, I propose always using the term ‟void” for what makunbound sets, and using the ‟bind/unbind” terminology exclusively for the kind of binding that «let» does.

Emacs already uses the term ‟void” sometimes. For example, if you try to evaluate a symbol after doing makunbound on it, then assuming there isn't a lexical variable of it in scope, it signals the error ⌜void-variable⌝. And the docstrings for makunbound and boundp use that term. Consistently using it would make the code more clear.

So, I propose a global search/replace in the source code of ⌜Qunbound⌝ to ⌜Qvoid⌝. Maybe you'll think that in some places, it really makes more sense to interpret it as «unbound», not «void». However, that would be inconsistent, if makunbound is allowed to function on non-global (i.e. let-bound or buffer-local) variables. To make it consistent, leave ⌜Qunbound⌝ the way it is, and make makunbound stop functioning on non-global variables; that's what I prefer, but since nobody else does, the alternative way of being consistent is to change ⌜Qunbound⌝ to ⌜Qvoid⌝.

A search-replace of ⌜unbound⌝ to ⌜void⌝ in the «el» files would also be in order, but it would break code that depends on the current names, and apparently just making aliases for the old names isn't an acceptable solution. Currently, the Elisp code is inconsistent; it uses ‟void” in a few places, and ‟unbound” in most others. It doesn't look like it ever uses ‟unbound” in the sense of let-binding. In the manual, it's the opposite: it usually uses ‟void” for the special value, and uses ‟unbound” in the sense of let-binding in a couple places.

Just to be clear, the only reason there's any ambiguity in the first place is because makunbound operates on non-global variables. If it didn't do that, there would be no need for «void» as a separate concept. (Being bound to void vs. being unbound is indistinguishable for globals, since they can't shadow anything.) ‟Unbound” would be all that's needed, ‟void” would just be a synonym, and the names of makunbound and boundp would be appropriate for those functions.

Re: The purpose of makunbound

[Stefan Monnier]

> To get rid of the ambiguity, I propose always using the term ‟void” for what
> makunbound sets, and using the ‟bind/unbind” terminology exclusively for the
> kind of binding that «let» does.

Here's the problem: I don't see any difference between those two kinds
of bindings and the implementation doesn't see them as different either.
So if we try and distinguish them, we'll probably end up describing
a semantics that doesn't match the implementation.

> Just to be clear, the only reason there's any ambiguity in the first place
> is because makunbound operates on non-global variables.

Dynamically-scoped vars are global.


        Stefan

Re: The purpose of makunbound

[Kelly Dean]

Stefan Monnier wrote:
>> To get rid of the ambiguity, I propose always using the term ‟void” for what
>> makunbound sets, and using the ‟bind/unbind” terminology exclusively for the
>> kind of binding that «let» does.
>
> Here's the problem: I don't see any difference between those two kinds
> of bindings and the implementation doesn't see them as different either.

The implementation does see them as different, and the difference is exposed to the Lisp world too. The difference is not dependent on the interpretation of whether «let» creates a new variable or temporarily changes the value of the global variable.

The difference is because makunbound works for let-bound and buffer-local variables. Regardless of the interpretation, there are two different kinds of ‟unbind” events: the one caused by exit of a «let» (which unshadows/restores the outer/previous value), and the one caused by makunbound (which sets the value to void). These are different in the implementation and different in the Lisp world (observable by doing makunbound on a let-bound or buffer-local variable, then exiting the «let» or killing the buffer-local), yet the same terminology is used for both.

I'm proposing a difference in terminology to account for those two different kinds of events.

In contrast, if makunbound didn't work for let-bound or buffer-local variables, then there wouldn't be two different kinds of events in the Lisp world (though there would still be in the implementation), because setting non-shadowing/not-temporarily-changed non-buffer-local variables to void is indistinguishable from unbinding them in the let-binding sense (because there's no shadowed/previous value to restore). And because there would be only one kind of event, there would be no need for a difference in terminology to avoid ambiguity.

>> Just to be clear, the only reason there's any ambiguity in the first place
>> is because makunbound operates on non-global variables.
>
> Dynamically-scoped vars are global.

In my message, I wrote that by ‟non-global” I meant ‟let-bound or buffer-local”, so I could use ‟non-global” and ‟global” as shorthand in the rest of the message. For your preferred interpretation of what let-binding does, just pretend I spelled out ‟let-bound-or-buffer-local” and ‟not-let-bound-and-not-buffer-local”.

Re: The purpose of makunbound

[Stefan Monnier]

> variables. Regardless of the interpretation, there are two different kinds
> of ‟unbind” events: the one caused by exit of a «let» (which
> unshadows/restores the outer/previous value), and the one caused by
> makunbound (which sets the value to void).

Ah, I think I understand a bit better.  I think treating "void" as an
event is the error: it's a state.  OTOH, the "revert to previous
state" done at the end of `let' is indeed an event.

So one is an action and the other is a state.

> or killing the buffer-local), yet the same terminology is used for both.

I don't see "unbind" used anywhere in the description of what `let'
does, nor is it a term that I've heard used (other than by yourself) to
refer to what happens at the end of a "let".


        Stefan

Re: The purpose of makunbound

[Kelly Dean]

Stefan Monnier wrote:
> Ah, I think I understand a bit better.  I think treating "void" as an
> event is the error: it's a state.  OTOH, the "revert to previous
> state" done at the end of `let' is indeed an event.

I'm not treating void as an event. Of course it's a state. I'm treating «set the state to void» as an event, and «exit let» as an event, and my first point is that since they're different events with different resultant states, different terminology should be used for them.

My second point is that if makunbound didn't work for let-bound or buffer-local variables, then the void state could never shadow a non-void state, which means an «exit let» event could never change a void state to a non-void state. Therefore a «set the state to void» event would be indistinguishable from an «exit let» event in which no non-void state had been shadowed, and of course there's no need for different terminology for indistinguishable events. In contrast, with makunbound's current behavior, those two events are distinguishable: the first one leaves open the possibility that a subsequent «exit let» event will produce a non-void state, but the second one doesn't.

> I don't see "unbind" used anywhere in the description of what `let'
> does, nor is it a term that I've heard used (other than by yourself) to
> refer to what happens at the end of a "let".

The definition of «let» in eval.c calls the function «unbind_to» at the end.

Also see the Emacs 24.4 Elisp manual:
(elisp) Catch and Throw (section 10.5.1):
⌜Executing `throw' exits all Lisp constructs up to the matching
`catch', including function calls.  When binding constructs such as
`let' or function calls are exited in this way, the bindings are
unbound, just as they are when these constructs exit normally⌝

(elisp) Named Features (section 15.7):
⌜Loading a library while its variables are let-bound can have
unintended consequences, namely the variables becoming unbound after
the let exits.⌝

(elisp) Warning Variables (section 37.5.2):
⌜Programs can bind this variable to `t' to say that the next
warning should begin a series.
...
The series ends when the local binding
is unbound and `warning-series' becomes `nil' again.⌝

(elisp) Intro to Buffer-Local (section 11.10.1):
⌜if you exit the `let'
while still in the other buffer, you won't see the unbinding occur⌝

Re: The purpose of makunbound

[Stefan Monnier]

> The definition of «let» in eval.c calls the function «unbind_to» at the end.

This is completely invisible to the Elisp programmer, so it's of
no importance.

> Also see the Emacs 24.4 Elisp manual:
> (elisp) Catch and Throw (section 10.5.1):
> ⌜Executing `throw' exits all Lisp constructs up to the matching
> `catch', including function calls.  When binding constructs such as
> `let' or function calls are exited in this way, the bindings are
> unbound, just as they are when these constructs exit normally⌝

This "unbound" is indeed incorrect, it should be "undone" (if anything
is "unbound" it could be the variables, but definitely not the bindings).

> (elisp) Named Features (section 15.7):
> ⌜Loading a library while its variables are let-bound can have
> unintended consequences, namely the variables becoming unbound after
> the let exits.⌝

This "becoming unbound" here really means "the variables will end up
with a Qunbound value".  BTW, the problem described in this little bit
of text has been mostly eliminated by recentish changes which made
`defvar' use the new `set-default-toplevel-value'.

> (elisp) Warning Variables (section 37.5.2):
> ⌜Programs can bind this variable to `t' to say that the next
> warning should begin a series.
> ...
> The series ends when the local binding
> is unbound and `warning-series' becomes `nil' again.⌝

Here again "unbound" should be "undone".

> (elisp) Intro to Buffer-Local (section 11.10.1):
> ⌜if you exit the `let'
> while still in the other buffer, you won't see the unbinding occur⌝

This one is admittedly a real case where "unbind" is used to mean "undo
a let binding".


        Stefan

Re: The purpose of makunbound

[Stephen J. Turnbull]

Kelly Dean writes:

 > The implementation does see [the two forms of unbinding] as
 > different,

I don't understand what you mean.  The unbinding is the same, and has
the conventional semantics: this value can no longer be accessed by
that name (== symbol).  Exit from let has *additional semantics* of
restoring the previous binding.

The void or unbound pseudo-value is an implementation detail, useful
in so-called shallow-binding implementations.  You can't observe that
symbol from Lisp without making an end-run around the Lisp engine (eg,
via an FFI).  It's uninterned so it's not possible to find it and then
store it somewhere, and all slot-accessing internal functions check
for it and error if it's found.

 > In contrast, if makunbound didn't work for let-bound or
 > buffer-local variables,

If there's a reason for makunbound in the top-level environment, the
same reason will apply to let-bound environments.  I can't support
your proposal to error on makunbound of let-bound variables.

Re: The purpose of makunbound

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So, I propose a global search/replace in the source code of
  > ⌜Qunbound⌝ to ⌜Qvoid⌝. Maybe you'll think that in some places, it
  > really makes more sense to interpret it as «unbound», not
  > «void». However, that would be inconsistent, if makunbound is
  > allowed to function on non-global (i.e. let-bound or buffer-local)
  > variables.

I don't see an inconsistency.  I think the term "void" is fine with
the existing behavior.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

Re: The purpose of makunbound

[Kelly Dean]

Richard Stallman wrote:
>> So, I propose a global search/replace in the source code of
>> ⌜Qunbound⌝ to ⌜Qvoid⌝. Maybe you'll think that in some places, it
>> really makes more sense to interpret it as «unbound», not
>> «void». However, that would be inconsistent, if makunbound is
>> allowed to function on non-global (i.e. let-bound or buffer-local)
>> variables.
>
> I don't see an inconsistency.  I think the term "void" is fine with
> the existing behavior.

Ok, great! Since the term ‟void” is fine, let's use ⌜Qvoid⌝ for the corresponding constant in the source code.

Re: The purpose of makunbound

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Ok, great! Since the term ‟void” is fine, let's use ⌜Qvoid⌝ for the corresponding constant in the source code.

I won't argue against it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! See stallman.org/skype.html.

Re: The purpose of makunbound

[Stephen J. Turnbull]

Richard Stallman writes:

 >   > Ok, great! Since the term ‟void” is fine, let's use ⌜Qvoid⌝ for
 >   > the corresponding constant in the source code.
 > 
 > I won't argue against it.

I will.

It's a gratuitous change which pleases at most one person in the whole
world, while making working with the code a little bit more difficult
for people who have code bases containing the old name.  (Of course
I'm thinking of XEmacs, but the same reason applies to anybody with a
branch planned for merger in the future.)

OTOH, "Qunbound" corresponds to perfectly good English usage.

@Stefan: Please refuse to make this change.

Re: The purpose of makunbound

[Kelly Dean]

Stephen J. Turnbull wrote:
>Richard Stallman writes:
>
>>> Ok, great! Since the term ‟void” is fine, let's use ⌜Qvoid⌝ for
>>> the corresponding constant in the source code.
>>
>> I won't argue against it.
>
> I will.
[snip]
> @Stefan: Please refuse to make this change.

@Stefan:
Stephen objects to the change of Qunbound to Qvoid. Richard does not object.

The change would make the source code more consistent with the manual, which almost always uses the term ‟void” for the Qunbound state. Even the docstrings for makunbound and boundp use ‟void”, not ‟unbound”.

Section 11.4 ((elisp) Void Variables) even has four occurrences of ‟void” as a verb meaning «to make the state void», and uses ‟voidness” to mean «the state of being void».

The change would also make the source code more consistent with itself, since it already has Qvoid_variable, not Qunbound_variable, as the corresponding error code.

Stefan, may I make the change?

Re: The purpose of makunbound

[Andreas Schwab]

Kelly Dean <kelly@prtime.org> writes:

> may I make the change?

No.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

I wrote:
> it now spends half the time doing garbage collection.

I figured out how to solve the voidness problem without consing or specbinding every time a hooked variable is set, and without usurping a keyword to represent void. Simply use an uninterned symbol to represent void. Specifically:
  DEFSYM (Qvoid_sentinel, "void-sentinel");
  DEFVAR_LISP ("void-sentinel", Vvoid_sentinel,
	       doc: /* Representation of voidness for hooked variables.
The value of this constant is an uninterned Lisp symbol that represents void
when passed to or returned from `symbol-setter-function'.
When a variable is hooked, Emacs can't distinguish between setting it to this
value and making it unbound. Therefore, to prevent a difference of behavior
for hooked and unhooked variables, don't set any variable to this value.  */);
  Vvoid_sentinel = Fmake_symbol (build_string ("::void::"));
  XSYMBOL (Qvoid_sentinel)->declared_special = true;
  XSYMBOL (Qvoid_sentinel)->vetted = SYM_CONST;

Then at the end of run_varhook:
  oldval = EQ (oldval, Qunbound) ? Vvoid_sentinel : oldval;
  newval = EQ (newval, Qunbound) ? Vvoid_sentinel : newval;
  newval = call4 (Vsymbol_setter_function, symbol, env, oldval, newval);
  return EQ (newval, Vvoid_sentinel) ? Qunbound : newval;

Semantically, that's worse than consing or specbinding, but not as bad as usurping a keyword. Is it ok to do it that way?

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> Semantically, that's worse than consing or specbinding, but not as bad
> as usurping a keyword. Is it ok to do it that way?

Fine by me, yes,


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> Surely you can't say with a straight face that binding to unboundedness is
> not an absurd concept.

Where does the manual say such a thing?  AFAICT this absurd concept
comes from your interpretation of what's going on, and just shows that
your interpretation is not a good model.

> Either way, the behavior of makunbound is wrong.

You've said so enough times, but I haven't heard any concrete
alternative proposal so far.  Not that I'd advise you to spend time on
it, since this is a complete non-problem as far as I'm concerned.

>>>> - OLDVAL is either a list of one element containing the old value, or
>>>> nil (when that old value is Qunbound).
>>> Then run_varhook must cons. That'll generate a lot of garbage if you use it
>>> for profiling, or for debugging in a way where you don't just pause to
>>> inspect every hooked variable. Is that ok?
>> I think it's OK, yes.
> This seems to be the least-bad option, so I did it this way, even though it
> makes the API a bit gross. Unfortunately, when you do:
> (require 'cl)
> (setq x 0)
> (symbol-hook 'x)
> (benchmark-run-compiled 100000 (incf x))
> it now spends half the time doing garbage collection. That's a high price to
> pay to cater to the brain-dead misbehavior of makunbound.

Agreed.  Another option is to pass 2 more arguments, i.e. pass "OLDVAL
OLDBOUND NEWVAL NEWBOUND".  You still have the problem of returning
whether the actual new value should be bound or not.

I see you decided to use a special (uninterned symbol) value instead.
That's OK with me.

> so to account for the inversion, it would have to change to:
> if (!writeable) a...;
> else b...;
> which introduces an extra «not» operator.

This operator has 0 cost since any decent compiler will just swap the
two branches of the `if' instead of performing an actual `not' computation.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> Where does the manual say such a thing?

Er, it seems it doesn't. ;-) It says ‟void”, not ‟unbound”. Except in just one place (17.2.7, (elisp) Trapping Errors) where it has ⌜unbound variable⌝ to mean void variable.

I was fooled by the void value being named Qunbound.

> AFAICT this absurd concept
> comes from your interpretation of what's going on, and just shows that
> your interpretation is not a good model.

Regardless of the name, the interpretation is the right one for multi-threaded code, and is just as valid as the alternative for single-threaded code.

>> Either way, the behavior of makunbound is wrong.
>
> You've said so enough times, but I haven't heard any concrete
> alternative proposal so far.

Oh, I thought that was clear. Have makunbound signal an error if let_shadows_global_binding_p or Flocal_variable_p is true. My theory is that this would only catch bugs, not catch intentional uses.

> Not that I'd advise you to spend time on
> it, since this is a complete non-problem as far as I'm concerned.

Understood.

> I see you decided to use a special (uninterned symbol) value instead.
> That's OK with me.

Ok, I'll have an updated and hopefully finalized patch shortly.

> This operator has 0 cost since any decent compiler will just swap the
> two branches of the `if' instead of performing an actual `not' computation.

Duh, oops.

Well, ⌜vetted⌝ works just as well as ⌜writable⌝ for the field name, so I'm inclined to leave it that way, but I'll change it if you want.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

>> AFAICT this absurd concept comes from your interpretation of what's
>> going on, and just shows that your interpretation is not
>> a good model.
> Regardless of the name, the interpretation is the right one for
> multi-threaded code, and is just as valid as the alternative for
> single-threaded code.

No, the interpretation based on the current implementation (i.e. "void"
is just a special value) works just as well in the multithreaded case.

> Oh, I thought that was clear. Have makunbound signal an error if
> let_shadows_global_binding_p or Flocal_variable_p is true. My theory is that
> this would only catch bugs, not catch intentional uses.

Based on my experience with emitting messages in similar oddball cases
(i.e. making a variable buffer-local while it's globally let-bound), my
guess is that it would catch a few very rare oddball cases indeed, but:
1- those oddball cases end up harmless.
2- there's no easy "fix" for those cases (because the let-binding and the
   makunbound happen in completely unrelated code which usually aren't
   used at the same time and what *really* should happen is at best
   unclear, or is otherwise exactly what happens with the current
   semantics you don't like).

> Well, ⌜vetted⌝ works just as well as ⌜writable⌝ for the field name, so I'm
> inclined to leave it that way, but I'll change it if you want.

Can I have my bikeshed transparent?


        Stefan

[PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 1994 bytes --]

Stefan Monnier wrote:
> Can I have my bikeshed transparent?

Absolutely. I'll untabify all the source code, then rename this variable to the tab character, and patch GCC to interpret tab as a variable. ;-)

In the meantime, an updated patch for varhook is attached. Changes from the previous version:

Use void-sentinel instead of consing in run_varhook.

Disallow hooking of constants. Duh.

Properly handle a case that was introduced when I added support for blocking/overriding the variable setting: if you hook a variable and override its setting, then the return value of set, setq, etc should be the override value instead of the originally attempted value, so that the override will cascade through e.g. (setq x (setq y foo)) and (if (setq z foo) ...). Otherwise, during debugging if e.g. you hook foo, and discover that it's nil there, but you want to try running that «if» condition, so you override the setq and set z to t, it would be annoying if the setq returned nil anyway.

Because of the previous change, I had to remove the handler function's capability of setting a variable to void when the setter just attempted to set a regular value. This is because otherwise, if x is lexical and y is special and hooked, and you override the setting of y by setting it to void, then (setq x (setq y foo)) would result in x being set to void, which isn't allowed.

But forcibly voiding a variable when it's set would be a pathological thing to do, so I don't think removing that capability is a problem. You can still do the opposite thing, i.e. override makunbound by setting a non-void value, and of course makunbound still works as normal (even on hooked variables) if you don't override it.

Removing that capability has a fortunate side effect: even if a variable is hooked, Emacs can now distinguish between setting it to the value of void-sentinel (though there's no reason to ever do that) and doing makunbound on it.


Any other changes I should make?



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: varhook-advice-3.patch --]
[-- Type: text/x-diff, Size: 30315 bytes --]

--- src/lisp.h
+++ src/lisp.h
@@ -290,6 +290,15 @@
 # define GCALIGNED /* empty */
 #endif
 
+/* These are the masks for the vetted field of Lisp_Symbol.
+   Bit 0 stores the constant field.  Bit 1 stores the hooked field.  */
+#define SYM_CONST 1
+#define SYM_HOOKED 2
+
+# define SYM_CONST_P(sym) (((sym)->vetted) & SYM_CONST)
+# define SYM_HOOKED_P(sym) (((sym)->vetted) & SYM_HOOKED)
+
+
 /* Some operations are so commonly executed that they are implemented
    as macros, not functions, because otherwise runtime performance would
    suffer too much when compiling with GCC without optimization.
@@ -344,7 +353,7 @@
 #define lisp_h_NILP(x) EQ (x, Qnil)
 #define lisp_h_SET_SYMBOL_VAL(sym, v) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value = (v))
-#define lisp_h_SYMBOL_CONSTANT_P(sym) (XSYMBOL (sym)->constant)
+#define lisp_h_SYMBOL_CONSTANT_P(sym) (SYM_CONST_P (XSYMBOL (sym)))
 #define lisp_h_SYMBOL_VAL(sym) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value)
 #define lisp_h_SYMBOLP(x) (XTYPE (x) == Lisp_Symbol)
@@ -659,10 +668,13 @@
      3 : it's a forwarding variable, the value is in `forward'.  */
   ENUM_BF (symbol_redirect) redirect : 3;
 
-  /* Non-zero means symbol is constant, i.e. changing its value
-     should signal an error.  If the value is 3, then the var
-     can be changed, but only by `defconst'.  */
-  unsigned constant : 2;
+  /* When masked with SYM_CONST, non-zero means symbol is constant,
+     i.e. changing its value should signal an error.
+     When masked with SYM_HOOKED, non-zero means setting symbol will
+     run varhook.  These two fields are combined into one in order
+     to optimize the fast path of non-hooked non-constants by
+     having only one conditional branch for that case.  */
+  unsigned vetted : 2;
 
   /* Interned state of the symbol.  This is an enumerator from
      enum symbol_interned.  */
@@ -3463,6 +3475,15 @@
 }
 
 /* Defined in data.c.  */
+typedef enum
+  { /* See set_internal for a description of these values.  */
+    Dyn_Makvoid = -2,
+    Dyn_Unbind = -1,
+    Dyn_Current = 0,
+    Dyn_Bind = 1,
+    Dyn_Skip = 2,
+    Dyn_Global = 3
+  } Dyn_Bind_Env;
 extern Lisp_Object indirect_function (Lisp_Object);
 extern Lisp_Object find_symbol_value (Lisp_Object);
 enum Arith_Comparison {
@@ -3509,7 +3530,15 @@
 extern _Noreturn void args_out_of_range_3 (Lisp_Object, Lisp_Object,
 					   Lisp_Object);
 extern Lisp_Object do_symval_forwarding (union Lisp_Fwd *);
-extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool);
+extern Lisp_Object run_varhook (struct Lisp_Symbol*, bool, Dyn_Bind_Env,
+				Lisp_Object, Lisp_Object);
+extern Lisp_Object set_internal_vetted (Lisp_Object, Lisp_Object, Lisp_Object, bool,
+					 Dyn_Bind_Env, struct Lisp_Symbol *);
+extern Lisp_Object set_internal_localized_or_forwarded (Lisp_Object, Lisp_Object,
+						 Lisp_Object, bool,
+						 Dyn_Bind_Env,
+						 struct Lisp_Symbol *);
+extern Lisp_Object set_default_internal (Lisp_Object, Lisp_Object, Dyn_Bind_Env);
 extern void syms_of_data (void);
 extern void swap_in_global_binding (struct Lisp_Symbol *);
 
@@ -4776,6 +4805,51 @@
     return false;
 }
 
+/* Store the value NEWVAL into SYMBOL.
+   If buffer/frame-locality is an issue, WHERE specifies which context to use.
+   (nil stands for the current buffer/frame).
+
+   If BINDFLAG is false, then if this symbol is supposed to become
+   local in every buffer where it is set, then we make it local.
+   If BINDFLAG is true, we don't do that.
+
+   ENV indicates the dynamic environment for this function call, i.e. whether
+   this call is due to a variable binding (Dyn_Bind), an unbinding (Dyn_Unbind),
+   or neither (Dyn_Current). As special cases, a value of Dyn_Skip is a flag
+   to disable run_varhook so that varhooks aren't run during backtraces, and
+   a value of Dyn_Global is a flag indicating that this function call is due
+   to set_default, which allows run_varhook to distinguish beween the global
+   and the dyn-local binding. And Dyn_Makvoid is a flag indicating that this
+   function call is due to makunbound, which tells run_varhook to allow
+   setting the variable to void.  */
+
+INLINE Lisp_Object
+set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+	      bool bindflag, Dyn_Bind_Env env)
+{
+  struct Lisp_Symbol *sym;
+
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  if (sym->vetted)
+    return set_internal_vetted (symbol, newval, where, bindflag, env, sym);
+
+ start:
+  switch (sym->redirect)
+    {
+    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym, newval); return newval;
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+    default: return set_internal_localized_or_forwarded
+	(symbol, newval, where, bindflag, env, sym);
+    }
+}
+
+#define MAYBE_RUN_VARHOOK(result, sym, buf_local, env, oldval, newval)	\
+  {									\
+    if (SYM_HOOKED_P (sym))						\
+      (result) = run_varhook (sym, buf_local, env, oldval, newval);	\
+  }
+
 INLINE_HEADER_END
 
 #endif /* EMACS_LISP_H */
--- src/eval.c
+++ src/eval.c
@@ -250,7 +250,7 @@
   max_lisp_eval_depth = XINT (XCDR (data));
 }
 
-static void grow_specpdl (void);
+static inline void grow_specpdl (void);
 
 /* Call the Lisp debugger, giving it argument ARG.  */
 
@@ -530,7 +530,8 @@
 			= Fassq (sym, Vinternal_interpreter_environment)))
 	    XSETCDR (lex_binding, val); /* SYM is lexically bound.  */
 	  else
-	    Fset (sym, val);	/* SYM is dynamically bound.  */
+	    /* SYM is dynamically bound.  */
+	    val = set_internal (sym, val, Qnil, false, Dyn_Current);
 
 	  args_left = Fcdr (XCDR (args_left));
 	}
@@ -616,7 +617,7 @@
 
   sym = XSYMBOL (new_alias);
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     /* Not sure why, but why not?  */
     error ("Cannot make a constant an alias");
 
@@ -633,7 +634,7 @@
      so that old-code that affects n_a before the aliasing is setup
      still works.  */
   if (NILP (Fboundp (base_variable)))
-    set_internal (base_variable, find_symbol_value (new_alias), Qnil, 1);
+    set_internal (base_variable, find_symbol_value (new_alias), Qnil, true, Dyn_Current);
 
   {
     union specbinding *p;
@@ -648,7 +649,7 @@
   XSYMBOL (base_variable)->declared_special = 1;
   sym->redirect = SYMBOL_VARALIAS;
   SET_SYMBOL_ALIAS (sym, XSYMBOL (base_variable));
-  sym->constant = SYMBOL_CONSTANT_P (base_variable);
+  sym->vetted = SYMBOL_CONSTANT_P (base_variable);
   LOADHIST_ATTACH (new_alias);
   /* Even if docstring is nil: remove old docstring.  */
   Fput (new_alias, Qvariable_documentation, docstring);
@@ -2006,7 +2007,7 @@
    never-used entry just before the bottom of the stack; sometimes its
    address is taken.  */
 
-static void
+static inline void
 grow_specpdl (void)
 {
   specpdl_ptr++;
@@ -3038,8 +3039,6 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS:
-      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
     case SYMBOL_PLAINVAL:
       /* The most common case is that of a non-constant symbol with a
 	 trivial value.  Make that as fast as we can.  */
@@ -3047,11 +3046,15 @@
       specpdl_ptr->let.symbol = symbol;
       specpdl_ptr->let.old_value = SYMBOL_VAL (sym);
       grow_specpdl ();
-      if (!sym->constant)
-	SET_SYMBOL_VAL (sym, value);
+      if (!sym->vetted) SET_SYMBOL_VAL (sym, value);
+      else if (SYM_HOOKED_P (sym))
+	SET_SYMBOL_VAL (sym, run_varhook
+			(sym, false, Dyn_Bind, sym->val.value, value));
       else
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, true, Dyn_Bind);
       break;
+    case SYMBOL_VARALIAS:
+      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
     case SYMBOL_LOCALIZED:
       if (SYMBOL_BLV (sym)->frame_local)
 	error ("Frame-local vars cannot be let-bound");
@@ -3082,7 +3085,7 @@
 	      {
 		specpdl_ptr->let.kind = SPECPDL_LET_DEFAULT;
 		grow_specpdl ();
-		Fset_default (symbol, value);
+		set_default_internal (symbol, value, Dyn_Bind);
 		return;
 	      }
 	  }
@@ -3090,7 +3093,7 @@
 	  specpdl_ptr->let.kind = SPECPDL_LET;
 
 	grow_specpdl ();
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, true, Dyn_Bind);
 	break;
       }
     default: emacs_abort ();
@@ -3225,7 +3228,9 @@
 	    struct Lisp_Symbol *sym = XSYMBOL (specpdl_symbol (specpdl_ptr));
 	    if (sym->redirect == SYMBOL_PLAINVAL)
 	      {
-		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
+		Lisp_Object oldval = specpdl_old_value (specpdl_ptr);
+		MAYBE_RUN_VARHOOK (oldval, sym, false, Dyn_Unbind, sym->val.value, oldval);
+		SET_SYMBOL_VAL (sym, oldval);
 		break;
 	      }
 	    else
@@ -3235,8 +3240,8 @@
 	      }
 	  }
 	case SPECPDL_LET_DEFAULT:
-	  Fset_default (specpdl_symbol (specpdl_ptr),
-			specpdl_old_value (specpdl_ptr));
+	  set_default_internal (specpdl_symbol (specpdl_ptr),
+				specpdl_old_value (specpdl_ptr), Dyn_Unbind);
 	  break;
 	case SPECPDL_LET_LOCAL:
 	  {
@@ -3248,7 +3253,7 @@
 	    /* If this was a local binding, reset the value in the appropriate
 	       buffer, but only if that buffer's binding still exists.  */
 	    if (!NILP (Flocal_variable_p (symbol, where)))
-	      set_internal (symbol, old_value, where, 1);
+	      set_internal (symbol, old_value, where, true, Dyn_Unbind);
 	  }
 	  break;
 	}
@@ -3454,7 +3459,7 @@
 	    Lisp_Object sym = specpdl_symbol (tmp);
 	    Lisp_Object old_value = specpdl_old_value (tmp);
 	    set_specpdl_old_value (tmp, Fdefault_value (sym));
-	    Fset_default (sym, old_value);
+	    set_default_internal (sym, old_value, Dyn_Skip);
 	  }
 	  break;
 	case SPECPDL_LET_LOCAL:
@@ -3470,7 +3475,7 @@
 	      {
 		set_specpdl_old_value
 		  (tmp, Fbuffer_local_value (symbol, where));
-		set_internal (symbol, old_value, where, 1);
+		set_internal (symbol, old_value, where, true, Dyn_Skip);
 	      }
 	  }
 	  break;
@@ -3746,6 +3751,66 @@
 still determine whether to handle the particular condition.  */);
   Vdebug_on_signal = Qnil;
 
+  DEFSYM (Qvoid_sentinel, "void-sentinel");
+  DEFVAR_LISP ("void-sentinel", Vvoid_sentinel,
+	       doc: /* Representation of voidness for hooked variables.
+The value of this constant is an uninterned Lisp symbol that represents void
+when passed to or returned from `symbol-setter-function'.  */);
+  Vvoid_sentinel = Fmake_symbol (build_string ("::void::"));
+  XSYMBOL (Vvoid_sentinel)->declared_special = true;
+  XSYMBOL (Vvoid_sentinel)->vetted = SYM_CONST;
+  XSYMBOL (Qvoid_sentinel)->declared_special = true;
+  XSYMBOL (Qvoid_sentinel)->vetted = SYM_CONST;
+
+  DEFVAR_LISP ("symbol-setter-function", Vsymbol_setter_function,
+       doc: /* This function is called whenever a hooked variable is set.
+It takes four arguments: SYMBOL, ENV, OLDVAL, NEWVAL. By default, it just
+returns NEWVAL unchanged.
+
+SYMBOL is the symbol being set. ENV is the environment is which it's being
+set. OLDVAL is the current value, or if the current value is void, then OLDVAL
+is the value of `void-sentinel'. NEWVAL is the new value to which the setter,
+i.e. the caller of a function such as `setq', is attempting to set the
+variable, or the value of void-sentinel if the setter called `makunbound'.
+The actual new value to which the variable will be set is return value of
+this function, unless the setter called makunbound and this function returns
+the value of void-sentinel, in which case the variable will be set to void.
+The return value is NEWVAL if this function lacks advice that overrides it.
+
+The possible values of ENV are these symbols, with these meanings:
+global: The global environment.
+buf-local: The setter's buffer-local environment.
+dyn-local: The innermost dynamic environment in which SYMBOL is bound.
+dyn-bind: A new dynamic environment, such as creatable using `let'.
+dyn-unbind: The next-outer dynamic environment in which SYMBOL is still bound,
+unshadowed due to destruction of the setter's current dynamic environment,
+such as due to exit of a `let' form, or the buffer-local environment if SYMBOL
+is not bound in any dynamic environment, or the global environment is SYMBOL
+is not in the buffer-local environment.
+
+If you use overriding advice, your advice must return the value to which to
+set the variable. To avoid overriding the setter's attempt to set the variable
+to NEWVAL, return NEWVAL. To block the attempt, and leave the variable
+unchanged, return OLDVAL. If ENV is dyn-bind or dyn-unbind, you can block
+the change of value, but you can't prevent the corresponding creation or
+destruction of a dynamic environment. Therefore, blocking when ENV is
+dyn-bind will set SYMBOL in the new environment to its value in the outer
+environment, and blocking when ENV is dyn-unbind will set SYMBOL in the
+outer environment to its value in the environment being destroyed. If OLDVAL
+is the value of void-sentinel but NEWVAL is not, you can override the new
+value, but you can't prevent the variable from being set to a non-void value.
+
+Don't set the variable in your advice; that would cause a recursive call
+to this function, and even if you terminate the recursion, your setting
+would be overridden by the return value of this function. Instead, if your
+advice needs to set the variable, use `add-function' with overriding advice.
+
+To hook all variables of a symbol, use `symbol-hook'. To unhook them,
+use `symbol-unhook'. If you only want to watch or override some variables
+of a symbol, then filter according to ENV, and if you use overriding advice,
+simply return NEWVAL for the ones you don't want to process.  */);
+  Vsymbol_setter_function = Qnil; /* Set in subr.el */
+
   /* When lexical binding is being used,
    Vinternal_interpreter_environment is non-nil, and contains an alist
    of lexically-bound variable, or (t), indicating an empty
--- src/data.c
+++ src/data.c
@@ -574,6 +574,20 @@
 \f
 /* Extract and set components of symbols.  */
 
+DEFUN ("symbol-hooked-p", Fsymbol_hooked_p, Ssymbol_hooked_p, 1, 1, 0,
+       doc: /* Return t if SYMBOL is hooked.
+To hook and unhook it, use `symbol-hook' and `symbol-unhook'.
+When hooked, setting SYMBOL will run `symbol-setter-function'.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    sym = indirect_variable (sym);
+  return SYM_HOOKED_P (sym) ? Qt : Qnil;
+}
+
 DEFUN ("boundp", Fboundp, Sboundp, 1, 1, 0,
        doc: /* Return t if SYMBOL's value is not void.
 Note that if `lexical-binding' is in effect, this refers to the
@@ -623,6 +637,48 @@
   return NILP (XSYMBOL (symbol)->function) ? Qnil : Qt;
 }
 
+DEFUN ("symbol-hook", Fsymbol_hook, Ssymbol_hook, 1, 1, 0,
+       doc: /* Hook SYMBOL.
+When hooked, setting it will run `symbol-setter-function'.
+To unhook it, use `symbol-unhook'.
+To test whether it's hooked, use `symbol-hooked-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  if (SYMBOL_CONSTANT_P (symbol))
+    xsignal1 (Qsetting_constant, symbol);
+  sym = XSYMBOL (symbol);
+  sym->vetted |= SYM_HOOKED;
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->vetted |= SYM_HOOKED;
+    }
+  return symbol;
+}
+
+DEFUN ("symbol-unhook", Fsymbol_unhook, Ssymbol_unhook, 1, 1, 0,
+       doc: /* Unhook SYMBOL.
+When unhooked, setting it will not run `symbol-setter-function'.
+To hook it, use `symbol-hook'.
+To test whether it's hooked, use `symbol-hooked-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  sym->vetted &= (SYM_HOOKED ^ -1);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->vetted &= (SYM_HOOKED ^ -1);
+    }
+  return symbol;
+}
+
 DEFUN ("makunbound", Fmakunbound, Smakunbound, 1, 1, 0,
        doc: /* Make SYMBOL's value be void.
 Return SYMBOL.  */)
@@ -631,7 +685,7 @@
   CHECK_SYMBOL (symbol);
   if (SYMBOL_CONSTANT_P (symbol))
     xsignal1 (Qsetting_constant, symbol);
-  Fset (symbol, Qunbound);
+  set_internal (symbol, Qunbound, Qnil, false, Dyn_Makvoid);
   return symbol;
 }
 
@@ -1171,8 +1225,8 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
@@ -1201,54 +1255,116 @@
   xsignal1 (Qvoid_variable, symbol);
 }
 
+/* For the symbol S being set, run symbol-setter-function with these arguments:
+   0. S
+   1. A symbol indicating the environment in which S is being set.
+   2. The current value of S in that environment.
+   3. The value to which the setter is attempting to set the variable.
+
+   If argument #2 or #3 is Qunbound, it's replaced by the value of
+   Vvoid_sentinel.
+
+   Return the result of symbol-setter-function, or if it's the value of
+   Vvoid_sentinel and RAWENV is Dyn_Makvoid, return Qunbound; this avoids
+   blocking the setter's call to makunbound. The variable will be set
+   (by code that calls run_varhook) to that return value, overriding
+   the value to which the setter attempted to set the variable.  */
+
+Lisp_Object
+run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Env rawenv,
+	     Lisp_Object oldval, Lisp_Object newval)
+{
+  Lisp_Object symbol;
+  Lisp_Object env;
+  if (rawenv == Dyn_Skip) /* From backtrace_eval_unrewind */
+    return newval;
+  XSETSYMBOL (symbol, sym);
+  switch (rawenv) /* Disambiguate Dyn_Current and Dyn_Global */
+    {
+    case Dyn_Makvoid:
+    case Dyn_Current:
+      {
+	bool shadowed = (buf_local ? let_shadows_buffer_binding_p (sym)
+                         : let_shadows_global_binding_p (symbol));
+	if (shadowed) env = Qdyn_local;
+	else if (buf_local) env = Qbuf_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Global:
+      {
+	/* let_shadows_buffer_binding_p doesn't disambiguate this case */
+	if (let_shadows_global_binding_p (symbol) &&
+	    NILP (Flocal_variable_p (symbol, Qnil)))
+	  env = Qdyn_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Bind: env = Qdyn_bind; break;
+    case Dyn_Unbind: env = Qdyn_unbind; break;
+    default: emacs_abort ();
+    }
+  oldval = EQ (oldval, Qunbound) ? Vvoid_sentinel : oldval;
+  newval = EQ (newval, Qunbound) ? Vvoid_sentinel : newval;
+  newval = call4 (Vsymbol_setter_function, symbol, env, oldval, newval);
+  if (rawenv == Dyn_Makvoid && EQ (newval, Vvoid_sentinel))
+    return Qunbound; /* Converting setq, etc to makunbound is prohibited. */
+  return newval; /* So void_sentinel is ignored except for makunbound. */
+}
+
 DEFUN ("set", Fset, Sset, 2, 2, 0,
        doc: /* Set SYMBOL's value to NEWVAL, and return NEWVAL.  */)
   (register Lisp_Object symbol, Lisp_Object newval)
 {
-  set_internal (symbol, newval, Qnil, 0);
-  return newval;
+  return set_internal (symbol, newval, Qnil, false, Dyn_Current);
 }
 
-/* Store the value NEWVAL into SYMBOL.
-   If buffer/frame-locality is an issue, WHERE specifies which context to use.
-   (nil stands for the current buffer/frame).
-
-   If BINDFLAG is false, then if this symbol is supposed to become
-   local in every buffer where it is set, then we make it local.
-   If BINDFLAG is true, we don't do that.  */
+/* set_internal is in lisp.h due to being inlined.  */
 
-void
-set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
-	      bool bindflag)
-{
-  bool voide = EQ (newval, Qunbound);
-  struct Lisp_Symbol *sym;
-  Lisp_Object tem1;
+/* Factored out from set_internal to avoid inlining the non-hotpath.  */
 
-  /* If restoring in a dead buffer, do nothing.  */
-  /* if (BUFFERP (where) && NILP (XBUFFER (where)->name))
-      return; */
-
-  CHECK_SYMBOL (symbol);
-  if (SYMBOL_CONSTANT_P (symbol))
+Lisp_Object
+set_internal_vetted (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+			     bool bindflag, Dyn_Bind_Env env, struct Lisp_Symbol *sym)
+{
+  if (SYM_HOOKED_P (sym))
     {
+    start:
+      switch (sym->redirect)
+	{
+	case SYMBOL_PLAINVAL:
+	  newval = run_varhook (sym, false, env, sym->val.value, newval);
+	  SET_SYMBOL_VAL (sym, newval);
+	  return newval;
+	case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+	default:
+	  return set_internal_localized_or_forwarded
+	    (symbol, newval, where, bindflag, env, sym);
+	}
+    }
       if (NILP (Fkeywordp (symbol))
 	  || !EQ (newval, Fsymbol_value (symbol)))
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
-	return;
-    }
+	return newval;
+}
 
-  sym = XSYMBOL (symbol);
+/* Split from set_internal to avoid code duplication, because both set_internal and
+   set_internal_vetted must call this function.  */
 
- start:
+Lisp_Object
+set_internal_localized_or_forwarded (Lisp_Object symbol, Lisp_Object newval,
+				     Lisp_Object where, bool bindflag,
+				     Dyn_Bind_Env env, struct Lisp_Symbol *sym)
+{
+  bool voide;
+  Lisp_Object tem1;
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
     case SYMBOL_LOCALIZED:
       {
+	bool buf_local = true;
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 	if (NILP (where))
 	  {
@@ -1292,6 +1408,7 @@
 		   indicating that we're seeing the default value.
 		   Likewise if the variable has been let-bound
 		   in the current buffer.  */
+		buf_local = false;
 		if (bindflag || !blv->local_if_set
 		    || let_shadows_buffer_binding_p (sym))
 		  {
@@ -1319,6 +1436,9 @@
 	    set_blv_valcell (blv, tem1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym, buf_local, env, blv_value (blv), newval);
+	voide = EQ (newval, Qunbound);
+
 	/* Store the new value in the cons cell.  */
 	set_blv_value (blv, newval);
 
@@ -1350,6 +1470,11 @@
 	      SET_PER_BUFFER_VALUE_P (buf, idx, 1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym,
+			   (XFWDTYPE (innercontents)) == Lisp_Fwd_Buffer_Obj,
+			   env, do_symval_forwarding (innercontents), newval);
+	voide = EQ (newval, Qunbound);
+
 	if (voide)
 	  { /* If storing void (making the symbol void), forward only through
 	       buffer-local indicator, not through Lisp_Objfwd, etc.  */
@@ -1362,7 +1487,7 @@
       }
     default: emacs_abort ();
     }
-  return;
+  return newval;
 }
 \f
 /* Access or set a buffer-local symbol's default value.  */
@@ -1381,8 +1506,8 @@
  start:
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
     case SYMBOL_LOCALIZED:
       {
 	/* If var is set up for a buffer that lacks a local value for it,
@@ -1447,6 +1572,16 @@
 for this variable.  */)
   (Lisp_Object symbol, Lisp_Object value)
 {
+  return set_default_internal (symbol, value, Dyn_Global);
+}
+
+/* Like Fset_default, but with ENV argument. See set_internal for
+   a description of this argument. */
+
+Lisp_Object
+set_default_internal (Lisp_Object symbol, Lisp_Object value,
+		      Dyn_Bind_Env env)
+{
   struct Lisp_Symbol *sym;
 
   CHECK_SYMBOL (symbol);
@@ -1464,12 +1599,17 @@
  start:
   switch (sym->redirect)
     {
+    case SYMBOL_PLAINVAL:
+      {
+	return set_internal (symbol, value, Qnil, false, env);
+      }
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: return Fset (symbol, value);
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 
+	MAYBE_RUN_VARHOOK (value, sym, false, env, XCDR (blv->defcell), value);
+
 	/* Store new value into the DEFAULT-VALUE slot.  */
 	XSETCDR (blv->defcell, value);
 
@@ -1490,6 +1630,8 @@
 	    int offset = XBUFFER_OBJFWD (valcontents)->offset;
 	    int idx = PER_BUFFER_IDX (offset);
 
+	    MAYBE_RUN_VARHOOK (value, sym, false, env, per_buffer_default (offset), value);
+
 	    set_per_buffer_default (offset, value);
 
 	    /* If this variable is not always local in all buffers,
@@ -1504,8 +1646,7 @@
 	      }
 	    return value;
 	  }
-	else
-	  return Fset (symbol, value);
+	else return set_internal (symbol, value, Qnil, false, env);
       }
     default: emacs_abort ();
     }
@@ -1536,7 +1677,7 @@
     {
       val = eval_sub (Fcar (XCDR (args_left)));
       symbol = XCAR (args_left);
-      Fset_default (symbol, val);
+      val = Fset_default (symbol, val);
       args_left = Fcdr (XCDR (args_left));
     }
 
@@ -1633,7 +1774,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be buffer-local", SDATA (SYMBOL_NAME (variable)));
 
   if (!blv)
@@ -1706,7 +1847,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be buffer-local",
 	   SDATA (SYMBOL_NAME (variable)));
 
@@ -1895,7 +2036,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));
 
   blv = make_blv (sym, forwarded, valcontents);
@@ -3474,6 +3615,12 @@
   DEFSYM (Qad_advice_info, "ad-advice-info");
   DEFSYM (Qad_activate_internal, "ad-activate-internal");
 
+  DEFSYM (Qglobal, "global");
+  DEFSYM (Qbuf_local, "buf-local");
+  DEFSYM (Qdyn_local, "dyn-local");
+  DEFSYM (Qdyn_bind, "dyn-bind");
+  DEFSYM (Qdyn_unbind, "dyn-unbind");
+
   error_tail = pure_cons (Qerror, Qnil);
 
   /* ERROR is used as a signaler for random errors for which nothing else is
@@ -3609,8 +3756,11 @@
   defsubr (&Sindirect_function);
   defsubr (&Ssymbol_plist);
   defsubr (&Ssymbol_name);
+  defsubr (&Ssymbol_hook);
+  defsubr (&Ssymbol_unhook);
   defsubr (&Smakunbound);
   defsubr (&Sfmakunbound);
+  defsubr (&Ssymbol_hooked_p);
   defsubr (&Sboundp);
   defsubr (&Sfboundp);
   defsubr (&Sfset);
@@ -3677,10 +3827,10 @@
   DEFVAR_LISP ("most-positive-fixnum", Vmost_positive_fixnum,
 	       doc: /* The largest value that is representable in a Lisp integer.  */);
   Vmost_positive_fixnum = make_number (MOST_POSITIVE_FIXNUM);
-  XSYMBOL (intern_c_string ("most-positive-fixnum"))->constant = 1;
+  XSYMBOL (intern_c_string ("most-positive-fixnum"))->vetted = SYM_CONST;
 
   DEFVAR_LISP ("most-negative-fixnum", Vmost_negative_fixnum,
 	       doc: /* The smallest value that is representable in a Lisp integer.  */);
   Vmost_negative_fixnum = make_number (MOST_NEGATIVE_FIXNUM);
-  XSYMBOL (intern_c_string ("most-negative-fixnum"))->constant = 1;
+  XSYMBOL (intern_c_string ("most-negative-fixnum"))->vetted = SYM_CONST;
 }
--- src/alloc.c
+++ src/alloc.c
@@ -3350,7 +3350,7 @@
   set_symbol_next (val, NULL);
   p->gcmarkbit = false;
   p->interned = SYMBOL_UNINTERNED;
-  p->constant = 0;
+  p->vetted = 0;
   p->declared_special = false;
   p->pinned = false;
 }
--- src/lread.c
+++ src/lread.c
@@ -3755,7 +3755,7 @@
 
   if (SREF (SYMBOL_NAME (sym), 0) == ':' && EQ (obarray, initial_obarray))
     {
-      XSYMBOL (sym)->constant = 1;
+      XSYMBOL (sym)->vetted = SYM_CONST;
       XSYMBOL (sym)->redirect = SYMBOL_PLAINVAL;
       SET_SYMBOL_VAL (XSYMBOL (sym), sym);
     }
@@ -4041,12 +4041,12 @@
 
   DEFSYM (Qnil, "nil");
   SET_SYMBOL_VAL (XSYMBOL (Qnil), Qnil);
-  XSYMBOL (Qnil)->constant = 1;
+  XSYMBOL (Qnil)->vetted = SYM_CONST;
   XSYMBOL (Qnil)->declared_special = true;
 
   DEFSYM (Qt, "t");
   SET_SYMBOL_VAL (XSYMBOL (Qt), Qt);
-  XSYMBOL (Qt)->constant = 1;
+  XSYMBOL (Qt)->vetted = SYM_CONST;
   XSYMBOL (Qt)->declared_special = true;
 
   /* Qt is correct even if CANNOT_DUMP.  loadup.el will set to nil at end.  */
--- src/buffer.c
+++ src/buffer.c
@@ -5690,7 +5690,7 @@
 This variable is buffer-local but you cannot set it directly;
 use the function `set-buffer-multibyte' to change a buffer's representation.
 See also Info node `(elisp)Text Representations'.  */);
-  XSYMBOL (intern_c_string ("enable-multibyte-characters"))->constant = 1;
+  XSYMBOL (intern_c_string ("enable-multibyte-characters"))->vetted = SYM_CONST;
 
   DEFVAR_PER_BUFFER ("buffer-file-coding-system",
 		     &BVAR (current_buffer, buffer_file_coding_system), Qnil,
--- src/bytecode.c
+++ src/bytecode.c
@@ -843,7 +843,7 @@
 	    else
 	      {
 		BEFORE_POTENTIAL_GC ();
-		set_internal (sym, val, Qnil, 0);
+		set_internal (sym, val, Qnil, false, Dyn_Current);
 		AFTER_POTENTIAL_GC ();
 	      }
 	  }
--- src/font.c
+++ src/font.c
@@ -5249,19 +5249,19 @@
     [NUMERIC-VALUE SYMBOLIC-NAME ALIAS-NAME ...]
 NUMERIC-VALUE is an integer, and SYMBOLIC-NAME and ALIAS-NAME are symbols. */);
   Vfont_weight_table = BUILD_STYLE_TABLE (weight_table);
-  XSYMBOL (intern_c_string ("font-weight-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-weight-table"))->vetted = SYM_CONST;
 
   DEFVAR_LISP_NOPRO ("font-slant-table", Vfont_slant_table,
 	       doc: /*  Vector of font slant symbols vs the corresponding numeric values.
 See `font-weight-table' for the format of the vector. */);
   Vfont_slant_table = BUILD_STYLE_TABLE (slant_table);
-  XSYMBOL (intern_c_string ("font-slant-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-slant-table"))->vetted = SYM_CONST;
 
   DEFVAR_LISP_NOPRO ("font-width-table", Vfont_width_table,
 	       doc: /*  Alist of font width symbols vs the corresponding numeric values.
 See `font-weight-table' for the format of the vector. */);
   Vfont_width_table = BUILD_STYLE_TABLE (width_table);
-  XSYMBOL (intern_c_string ("font-width-table"))->constant = 1;
+  XSYMBOL (intern_c_string ("font-width-table"))->vetted = SYM_CONST;
 
   staticpro (&font_style_table);
   font_style_table = make_uninit_vector (3);
--- lisp/subr.el
+++ lisp/subr.el
@@ -2546,6 +2546,9 @@
 Note that this should end with a directory separator.
 See also `locate-user-emacs-file'.")
 \f
+(setq symbol-setter-function ; Defined in eval.c
+      (lambda (_sym _env _oldval newval) newval))
+\f
 ;;;; Misc. useful functions.
 
 (defsubst buffer-narrowed-p ()

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> blocking/overriding the variable setting: if you hook a variable and
> override its setting, then the return value of set, setq, etc should be the
> override value instead of the originally attempted value, so that the
> override will cascade through e.g. (setq x (setq y foo)) and (if (setq
> z foo) ...).

I think either behavior is equally correct and wrong (sometimes you'd
rather want one and sometimes you'd rather want the other), so I'd
rather choose based on which of the two is more efficient/easy to
implement (and if it's not on the critical path (i.e. the path used when
the variable is not hooked), then efficiency doesn't matter).

In either case, `setq' should never return Qunbound.

> Any other changes I should make?

Well, now that you ask:

Regarding one of the bikesheds, I think "constant" is a better name than
"vetted" because the name really doesn't matter that much, "vetted" is
not very self-explanatory, and "constant" has the major advantage of
making the patch more readable by eliminating "irrelevant" changes.
We can change the name later to something actually better, but for now,
let's just stick to "constant".

> +/* These are the masks for the vetted field of Lisp_Symbol.
> +   Bit 0 stores the constant field.  Bit 1 stores the hooked field.  */
> +#define SYM_CONST 1
> +#define SYM_HOOKED 2

constant and hooked at the same time is nonsensical, so these aren't two
independent bits, instead `vetted' is a 3-valued field.  An `enum'
sounds right.

> -static void grow_specpdl (void);
> +static inline void grow_specpdl (void);

What happens if we don't inline grow_specpdl?

> +      if (!sym->vetted) SET_SYMBOL_VAL (sym, value);
> +      else if (SYM_HOOKED_P (sym))

Please don't put the "then" branch on the same line as the test, this is
contrary to our coding conventions.

> +    case SYMBOL_VARALIAS:
> +      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;

Why did you have to move this?

> +	      set_internal (symbol, old_value, where, true, Dyn_Unbind);

If using "unbind" when referring to "undoing a let-binding" bothers you,
you could use "Dyn_Unwind" to mean that it's set as part of unwinding
the stack of bindings.  Personally I have no problem with using "unbind"
sometimes for "makunbound" and sometimes for "undo a let".

> +  DEFSYM (Qvoid_sentinel, "void-sentinel");
> +  DEFVAR_LISP ("void-sentinel", Vvoid_sentinel,
> +	       doc: /* Representation of voidness for hooked variables.
> +The value of this constant is an uninterned Lisp symbol that represents void
> +when passed to or returned from `symbol-setter-function'.  */);
> +  Vvoid_sentinel = Fmake_symbol (build_string ("::void::"));

I'd give it a more verbose name, to make sure it doesn't clash with
someone else's var.  After all, it's not like this is going to be used
very often, so the length doesn't matter.  `symbol-setter-void-value'
sounds fine.

> +  XSYMBOL (Vvoid_sentinel)->declared_special = true;
> +  XSYMBOL (Vvoid_sentinel)->vetted = SYM_CONST;

This value is just a symbol, not a variable, so there's no point marking
it as special and/or constant.

> +buf-local: The setter's buffer-local environment.
> +dyn-local: The innermost dynamic environment in which SYMBOL is bound.
> +dyn-bind: A new dynamic environment, such as creatable using `let'.

I think this needs clarification, because when I read it, I don't know
for sure which one I'd get in which case.

I think the info I'd want to get is:
- does it affect only the current buffer, or does it affect the default-value?
- is it a "set", or a "bind" or an "unbind"?
Note that those 2 aspects are orthogonal, so maybe we'd want 2 args
rather than 1.

> +If you use overriding advice, your advice must return the value to which to
> +set the variable.

Don't talk about advice here, just talk about what the function needs
to do.  After all, it may be modified without using advice.

> +If OLDVAL
> +is the value of void-sentinel but NEWVAL is not, you can override the new
> +value, but you can't prevent the variable from being set to a non-void value.

Why not?

> +To hook all variables of a symbol, use `symbol-hook'.

"all variables of a symbol" sounds really odd.  Whether there are
several variables for a single symbol is really a philosophical
question, and usually people seem to prefer thinking that it's not
the case.

>+  DEFVAR_LISP ("symbol-setter-function", Vsymbol_setter_function,
> +DEFUN ("symbol-hooked-p", Fsymbol_hooked_p, Ssymbol_hooked_p, 1, 1, 0,
> +DEFUN ("symbol-hook", Fsymbol_hook, Ssymbol_hook, 1, 1, 0,
> +DEFUN ("symbol-unhook", Fsymbol_unhook, Ssymbol_unhook, 1, 1, 0,

I'd much prefer to keep all the new functions/vars of this feature under
a common (and new) prefix.  I'll let you choose whether it should be
`symbol-setter-', `symbol-watcher-`, `symbol-hook-', or whatnot.

> -    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
>      case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
> +    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;

Why was this change needed?

> +  if (rawenv == Dyn_Makvoid && EQ (newval, Vvoid_sentinel))
> +    return Qunbound; /* Converting setq, etc to makunbound is prohibited. */
> +  return newval; /* So void_sentinel is ignored except for makunbound. */

Why disallow Qunbound for all but makunbound?
I understand that it might be strange to return Qunbound in the `setq'
case, but I'm not sure it's worth the trouble trying to disallow it.

And it seems positively wrong in the case of unwinding the topmost let
binding of a variable, so even if we do want to disallow it, the test
shouldn't be "rawenv == Dyn_Makvoid" but "orig_newval == Qunbound".
So, I think we don't need Dyn_Makvoid (hence we don't need to touch
Fmakunbound).

> -  set_internal (symbol, newval, Qnil, 0);
> -  return newval;
> +  return set_internal (symbol, newval, Qnil, false, Dyn_Current);

Since writing the above, I think I'm beginning to like the idea of
(setq <var> <val>) always returning <val> even if <var> ends up set to
something else.  After all, that's what we've been doing so far.

Try M-: (list (setq auto-hscroll-mode 2) auto-hscroll-mode) RET
 
> -    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
>      case SYMBOL_PLAINVAL: return SYMBOL_VAL (sym);
> +    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;

Hmm... same kind of change here.  I must be missing something.

> -  if (sym->constant)
> +  if (SYM_CONST_P (sym))
>      error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));

I think we can keep sym->constant here.  If things don't work 100% for
frame-local vars, it's a good thing because we want to inflict pain on
the few rare remaining users of frame-local.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> I think either behavior is equally correct and wrong (sometimes you'd
> rather want one and sometimes you'd rather want the other), so I'd
> rather choose based on which of the two is more efficient/easy to
> implement (and if it's not on the critical path (i.e. the path used when
> the variable is not hooked), then efficiency doesn't matter).

It's not on the critical path, and they're equally efficient and easy to implement. Returning the override value results in a few less lines of code (thus smaller patch), since in some places I don't have to put return statements on separate lines from the calls to run_varhook, and in some places that lets me eliminate a pair of curly brackets too.

> In either case, `setq' should never return Qunbound.

Right. It doesn't.

> Regarding one of the bikesheds,
[snip]
> We can change the name later to something actually better, but for now,
> let's just stick to "constant".

Ok, I'll change it back.

>> +#define SYM_CONST 1
>> +#define SYM_HOOKED 2
> constant and hooked at the same time is nonsensical, so these aren't two
> independent bits, instead `vetted' is a 3-valued field.  An `enum'
> sounds right.

The compiler will encode the enum into Lisp_Symbol's bitfield, so that will produce no change in the compiled code. And SYM_CONST and SYM_HOOKED are better than sprinkling the code with 1 and 2 as magic numbers, for the same reason that «false» and «true» are better than 0 and 1 for booleans.

But I'll put those constants into an enum.

Note that those constant definitions have been there ever since I originally combined the constant and hooked fields in the patch I submitted on Feb 9th.

>> -static void grow_specpdl (void);
>> +static inline void grow_specpdl (void);
>
> What happens if we don't inline grow_specpdl?

That's just an optimization since it's in the critical path of specbind, as I noted in my message when I made this change on Feb 9th. The result is a small but measurable performance improvement, at the cost of increasing the executable's size by a few kB (i.e. less than a tenth of a percent, since Emacs is nearly 20MB already).

If that's an undesirable tradeoff, I'll remove the inline.

>> +    case SYMBOL_VARALIAS:
>> +      sym = indirect_variable (sym); XSETSYMBOL (symbol, sym); goto start;
>
> Why did you have to move this?
[snip]
> Hmm... same kind of change here.  I must be missing something.

I didn't have to. It's just a minor optimization to avoid an extra conditional branch before the common case (SYMBOL_PLAINVAL) on the critical path, in case the compiler implements the switch statement as a series of «if», «else if», «else if» statements. I decided it was worth bothering to make this change because your original reason for wanting to combine the constant and hooked fields was just to avoid an extra conditional branch on this same critical path in set_internal.

Same thing for the other two places (specbind and find_symbol_value) where I made this same change.

These have been here since Feb 9th, and on Feb 7th I explained why.

>> +	      set_internal (symbol, old_value, where, true, Dyn_Unbind);
>
> If using "unbind" when referring to "undoing a let-binding" bothers you,

That doesn't bother me. ‟Unbind” is the right term for that. What bothers me is saying that a variable is ‟unbound” when it's actually bound to void and the binding shadows a non-void binding.

> Personally I have no problem with using "unbind"
> sometimes for "makunbound" and sometimes for "undo a let".

Neither would I, if void could never shadow non-void.

> I'd give it a more verbose name, to make sure it doesn't clash with
> someone else's var.  After all, it's not like this is going to be used
> very often, so the length doesn't matter.  `symbol-setter-void-value'
> sounds fine.

Ok, I'll change it.

>> +  XSYMBOL (Vvoid_sentinel)->declared_special = true;
>> +  XSYMBOL (Vvoid_sentinel)->vetted = SYM_CONST;
>
> This value is just a symbol, not a variable, so there's no point marking
> it as special and/or constant.

If the user does something stupid such as (set void-sentinel 'foo), it might as well barf. Especially since the print name looks like a keyword. Thus marked constant.

I don't see why mark it special, but t and nil are marked special, so I decided to do this for the sake of consistency, in case there was some reason constants should be marked special.

>> +buf-local: The setter's buffer-local environment.
>> +dyn-local: The innermost dynamic environment in which SYMBOL is bound.
>> +dyn-bind: A new dynamic environment, such as creatable using `let'.
>
> I think this needs clarification, because when I read it, I don't know
> for sure which one I'd get in which case.

Ok, I'll elaborate.

>> +If OLDVAL
>> +is the value of void-sentinel but NEWVAL is not, you can override the new
>> +value, but you can't prevent the variable from being set to a non-void value.
>
> Why not?

Because if you could, then as I explained in my previous message, you could convert a setq into a makunbound, and since I changed setq to return the override value, that means setq could return Qunbound, which is something it should never do.

>> +To hook all variables of a symbol, use `symbol-hook'.
>
> "all variables of a symbol" sounds really odd.

Ok, I'll change it.

> Whether there are
> several variables for a single symbol is really a philosophical
> question,

Until you add multithreading to Emacs. ;-)

> I'd much prefer to keep all the new functions/vars of this feature under
> a common (and new) prefix.  I'll let you choose whether it should be
> `symbol-setter-', `symbol-watcher-`, `symbol-hook-', or whatnot.

Ok, I'll change it.

> Why disallow Qunbound for all but makunbound?
> I understand that it might be strange to return Qunbound in the `setq'
> case, but I'm not sure it's worth the trouble trying to disallow it.

Because I changed setq to return the override value. And as I noted, it also has the fortunate side effect of making (setq x void-sentinel) behave the same regardless of whether x is hooked. IOW, technically, allowing Qunbound for setq would be a bug (even if setq returned its argument, instead of the override value), because hooking a symbol must not change the behavior of Emacs.

> And it seems positively wrong in the case of unwinding the topmost let
> binding of a variable, so even if we do want to disallow it, the test
> shouldn't be "rawenv == Dyn_Makvoid" but "orig_newval == Qunbound".
> So, I think we don't need Dyn_Makvoid (hence we don't need to touch
> Fmakunbound).

Yup, that's a bug. Oops.

> Since writing the above, I think I'm beginning to like the idea of
> (setq <var> <val>) always returning <val> even if <var> ends up set to
> something else.

It would be annoying in the case of (if (setq x y) ...), if you override the setting of x but the «if» doesn't respect your choice.

> Try M-: (list (setq auto-hscroll-mode 2) auto-hscroll-mode) RET

Whether or not hooked, you get (2 t). If hooked, and you override to 3, you get (3 t), not (t t), because the return value of setq is the override value; setq doesn't set the variable and then read it back.

>> -  if (sym->constant)
>> +  if (SYM_CONST_P (sym))
>>      error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));
>
> I think we can keep sym->constant here.

That would be a bug, since it would signal an error if the symbol is hooked. Hooking a symbol must not change the behavior of Emacs.

> we want to inflict pain on
> the few rare remaining users of frame-local.

Just deleting all the code for frame-local would accomplish that more effectively.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

>>> +#define SYM_CONST 1
>>> +#define SYM_HOOKED 2
>> constant and hooked at the same time is nonsensical, so these aren't two
>> independent bits, instead `vetted' is a 3-valued field.  An `enum'
>> sounds right.
> The compiler will encode the enum into Lisp_Symbol's bitfield, so that will
> produce no change in the compiled code.

I know.  My objection is mostly against the comment(s).
They're basically a left over from when you had 2 separate fields but
are out-of-date w.r.t the current code.

> And SYM_CONST and SYM_HOOKED are better than sprinkling the code with
> 1 and 2 as magic numbers, for the same reason that «false» and «true»
> are better than 0 and 1 for booleans.

I definitely don't want magic constants, which is why I said "An `enum'
sounds right".

> Note that those constant definitions have been there ever since I originally
> combined the constant and hooked fields in the patch I submitted on Feb 9th.

I know.

>>> -static void grow_specpdl (void);
>>> +static inline void grow_specpdl (void);
>> What happens if we don't inline grow_specpdl?
> That's just an optimization since it's in the critical path of specbind, as
> I noted in my message when I made this change on Feb 9th.

But that's orthogonal to the "variable hook" functionality, right?
So I think we can keep this for some later changes.

> I didn't have to. It's just a minor optimization to avoid an extra
> conditional branch before the common case (SYMBOL_PLAINVAL) on the critical

Ah, you're moving the expected common case to the first position.
That sounds OK.  But maybe we can keep this for some later changes, tho.

> Neither would I, if void could never shadow non-void.

It definitely can.  It almost never does, tho.

> I don't see why mark it special, but t and nil are marked special, so
> I decided to do this for the sake of consistency, in case there was some
> reason constants should be marked special.

I'm not sure exactly why t and nil are marked special, but the effect it
has is to prevent their use a lexical vars, and hence prevent their use
as let-bound vars altogether.

The "constant" bit prevents a dynamically scoped (let ((t <foo>)) ...)
while the "special" bit prevents a lexically scoped (let ((t <foo>)) ...).

The first would be dangerous, while the second would be perfectly safe,
but it might be a bit confusing for the programmer who expects t and nil
to always refer to the special predefined constants.

I can't imagine how a chunk of code you try to let-bind the symbol
that's the void-value, marking it "special" is pretty paranoid.

> Because if you could, then as I explained in my previous message, you could
> convert a setq into a makunbound,

I don't think that's a serious problem.

> and since I changed setq to return the override value,

By now, I think I've convinced myself that this an error.

> It would be annoying in the case of (if (setq x y) ...), if you override the
> setting of x but the «if» doesn't respect your choice.

It all depends on what you mean by "override the setting".  In the above
code, the usual intention is "compute y, test the result, and remember
it in x for later reuse".  If the setter function decides to put some
other value into `x', that doesn't mean that the test should also return
another value.

Here's another case:

   (setq x (setq y <foo>))
vs
   (setq y (setq x <foo>))

if you hook `x' and change the value to which it's set, do you really
think the programmer expect the above two expressions to
behave differently?

>> Try M-: (list (setq auto-hscroll-mode 2) auto-hscroll-mode) RET
> Whether or not hooked, you get (2 t).

That's not the point: the `setq' returned the value passed to it, and
not the value to which the variable was set.  This is not done with your
variable hooks, but by some pre-existing code instead, but the principle
is the same.  And in that pre-existing similar situation we made the
opposite choice to the one you're making now.

>>> -  if (sym->constant)
>>> +  if (SYM_CONST_P (sym))
>>> error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));
>> I think we can keep sym->constant here.
> That would be a bug, since it would signal an error if the symbol is
> hooked. Hooking a symbol must not change the behavior of Emacs.

No, as I said, I'm happy to introduce such "bugs" for frame-local vars.

>> we want to inflict pain on the few rare remaining users of
>> frame-local.
> Just deleting all the code for frame-local would accomplish that
> more effectively.

I've done that in my local branch, and every new release reduces the
support of frame-local vars to some extent.  I haven't thought enough
about when we should finally remove support for them altogether (maybe
25.1 is a bit early still), but in the mean time, there's no reason to
support interaction between them and new features.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

Stefan Monnier wrote:
> I definitely don't want magic constants, which is why I said "An `enum'
> sounds right".

If no magic constants, then all the places in trunk that have ⌜->constant = 1⌝ must change to ⌜->constant = SYM_CONST⌝ (and my patch already does this, but it changes the field name too). That's 9 lines of code.

By retaining the name ⌜constant⌝ for the field name, I can avoid touching 11 lines of code. But 9 of those are the aforementioned lines, which means if magic constants aren't allowed, then retaining the name ⌜constant⌝ saves changing just two lines of code. That's a negligible reduction in the size of the patch, at the cost of retaining a field name that would no longer be appropriate.

Leaving all 11 lines unchanged would leave us with both an inappropriate field name and magic constants, and still be only a small reduction in the size of the patch.

>>> What happens if we don't inline grow_specpdl?
>> That's just an optimization since it's in the critical path of specbind, as
>> I noted in my message when I made this change on Feb 9th.
>
> But that's orthogonal to the "variable hook" functionality, right?
> So I think we can keep this for some later changes.

You originally objected to my patch slowing down Emacs. So I looked for optimization opportunities to ensure that my patch paid for its performance costs. I was successful, and not only did I offset the costs, I even produced a net improvement in speed, and provided benchmarks to prove it.

If I remove my optimizations, then my patch will slow down Emacs. And then you can once again object to the performance impact.

And the above minor optimization is a trivial change in just two lines of code, so removing it would produce a negligible reduction in the size of the patch anyway. Removing the even-more-minor optimization that you originally insisted on (i.e. combining the constant and hooked fields to save a conditional branch) would produce a much larger reduction in the size of the patch, so if you're so worried about the patch size, how about we remove the constant-hooked combination? My other optimizations, which touch fewer lines, actually improve the speed more than the constant-hooked combination does.

>> I didn't have to. It's just a minor optimization to avoid an extra
>> conditional branch before the common case (SYMBOL_PLAINVAL) on the critical
>
> Ah, you're moving the expected common case to the first position.
> That sounds OK.  But maybe we can keep this for some later changes, tho.

Same here. Just three occurrences of moving one line of code. And I have to move one of those lines anyway, as part of the process of combining the constant and hooked fields into one field.

> I can't imagine how a chunk of code you try to let-bind the symbol
> that's the void-value, marking it "special" is pretty paranoid.

Ok, I'll un-mark it as special.

>> Because if you could, then as I explained in my previous message, you could
>> convert a setq into a makunbound,
>
> I don't think that's a serious problem.

The tradeoff is providing a new capability (not previously present in Emacs) that's completely unnecessary (even for debugging) and pathological, versus strictly abiding by the rule that hooking a symbol must not change the behavior of Emacs. Either option is equally easy to implement, and the latter is obviously the right one.

>> and since I changed setq to return the override value,
>
> By now, I think I've convinced myself that this an error.

Ok, I'll change it back. Note that changing it back doesn't require enabling conversion of setq into makunbound, so I'll still prevent the latter, for the sake of correctness.

> No, as I said, I'm happy to introduce such "bugs" for frame-local vars.

Ok, if the alternative to intentionally introducing a bug is that my patch gets rejected, then I'll introduce the bug. But I'll add a comment that it's an intentional bug.

Re: [PATCH] (Updated) Run hook when variable is set

[Stephen J. Turnbull]

Kelly Dean writes:

 > > So I think we can keep this for some later changes.
 > 
 > You originally objected to my patch slowing down Emacs.

I think what Stefan is suggesting is factoring your patch into a
series of patches (ie, a branch) so that each thematic change can be
reviewed separately.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

>> But that's orthogonal to the "variable hook" functionality, right?
>> So I think we can keep this for some later changes.
> You originally objected to my patch slowing down Emacs.  So I looked for
> optimization opportunities to ensure that my patch paid for its performance
> costs.  I was successful, and not only did I offset the costs, I even
> produced a net improvement in speed, and provided benchmarks to prove it.

I'm sorry, but I consider this a form of lying.  It makes it sound like
"the new var-hook functionality actually speeds things up", even though
it's not this new functionality but some unrelated (tho bundled) change
which does it (and which could be applied independently).

> If I remove my optimizations, then my patch will slow down Emacs.
> And then you can once again object to the performance impact.

I'm quite willing to have Emacs be slightly slower for this new functionality.
I just want to reduce this speed impact to the minimum.

Offsetting this speed impact by adding unrelated optimization just hides
the speed impact.

> of the patch, so if you're so worried about the patch size, how about we

The issue is not patch size per se, but just keeping the patch focused
on its core purpose.

Those optimizations you found are good changes.  I'm just asking you to
put them in a separate patch.

>>> Because if you could, then as I explained in my previous message, you could
>>> convert a setq into a makunbound,
>> I don't think that's a serious problem.
> The tradeoff is providing a new capability (not previously present in Emacs)
> that's completely unnecessary (even for debugging) and pathological,

"unnecessary" and "pathological" are judgments which are actually hard
to make for such a generic hooking functionality where we don't (want
to) know what the applications will be.

As a general design principle Emacs doesn't really try to prevent you
from shooting yourself in the foot.

> Ok, I'll change it back.

Thanks,


        Stefan

[PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

[-- Attachment #1: Type: text/plain, Size: 3616 bytes --]

Stefan Monnier wrote:
>> You originally objected to my patch slowing down Emacs.  So I looked for
>> optimization opportunities to ensure that my patch paid for its performance
>> costs.  I was successful, and not only did I offset the costs, I even
>> produced a net improvement in speed, and provided benchmarks to prove it.
>
> I'm sorry, but I consider this a form of lying.  It makes it sound like
> "the new var-hook functionality actually speeds things up", even though
> it's not this new functionality but some unrelated (tho bundled) change
> which does it (and which could be applied independently).

I didn't say the varhook functionality speeds things up. I explicitly said it has performance costs, and I found optimization opportunities to _pay for_ those costs.

And again, I didn't say I eliminated the costs; I said I _offset_ the costs, and did better than just break even.

I was completely honest about what I did, and why I did it. And you accuse me of lying?

> The issue is not patch size per se, but just keeping the patch focused
> on its core purpose.

The constant-hooked combination is an optimization, not part of the patch's core purpose, yet you told me to include it anyway. Varhook works just fine without that optimization (and the patch is simpler without it), as shown by the varhook-single.patch I submitted on Feb 5th.

The patch could be applied to trunk without that optimization, then the optimization applied later, along with all the rest, without affecting the functionality of Emacs in general or of the varhook feature in particular.

But I'm leaving it in the patch, just because you asked me to put it there. I don't want to bother to take it back out now, and IIUC, you still want me to leave it in.

> "unnecessary" and "pathological" are judgments which are actually hard
> to make for such a generic hooking functionality where we don't (want
> to) know what the applications will be.
>
> As a general design principle Emacs doesn't really try to prevent you
> from shooting yourself in the foot.

Fine, but adding the capability of converting setq, etc into makunbound has the cost of breaking correctness (because hooking a symbol would change the behavior of (setq foo void-sentinel)), not just the cost of enabling user errors. Remember, this capability isn't something I'm removing from Emacs; it's just an additional capability that varhook could add. The benefit isn't worth the cost.


Updated patch attached. Changes, as you requested:
setq, etc return the attempt value instead of the override value.
One bug fixed.
Another bug intentionally added.
The void sentinel value is un-marked as special, and the constant is renamed.
Function names changed yet again. Now they all start with ⌜symbol-hook⌝.
All mention of «advice» banished from the documentation.
The documentation is more explicit about the cases in which each environment is affected.
As much as possible without making the documentation incomprehensible, it now conflates symbols with global variables.
The field name in Lisp_Symbol changed back to the misleading name, just to avoid touching a few lines of code in the patch.
Magic constants added back in to the source code, just to avoid touching a few lines of code in the patch.
All optimizations removed, except for the particular one (combining constant and hooked) that you told me to include, which happens to be the most invasive one, yet with no more benefit than the others.

This version of the patch slows down Emacs.

IIUC, I've made all the changes you requested.


[-- Attachment #2: varhook-bikeshedded.patch --]
[-- Type: text/x-diff, Size: 23994 bytes --]

--- src/lisp.h
+++ src/lisp.h
@@ -290,6 +290,17 @@
 # define GCALIGNED /* empty */
 #endif
 
+enum symbol_constant
+{
+  SYM_UNVETTED = 0,
+  SYM_CONST = 1,
+  SYM_HOOKED = 2
+};
+
+# define SYM_CONST_P(sym) (((sym)->constant) == SYM_CONST)
+# define SYM_HOOKED_P(sym) (((sym)->constant) == SYM_HOOKED)
+
+
 /* Some operations are so commonly executed that they are implemented
    as macros, not functions, because otherwise runtime performance would
    suffer too much when compiling with GCC without optimization.
@@ -344,7 +355,7 @@
 #define lisp_h_NILP(x) EQ (x, Qnil)
 #define lisp_h_SET_SYMBOL_VAL(sym, v) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value = (v))
-#define lisp_h_SYMBOL_CONSTANT_P(sym) (XSYMBOL (sym)->constant)
+#define lisp_h_SYMBOL_CONSTANT_P(sym) (SYM_CONST_P (XSYMBOL (sym)))
 #define lisp_h_SYMBOL_VAL(sym) \
    (eassert ((sym)->redirect == SYMBOL_PLAINVAL), (sym)->val.value)
 #define lisp_h_SYMBOLP(x) (XTYPE (x) == Lisp_Symbol)
@@ -659,10 +670,12 @@
      3 : it's a forwarding variable, the value is in `forward'.  */
   ENUM_BF (symbol_redirect) redirect : 3;
 
-  /* Non-zero means symbol is constant, i.e. changing its value
-     should signal an error.  If the value is 3, then the var
-     can be changed, but only by `defconst'.  */
-  unsigned constant : 2;
+  /* SYM_CONST means symbol is constant, i.e. changing its value should signal
+     an error. SYM_HOOKED means setting symbol will run varhook. These two
+     attributes are combined into one field to optimize the fast path of
+     non-hooked non-constants by having only one conditional branch for that
+     case. The name of this field is ⌜constant⌝ for historical reasons.  */
+  ENUM_BF (symbol_constant) constant : 2;
 
   /* Interned state of the symbol.  This is an enumerator from
      enum symbol_interned.  */
@@ -3463,6 +3476,14 @@
 }
 
 /* Defined in data.c.  */
+typedef enum
+  { /* See set_internal for a description of these values.  */
+    Dyn_Unbind = -1,
+    Dyn_Current = 0,
+    Dyn_Bind = 1,
+    Dyn_Skip = 2,
+    Dyn_Global = 3
+  } Dyn_Bind_Env;
 extern Lisp_Object indirect_function (Lisp_Object);
 extern Lisp_Object find_symbol_value (Lisp_Object);
 enum Arith_Comparison {
@@ -3509,7 +3530,16 @@
 extern _Noreturn void args_out_of_range_3 (Lisp_Object, Lisp_Object,
 					   Lisp_Object);
 extern Lisp_Object do_symval_forwarding (union Lisp_Fwd *);
-extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool);
+extern void set_internal (Lisp_Object, Lisp_Object, Lisp_Object, bool, Dyn_Bind_Env);
+extern Lisp_Object run_varhook (struct Lisp_Symbol*, bool, Dyn_Bind_Env,
+				Lisp_Object, Lisp_Object);
+extern void set_internal_vetted (Lisp_Object, Lisp_Object, Lisp_Object, bool,
+					 Dyn_Bind_Env, struct Lisp_Symbol *);
+extern void set_internal_localized_or_forwarded (Lisp_Object, Lisp_Object,
+						 Lisp_Object, bool,
+						 Dyn_Bind_Env,
+						 struct Lisp_Symbol *);
+extern void set_default_internal (Lisp_Object, Lisp_Object, Dyn_Bind_Env);
 extern void syms_of_data (void);
 extern void swap_in_global_binding (struct Lisp_Symbol *);
 
@@ -4776,6 +4806,12 @@
     return false;
 }
 
+#define MAYBE_RUN_VARHOOK(result, sym, buf_local, env, oldval, newval)	\
+  {									\
+    if (SYM_HOOKED_P (sym))						\
+      (result) = run_varhook (sym, buf_local, env, oldval, newval);	\
+  }
+
 INLINE_HEADER_END
 
 #endif /* EMACS_LISP_H */
--- src/eval.c
+++ src/eval.c
@@ -616,7 +616,7 @@
 
   sym = XSYMBOL (new_alias);
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     /* Not sure why, but why not?  */
     error ("Cannot make a constant an alias");
 
@@ -633,7 +633,7 @@
      so that old-code that affects n_a before the aliasing is setup
      still works.  */
   if (NILP (Fboundp (base_variable)))
-    set_internal (base_variable, find_symbol_value (new_alias), Qnil, 1);
+    set_internal (base_variable, find_symbol_value (new_alias), Qnil, true, Dyn_Current);
 
   {
     union specbinding *p;
@@ -3049,8 +3049,11 @@
       grow_specpdl ();
       if (!sym->constant)
 	SET_SYMBOL_VAL (sym, value);
+      else if (SYM_HOOKED_P (sym))
+	SET_SYMBOL_VAL (sym, run_varhook
+			(sym, false, Dyn_Bind, sym->val.value, value));
       else
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, true, Dyn_Bind);
       break;
     case SYMBOL_LOCALIZED:
       if (SYMBOL_BLV (sym)->frame_local)
@@ -3082,7 +3085,7 @@
 	      {
 		specpdl_ptr->let.kind = SPECPDL_LET_DEFAULT;
 		grow_specpdl ();
-		Fset_default (symbol, value);
+		set_default_internal (symbol, value, Dyn_Bind);
 		return;
 	      }
 	  }
@@ -3090,7 +3093,7 @@
 	  specpdl_ptr->let.kind = SPECPDL_LET;
 
 	grow_specpdl ();
-	set_internal (symbol, value, Qnil, 1);
+	set_internal (symbol, value, Qnil, true, Dyn_Bind);
 	break;
       }
     default: emacs_abort ();
@@ -3225,7 +3228,9 @@
 	    struct Lisp_Symbol *sym = XSYMBOL (specpdl_symbol (specpdl_ptr));
 	    if (sym->redirect == SYMBOL_PLAINVAL)
 	      {
-		SET_SYMBOL_VAL (sym, specpdl_old_value (specpdl_ptr));
+		Lisp_Object oldval = specpdl_old_value (specpdl_ptr);
+		MAYBE_RUN_VARHOOK (oldval, sym, false, Dyn_Unbind, sym->val.value, oldval);
+		SET_SYMBOL_VAL (sym, oldval);
 		break;
 	      }
 	    else
@@ -3235,8 +3240,8 @@
 	      }
 	  }
 	case SPECPDL_LET_DEFAULT:
-	  Fset_default (specpdl_symbol (specpdl_ptr),
-			specpdl_old_value (specpdl_ptr));
+	  set_default_internal (specpdl_symbol (specpdl_ptr),
+				specpdl_old_value (specpdl_ptr), Dyn_Unbind);
 	  break;
 	case SPECPDL_LET_LOCAL:
 	  {
@@ -3248,7 +3253,7 @@
 	    /* If this was a local binding, reset the value in the appropriate
 	       buffer, but only if that buffer's binding still exists.  */
 	    if (!NILP (Flocal_variable_p (symbol, where)))
-	      set_internal (symbol, old_value, where, 1);
+	      set_internal (symbol, old_value, where, true, Dyn_Unbind);
 	  }
 	  break;
 	}
@@ -3454,7 +3459,7 @@
 	    Lisp_Object sym = specpdl_symbol (tmp);
 	    Lisp_Object old_value = specpdl_old_value (tmp);
 	    set_specpdl_old_value (tmp, Fdefault_value (sym));
-	    Fset_default (sym, old_value);
+	    set_default_internal (sym, old_value, Dyn_Skip);
 	  }
 	  break;
 	case SPECPDL_LET_LOCAL:
@@ -3470,7 +3475,7 @@
 	      {
 		set_specpdl_old_value
 		  (tmp, Fbuffer_local_value (symbol, where));
-		set_internal (symbol, old_value, where, 1);
+		set_internal (symbol, old_value, where, true, Dyn_Skip);
 	      }
 	  }
 	  break;
@@ -3746,6 +3751,63 @@
 still determine whether to handle the particular condition.  */);
   Vdebug_on_signal = Qnil;
 
+  DEFSYM (Qsymbol_hook_void_value, "symbol-hook-void-value");
+  DEFVAR_LISP ("symbol-hook-void-value", Vsymbol_hook_void_value,
+	       doc: /* Representation of voidness for hooked variables.
+The value of this constant is an uninterned Lisp symbol that represents void
+when passed to or returned from `symbol-hook-function'.  */);
+  Vsymbol_hook_void_value = Fmake_symbol (build_string ("::void::"));
+  XSYMBOL (Vsymbol_hook_void_value)->constant = SYM_CONST;
+  XSYMBOL (Qsymbol_hook_void_value)->declared_special = true;
+  XSYMBOL (Qsymbol_hook_void_value)->constant = SYM_CONST;
+
+  DEFVAR_LISP ("symbol-hook-function", Vsymbol_hook_function,
+       doc: /* This function is called whenever a hooked variable is set.
+It takes four arguments: SYMBOL, ENV, OLDVAL, NEWVAL. By default, it just
+returns NEWVAL unchanged.
+
+SYMBOL is the symbol being set. ENV is the environment is which it's being
+set. OLDVAL is the current value, or if the current value is void, then OLDVAL
+is the value of `symbol-hook-void-value'. NEWVAL is the new value to which the setter,
+i.e. the caller of a function such as `setq', is attempting to set the
+variable, or the value of symbol-hook-void-value if the setter called `makunbound'.
+The actual new value to which the variable will be set is return value of
+this function, unless the setter called makunbound and this function returns
+the value of symbol-hook-void-value, in which case the variable will be set to void.
+
+The possible values of ENV are these symbols, with these meanings:
+global: The global environment.
+buf-local: The setter's buffer-local environment. ENV is this value if the
+setter sets the buffer-local variable.
+dyn-local: The innermost dynamic environment in which SYMBOL is bound. ENV
+is this value if the setter sets a dynamic local variable.
+dyn-bind: A new dynamic environment. ENV is this value if the setter creates
+a new dynamic environment, such as by using `let'.
+dyn-unbind: The next-outer dynamic environment in which SYMBOL is still bound,
+unshadowed due to destruction of the setter's current dynamic environment,
+such as due to exit of a `let' form, or the buffer-local environment if SYMBOL
+is not bound in any dynamic environment, or the global environment is SYMBOL
+is not in the buffer-local environment.
+
+To avoid overriding the setter's attempt to set the variable
+to NEWVAL, return NEWVAL. To block the attempt, and leave the variable
+unchanged, return OLDVAL. If ENV is dyn-bind or dyn-unbind, you can block
+the change of value, but you can't prevent the corresponding creation or
+destruction of a dynamic environment. Therefore, blocking when ENV is
+dyn-bind will set SYMBOL in the new environment to its value in the outer
+environment, and blocking when ENV is dyn-unbind will set SYMBOL in the
+outer environment to its value in the environment being destroyed. If OLDVAL
+is the value of symbol-hook-void-value but NEWVAL is not, you can override the new
+value, but you can't prevent the variable from being set to a non-void value.
+
+Don't set the variable in this function; that would cause a recursive call
+to this function, and even if you terminate the recursion, your setting
+would be overridden by the return value of this function. Instead, if you
+need to set the variable, return the value from this function.
+
+See also `symbol-hook-set' and `symbol-hook-unset'.  */);
+  Vsymbol_hook_function = Qnil; /* Set in subr.el */
+
   /* When lexical binding is being used,
    Vinternal_interpreter_environment is non-nil, and contains an alist
    of lexically-bound variable, or (t), indicating an empty
--- src/data.c
+++ src/data.c
@@ -574,6 +574,20 @@
 \f
 /* Extract and set components of symbols.  */
 
+DEFUN ("symbol-hook-p", Fsymbol_hook_p, Ssymbol_hook_p, 1, 1, 0,
+       doc: /* Return t if SYMBOL is hooked.
+To hook and unhook it, use `symbol-hook-set' and `symbol-hook-unset'.
+When hooked, setting SYMBOL will run `symbol-hook-function'.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  sym = XSYMBOL (symbol);
+  while (sym->redirect == SYMBOL_VARALIAS)
+    sym = indirect_variable (sym);
+  return SYM_HOOKED_P (sym) ? Qt : Qnil;
+}
+
 DEFUN ("boundp", Fboundp, Sboundp, 1, 1, 0,
        doc: /* Return t if SYMBOL's value is not void.
 Note that if `lexical-binding' is in effect, this refers to the
@@ -623,6 +637,50 @@
   return NILP (XSYMBOL (symbol)->function) ? Qnil : Qt;
 }
 
+DEFUN ("symbol-hook-set", Fsymbol_hook_set, Ssymbol_hook_set, 1, 1, 0,
+       doc: /* Hook SYMBOL.
+When hooked, setting it will run `symbol-hook-function'.
+To unhook it, use `symbol-hook-unset'.
+To test whether it's hooked, use `symbol-hook-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  if (SYMBOL_CONSTANT_P (symbol))
+    xsignal1 (Qsetting_constant, symbol);
+  sym = XSYMBOL (symbol);
+  sym->constant = SYM_HOOKED;
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->constant = SYM_HOOKED;
+    }
+  return symbol;
+}
+
+DEFUN ("symbol-hook-unset", Fsymbol_hook_unset, Ssymbol_hook_unset, 1, 1, 0,
+       doc: /* Unhook SYMBOL.
+When unhooked, setting it will not run `symbol-hook-function'.
+To hook it, use `symbol-hook-set'.
+To test whether it's hooked, use `symbol-hook-p'.
+Return SYMBOL.  */)
+  (register Lisp_Object symbol)
+{
+  struct Lisp_Symbol *sym;
+  CHECK_SYMBOL (symbol);
+  if (SYMBOL_CONSTANT_P (symbol))
+    return symbol; /* Unhooking a constant is a harmless no-op. */
+  sym = XSYMBOL (symbol);
+  sym->constant = SYM_UNVETTED;
+  while (sym->redirect == SYMBOL_VARALIAS)
+    {
+      sym = indirect_variable (sym);
+      sym->constant = SYM_UNVETTED;
+    }
+  return symbol;
+}
+
 DEFUN ("makunbound", Fmakunbound, Smakunbound, 1, 1, 0,
        doc: /* Make SYMBOL's value be void.
 Return SYMBOL.  */)
@@ -1201,11 +1259,65 @@
   xsignal1 (Qvoid_variable, symbol);
 }
 
+/* For the symbol S being set, run symbol-hook-function with these arguments:
+   0. S
+   1. A symbol indicating the environment in which S is being set.
+   2. The current value of S in that environment.
+   3. The value to which the setter is attempting to set the variable.
+
+   If argument #2 or #3 is Qunbound, it's replaced by the value of
+   Vsymbol_hook_void_value.
+
+   Return the result of symbol-hook-function, or if it's the value of
+   Vsymbol_hook_void_value and ATTEMPTED_VAL is Qunbound, return Qunbound. The variable
+   will be set (by code that calls run_varhook) to that return value,
+   overriding the value to which the setter attempted to set the variable.  */
+
+Lisp_Object
+run_varhook (struct Lisp_Symbol* sym, bool buf_local, Dyn_Bind_Env rawenv,
+	     Lisp_Object oldval, Lisp_Object attempted_val)
+{
+  Lisp_Object symbol, env, newval;
+  if (rawenv == Dyn_Skip) /* From backtrace_eval_unrewind */
+    return attempted_val;
+  XSETSYMBOL (symbol, sym);
+  switch (rawenv) /* Disambiguate Dyn_Current and Dyn_Global */
+    {
+    case Dyn_Current:
+      {
+	bool shadowed = (buf_local ? let_shadows_buffer_binding_p (sym)
+                         : let_shadows_global_binding_p (symbol));
+	if (shadowed) env = Qdyn_local;
+	else if (buf_local) env = Qbuf_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Global:
+      {
+	/* let_shadows_buffer_binding_p doesn't disambiguate this case */
+	if (let_shadows_global_binding_p (symbol) &&
+	    NILP (Flocal_variable_p (symbol, Qnil)))
+	  env = Qdyn_local;
+	else env = Qglobal;
+	break;
+      }
+    case Dyn_Bind: env = Qdyn_bind; break;
+    case Dyn_Unbind: env = Qdyn_unbind; break;
+    default: emacs_abort ();
+    }
+  oldval = EQ (oldval, Qunbound) ? Vsymbol_hook_void_value : oldval;
+  newval = EQ (attempted_val, Qunbound) ? Vsymbol_hook_void_value : attempted_val;
+  newval = call4 (Vsymbol_hook_function, symbol, env, oldval, newval);
+  if (attempted_val == Qunbound && EQ (newval, Vsymbol_hook_void_value))
+    return Qunbound; /* Converting setq, etc to makunbound is prohibited. */
+  return newval; /* So symbol_hook_void_value is ignored if Qunbound wasn't attempted. */
+}
+
 DEFUN ("set", Fset, Sset, 2, 2, 0,
        doc: /* Set SYMBOL's value to NEWVAL, and return NEWVAL.  */)
   (register Lisp_Object symbol, Lisp_Object newval)
 {
-  set_internal (symbol, newval, Qnil, 0);
+  set_internal (symbol, newval, Qnil, false, Dyn_Current);
   return newval;
 }
 
@@ -1215,40 +1327,85 @@
 
    If BINDFLAG is false, then if this symbol is supposed to become
    local in every buffer where it is set, then we make it local.
-   If BINDFLAG is true, we don't do that.  */
+   If BINDFLAG is true, we don't do that.
+
+   ENV indicates the dynamic environment for this function call, i.e. whether
+   this call is due to a variable binding (Dyn_Bind), an unbinding (Dyn_Unbind),
+   or neither (Dyn_Current). As special cases, a value of Dyn_Skip is a flag
+   to disable run_varhook so that varhooks aren't run during backtraces, and
+   a value of Dyn_Global is a flag indicating that this function call is due
+   to set_default, which allows run_varhook to distinguish beween the global
+   and the dyn-local binding.  */
 
 void
 set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
-	      bool bindflag)
+	      bool bindflag, Dyn_Bind_Env env)
 {
-  bool voide = EQ (newval, Qunbound);
   struct Lisp_Symbol *sym;
-  Lisp_Object tem1;
-
-  /* If restoring in a dead buffer, do nothing.  */
-  /* if (BUFFERP (where) && NILP (XBUFFER (where)->name))
-      return; */
 
   CHECK_SYMBOL (symbol);
-  if (SYMBOL_CONSTANT_P (symbol))
+  sym = XSYMBOL (symbol);
+  if (sym->constant)
     {
+      set_internal_vetted (symbol, newval, where, bindflag, env, sym);
+      return;
+    }
+
+ start:
+  switch (sym->redirect)
+    {
+    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym, newval); return;
+    default: set_internal_localized_or_forwarded
+	(symbol, newval, where, bindflag, env, sym);
+    }
+}
+
+void
+set_internal_vetted (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
+			     bool bindflag, Dyn_Bind_Env env, struct Lisp_Symbol *sym)
+{
+  if (SYM_HOOKED_P (sym))
+    {
+    start:
+      switch (sym->redirect)
+	{
+	case SYMBOL_PLAINVAL:
+	  newval = run_varhook (sym, false, env, sym->val.value, newval);
+	  SET_SYMBOL_VAL (sym, newval);
+	  return;
+	case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
+	default:
+	  {
+	    set_internal_localized_or_forwarded
+	      (symbol, newval, where, bindflag, env, sym);
+	    return;
+	  }
+	}
+    }
       if (NILP (Fkeywordp (symbol))
 	  || !EQ (newval, Fsymbol_value (symbol)))
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
 	return;
-    }
+}
 
-  sym = XSYMBOL (symbol);
+/* Split from set_internal to avoid code duplication, because both set_internal and
+   set_internal_vetted must call this function.  */
 
- start:
+void
+set_internal_localized_or_forwarded (Lisp_Object symbol, Lisp_Object newval,
+				     Lisp_Object where, bool bindflag,
+				     Dyn_Bind_Env env, struct Lisp_Symbol *sym)
+{
+  bool voide;
+  Lisp_Object tem1;
   switch (sym->redirect)
     {
-    case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: SET_SYMBOL_VAL (sym , newval); return;
     case SYMBOL_LOCALIZED:
       {
+	bool buf_local = true;
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 	if (NILP (where))
 	  {
@@ -1292,6 +1449,7 @@
 		   indicating that we're seeing the default value.
 		   Likewise if the variable has been let-bound
 		   in the current buffer.  */
+		buf_local = false;
 		if (bindflag || !blv->local_if_set
 		    || let_shadows_buffer_binding_p (sym))
 		  {
@@ -1319,6 +1477,9 @@
 	    set_blv_valcell (blv, tem1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym, buf_local, env, blv_value (blv), newval);
+	voide = EQ (newval, Qunbound);
+
 	/* Store the new value in the cons cell.  */
 	set_blv_value (blv, newval);
 
@@ -1350,6 +1511,11 @@
 	      SET_PER_BUFFER_VALUE_P (buf, idx, 1);
 	  }
 
+	MAYBE_RUN_VARHOOK (newval, sym,
+			   (XFWDTYPE (innercontents)) == Lisp_Fwd_Buffer_Obj,
+			   env, do_symval_forwarding (innercontents), newval);
+	voide = EQ (newval, Qunbound);
+
 	if (voide)
 	  { /* If storing void (making the symbol void), forward only through
 	       buffer-local indicator, not through Lisp_Objfwd, etc.  */
@@ -1447,6 +1613,17 @@
 for this variable.  */)
   (Lisp_Object symbol, Lisp_Object value)
 {
+  set_default_internal (symbol, value, Dyn_Global);
+  return value;
+}
+
+/* Like Fset_default, but with ENV argument. See set_internal for
+   a description of this argument. */
+
+void
+set_default_internal (Lisp_Object symbol, Lisp_Object value,
+		      Dyn_Bind_Env env)
+{
   struct Lisp_Symbol *sym;
 
   CHECK_SYMBOL (symbol);
@@ -1457,7 +1634,7 @@
 	xsignal1 (Qsetting_constant, symbol);
       else
 	/* Allow setting keywords to their own value.  */
-	return value;
+	return;
     }
   sym = XSYMBOL (symbol);
 
@@ -1465,18 +1642,24 @@
   switch (sym->redirect)
     {
     case SYMBOL_VARALIAS: sym = indirect_variable (sym); goto start;
-    case SYMBOL_PLAINVAL: return Fset (symbol, value);
+    case SYMBOL_PLAINVAL:
+      {
+	set_internal (symbol, value, Qnil, false, env);
+	return;
+      }
     case SYMBOL_LOCALIZED:
       {
 	struct Lisp_Buffer_Local_Value *blv = SYMBOL_BLV (sym);
 
+	MAYBE_RUN_VARHOOK (value, sym, false, env, XCDR (blv->defcell), value);
+
 	/* Store new value into the DEFAULT-VALUE slot.  */
 	XSETCDR (blv->defcell, value);
 
 	/* If the default binding is now loaded, set the REALVALUE slot too.  */
 	if (blv->fwd && EQ (blv->defcell, blv->valcell))
 	  store_symval_forwarding (blv->fwd, value, NULL);
-	return value;
+	return;
       }
     case SYMBOL_FORWARDED:
       {
@@ -1490,6 +1673,8 @@
 	    int offset = XBUFFER_OBJFWD (valcontents)->offset;
 	    int idx = PER_BUFFER_IDX (offset);
 
+	    MAYBE_RUN_VARHOOK (value, sym, false, env, per_buffer_default (offset), value);
+
 	    set_per_buffer_default (offset, value);
 
 	    /* If this variable is not always local in all buffers,
@@ -1502,10 +1687,13 @@
 		  if (!PER_BUFFER_VALUE_P (b, idx))
 		    set_per_buffer_value (b, offset, value);
 	      }
-	    return value;
+	    return;
 	  }
 	else
-	  return Fset (symbol, value);
+	  {
+	    set_internal (symbol, value, Qnil, false, env);
+	    return;
+	  }
       }
     default: emacs_abort ();
     }
@@ -1633,7 +1821,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be buffer-local", SDATA (SYMBOL_NAME (variable)));
 
   if (!blv)
@@ -1706,7 +1894,7 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  if (SYM_CONST_P (sym))
     error ("Symbol %s may not be buffer-local",
 	   SDATA (SYMBOL_NAME (variable)));
 
@@ -1895,7 +2083,8 @@
     default: emacs_abort ();
     }
 
-  if (sym->constant)
+  /* Intentional bug, at Stefan's insistence. */
+  if (sym->constant) /* This should be: if (SYM_CONST_P (sym)) */
     error ("Symbol %s may not be frame-local", SDATA (SYMBOL_NAME (variable)));
 
   blv = make_blv (sym, forwarded, valcontents);
@@ -3474,6 +3663,12 @@
   DEFSYM (Qad_advice_info, "ad-advice-info");
   DEFSYM (Qad_activate_internal, "ad-activate-internal");
 
+  DEFSYM (Qglobal, "global");
+  DEFSYM (Qbuf_local, "buf-local");
+  DEFSYM (Qdyn_local, "dyn-local");
+  DEFSYM (Qdyn_bind, "dyn-bind");
+  DEFSYM (Qdyn_unbind, "dyn-unbind");
+
   error_tail = pure_cons (Qerror, Qnil);
 
   /* ERROR is used as a signaler for random errors for which nothing else is
@@ -3609,8 +3804,11 @@
   defsubr (&Sindirect_function);
   defsubr (&Ssymbol_plist);
   defsubr (&Ssymbol_name);
+  defsubr (&Ssymbol_hook_set);
+  defsubr (&Ssymbol_hook_unset);
   defsubr (&Smakunbound);
   defsubr (&Sfmakunbound);
+  defsubr (&Ssymbol_hook_p);
   defsubr (&Sboundp);
   defsubr (&Sfboundp);
   defsubr (&Sfset);
--- src/bytecode.c
+++ src/bytecode.c
@@ -843,7 +843,7 @@
 	    else
 	      {
 		BEFORE_POTENTIAL_GC ();
-		set_internal (sym, val, Qnil, 0);
+		set_internal (sym, val, Qnil, false, Dyn_Current);
 		AFTER_POTENTIAL_GC ();
 	      }
 	  }
--- lisp/subr.el
+++ lisp/subr.el
@@ -2546,6 +2546,9 @@
 Note that this should end with a directory separator.
 See also `locate-user-emacs-file'.")
 \f
+(setq symbol-hook-function ; Defined in eval.c
+      (lambda (_sym _env _oldval newval) newval))
+\f
 ;;;; Misc. useful functions.
 
 (defsubst buffer-narrowed-p ()

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

>> I'm sorry, but I consider this a form of lying.  It makes it sound like
>> "the new var-hook functionality actually speeds things up", even though
>> it's not this new functionality but some unrelated (tho bundled) change
>> which does it (and which could be applied independently).
> I didn't say the varhook functionality speeds things up.  I explicitly said
> it has performance costs, and I found optimization opportunities to _pay
> for_ those costs.

I know.  I said "makes it sound like".

> I was completely honest about what I did,

You were, indeed.

>> The issue is not patch size per se, but just keeping the patch focused
>> on its core purpose.
> The constant-hooked combination is an optimization,

I don't consider it an optimization.  I consider the functionality to be
an extension of the "constant" bit into a more nuanced 3-valued field.

> The patch could be applied to trunk without that optimization,

No because it's fundamentally wrong to have `constant' in one field and
`hooked' in another when `hooked' can only be used when constant==0.

> Fine, but adding the capability of converting setq, etc into makunbound has
> the cost of breaking correctness (because hooking a symbol would change the
> behavior of (setq foo void-sentinel)), not just the cost of enabling user
> errors.

I think that would be OK because (setq foo void-sentinel) can be
considered as erroneous code.

This said, I don't care very much about allowing/disallowing a setq to
turn into a makunbound.  I think it's not worth spending any code on this,
so I similarly shouldn't spend much time arguing about it.

> Magic constants added back in to the source code, just to avoid
> touching a few lines of code in the patch.

Actually, this is a mistake.  Using 1 was OK back when the code was
written (it should be `true' instead nowadays), but with the new
definition of the field, it should be SYM_CONST.

> This version of the patch slows down Emacs.

How much, where, and why?  Clearly, there is an added cost in the
unbind_to case since we now have to check sym->constant, but otherwise,
where the extra cost come from?  Is it just the extra `env' argument to
set_internal?

> IIUC, I've made all the changes you requested.

Indeed, thank you.  See included the remaining nitpicks below.

The only significant comments below is the one about the ENV arg, since
I think we really should be able to distinguish all buffer-local changes
from the non-buffer-local ones (I think this is more important than
distinguishing `setq' to the toplevel value vs `setq' within a `let',
which we could even distinguish without any special ENV value by
exporting let_shadows_buffer_binding_p and/or
let_shadows_global_binding_p to Elisp).


        Stefan


> +SYMBOL is the symbol being set. ENV is the environment is which it's being
> +set.

Actually, ENV is not an environment (it's just a symbol which describes
in which environment the modification is made).

> +The possible values of ENV are these symbols, with these meanings:
> +global: The global environment.

So, this is for a `setq-default' or a `setq' when there's no
buffer-local value, and no active let-binding.

> +buf-local: The setter's buffer-local environment. ENV is this value if the
> +setter sets the buffer-local variable.

So, IIUC this is for a `setq' when the variable has been made buffer
local in this buffer.
And IIUC this is for the case where there's no active let-binding for
this variable, right?
Does it distinguish between let-bindings for the buffer-local part of
this variable, and let-bindings for other parts of the variable (e.g. in
other buffers, or let-binding for the global part of the var)?

> +dyn-local: The innermost dynamic environment in which SYMBOL is bound. ENV
> +is this value if the setter sets a dynamic local variable.

IIUC this is for a `setq' to a variable that has an active let-binding.
Does it distinguish between the case where the let-binding is
buffer-local and where it isn't?

> +dyn-bind: A new dynamic environment. ENV is this value if the setter creates
> +a new dynamic environment, such as by using `let'.

This is for a `let'.  But IIUC we don't get to know if this let is
buffer-local or global, right?

> +to NEWVAL, return NEWVAL. To block the attempt, and leave the variable
                           ^^^
BTW, we use the `sentence-end-double-space' convention in our comments
and docstrings.

> +is the value of symbol-hook-void-value but NEWVAL is not, you can
                   ^^^^^^^^^^^^^^^^^^^^^^
Please wrap this in `...' so C-h f can turn it into a hyperlink.

> +       doc: /* Return t if SYMBOL is hooked.
                        ^^^
We usually prefer to say "non-nil" for such predicate return value (and
callers shouldn't assume that it's necessarily t).


        Stefan

Proposal for debugging/testing option

[Kelly Dean]

Stefan Monnier wrote:
> Based on my experience with emitting messages in similar oddball cases
> (i.e. making a variable buffer-local while it's globally let-bound), my
> guess is that it would catch a few very rare oddball cases indeed, but:
> 1- those oddball cases end up harmless.
> 2- there's no easy "fix" for those cases (because the let-binding and the
>    makunbound happen in completely unrelated code which usually aren't
>    used at the same time and what *really* should happen is at best
>    unclear, or is otherwise exactly what happens with the current
>    semantics you don't like).

Would it be ok to add an option to barf in those cases instead of just emitting warning messages? Even if users won't use the option, it would be appropriate during development and testing.

I.e. barf-if-probable-bug, with the possible values t, 'warn, and nil. With 'warn as the default.

Then, in the rare cases where it really is not a bug, the barf option can be let-bound to nil (i.e. a runtime analog of the with-no-warnings wrapper), which both suppresses the warning messages for known-correct usage and explicitly indicates in the code that the case has been reviewed and determined to not be a bug.

Re: Proposal for debugging/testing option

[Stefan Monnier]

> Would it be ok to add an option to barf in those cases instead of just
> emitting warning messages?

I'm more in favor of just removing those messages since they still
haven't found a useful case where they diagnosed a problem we could fix.
Adding an option to turn them into errors would be over-engineering it.

Of course, your varhook feature should make it possible to implement
such a system all in Elisp.


        Stefan

Re: [PATCH] (Updated) Run hook when variable is set

[Johan Bockgård]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> [ BTW, even for 30KB patches, I prefer them uncompressed, because
>   I typically read them directly in Gnus.  ]

Use `K i' from the summary buffer, or `i' on the MIME button in the
article buffer.

Re: [PATCH] (Updated) Run hook when variable is set

[Stefan Monnier]

> Use `K i' from the summary buffer, or `i' on the MIME button in the
> article buffer.

Oh, cool, I hadn't even tried but indeed it works!
OK, so feel free to send compressed patches from now on,


        Stefan

Patches: inline vs. attachment, compressed vs. uncompressed. [was: Run hook when variable is set]

[Alan Mackenzie]

On Sun, Feb 15, 2015 at 02:36:32PM -0500, Stefan Monnier wrote:
> > Use `K i' from the summary buffer, or `i' on the MIME button in the
> > article buffer.

> Oh, cool, I hadn't even tried but indeed it works!
> OK, so feel free to send compressed patches from now on,

Please don't.  Compressed patches are more troublesome to read than
normal ones.  Not everybody uses the same setup.

For that matter, why are so many patches being sent as attachments these
days rather than being inline in the text?  The patch program is quite
capable of extracting an inline patch from extraneous matter.

Attachments are more troublesome to read than inline patches.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [PATCH] (Updated) Run hook when variable is set

[Kelly Dean]

As a followup to my previous message, here are some benchmarks to test the worst-case performance impact of the varhook-single patch. This is without any changes to the patch; the «hooked» bit is still separate from «constant», so try_run_varhook is executed every time a non-lexical variable is set.

This is on an Intel Core 2 (more than 8 years old, so not taking advantage of any recent CPU optimizations, but it does have a branch predictor, which is an ancient feature). Emacs is compiled with its default optimization settings, with and without the patch.

Note, to compile 24.4 with the patch, you have to add to the top of data.c the line:
static Lisp_Object Qvarhook, Qglobal, Qbuf_local, Qdyn_local, Qdyn_bind, Qdyn_unbind, Qinvalid;

Do emacs -Q, then:
(require 'cl)
(setq x 0)
(defun test ()
  (benchmark-run-compiled 100000 (incf x)))

With Emacs 24.4, without the patch:
(test) → (0.009720718000000003 0 0.0)
(test) → (0.007469938999999995 0 0.0)
(test) → (0.007450257000000002 0 0.0)

With the patch:
(test) → (0.006218317000000001 0 0.0)
(test) → (0.0055950570000000005 0 0.0)
(test) → (0.005777339000000006 0 0.0)

IOW, varhook miraculously makes Emacs faster! ;-)

Ok, another couple test runs:

Without the patch:
(test) → (0.006397249999999993 0 0.0)
(test) → (0.006253384000000001 0 0.0)
(test) → (0.006264767000000004 0 0.0)

With the patch:
(test) → (0.007369438000000006 0 0.0)
(test) → (0.006640647999999999 0 0.0)
(test) → (0.007056691999999989 0 0.0)

With multiple runs, the difference is below the accuracy of measurement.

Continuing onward with the patch:
(hook 'x) ; But «varhook» is still nil (i.e. there's no handler function)
;; Now, run_varhook will be called each time x is incremented.
(test) → (0.04964255199999999 0 0.0)
(test) → (0.046831024 0 0.0)
(test) → (0.047073512 0 0.0)
(add-hook 'varhook (lambda (_sym _env _val) nil))
(test) → (0.07824081199999999 0 0.0)
(test) → (0.07892686900000001 0 0.0)
(test) → (0.07999398 0 0.0)
(unhook 'x)
(test) → (0.00680763899999999 0 0.0)
(test) → (0.005905495999999996 0 0.0)
(test) → (0.006095260000000005 0 0.0)

All as expected. After unhooking, the results are again indistinguishable from the results of Emacs without the patch.

Another test:
(require 'cl)
(defun test1 ()
  (setq x 0)
  (benchmark-run-compiled
      (do () ((> x 100000)) (incf x))))

Without the patch:
(test1) → (0.010726432 0 0.0)
(test1) → (0.010739914 0 0.0)
(test1) → (0.010754511 0 0.0)

With the patch:
(test1) → (0.011321344 0 0.0)
(test1) → (0.011568507 0 0.0)
(test1) → (0.011352074 0 0.0)

Without the patch:
(test1) → (0.011268476 0 0.0)
(test1) → (0.011142896 0 0.0)
(test1) → (0.011574237 0 0.0)

With the patch:
(test1) → (0.011327352 0 0.0)
(test1) → (0.011323509 0 0.0)
(test1) → (0.011337811 0 0.0)

The difference is again below the accuracy of measurement.

Based on these results, I recommend against putting the «hooked» bit into the «constant» field. It wouldn't accomplish anything, and it would reduce the clarity of the source code.

Re: [PATCH] Run hook when variable is set

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Hooks on setting variables is a fundamentally bad idea
because it means that Lisp code which appears to just bind variables
can call functions where you did not expect it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

Richard Stallman wrote:
> Hooks on setting variables is a fundamentally bad idea
> because it means that Lisp code which appears to just bind variables
> can call functions where you did not expect it.

But what if it's documented? For example, it's documented that setting the variable cursor-type not only sets a variable, but also has the side effect of changing how the cursor is displayed.

Disabling dynamic-cursor-mode could simply be another documented side effect of setting cursor-type. And DCM itself should also document that it's disabled when cursor-type is set, which is why I already included that in the docstring in the patch I posted.

Besides that, hooks on setting variables would be useful for debugging.

Re: [PATCH] Run hook when variable is set

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But what if it's documented? For example, it's documented that
  > setting the variable cursor-type not only sets a variable, but
  > also has the side effect of changing how the cursor is displayed.

That is safe because it does not run Lisp code when you set the
variable.  Basically, redisplay looks at the value of the variable,
and that happens only when redisplay gets called.

  > Disabling dynamic-cursor-mode could simply be another documented
  > side effect of setting cursor-type.

If you do it by having redisplay check cursor-type and decide
what to do about the cursor mode, I have nothing against it.

However, having any variable, the binding of which can run Lisp code,
is an absolute disaster.  If the price we pay for to avoid that disaster
is that we don't have the feature you would like, so be it.

  > Besides that, hooks on setting variables would be useful for debugging.

No matter what they would be useful for, it is not worth the chaos
they would cause.

Maybe a specialized feature solely for debugging could be made safe.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

Richard Stallman wrote:
> having any variable, the binding of which can run Lisp code,
> is an absolute disaster.
[snip]
> Maybe a specialized feature solely for debugging could be made safe.

So, the debugging handler functions should be writeable only in C, solely for the purpose of depriving programmers of the ability to write them in Elisp?

Re: [PATCH] Run hook when variable is set

[Alexis]

Stefan Monnier writes:

> For the same reason I want to kill the `intangible' text-property:
> this operates at too-low a level, so it's bound to introduce buggy
> interactions between unsuspecting packages.

So what is now the best way to achieve the same effects/behaviours as
the 'intangible' text-property? One of the packages i maintain uses
'intangible' on some text in a minibuffer, to facilitate movement
between the two fields whose contents users can modify: left- and
right-movement doesn't move point through individual characters of
non-modifiable text, but moves point past the entirety of that text.


Alexis.

Re: [PATCH] Run hook when variable is set

[Stefan Monnier]

>> For the same reason I want to kill the `intangible' text-property:
>> this operates at too-low a level, so it's bound to introduce buggy
>> interactions between unsuspecting packages.
> So what is now the best way to achieve the same effects/behaviours as
> the 'intangible' text-property?

You can use a post-command-hook or pre-redisplay-function, for example.

We already have code to do those kinds of things for properties such as
`invisible' (run right after post-command-hook, under the control of
disable-point-adjustment) and we should extend it for a new property,
which we could call `cursor-intangible'.

> One of the packages i maintain uses 'intangible' on some text in
> a minibuffer, to facilitate movement between the two fields whose
> contents users can modify: left- and right-movement doesn't move point
> through individual characters of non-modifiable text, but moves point
> past the entirety of that text.

Exactly: like 99.99% of all uses of `intangible', you use it to affect
cursor motion, i.e. motion at the command level, whereas intangible
affects motion at the function level (i.e. motions that happen within
commands, sometimes thousands of such motions within a single command,
some of those motions may happen in different buffers).


        Stefan

Re: [PATCH] Run hook when variable is set

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> For the same reason I want to kill the `intangible' text-property:
>>> this operates at too-low a level, so it's bound to introduce buggy
>>> interactions between unsuspecting packages.
>> So what is now the best way to achieve the same effects/behaviours as
>> the 'intangible' text-property?
>
> You can use a post-command-hook or pre-redisplay-function, for example.
>
> We already have code to do those kinds of things for properties such as
> `invisible' (run right after post-command-hook, under the control of
> disable-point-adjustment) and we should extend it for a new property,
> which we could call `cursor-intangible'.

The cursor does not "tangle" anything anyway.  It's more like a
"no-parking" or "get-off-my" property.

> Exactly: like 99.99% of all uses of `intangible', you use it to affect
> cursor motion, i.e. motion at the command level, whereas intangible
> affects motion at the function level (i.e. motions that happen within
> commands, sometimes thousands of such motions within a single command,
> some of those motions may happen in different buffers).

preview-latex (part of AUCTeX) implements such machinery.  Some of it
probably is no longer strictly necessary as Emacs has gained some more
automatic move-out semantics since early Emacs-21.  But it may well be
needed as fallback for XEmacs.

-- 
David Kastrup

Re: [PATCH] Run hook when variable is set

[Eli Zaretskii]

> From: Kelly Dean <kelly@prtime.org>
> Date: Thu, 29 Jan 2015 08:20:27 +0000
> Cc: emacs-devel@gnu.org
> 
> For just one example, without varhook, what solution would you
> propose for bug #19068?

The canonical way of solving those is to have a :set function in the
defcustom (and tell in the doc string that setting the value directly
doesn't work).

IOW, for defcustoms we already have this solved.

Re: [PATCH] Run hook when variable is set

[Kelly Dean]

>> For just one example, without varhook, what solution would you
>> propose for bug #19068?
>
> The canonical way of solving those is to have a :set function in the
> defcustom (and tell in the doc string that setting the value directly
> doesn't work).
>
> IOW, for defcustoms we already have this solved.

Oh, I missed the :set function for defcustom.

I'm probably just being dense, but the customization interface in Emacs is a bit confusing for me, so I've always just put «setq»s in my init file. Requiring use of the customization interface for users who are accustomed to directly editing init seems a bit Windows-esque, but at least if the docstring says it's required, then it won't be misleading.

Re: [PATCH] Run hook when variable is set

[Eli Zaretskii]

> From: Kelly Dean <kelly@prtime.org>
> CC: emacs-devel@gnu.org
> Date: Fri, 30 Jan 2015 07:14:19 +0000
> 
> Oh, I missed the :set function for defcustom.
> 
> I'm probably just being dense, but the customization interface in Emacs is a bit confusing for me, so I've always just put «setq»s in my init file. Requiring use of the customization interface for users who are accustomed to directly editing init seems a bit Windows-esque, but at least if the docstring says it's required, then it won't be misleading.

Some options need non-trivial processing when their value changes.
The alternative is to tell users to do that by hand, which is not my
idea of a good customization UI.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Stefan Monnier]

>>> +      (if dynamic-cursor-mode (setq cursor-type 'bar))
>> (if (and dynamic-cursor-mode (eq cursor-type t))
>> (setq cursor-type 'bar dynamic-cursor-mode--set t))
> That's even uglier! ;-) You still have a global minor mode (and its
> variable), and a separately-named buffer-local variable, so it's no
> simpler.

It's a lot simpler because the buffer-local var is internal.

> (setq-default cursor-type 'bar)
> A few days later in the same Emacs session...
> (dynamic-cursor-mode)

No, because dynamic-cursor-mode can do (setq-default cursor-type t).
That's one of the advantages of a minor mode over a plain defcustom.

>> So you don't need a buffer-local activation of dynamic-cursor-mode.
> If (global) dynamic-cursor-mode is enabled, your code provides no way
> to prevent it from operating in a buffer in which cursor-type happens
> to be t.

Of course it does:

   (add-hook 'foo-mode-hook (lambda () (setq-local dynamic-cursor-mode nil)))

> Modifying the «set» family of functions to provide triggers would be the
> ideal solution, as I described in my previous message.

The difference between a boolean defcustom and a global minor-mode is
pretty much that the global minor mode offers the "set triggers".


        Stefan

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

Stefan Monnier wrote:
> You still have a global minor mode (and its
>> variable), and a separately-named buffer-local variable, so it's no
>> simpler.
>
> It's a lot simpler because the buffer-local var is internal.

What do you mean by ‟internal”? Just the double-dash naming convention? If by ‟simpler” you mean that no ‟external” (no double-dash, and intended to be user-level) var has to be used buffer-locally, then your solution isn't simpler; you still need (setq-local dynamic-cursor-mode nil). See below.

>> If (global) dynamic-cursor-mode is enabled, your code provides no way
>> to prevent it from operating in a buffer in which cursor-type happens
>> to be t.
>
> Of course it does:
>
>    (add-hook 'foo-mode-hook (lambda () (setq-local dynamic-cursor-mode nil)))

I didn't know it was appropriate for any code other than the mode itself to set a mode's variable. I thought it was supposed to be treated as read-only by all other code.

>> (setq-default cursor-type 'bar)
>> A few days later in the same Emacs session...
>> (dynamic-cursor-mode)
>
> No, because dynamic-cursor-mode can do (setq-default cursor-type t).
> That's one of the advantages of a minor mode over a plain defcustom.

You mean do that when dynamic-cursor-mode turns on? In that case, what if I want to enable dynamic-cursor-mode buffer-locally, rather than globally?

If I were to turn it on globally by doing (dynamic-cursor-mode), then it would (once, at the time of turn-on) set the cursor type in all buffers that are using the global value of cursor-type (i.e. that have no buffer-local value for it), which is the wrong thing to do, since I was only trying to enable dynamic-cursor-mode buffer-locally.

If I were to instead do (setq-local dynamic-cursor-mode t), then would I lose the supposed advantage of a minor mode over a plain defcustom, and the original failure mode I described would therefore remain.

> The difference between a boolean defcustom and a global minor-mode is
> pretty much that the global minor mode offers the "set triggers".

But only in one direction: dynamic-cursor-mode can set cursor-type when the former is turned on (though this doesn't do what's needed, as described above), but setting cursor-type can't turn off dynamic-cursor-mode (which is what's actually needed to enable defaulting dynamic-cursor-mode to t without causing conflict).

It still seems my original patch, except with the ⌜-mode⌝ suffix removed and the default changed to nil, is the cleanest way to implement dynamic-cursor. Even if you want it to formally be a minor mode (even though it has no advantage in this case), the cleanest implementation is with the simple additions to de/activate-mark in my patches (all three patches have these same additions), and nothing in the minor mode itself except the variable, which I'll set buffer-locally when necessary since you say that's appropriate.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Tassilo Horn]

Hi,

I like that mode and actually have a pretty similar hack in my ~/.emacs.
I just wanted to mention that indicating `overwrite-mode' with that
feature (maybe as hbar cursor) might be a worthwhile addition.

Bye,
Tassilo

RE: [PATCH] Proposal to change cursor appearance to indicate region activation

[Drew Adams]

> I just wanted to mention that indicating `overwrite-mode' with that
> feature (maybe as hbar cursor) might be a worthwhile addition.

FWIW, that is what the `cursor-chg.el' code does.  You can choose
the cursor types used.

code: http://www.emacswiki.org/emacs-en/download/cursor-chg.el
description: http://www.emacswiki.org/ChangingCursorDynamically

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Eli Zaretskii]

> From: Kelly Dean <kelly@prtime.org>
> Date: Fri, 23 Jan 2015 03:08:33 +0000
> Cc: emacs-devel@gnu.org
> 
> +(add-hook 'emacs-startup-hook #'maybe-disable--dynamic-cursor)

Isn't there a way to do this that avoids using hooks?  I think using
hooks in standard features should be avoided.  Hooks are for
customizing Emacs.

Thanks.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

Eli Zaretskii wrote:
>> +(add-hook 'emacs-startup-hook #'maybe-disable--dynamic-cursor)
>
> Isn't there a way to do this that avoids using hooks?  I think using
> hooks in standard features should be avoided.  Hooks are for
> customizing Emacs.

I don't know, but there's already a bunch of stuff in Emacs that uses emacs-startup-hook and after-init-hook, so I assumed that's the way I'm supposed to do it.

But Stefan and I agree that maybe-disable--dynamic-cursor in particular is ugly, and I recommend not using it. I just wrote it to satisfy a requirement he gave (avoiding trampling on the user's setting of cursor-type in his init file) if dynamic-cursor defaults to t.

I recommend defaulting dynamic-cursor to nil, which avoids the need for maybe-disable--dynamic-cursor, because the latter doesn't solve the general problem (the user might set cursor-type even after startup) and Emacs provides no way to solve it. I originally recommended defaulting dynamic-cursor to t, because I missed the problem (the trampling) that he pointed out.

Re: [PATCH] Proposal to change cursor appearance to indicate region activation

[Eli Zaretskii]

> From: Kelly Dean <kelly@prtime.org>
> CC: emacs-devel@gnu.org
> Date: Fri, 23 Jan 2015 11:40:55 +0000
> 
> Eli Zaretskii wrote:
> >> +(add-hook 'emacs-startup-hook #'maybe-disable--dynamic-cursor)
> >
> > Isn't there a way to do this that avoids using hooks?  I think using
> > hooks in standard features should be avoided.  Hooks are for
> > customizing Emacs.
> 
> I don't know, but there's already a bunch of stuff in Emacs that uses emacs-startup-hook and after-init-hook, so I assumed that's the way I'm supposed to do it.

emacs-startup-hook is currently used in exactly 2 places, one of which
(edt.el) is an emulator, so I could understand why it needs something
like that.

Anyway, the key word here is "avoid", so if there's no cleaner way to
do what you need, it's not *verboten* to use that hook.

Re: Proposal to change cursor appearance to indicate region activation

[Kelly Dean]

Drew Adams wrote on 20 Apr 2013:
> A block cursor itself is the cause of any such confusion, I think, regardless of
> inverse video.  It sits on top of a char, so a priori it is ambiguous whether
> the region includes that character or not.

Right. That's why it makes sense to not use a block cursor when visually indicating the region.

> And a moment's thought would anyway reach the conclusion that choosing to
> include the char under the cursor would mean that the active region always has
> at least one char (the one under the cursor).

Right. That would be a bad design, which is why Emacs excludes that char. Using a bar cursor to visually indicate the region makes it clear that Emacs is excluding that char.

> I wouldn't have a problem with that being the default behavior, but I don't see
> a crying need for it either.

Ok. My patch has a boolean defcustom for it.

> FWIW, I use code[*] that indicates the size of the active region in the mode
> line (using face `region').

That's a good idea, and I think Emacs should do it by default. But when I'm making a series of edits involving frequent activation/deactivation of the region, and my focus of attention is near the cursor, I'm not paying attention to the mode line. And a bar cursor for an active region still makes sense for the reason above. So your feature is a complement, not a substitute, for mine.

> I also use a dynamically changing cursor[**] for other purposes than indicating
> the active region:
[snip]

With my feature turned off, it won't interfere with other uses of cursor-type. You can also enable/disable it buffer-locally, and it won't interfere with cursor-type in other buffers.

> But there are multiple ways to skin the cat.  I'm neutral on your proposal wrt
> the resulting behavior, and slightly negative wrt the mechanism (hooks).

Ok. Stefan also suggested not using hooks, so my patch doesn't.


Drew Adams wrote on 23 Nov 2013:
> The proposal says that this "(speaking from experience) an Emacs
> newbie finds distracting, since it seems to indicate that that
> character is also part of the region, even though it actually
> isn't."
>
> In what way does it "seem to indicate" that the character under
> the cursor is part of the region?

The character under the cursor, like text in a highlighted region, is displayed with a different background than the rest of the text in the buffer is. And with the cursor at the end of the region, that non-standard background is visually suggestive of the character being in the region.

> Emacs provides plenty of visual feedback about what is the
> correct impression.  It doesn't take any thought to find out what
> is in the region and what is not.  It just takes one or two
> selections - an experimentation of maybe 20 seconds or so.
> Seriously, how long did it take you to figure out the truth here?

Of course, the truth quickly becomes clear. But it remains visually grating. Since this is just a matter of preference, there's an option to turn it on or off.

> 2. The 2nd reason given for the proposal: "Neither is there
> indication of the active region if point is one less than mark and
> blink-cursor-mode is off."
>
> I don't see that.  For me, the active region is highlighted in that
> case.  Am I missing something?

You're not missing anything. I was.

> you can see the `region'
> face applied to it as foreground.  I.e., you see the character
> highlighted as part of the region, but you see it in inverse
> video.
>
> Admittedly, the `region' face as background instead of
> foreground is more obvious.  I'll grant you that.

Indeed, the region face as foreground is so un-obvious that I missed it until you pointed it out.

> But that alone
> does not justify this feature being hardcoded into Emacs, IMO.

You and Stefan both suggested not using hooks, so I don't see any other obvious way to implement it except by modifying activate-mark and deactivate-mark.

Re: Proposal to change cursor appearance to indicate region activation

[Stefan Monnier]

> (add-hook 'deactivate-mark-hook (lambda () (setq cursor-type t)))
> (add-hook 'activate-mark-hook (lambda () (setq cursor-type 'bar)))

I like the idea, but this collides with other uses of cursor-type.
Could someone cook up a patch which only does the above if cursor-type
has not been modified (and which lets users opt-out if they prefer).
Also, the patch should directly modify deactivate-mark and activate-mark
rather than using the hooks.


        Stefan

RE: Proposal to change cursor appearance to indicate region activation

[Drew Adams]

> > (add-hook 'deactivate-mark-hook (lambda () (setq cursor-type t)))
> > (add-hook 'activate-mark-hook (lambda () (setq cursor-type 'bar)))
> 
> I like the idea, but this collides with other uses of cursor-type.
> Could someone cook up a patch which only does the above if cursor-
> type has not been modified (and which lets users opt-out if they
> prefer).  Also, the patch should directly modify deactivate-mark
> and activate-mark rather than using the hooks.

If you do anything like this, please put it in a minor mode or make
it otherwise customizable (e.g. trivial to turn off).  I would even
argue against it being on by default (but not strongly) - I prefer
that the default behavior not be changed.

It is simple for a user or library to change the `cursor-type' on
demand or based on some dynamic condition.  Why put this on mark
(de)activation hooks unconditionally, or "directly modify
deactivate-mark and activate-mark"?  That sounds like a mistake.

For instance, I have the cursor type change to `box' for overwrite
mode and read-only buffers, and use type `bar' as the default.  And
the type changes to `box' when Emacs is idle.  But these things are
user choices.  They happen only if a user chooses them, and the
particular types used for each state are also customizable.

Emacs should do likewise: make such behavior optional.  You speak
of letting users opt out, but only if the `cursor-type' has been
modified, whatever that might mean (does it matter how it was
modified or how long ago?).

The problems this proposal purports to solve are exaggerated, IMO:

1. The 1st reason given for the proposal is to let users tell when
the active region is empty: "when the region is active but empty,
there's no visual indication of this status."

A better way to do that is to include the code from library
`modeline-posn.el' for this: show the region size, highlighted
with face `region', in the mode line in place of the buffer size
whenever the region is active.

2. The 2nd reason given for the proposal: "Neither is there
indication of the active region if point is one less than mark and
blink-cursor-mode is off."

I don't see that.  For me, the active region is highlighted in that
case.  Am I missing something?

And again, the `modeline-posn.el' region-size indication shows
clearly when the region size is 1, regardless of the where point
is relative to mark.

3. The 3rd reason given for the proposal: "Also, if point is
greater than mark, the active region is highlighted, but the block
cursor does an inverse-video highlight of the character following
the region."

So?  The cursor, block or otherwise, indicates point: where text
will insert.  A block cursor is on top of the character after
point, but that does not mean or suggest that that character is
selected, i.e., is in the region.

The proposal says that this "(speaking from experience) an Emacs
newbie finds distracting, since it seems to indicate that that
character is also part of the region, even though it actually
isn't."

In what way does it "seem to indicate" that the character under
the cursor is part of the region?  It never indicated that to me,
and I don't see how it would for others.

And even if some newbie did mistakenly get that impression, how
long does it take to realize that that impression is incorrect?
Emacs provides plenty of visual feedback about what is the
correct impression.  It doesn't take any thought to find out what
is in the region and what is not.  It just takes one or two
selections - an experimentation of maybe 20 seconds or so.
Seriously, how long did it take you to figure out the truth here?

4. The 4th and final reason given for this proposal: "Adding to
the confusion, the inverse-video highlighted character actually
is part of the region in the case that point is less than mark."

So?  The position of the cursor does not tell you anything about
whether the character after it (for `bar') or under it (for
`box') is part of the region.  For `bar', you can see the
character after the cursor, so you can tell.  OK.  For `box',
you can see the character also, and you can see the `region'
face applied to it as foreground.  I.e., you see the character
highlighted as part of the region, but you see it in inverse
video.

Admittedly, the `region' face as background instead of
foreground is more obvious.  I'll grant you that. But that alone
does not justify this feature being hardcoded into Emacs, IMO.

If this feature is added, please make it optional, easy to
change (turn off), and preferably not on by default.  And you
are welcome to consider incorporating the behavior of
`modeline-posn.el'.
http://www.emacswiki.org/emacs-en/download/modeline-posn.el