Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Stefan Monnier]

> @@ -1240,6 +1240,7 @@ set_internal (Lisp_Object symbol, Lisp_Object newval, Lisp_Object where,
>  	return;
>      }
 
> +  maybe_set_redisplay (symbol);
>    sym = XSYMBOL (symbol);
 
>   start:

In dynamic-binding code, this is a chunk of code that's executed *very*
often, so I expect your change could have a measurable performance impact
(e.g. on Gnus code).

The write-barrier discussed in the past moved this check to within the
SYMBOL_CONSTANT_P test.  More specifically, it turned the
SYMBOL_CONSTANT_P boolean into a field with 3 different values,
depending on whether a write can:
- go through at full-speed
- goes through a watcher function (or call it a "barrier")
- is not allowed at all

This way, assignments to most variables is unaffected.


        Stefan

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Sat, 07 Nov 2015 18:40:30 -0500
> 
> The write-barrier discussed in the past moved this check to within the
> SYMBOL_CONSTANT_P test.  More specifically, it turned the
> SYMBOL_CONSTANT_P boolean into a field with 3 different values,
> depending on whether a write can:
> - go through at full-speed
> - goes through a watcher function (or call it a "barrier")
> - is not allowed at all
> 
> This way, assignments to most variables is unaffected.

Thanks.  AFAIU, the patch posted in that discussion cannot be used due
to copyright reasons.  So Someone™ will have to reimplement the idea
from scratch.  I cannot do that: I understand the idea, but don't know
enough about the machinery that's involved in binding symbols to work
out the low-level details.  Would you like to do that?

Also, we have this strong objection to the idea from Richard:

> Hooks on setting variables is a fundamentally bad idea
> because it means that Lisp code which appears to just bind variables
> can call functions where you did not expect it.
> [...]
> However, having any variable, the binding of which can run Lisp code,
> is an absolute disaster.  If the price we pay for to avoid that disaster
> is that we don't have the feature you would like, so be it.
> 
>   > Besides that, hooks on setting variables would be useful for debugging.
> 
> No matter what they would be useful for, it is not worth the chaos
> they would cause.
> 
> Maybe a specialized feature solely for debugging could be made safe.

AFAICT, these objections were never addressed in the discussions.  And
we _are_ talking about using such a facility for purposes other than
debugging, albeit internal purposes.  Maybe calling a C function for
"hooked" symbols takes care of Richard's objections, but then such a
function will probably have to consult some Lisp data to know what to
do with each "hooked" symbol.  Not sure if this is okay or sufficient.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Stefan Monnier]

> AFAICT, these objections were never addressed in the discussions.

That's just a disagreement.  I don't think that can be addressed.
I fully agree with Richard that it's a facility with which you can shoot
yourself in the foot in many different ways, but I don't find it to be
a reason to disallow it.

After all, this same objection can be raised against lots of other "hooks"
we have.  To me it just means that it's a feature that needs to be used
with care, not that it's a feature we should reject.

> And we _are_ talking about using such a facility for purposes other
> than debugging, albeit internal purposes.

That's exactly why I think this feature should be generic (and hence
dangerous): there are several different potential valid applications,
and the reason why it's safe/acceptable can completely vary from one use
to another (e.g. in the debugging case you'll probably want to allow
the hook to make random changes, whereas in general this would be
completely unacceptable.  In the present case, the hook would be
perfectly acceptable because its effect is "unnoticeable" since all it
does is set a flag somewhere).


        Stefan

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[John Wiegley]

I am unaware of the historical context for this argument. To chime in based on
what I've read from Eli:

Allowing dynamic behavior to trigger on modifying a variable could be very
valuable during debugging. GDB allows this in the form of watchpoints.

As long as we never allow the use of this facility to creep into Emacs itself,
I can see value in giving users another way to debug and/or customize their
environment, as long as they are fully informed of the potential costs.

John

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> Also, we have this strong objection to the idea from Richard:
>> Hooks on setting variables is a fundamentally bad idea
>> because it means that Lisp code which appears to just bind variables
>> can call functions where you did not expect it.
>> [...]
>> However, having any variable, the binding of which can run Lisp code,
>> is an absolute disaster.  If the price we pay for to avoid that disaster
>> is that we don't have the feature you would like, so be it.
>> 
>> > Besides that, hooks on setting variables would be useful for debugging.
>> 
>> No matter what they would be useful for, it is not worth the chaos
>> they would cause.
>> 
>> Maybe a specialized feature solely for debugging could be made safe.

> AFAICT, these objections were never addressed in the discussions. And we
> _are_ talking about using such a facility for purposes other than debugging,
> albeit internal purposes. Maybe calling a C function for "hooked" symbols
> takes care of Richard's objections, but then such a function will probably
> have to consult some Lisp data to know what to do with each "hooked" symbol.
> Not sure if this is okay or sufficient.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Mon, 09 Nov 2015 11:06:47 -0800
> 
> I am unaware of the historical context for this argument.

I can post the URL of the discussion.  It was a very long discussion,
most of it dedicated to Stefan's review of the proposed patch.  That
patch was eventually not admitted, because the author won't sign legal
papers as required by the FSF rules.

> Allowing dynamic behavior to trigger on modifying a variable could be very
> valuable during debugging. GDB allows this in the form of watchpoints.
> 
> As long as we never allow the use of this facility to creep into Emacs itself,
> I can see value in giving users another way to debug and/or customize their
> environment, as long as they are fully informed of the potential costs.

I agree, but the problem is that now someone needs to re-implement
that changeset from scratch, without looking too much at the patch
posted back then.  I cannot be that someone, unfortunately.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I fully agree with Richard that it's a facility with which you can shoot
  > yourself in the foot in many different ways, 

It's worse that that; what I said is stronger than that.

If you write private code that creates advice, at worst you shoot
yourself in the foot.  But if you insert code to create advice into
the code of Emacs, you shoot anyone else that needs to debug cases
where that advice has an effect.

						 but I don't find it to be
  > a reason to disallow it.

There's no need to disallow shooting yourself in the foot, but when I
was Emacs maintainer, I told people not to insert code that would
shoot my foot.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> I agree, but the problem is that now someone needs to re-implement that
> changeset from scratch, without looking too much at the patch posted back
> then. I cannot be that someone, unfortunately.

OK. Without a contributor on proper legal footing, we'll leave it be.

John

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Stefan Monnier]

>> I fully agree with Richard that it's a facility with which you can shoot
>> yourself in the foot in many different ways, 

> It's worse that that; what I said is stronger than that.
> If you write private code that creates advice, at worst you shoot
> yourself in the foot.  But if you insert code to create advice into
> the code of Emacs, you shoot anyone else that needs to debug cases
> where that advice has an effect.

You lost me here.  What I'm suggesting is to add a hook that's called
when some variables are modified.  So to me it sounds just like advice: as
long as you don't use this hook, Emacs's behavior is unchanged.
And just like with an advice, if you use such a hook in your ~/.emacs
you can shoot yourself in the foot, but if you use this is one of your
package, you might shoot the foot of your users.


        Stefan

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You lost me here.  What I'm suggesting is to add a hook that's called
  > when some variables are modified.  So to me it sounds just like advice: as
  > long as you don't use this hook, Emacs's behavior is unchanged.

Yes, the two issues are similar.

  > And just like with an advice, if you use such a hook in your ~/.emacs
  > you can shoot yourself in the foot, but if you use this is one of your
  > package, you might shoot the foot of your users.

Not just them.  You might shoot the foot of anyone that tries
to debug a problem that is reported by the users of your package.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Stefan Monnier]

>> And just like with an advice, if you use such a hook in your ~/.emacs
>> you can shoot yourself in the foot, but if you use this is one of your
>> package, you might shoot the foot of your users.

> Not just them.  You might shoot the foot of anyone that tries
> to debug a problem that is reported by the users of your package.

Same holds with `advice', of course.


        Stefan

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Noam Postavsky]

On Mon, Nov 9, 2015 at 2:17 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: John Wiegley <jwiegley@gmail.com>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
>> Date: Mon, 09 Nov 2015 11:06:47 -0800
>>
>> Allowing dynamic behavior to trigger on modifying a variable could be very
>> valuable during debugging. GDB allows this in the form of watchpoints.
>>
>> As long as we never allow the use of this facility to creep into Emacs itself,
>> I can see value in giving users another way to debug and/or customize their
>> environment, as long as they are fully informed of the potential costs.
>
> I agree, but the problem is that now someone needs to re-implement
> that changeset from scratch, without looking too much at the patch
> posted back then.  I cannot be that someone, unfortunately.
>

This functionality sounds very useful; I have definitely wanted
something like this while debugging in the past.

Where should I look to get an idea of how to implement it?

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Eli Zaretskii]

> Date: Thu, 12 Nov 2015 19:31:29 -0500
> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Cc: John Wiegley <jwiegley@gmail.com>, Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> > I agree, but the problem is that now someone needs to re-implement
> > that changeset from scratch, without looking too much at the patch
> > posted back then.  I cannot be that someone, unfortunately.
> >
> 
> This functionality sounds very useful; I have definitely wanted
> something like this while debugging in the past.
> 
> Where should I look to get an idea of how to implement it?

In the discussion I pointed to.  You will have to make a conscious
effort to read just the discussion parts and skip the code attachments
parts.  That's not very easy, but doable: just don't open any
attachments, and whenever you see "diff --git" move one to the next
message in the thread.

Alternatively, Someone™ could re-read the discussion and post the
description of what the posted code did (without showing any actual
code), then you could write the code using only that description,
without looking at the actual code posted back then.  This is a better
method, I think.  Are there any volunteers for the first part, of
coming up with the description?  If no one steps forward, I could do
this myself, but I think this is a good opportunity for someone to
study a somewhat complicated issue related to how the Emacs Lisp
interpreter works.

TIA.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Stefan Monnier]

> Alternatively, Someone™ could re-read the discussion and post the
> description of what the posted code did (without showing any actual
> code), then you could write the code using only that description,
> without looking at the actual code posted back then.  This is a better
> method, I think.  Are there any volunteers for the first part, of
> coming up with the description?  If no one steps forward, I could do
> this myself, but I think this is a good opportunity for someone to
> study a somewhat complicated issue related to how the Emacs Lisp
> interpreter works.

Actually, I think it's not that complicated:
- Look at the definition of SYMBOL_CONSTANT_P.
- Change its name to SYMBOL_SLOWWRITE_P.
- Change the field it tests from being a boolean to being a 3-valued
  thingy, with values "fullspeed", "hooked", and "readonly".
- Look at all users of that macro and update them accordingly.
- Add new primitives to set/unset/test the field (probably only
  allowing to change it from fullspeed to hooked and back).

You get to choose what "hooked" really means, of course.
For that, you should take into account the applications we have in mind:
- tickle the debugger.
- mark the mode-line for redisplay.
There could be others, of course.  Tho I'd expect most others would look
like the redisplay case, i.e. the hook function probably wouldn't do
much more than "mark somewhere that the assignment took place and keep
going".
Maybe it'd be nice to make sure that "defvaralias" could be
re-implemented on top of those hooks, even though I don't think such
a reimplementation is desirable at this point.


        Stefan

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Noam Postavsky]

On Fri, Nov 13, 2015 at 3:06 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> In the discussion I pointed to.

I think you haven't pointed to it yet (you said earlier "I can post
the URL", but you didn't post it). But perhaps Stefan's description
will be enough by itself anyway, I'll take a stab at it this weekend.

Re: [Emacs-diffs] master 19e09cf: Ensure redisplay after evaluation

[Eli Zaretskii]

> Date: Fri, 13 Nov 2015 11:36:59 -0500
> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Cc: John Wiegley <jwiegley@gmail.com>, Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> On Fri, Nov 13, 2015 at 3:06 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > In the discussion I pointed to.
> 
> I think you haven't pointed to it yet (you said earlier "I can post
> the URL", but you didn't post it).

Sorry, here it is:

   http://lists.gnu.org/archive/html/emacs-devel/2015-01/msg00974.html

It continues into February.

> But perhaps Stefan's description will be enough by itself anyway,
> I'll take a stab at it this weekend.

Thanks.