New hook before-region-change-functions wanted

New hook before-region-change-functions wanted

[Alan Mackenzie]

Hello, Emacs.

I would like to implement a new abnormal hook,
before-change-region-functions.  It would be called just before a change
in the buffer's restriction (i.e., from widen and from narrow-to-region)
and each function on it would take two parameters, the beginning and end
of the new region being set.

The reason I want it is as part of the solution to bug #22983
(syntax-ppss returns wrong result).  I envisage two (or possibly more)
mutually independent caches, and a switch being made to the appropriate
cache when the region is changed.  This switch would be made inside a
function on before-change-region-functions.

While it is true that this hook is not absolutely necessary, in that the
cache switch could be made by the first call to syntax-ppss after the
region change, it makes the cache switch clean.  In particular, the
cache will always be in synch with the region, and any functions which
examine the cache at an arbitrary time (for example, jit-lock
functions), will get the right cache.  The mechanism will also work a
little faster with the new hook than without it, given that changes of
restriction are more rarely effected than syntax-ppss is called.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: New hook before-region-change-functions wanted

[John Wiegley]

>>>>> "AM" == Alan Mackenzie <acm@muc.de> writes:

AM> I would like to implement a new abnormal hook,
AM> before-change-region-functions. It would be called just before a change in
AM> the buffer's restriction (i.e., from widen and from narrow-to-region) and
AM> each function on it would take two parameters, the beginning and end of
AM> the new region being set.

I'd call it before-change-restriction-functions, if we wanted the hook.  It
could have interesting uses beyond syntax-ppss too. I'm not sure, though.

AM> The reason I want it is as part of the solution to bug #22983 (syntax-ppss
AM> returns wrong result). I envisage two (or possibly more) mutually
AM> independent caches, and a switch being made to the appropriate cache when
AM> the region is changed. This switch would be made inside a function on
AM> before-change-region-functions.

I was talking to Stefan about syntax-ppss yesterday (in person!). He
acknowledged the bug in syntax-ppss, and said he would like to find a better
way to set the right stage for the fix, involving a more rational scheme for
narrowing (if I understood correctly). Perhaps I can encourage him to chime in
with what he was thinking, because I'd like to hear that before committing to
a new hook that solves one aspect of problem.

It'll likely be a few days before he's done traveling home and gets through
his backlog before we hear from him.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: New hook before-region-change-functions wanted

[Eli Zaretskii]

> Date: Fri, 8 Sep 2017 14:46:57 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> I would like to implement a new abnormal hook,
> before-change-region-functions.  It would be called just before a change
> in the buffer's restriction (i.e., from widen and from narrow-to-region)
> and each function on it would take two parameters, the beginning and end
> of the new region being set.

If this hook will react to restriction, then
before-change-region-functions is not an appropriate name for it.
It's not the region you are changing.

Re: New hook before-region-change-functions wanted

[Alan Mackenzie]

Hello, Eli.

On Fri, Sep 08, 2017 at 18:10:31 +0300, Eli Zaretskii wrote:
> > Date: Fri, 8 Sep 2017 14:46:57 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > I would like to implement a new abnormal hook,
> > before-change-region-functions.  It would be called just before a change
> > in the buffer's restriction (i.e., from widen and from narrow-to-region)
> > and each function on it would take two parameters, the beginning and end
> > of the new region being set.

> If this hook will react to restriction, then
> before-change-region-functions is not an appropriate name for it.
> It's not the region you are changing.

How about before-change-restriction-functions?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: New hook before-region-change-functions wanted

[Dmitry Gutov]

On 9/8/17 5:46 PM, Alan Mackenzie wrote:

> While it is true that this hook is not absolutely necessary,

+1

> in that the
> cache switch could be made by the first call to syntax-ppss after the
> region change, it makes the cache switch clean.  In particular, the
> cache will always be in synch with the region, and any functions which
> examine the cache at an arbitrary time (for example, jit-lock
> functions), will get the right cache.

Do we ever want the jit-lock code examining the cache contents directly?

> The mechanism will also work a
> little faster with the new hook than without it, given that changes of
> restriction are more rarely effected than syntax-ppss is called.

I don't imagine the performance difference will be in any way 
noticeable. It's just one "if".

Re: New hook before-region-change-functions wanted

[Stefan Monnier]

>> The mechanism will also work a little faster with the new hook than
>> without it, given that changes of restriction are more rarely
>> effected than syntax-ppss is called.
> I don't imagine the performance difference will be in any way
> noticeable. It's just one "if".

Furthermore, this `if` will avoid two cache switches when
a save-restriction+widen happens between two syntax-ppss calls.

All in all, I think both approaches would work equally well,


        Stefan

Re: New hook before-region-change-functions wanted

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I would like to implement a new abnormal hook,
  > before-change-region-functions.  It would be called just before a change
  > in the buffer's restriction (i.e., from widen and from narrow-to-region)
  > and each function on it would take two parameters, the beginning and end
  > of the new region being set.

That kind of hook is pandora's box.  It is too powerful, and would
interact in complex ways.  There will be many cases that are hard to
implement correctly, or cause trouble.  To implement this probably
means future hassles for ourselves and for package developers.

  > The reason I want it is as part of the solution to bug #22983
  > (syntax-ppss returns wrong result).  I envisage two (or possibly more)
  > mutually independent caches, and a switch being made to the appropriate
  > cache when the region is changed.

For this, all you need to do is record, for each alternative cashe,
what buffer restrictions it corresponds to.  Then use the right cache.

That is a much safer way to do it.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New hook before-region-change-functions wanted

[Alan Mackenzie]

Hello, Richard.

On Fri, Sep 08, 2017 at 18:22:15 -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>   > I would like to implement a new abnormal hook,
>   > before-change-region-functions.  It would be called just before a change
>   > in the buffer's restriction (i.e., from widen and from narrow-to-region)
>   > and each function on it would take two parameters, the beginning and end
>   > of the new region being set.

> That kind of hook is pandora's box.  It is too powerful, and would
> interact in complex ways.  There will be many cases that are hard to
> implement correctly, or cause trouble.  To implement this probably
> means future hassles for ourselves and for package developers.

I actually implemented it, and it worked as intended.  It's not often I
get criticised for proposing something too powerful.

>   > The reason I want it is as part of the solution to bug #22983
>   > (syntax-ppss returns wrong result).  I envisage two (or possibly more)
>   > mutually independent caches, and a switch being made to the appropriate
>   > cache when the region is changed.

> For this, all you need to do is record, for each alternative cache,
> what buffer restrictions it corresponds to.  Then use the right cache.

> That is a much safer way to do it.

OK.  I will keep my new implementation under lock and key.  As you
suggest, managing without it is not difficult.

Thanks for the criticism.

> -- 
> Dr Richard Stallman
> President, Free Software Foundation (gnu.org, fsf.org)
> Internet Hall-of-Famer (internethalloffame.org)
> Skype: No way! See stallman.org/skype.html.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: New hook before-region-change-functions wanted

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > That kind of hook is pandora's box.  It is too powerful, and would
  > > interact in complex ways.  There will be many cases that are hard to
  > > implement correctly, or cause trouble.  To implement this probably
  > > means future hassles for ourselves and for package developers.

  > I actually implemented it, and it worked as intended.

I think you mean it works as intended in your use case.  I am talking
about something else.

My experience says that this kind of feature is likely to fail in
various peculiar cases, which would make it a maintenance burder.
Also, it is likely to show show strange confusing behavior which would
lead to trouble in debugging.  We are better off without it.

The other approach has the virtue of being entirely internal.  It will
not require an announcement, or adding to the Lisp manual.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New hook before-region-change-functions wanted

[Alan Mackenzie]

Hello, Richard.

On Sat, Sep 09, 2017 at 22:44:19 -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>   > > That kind of hook is pandora's box.  It is too powerful, and would
>   > > interact in complex ways.  There will be many cases that are hard to
>   > > implement correctly, or cause trouble.  To implement this probably
>   > > means future hassles for ourselves and for package developers.

>   > I actually implemented it, and it worked as intended.

> I think you mean it works as intended in your use case.  I am talking
> about something else.

Note I said "implemented", not "have implemented" and "worked", not
"works" - eveything in the past tense, not the present.

> My experience says that this kind of feature is likely to fail in
> various peculiar cases, which would make it a maintenance burden.
> Also, it is likely to show show strange confusing behavior which would
> lead to trouble in debugging.  We are better off without it.

I accept this, though not without some regret.  I am no longer pressing
ahead with this hook.

> The other approach has the virtue of being entirely internal.  It will
> not require an announcement, or adding to the Lisp manual.

Yes.  This other approach is slightly more difficult to debug with, but
not unreasonably so.

Again, thanks for interfering, here.  :-)

> -- 
> Dr Richard Stallman
> President, Free Software Foundation (gnu.org, fsf.org)
> Internet Hall-of-Famer (internethalloffame.org)
> Skype: No way! See stallman.org/skype.html.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: New hook before-region-change-functions wanted

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The other approach has the virtue of being entirely internal.  It will
  > > not require an announcement, or adding to the Lisp manual.

  > Yes.  This other approach is slightly more difficult to debug with, but
  > not unreasonably so.

Just from curiosity, why is it more difficult to debug with?
Since it has no effect on behavior, and its only effect is to speed
up some cases, I'd expect it to have no effect on debugging.

Is there something I am missing?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New hook before-region-change-functions wanted

[Alan Mackenzie]

Hello, Richard.

On Sun, Sep 10, 2017 at 21:17:57 -0400, Richard Stallman wrote:
>   > > The other approach has the virtue of being entirely internal.  It will
>   > > not require an announcement, or adding to the Lisp manual.

>   > Yes.  This other approach is slightly more difficult to debug with, but
>   > not unreasonably so.

> Just from curiosity, why is it more difficult to debug with?
> Since it has no effect on behavior, and its only effect is to speed
> up some cases, I'd expect it to have no effect on debugging.

> Is there something I am missing?

As I said, the difference was small.

With the hook in place, the filling and use of the cache was entirely
disjoint from swapping the caches in and out - the former occurred on
calling syntax-ppss, the latter on the hook called from
widen/narrow-to-region.

Using the actual solution, the swapping in and out of the caches
occurred at the beginning of syntax-ppss (if needed), and NOT on
widen/n-t-r.  So until I got used to it, I was calling n-t-r, then
forgetting a token invocation of syntax-ppss, and wondering why the
caches hadn't been swapped.

In that sense, the hook would have been a cleaner solution.  A pity
about its overwhelming disadvantages.

> -- 
> Dr Richard Stallman
> President, Free Software Foundation (gnu.org, fsf.org)
> Internet Hall-of-Famer (internethalloffame.org)
> Skype: No way! See stallman.org/skype.html.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: New hook before-region-change-functions wanted

[Andreas Röhler]

[-- Attachment #1: Type: text/plain, Size: 1186 bytes --]



On 10.09.2017 04:44, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>    > > That kind of hook is pandora's box.  It is too powerful, and would
>    > > interact in complex ways.  There will be many cases that are hard to
>    > > implement correctly, or cause trouble.  To implement this probably
>    > > means future hassles for ourselves and for package developers.
>
>    > I actually implemented it, and it worked as intended.
>
> I think you mean it works as intended in your use case.  I am talking
> about something else.
>
> My experience says that this kind of feature is likely to fail in
> various peculiar cases, which would make it a maintenance burder.
> Also, it is likely to show show strange confusing behavior which would
> lead to trouble in debugging.  We are better off without it.
>
> The other approach has the virtue of being entirely internal.  It will
> not require an announcement, or adding to the Lisp manual.
>


+1, likewise a related earlier message.


[-- Attachment #2: Type: text/html, Size: 1568 bytes --]

Re: New hook before-region-change-functions wanted

[Stefan Monnier]

> The reason I want it is as part of the solution to bug #22983
> (syntax-ppss returns wrong result).  I envisage two (or possibly more)
> mutually independent caches, and a switch being made to the appropriate
> cache when the region is changed.  This switch would be made inside a
> function on before-change-region-functions.

Regarding a hook placed on narrow-to-region (from where I stand, `widen`
is just a special call to narrow-to-region):
- As discussed in the past, narrow-to-region needs to be
  extended/adjusted/complemented with some way to distinguish "hard
  narrowing" (as is used in Info-mode, and might be used in cases of
  multiple-major-modes) from "soft narrowing", so there's a risk such
  changes would impact the desired behavior of this hook.
- syntax-ppss doesn't need this hook, and even if it were added it
  probably wouldn't benefit from it: you can just as easily change
  syntax-ppss to compare (point-min) with its last value in order to
  detect calls to narrow-to-region.  There are minor advantages to using
  your hooks, but there are also similarly minor advantages to
  performing the detection dynamically in syntax-ppss, and I think
  overall neither approach would be noticeably better than the other.

> While it is true that this hook is not absolutely necessary, in that the
> cache switch could be made by the first call to syntax-ppss after the
> region change, it makes the cache switch clean.  In particular, the
> cache will always be in synch with the region, and any functions which
> examine the cache at an arbitrary time (for example, jit-lock
> functions), will get the right cache.

AFAIK noone ever looks at syntax-ppss's cache except for syntax-ppss.
jit-lock definitely doesn't.


        Stefan

Re: New hook before-region-change-functions wanted

[Alan Mackenzie]

Hello, Stefan.

On Sat, Sep 09, 2017 at 09:44:20 -0400, Stefan Monnier wrote:
> > The reason I want it is as part of the solution to bug #22983
> > (syntax-ppss returns wrong result).  I envisage two (or possibly more)
> > mutually independent caches, and a switch being made to the appropriate
> > cache when the region is changed.  This switch would be made inside a
> > function on before-change-region-functions.

> Regarding a hook placed on narrow-to-region (from where I stand, `widen`
> is just a special call to narrow-to-region):
> - As discussed in the past, narrow-to-region needs to be
>   extended/adjusted/complemented with some way to distinguish "hard
>   narrowing" (as is used in Info-mode, and might be used in cases of
>   multiple-major-modes) from "soft narrowing", so there's a risk such
>   changes would impact the desired behavior of this hook.
> - syntax-ppss doesn't need this hook, and even if it were added it
>   probably wouldn't benefit from it: you can just as easily change
>   syntax-ppss to compare (point-min) with its last value in order to
>   detect calls to narrow-to-region.  There are minor advantages to using
>   your hooks, but there are also similarly minor advantages to
>   performing the detection dynamically in syntax-ppss, and I think
>   overall neither approach would be noticeably better than the other.

OK, first of all, I've given up the hook proposal after strong
objections from Richard.

`widen' is physically a separate function from `narrow-to-region'.

As I see it, there is no such thing as "hard" or "soft" narrowing.
There's just narrowing which is completely defined as it always has
been.  I'm in favour of it staying that way, having proposed alternative
measures for multiple major modes.

> > While it is true that this hook is not absolutely necessary, in that the
> > cache switch could be made by the first call to syntax-ppss after the
> > region change, it makes the cache switch clean.  In particular, the
> > cache will always be in synch with the region, and any functions which
> > examine the cache at an arbitrary time (for example, jit-lock
> > functions), will get the right cache.

> AFAIK noone ever looks at syntax-ppss's cache except for syntax-ppss.
> jit-lock definitely doesn't.

Sorry, yes, I got a little confused.  font-lock.el calls syntax-ppss,
and I think I was worried that it might do so between a change in
restriction and the syntax-ppss cache's getting changed.  Or something
like that.  Nothing to worry about.

Incidentally, I've now posted a patch for bug #22983.  I'm interested in
what you think of it.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: New hook before-region-change-functions wanted

[Stefan Monnier]

> `widen' is physically a separate function from `narrow-to-region'.

It currently isn't implemented by calling narrow-to-region, but that's
just a technical detail: it behaves exactly like

    (defun widen () (narrow-to-region 1 (1+ (buffer-size))))

> As I see it, there is no such thing as "hard" or "soft" narrowing.

Indeed, the issue of soft-vs-hard narrowing can also be looked at in
a different way, and maybe the best way to solve it will not involve
changing narrowing at all.  Of Then again, maybe it will.

> Incidentally, I've now posted a patch for bug #22983.  I'm interested in
> what you think of it.

I'll try and look at it soon, thank you,


        Stefan

RE: New hook before-region-change-functions wanted

[Drew Adams]

> (defun widen () (narrow-to-region 1 (1+ (buffer-size))))

(defun widen () (interactive) (narrow-to-region 1 (1+ (buffer-size))))