Re: Unbalanced change hooks (part 2) [Documentation fix still remaining]

[Daniel Colascione <dancol@dancol.org>]

On 08/28/2016 05:09 PM, Stefan Monnier wrote:
>> 1) b-c-f isn't currently balanced with a-c-f in the insdel.c implementation
>
> There's no guarantee that it be balanced, indeed.
>
>> 2) b-c-f isn't currently guaranteed to be called at all before any given
>> call to a-c-f (e.g., the C-x C-f case)
>
> It is guaranteed, modulo bugs (the insert-file-contents (with replace) case).

Sure. The current implementation provides neither property though.

>> 2) the documentation suggests that a-c-f and b-c-f are balanced
>
> I don't see where in the documentation it suggests it.

See below.

>> 3) many people, Alan and me included, were under the impression that b-c-f
>> and a-c-f were balanced and wrote code accordingly
>
> Ah, so you too have code which makes this assumption.  Could you point
> to it, explaining what it does and why it makes this assumption, so we
> have a better picture of why it could be useful?

Same as Alan's --- examining the buffer before the change to see which 
brace-delimited scopes need to be invalidated. It's not possible to do 
that in a-c-f because the text that was changed is unavailable in a-c-f. 
Emacs core also contains a lot of code that looks fragile under 
asymmetric b-c-f behavior: see tex-mode, org, and viper. (I'm not 
claiming that these modes are broken, but it's not obvious from the code 
that they aren't broken.)

Yes, it's always possible, with enough shadowing of buffer contents, to 
rely on a-c-f alone. I think. But since Emacs already *has* the b-c-f 
feature, since it's pretty widely used, and since it *almost* works as 
expected today, it's be best to just change the behavior instead of 
sending a "gotcha! The manual is correct only under close reading!" 
message to everyone who uses b-c-f.

>> AIUI, Eli's position is that the current behavior should not change at all,
>> because making changes at this level is too risky.
>
> I don't think he said "at all", but just that it's tricky to make such
> changes, so they had better be backed by really good reasons.
>
>> Your position is that #2 is a bug, but #1 is not, and the
>> documentation needs to change.
> Please trust me that the documentation misleads.
> Almost: I don't see the need to change the doc because I don't see where
> it misleads.

b-c-f and a-c-f are symmetric in name and signature. b-c-f is documented 
as "List of functions to call before each text change." a-c-f is 
documented as "List of functions to call after each text change." The 
elisp manual documentation is similarly symmetric. This symmetry 
produces an expectation that the behavior is symmetric, and this 
expectation is reinforced by how the observed behavior is almost always 
symmetric in practice. Symmetric behavior here is also what intuitively 
makes sense.

Please believe me when I say that the current documentation misleads.

I mean, imagine you were writing a home automation system and saw 
before-doorbell-functions and after-doorbell-functions hooks, read the 
documentation, and saw that the former is a list of functions to be 
called "before the doorbell chime rings" and the latter is a list of 
functions to be called "after the doorbell chime rings".

Imagine that when you tested these hooks, you saw that the b-d-f were 
indeed run before you heard the chime and a-d-f functions after. What on 
earth would make you suspect that this behavior wouldn't be the case in 
some circumstances? Wouldn't you treat a failure to run b-d-f if the 
doorbell butter were pressed forcefully as a bug? Wouldn't you think it 
strange if pressing the doorbell button quickly five times in a row 
after 9pm on Tuesdays produced five chimes, five calls to a-d-f, but 
only one call to b-d-f?

Sure, I guess you could argue that current Emacs behavior is consistent 
with the manual, but it's not what anyone would reasonably expect, and 
the current behavior is surprising even to people who have been writing 
elisp for a long time. It's like that joke about the mathematician who 
sees only half of one black sheep in Scotland.

A feature that almost works as expected --- except in certain weird 
cases where it corrupts your state --- is at best an attractive nuisance.

>> So what do you propose doing? Upthread, I explained why I think that fixing
>> both #1 and #2 is pretty low-risk.
>
> The problem is that changing the code w.r.t b/a-c-f might cause
> *other* changes.  Clearly making them perfectly balanced and paired
> should not *in itself* introduce any bugs: the bugs we're afraid of are
> the accompanying unintended changes.

I was under the impression that Eli's concern was with the effect of the 
b-c-f change itself, not with bugs that might be introduced by the 
change. I'm confident that with enough review, the core code could be 
changed to make b-c-f and a-c-f symmetric without causing weird bugs 
elsewhere. The necessary refactoring will probably make the logic 
cleaner as well.

Of course there's a risk that changing b-c-f will itself produce weird 
side effects, but I have a hard time seeing how any code could actually 
depend on the current surprising behavior.