bug#38406: 27.0.50; post-self-insert-hook does not hold its contract in cc-mode derived modes

[Alan Mackenzie <acm@muc.de>]

Hello, Eli.

On Sun, Dec 01, 2019 at 19:59:54 +0200, Eli Zaretskii wrote:
> > Date: Sun, 1 Dec 2019 15:07:38 +0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: 38406@debbugs.gnu.org

> > > > There are other possible "fixes", for example modifying these
> > > > functions so that they don't use self-insert-command at all, but
> > > > somehow I don't think that's what you want.

> > > I don't think that the code that is implemented against the
> > > contract listed in the hook documentation should be rewritten. If
> > > electric stuff is so that important and there is no way to disable
> > > it by default then at least a function to unbind the electric
> > > functionality the documentation of post-self-insert-hook should
> > > state: "Don't rely on this hook in cc derived modes because of
> > > {implementation details}. If you still want to use
> > > post-self-insert-hook disable use {implementation details} to turn
> > > electric off."

> > The problem you have stumbled over is more of a political problem
> > than a technical one.

> Can we please make it technical again?  Why can't the CC Mode function
> which temporarily disables post-self-insert-hook call the hook
> functions after it does its thing?  (I think I already asked this in
> the past, but I cannot find that question or any discussion of it.)

See bug #33794 (but a lot of the discussion is unedifying).

post-self-insert-hook's functions, unusually amongs hooks, interfere with
its triggering event.  This contrasts with, say, after-change-functions,
where the functions don't insert into or delete from the buffer, or
pre-redisplay-functions, where the functions don't try to prevent a
particular window getting displayed.  But post-s-i-h customarily makes
its own buffer changes such that self-insert-command no longer performs
its prime function, which is to insert exactly one copy (or N copies) of
the typed key into the buffer.  This makes it problematic for Lisp code
which calls self-insert-command.

It would be nice if functions on post-self-insert-hook could be split
into "disruptive" ones and "safe" ones, so that a function such as
c-electric-brace could elect to run just the "safe" ones.  I think Ivan's
functions would likely be classed as "safe".  How about this idea for
Emacs 28?

Anyhow back to your question: c-electric-brace carefully calls
electric-pair-post-self-insert-function testing various states afterwards
(such as the buffer reducing in size) so as to be able to complete
c-electric-brace's own processing.  Just calling post-self-insert-hook
instead could easily upset this balance.  Unfortunately this hook can
contain arbitrary code, and frequently does.

So to call this hook at the end of c-electric-brace would mean having to
filter the hook first (at the very least, to remove
electric-pair-post-self-insert-function), which just seems very hackish
and unsatisfactory.

As a statistic, there are approximately 111 occurrences of
(self-insert-command ...) in the Emacs Lisp sources.  Any of them might
be vulnerable to disruption by post-self-insert-hook.

-- 
Alan Mackenzie (Nuremberg, Germany).