bug#36474: Algorithm in electric-pair--unbalanced-strings-p unsuitable for CC Mode

[Alan Mackenzie <acm@muc.de>]

On Tue, Jul 02, 2019 at 15:13:35 +0100, João Távora wrote:
>  > Hello João and Emacs.

> Hello Alan and Emacs,

> On Tue, Jul 2, 2019 at 2:16 PM Alan Mackenzie <acm@muc.de> wrote:

> > This is a follow up bug to bug #36423: 27.0.50; electric-pair-mode not
> > working properly depending of file content.

> [did you mean to copy bug-gnu-emacs@gnu.org, or emacs-devel@gnu.org?
> I'm assuming the latter and correcting]

No, it's a bug, therefore I submitted a bug report.

> > Start the Emacs master (up to date state as of 2019-07-02T14:30 +0000)
> > with emacs -Q, put the following in a C++ Mode buffer and enable
> > electric-pair-mode:

> > "foo\n

> > .  Type a " at the end of foo.  electric-pair-mode wrongly inserts two
> > "s.

> > Diagnosis: electric-pair--unbalanced-strings-p works after the (single)
> > newly typed " has been stripped from the buffer.  It attempts to
> > determine whether there are any open strings after the point of
> > insertion.  It does this by using parse-partial-sexp, and checks (nth 3
> > <result>) as evidence of an open string.

> I'm afraid this is a (another?) direct consequence of the NL-terminated
> strings feature that you introduced more than one year ago.  If you
> remember, this had various consequences vis-a-vis balancing,
> broke a test (one that I disabled in the expectation that a fix would be
> made available, which I don't think happened). Here are some points of
> that thread:

> https://lists.gnu.org/archive/html/emacs-devel/2018-06/msg00551.html
> https://lists.gnu.org/archive/html/emacs-devel/2018-06/msg00580.html

> I think I made my views clear back then: NL-terminated strings are a
> misfeature. The only argument _for) them, that they mimic what some
> compilers do, is very weak because (1) the code is invalid in both
> situations (not in any way "slightly less" invalid in any of them) and
> (2) compilers don't edit code and so have different requirements.

> The arguments _against_ NL-terminated strings is that they (1) break
> longstanding features such as sexp-based navigation (e.g. `up-list`
> and friends) for modes such say, `js-mode` and (2) break features
> that expect this to work, most notably electric-pair-mode.

This isn't true.  If those other feature no longer work with an up to
date Emacs, they should be fixed.

The fontification that CC Mode does is natural and helpful, and users
haven't complained about it (except when there've been bugs).  There have
certainly been no complaints about using font-lock-warning-face for the
invalid string delimiters, and font-lock-string-face for valid ones.

> Moving forward:

> 1. We can consider that electric-pair-mode is doing the right thing.
> Indeed if NL is indeed terminating a string, then quote balance has been
> maintained after the double quote insertion, i.e. it has not worsened.
> That is the general contract of  `electric-pair-preserve-balance`.

There is a bug: on typing a " to close a string, two "s are inserted into
the buffer, the second one being invalid.  This make absolutely no sense
from a user point of view.

> 2.The NL-terminated string feature is removed (or, if you prefer, is
> made disableable). This would restore the behaviour that most users
> would expect coming over, from say python-mode or js-mode. Perhaps
> it can already be disabled with a couple of lines of emacs-lisp tweaking
> the syntax-table.

The invalid string feature is here to stay.  It is a positive user
feature.  CC Mode has often been a pioneer in inventing Emacs features,
and this is just such a feature.

> 3. Someone comes up with a suitable indirection that doesn't involve
> hardcoding `cc-mode` in elec-pair.el.  That indirection would
> presumably do what you want for modes `cc-mode` derived
> from cc-mode.

There is already a great deal of such indirection in electric-pair-mode.
(Look for "(funcall electric-pair-....)" in elec-pair.el.)  Maybe there
is enough there already to accomodate CC Mode, maybe an extra function
variable would need introducing.

For this, I think we would both rather that you amend elec-pair.el rather
than me.

> 4. Someone reinvents electric-pair-mode in cc-model.el.
> Let's not do this.

No, let's not do that!  :-)

> I prefer 2.

That isn't an option.  Unless you can come up with another workable
strategy that achieves the same effect.

> Thanks,
> João



> > This does not work in CC Mode, since although there is an open string
> > marker (with a string fence syntax-table property on it) this is
> > "closed" (from parse-partial-sexp's point of view) by the string fence
> > property on the newline at the end of the line.
> > electric-pair--unbalanced-strings-p thus returns the wrong result.

> > A more suitable algorithm might look something like this: check whether
> > the newly inserted " has a string fence syntax-table text property.
> > (Its insertion will have already triggered the before- and
> > after-change-functions which set this property.)  If so, there is an
> > open string.  Of course, this only applies to CC Mode modes.

> --
> João Távora

-- 
Alan Mackenzie (Nuremberg, Germany).