bug#36474: Algorithm in electric-pair--unbalanced-strings-p unsuitable for CC Mode

[Alan Mackenzie <acm@muc.de>]

Hello, João.

On Tue, Jul 02, 2019 at 18:22:34 +0100, João Távora wrote:
> On Tue, Jul 2, 2019 at 5:04 PM Alan Mackenzie <acm@muc.de> wrote:

> > > [did you mean to copy bug-gnu-emacs@gnu.org, or emacs-devel@gnu.org?
> > > I'm assuming the latter and correcting]
> > No, it's a bug, therefore I submitted a bug report.

> You should use the X-Debbugs-CC feature then (and why not continue
> in the existing bug 36423?)

I didn't know about the former, and as for the bug, it is a different
bug with differenct causes from #36423.

> Anyway, I insist this matter be brought to emacs-devel because it's a
> followup to a discussion that started there but never reached a
> suitable conclusion. For that reason, and because I provide a
> workaround for the bug  at the end of  this message, I'm cross-posting
> this one mail to both the bug list and emacs-devel.

> > > The arguments _against_ NL-terminated strings is that they (1) break
> > > longstanding features such as sexp-based navigation (e.g. `up-list`
> > > and friends) for modes such say, `js-mode` and (2) break features
> > > that expect this to work, most notably electric-pair-mode.

> > This isn't true.

> What "isn't true"? Have those features broken or not?

They may well be broken.  CC Mode hasn't broken them.  They made invalid
assumptions, which turned out to be unjustified.

> They worked before the fe06f643b commit and don't work after the
> commit. It sounds quite unrefutable to me.

I don't know what you're talking about.

> > If those other feature no longer work with an up to date Emacs, they
> > should be fixed.

> I've stated this repeatedly in the life of this discussion: it's not
> just about electric-pair-mode. If you try to M-x up-list from a
> multi-line string in emacs 26.1 it works just as well in js-mode and
> c++-mode.  In emacs master it does not in c++-mode. Same with
> forward-sexp on the starting delimiter, etc.

I've just tried these in a multiline string in C++ Mode.  Both up-list
and forward-sexp work just fine.  I don't know what you're doing.

> > The fontification that CC Mode does is natural and helpful, and users
> > haven't complained about it (except when there've been bugs).

> Yes, users haven't complained except when users have complained.

> > There have certainly been no complaints about using
> > font-lock-warning-face for the invalid string delimiters, and
> > font-lock-string-face for valid ones.

> That's because providing this annotation is perfectly fine.  The problem
> is providing it _at the expense of other features_.

If other features are broken (and your list of other broken features, so
far, is empty), they should be fixed.

> And _that's_ what they've complained about: an average user has no
> obvious way of telling that the particular implementation of the red
> annotation thingy is guilty of breaking his C-M-u or his
> electric-pair-mode.

That's groundless disparagement.  C-M-u works, and electric-pair-mode is
broken because it's broken.  In one place it's using scan-sexps to move
forward over whitespace, totally oblivious to the possibility of
syntax-table text properties (which have been in use since Emacs-21).
That's broken code.

> He/she might even judge the latter more vital than said red thingy
> , an annotation which he/she will get by other means if using
> very popular packages such as flycheck, or flymake, or eglot, or
> lsp-mode, etc. These normally call the compiler directly on the
> source code and highlight those and many other errors.

Irrelevant.

> On the other hand, if what you want is the red annotation, are you
> absolutely sure there isn't a better way to get it?

No, I'm not.  That's why I invited you to come up with a better way, if
you can.

> And if you are, are you also absolutely sure you need to put it in the
> code and and not provide an easy way to turn it off?

It's a core feature of the mode, not an option.

> > For this, I think we would both rather that you amend elec-pair.el rather
> > than me.

> I'll be "mulling this over". There are potentially many other points
> of breakage

"Potentially" many?  So far, there is precisely one, in
electric-pair--unbalanced-strings-p.  I thought I was doing you a favour
by diagnosing the trouble.  If I'd known I'd get the reaction from you
I've just got, I wouldn't have bothered.

> that would need such an indirection, and doing that to serve just a
> particular cc-mode quirk doesn't sound priority to me.

No, you'd be cleaning up your code, to conform with the reality that in
2019 major modes use syntax-table text properties.  Features from CC
Mode have a habit of migrating to the Emacs core.

> In the meantime, let others chime in.

> Also, in the meantime, for a user that is bothered by this bug,
> I'd recomend to put something like this in his/her .emacs file:

>   (defun c-unescaped-nls-in-string-p (&optional quote-pos) t)

It's free software, but that's a stupid thing to do.

> I had something more elaborate in my setup but just this
> seems to fix it in my testing.

> There is a also a very promising variable, c-multiline-string-start-char,
> that I think would be a good candidate for customizing this, ....

It is not a customisation variable.  It is a language definition
variable.

> .... but I haven't messed with it enough. Just setting it from .emacs
> doesn't do the trick. Perhaps in a mode hook?

Or, alternatively, actually fix the problems which have been festering
for years or decades, and are just now revealing themselves.  Thus far,
there's exactly one such problem in electric-pair--unbalanced-strings-p.

> --
> João Távora

-- 
Alan Mackenzie (Nuremberg, Germany).