scratch/fontify-open-string. [Was: CC Mode and electric-pair "problem".]

[Alan Mackenzie <acm@muc.de>]

Hello, Eli.

On Sun, Jul 01, 2018 at 18:22:48 +0300, Eli Zaretskii wrote:
> > Date: Sat, 30 Jun 2018 20:14:47 +0000
> > Cc: cpitclaudel@gmail.com, emacs-devel@gnu.org,
> >   stephen_leake@stephe-leake.org, monnier@IRO.UMontreal.CA,
> >   joaotavora@gmail.com
> > From: Alan Mackenzie <acm@muc.de>

[ .... ]

> > The initial problem I tried to solve was for CC Mode source files with
> > things like:

> >     char foo[] = "foo
> >     char bar[] = "bar";

> > Historically, the missing " on "foo has caused subsequent lines to have
> > their string quoting reversed.  This is not good.

> But not really a catastrophe, IMO.

Perhaps not, but it is nevertheless bad.  That it is so difficult to do
anything about is also bad.

> > What I'm now proposing, and implementing as a trial, is to enhance the
> > syntax table facilities to support unterminated strings.  There will be
> > an extra syntax flag `s' on newlines meaning "terminate any open string".
> > This is straightforward for forward scanning, but somewhat complicated
> > for backward scanning.  However, it does enable unterminated strings to
> > be easily fontified to EOL in any language, with minimal effort.

> > It should allow the desired fontification without causing problems for
> > electric-pair-mode.

> > Stefan is concerned that the extra functionality may not justify the
> > increase in complexity in syntax.c.

> So am I.  I'm also concerned that introducing this will slow down
> various syntax-related features, only to cater to what I consider a
> minor improvement at best.

> Of course, if the extra functionality turns out to be not as complex
> as Stefan fears and won't cause any significant slowdown that concerns
> me, then perhaps we should have it.  But is that a reasonable
> assumption?

It's no longer a matter of assumption.  Earlier on this afternoon, I
committed a preliminary working version of this change to the branch
scratch/fontify-open-string.

The most complicated part of the change is the new function
back_maybe_string in syntax.c.  This is a mere 137 lines long.  Even if
perhaps not fully fleshed out, it's not far off.  By contrast,
back_comment (which is also called at every newline when there're line
comments) is 289 lines long.

I have amended shell-script-mode to use this new strategy.  This required
changing just one line in sh-script.el.  To font-lock.el I have added an
optional feature to put warning-face on the opening ".

I think it is notable just how easy this new feature is to use.
Essentially any mode[*] can use it with a one line change (to the
syntax table code for \n).

[*] Except, currently, CC Mode.  ;-(

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).