Re: CC Mode and electric-pair "problem".

[Alan Mackenzie <acm@muc.de>]

Hello, Stefan.

On Fri, Jun 29, 2018 at 00:11:24 -0400, Stefan Monnier wrote:
> >> > How about this idea: we add a new syntax flag to Emacs, ", which
> >> > terminates any open string, the same way the syntax > terminates any
> >> > open comment.  We could then set this syntax flag on newline.
> > I've been making negative comments about this suggestion of mine over
> > the last day or two.  I now believe, again, that the proposal is sound;

> It's definitely sound.  And I very much agree that it could be cleaner
> than the current code on `master`.  I dislike this solution mainly
> because it requires changes to Emacs's core API, so it bumps against my
> feeling that the need is not clearly documented: you think the new
> behavior is more often beneficial than the old behavior but we have no
> actual data to verify it.

No, what I think is much less nuanced: that the old behaviour is simply
wrong; the new behaviour is likewise correct.  If one were to design an
editor's functionality from scratch, nobody would advocate the old
behaviour - it happened because it needed no implementation effort.

> FWIW, I do not know that the old behavior is more often beneficial
> either, but I'm definitely not convinced that the new behavior is
> often enough more beneficial to justify such changes to syntax-tables.

I am in the middle of writing a trial implementation (code speaks louder
than words).  Thus far, it has already worked in shell-script-mode
(which required a one-line change, this:

    -       ?\n ">#"
    +       ?\n ">#s"

the new `s' flag is how I've constructed it, so far).

> But that's for Eli to judge.

> So let's look at the technical issues:
> You suggest introducing a new syntax-table thingy similar to > but for
> strings.  Let's call it ]

As I noted above, I have implemented it as another flag, `s'.

> - This implies we'll need a new C-level function `back_string` to jump
>   backward over such a ]-terminated string, corresponding to
>   back_comment.

Yes.

>  `back_comment` has proved to be rather nasty, so while
>   we can learn from it, part of what we learn is that jumping backward
>   over such things is much easier ....

much less easy.  :-)

>   .... than jumping forward, so this
>   innocuous ] will be more costly than might meet the eye.

It requires the new function, which at the moment seems somewhat less
complicated than back_comment, and this requires to be called from
scan_lists.

> - In CC-mode, \n already has syntax > so it can't also have syntax ]
>   How do you intend to deal with that: will you mark those few \n that
>   terminate strings with syntax-table text-properties?

This is simple with the flag `s'.  NL would thus have end-comment syntax
_and_ the `s' flag.  In scan_lists, back_comment will be tried before
what I'm calling `back_maybe_string', since being a comment ender must have
precedence over being a string terminator.

>   If so, what's the benefit over using string-fences?

String-fence stopped the 'chomp facility of electric-pair-mode working
properly (for the currently accepted value of "properly").

> - Another approach would be to make it possible to mark \n as both ] and
>   > at the same time, which would make the CC-mode feature much cleaner
>   (no need to muck with syntax-table text-properties) but the cost of
>   yet more complexity in the syntax.c code.

That's what I'm doing with `s'.  The extra complexity in syntax.c
doesn't seem all that bad at the moment.  back_maybe_string is currently
137 lines long (including a macro analogous to INC_FROM, and a lossage:
clause modelled on the one in back_comment)), compared with
back_comment's 289 lines.  I'm planning on committing this new code to a
branch in the next few days, then you can judge better whether the new
facility is worth it.

[ .... ]

> > My suggestion has the strong advantage that it will benefit Emacs as a
> > whole, and there won't need to be separate implementations in CC Mode,
> > Python Mode, Ada Mode, .....  The need for a multilinne string to have
> > escaped NLs between its lines is actually a common pattern in the
> > languages Emacs handles.  Why can we not handle it in syntax.c?

> Emacs has handled it for the last 30 years or so.  You just want to
> handle it in a different way.  I agree that Emacs's core should ideally
> make it easy for a major mode to choose this "different way".

> But the way I see it, your suggestion is just adding one more wart to
> syntax-tables whereas we should instead work on "syntax-tables NG".

> IOW, I think that we should introduce a brand new replacement for
> syntax-tables (tho I don't really know what it should look like,
> otherwise I'd have coded it up already); something much more powerful
> and generic (probably based on a mix of a DFA at one level and some kind
> of push-down automata on top of it), and such a thing could/should
> easily accommodate such a feature without even needing any
> ad-hoc support.

"S-T-NG" may be fine for Emacs 28 or 29, but the syntax table is what we
have, and what we must work with in the short term.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).