Re: CC Mode and electric-pair "problem".

[Alan Mackenzie <acm@muc.de>]

On Sun, Jul 01, 2018 at 19:16:06 -0400, Stefan Monnier wrote:
> >> >> Why?    How 'bout:
> >> >>     char foo[] = "some unterminated // string
> >> > Bug compatibility with the current scan-sexps.
> >> I don't see why: currently, scan-sexps skips over the comment, but
> >> that's not a bug: it's exactly what it is documented to do.
> > There is no comment there, but scan-sexps skips to it nevertheless.

> The starting point is within the string (not according to the C language
> rules, of course, but according to the syntax-tables settings), and
> operations like scan-sexps are documented to work under the assumption
> that the starting point is outside of strings/comments, so it is very
> much correct for it to consider this "// string\n" to be a comment.

Yes.  Apologies for my misunderstanding.

[ .... ]

> ..... I see we agree.

Yes.

> >> It needs to work reliably for those languages where strings
> >> are indeed terminated by newline (e.g. jgraph-mode in GNU ELPA).
> > You mean, jgraph-mode is another use-case for `s'?  (I'm not familiar
> > with it.)

> I looked for existing use-cases and I indeed found one.  It's very much
> not high-profile, tho.  Also this use-case is slightly different in that
> the \n is really the normal/only way to terminate the string in jgraph.
> In case you're interested:

>     http://web.eecs.utk.edu/~plank/plank/jgraph/jgraph.html

> >> But that brings us back to "why not use string-fence?".
> > Yes.  String-fence interferes with syntactical stuff "inside" the
> > invalid string, whereas the `s' flag won't.

> Not sure how serious this "interferes with syntactical stuff" is
> in practice.

Maybe not very.

> >> But we usually don't make any effort to guess what the intended
> >> closest valid state might be, except where the user is actively
> >> editing the text (e.g. by proposing completion candidates for
> >> identifiers).
> > There's no need to guess.  The compiler defines the state, namely that
> > the (invalid) string ends at the EOL, and what follows is non-string.

> The compiler just makes an arbitrary choice, ....

No.  The compiler has no choice here.  Or does it?  Can you identify any
other sensible strategy a compiler could follow?

> .... just like we do and that has no bearing on what the intended
> valid state is (which is not something the compiler can discover
> either: it's only available in the head of the coder).

There may or may not be a unique "intended valid state".  I don't think
it's a helpful concept - it suggests that the states a buffer is in most
of the time are in some way unimportant.  I reaffirm my view that Emacs
should present optimal information about these normal (invalid) states,
and that they are very important indeed.

> > There's no need to guess.  back_maybe_comment is in the new
> > scratch/fontify-open-string branch.  It is NOT that complicated.

> Unsurprisingly it introduces a complexity which I find unjustified by
> the presented benefits.

> But it now occurs to me that maybe we can do better: have you tried to
> merge back_maybe_string into back_comment?  After all, back_comment
> already pays attention to strings (in order to try and correctly
> handle comment openers appearing within strings), so there's
> a possibility that back_comment might be able to handle your use case
> with much fewer changes (and in that case, the performance cost would
> be pretty close to 0, I think).

That's a good idea.  I think it's clear that such a merge could be done.
But it would need a lot of detailed painstaking work.  It's optimisation
(as in "don't do it yet!").  Once we decide to adopt the idea is the
time to do this merge, I think.  That's assuming some measurements show
it's worthwhile (which I think it would be).

In fact, in my modified shell-script-mode I timed (scan-sexps BONL -1) a
million times on the following text:

    "string" at the start of a line.

With the `s' flag in place: 1.9489 seconds.
Without the `s' flag:       1.3003 seconds.

This is an overhead of almost exactly 50%.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).