bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie <acm@muc.de>]

Hello, Dmitry.

On Mon, May 16, 2016 at 04:18:54PM +0300, Dmitry Gutov wrote:
> On 05/16/2016 01:20 PM, Alan Mackenzie wrote:

> > Note this convention is still active.

> The "convention" may be in place, but the underlying reasons for it  are 
> much weaker these days.

The convention is still needed, in particular in CC Mode.  We tried to
do without it for some time, and got complaints (from Martin Rudalics)
about its speed.

> Any relevant operation can use syntax-ppss. 

No, it can't.  Anything which uses back_comment can't.  That includes
scan-lists, backward-list, and so on, which are very widely used,
including in beginning-of-defun.

Or are you proposing to rewrite vast swathes of Emacs, expunging all
(backward) uses of scan-lists, etc.?

> >> We don't have to scan back to the beginning of the buffer, we can use
> >> syntax-ppss (and it's more reliable with bug#16247 fixed).

> > Sorry, this isn't true.  The scanning back to BOB is done at the C
> > level, in function back_comment.

> What I wrote is true: font-lock rules can use syntax-ppss, and often do.

Up to a point, you may be right.  Any time anybody uses
beginning-of-defun, etc., this scanning from BOB may happen.

Also, syntax-ppss will deliver the wrong value if font-lock-syntax-table
is non-nil and syntax-ppss is also used outside of font-lock.  This is
(one of) the problems with syntax-ppss - it ploughs on blindly,
regardless of changes to the syntax table, text-properties, etc.  But it
sort of works most of the time.

> > syntax-ppss isn't suitable for use
> > here (Stefan's view, not merely mine), because syntax-ppss doesn't react
> > to changes in the syntax table, and suchlike.

> Here where?

In back_comment.

[ .... ]

> > The scanning back to BOB which is slow doesn't just happen in font
> > lock; it can (and does) happen anywhere.

> Only in certain places, where the programmer didn't think to use the 
> cache provided by syntax-ppss.

This is simply false.  See above.

> > It's just font lock's job to warn the user about this, so
> > that she can correct it by adding in a backslash, for example.

> And it's the job of the programmer to avoid this problem altogether, 
> which is not too hard.

This is also false.  People have been struggling with the problem for
years, if not decades.

> > Things do get confused, for example see bug #22884, where there was an
> > open paren in column zero in our own C sources.

> Even if bug#22884 is somewhat related, it's actually irrelevant is the 
> current discussion because c-mode uses a non-default 
> beginning-of-defun-function. Which means font-lock-compile-keywords 
> won't add highlighting to 0-column parens in c-mode anyway.

Sadly true.  It ought to, though.  I can't see the connection between a
major mode determining its own BOD, and whether or not it wants parens
in column zero in strings and comments to get warning face.

> It seems the current code was designed with only Lisp modes in mind.

Not at all.  Read the manual.

> >> M-x beginning-of-defun does get confused, though. If *that* is problem
> >> what we want to detect, .....

> > Not particularly.  We want the user to be warned about things
> > potentially going wrong in back_comment, and anything which calls it.

> I don't see any reason to believe that the original author of this code 
> was concerned with back_comment specifically.

No, with things which call it, including scan-lists, beginning-of-defun,
etc.

> > No.  open-paren-in-column-0-is-defun-start is a variable that the user
> > can change at any time.

> I don't think it is, or should be, true. The major mode knows better 
> whether it can know where a defun starts, or not.

open-paren-in-... is a customisable option.  It is up to the user
whether she wants the speed of o-p-i-c-0-i-d-s set at t, or the accuracy
of it set at nil.

> E.g. js-mode and elisp-byte-code-mode set it to nil. If the user changes 
> that value in one of these modes, nothing good will happen.

Set it to nil or bind it to nil?  This may be a misuse of the variable
by these modes.

> > We can't make our font-locking dependent upon
> > what its value was at some time in the past.  If open-paren-... belongs
> > anywhere, it's in the form just beyond the end of your patch's text.

> I don't think so. I don't mind taking its comparison out altogether, but 
> then the predicate will become very simple.

Again, do you understand that comparison, and why all the components of
that `and' form are there?

> > Do you understand the consequences of taking out the check on
> > syntax-begin-function?  (I certainly don't.)  It would be good if Stefan
> > could express a view, here.

> Point is, there is no way to simply alter the check that it would accept 
> the current situation with syntax-begin-function, but still keep it 
> meaningful. If we accept the value nil (which it is emacs-lisp-mode 
> now), we should accept any syntax-begin-function, I think.

-- 
Alan Mackenzie (Nuremberg, Germany).