bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

Hello, Emacs.

In the Emacs manual page "Left Margin Paren", it says that:

   To help you catch violations of this convention, Font Lock mode
   highlights confusing opening delimiters (those that ought to be quoted)
   in bold red.

, where "this convention" is the convention of not putting opening parens
in column 0 when they aren't at the beginning of defuns.

In Emacs Lisp Mode, this highlighting isn't done.  It isn't in CC Mode,
either.

This is either a bug in the code, or a bug in the documentation.  I
rather tend to the view that this bold red highlighting should be done
when open-paren-in-column-0-is-defun-start is non-nil.  I think it did,
at one stage, but searching the emacs-devel archives doesn't reveal what
happened.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

In article <mailman.2066.1447172952.7904.bug-gnu-emacs@gnu.org> you wrote:
> In the Emacs manual page "Left Margin Paren", it says that:

>    To help you catch violations of this convention, Font Lock mode
>    highlights confusing opening delimiters (those that ought to be quoted)
>    in bold red.

> , where "this convention" is the convention of not putting opening parens
> in column 0 when they aren't at the beginning of defuns.

> In Emacs Lisp Mode, this highlighting isn't done.  It isn't in CC Mode,
> either.

> This is either a bug in the code, or a bug in the documentation.  I
> rather tend to the view that this bold red highlighting should be done
> when open-paren-in-column-0-is-defun-start is non-nil.  I think it did,
> at one stage, but searching the emacs-devel archives doesn't reveal what
> happened.

Thinking about it, this is surely a bug in the documentation.  The only
way Font Lock (or anything else) could pick out an offending paren would
be to scan a buffer from BOB.  This would rather defeat the point of the
paren in column 0 convention.

I'll patch the doc.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Glenn Morris]

Alan Mackenzie wrote:

> Thinking about it, this is surely a bug in the documentation.

I disagree, since it works as documented in Emacs 24.3.
It seems to have been broken since 24.4.

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

Hello, Glenn.

On Thu, Nov 12, 2015 at 11:36:55AM -0500, Glenn Morris wrote:
> Alan Mackenzie wrote:

> > Thinking about it, this is surely a bug in the documentation.

> I disagree, since it works as documented in Emacs 24.3.
> It seems to have been broken since 24.4.

Thanks for that tip.  The code for it (in font-lock-compile-keywords) is
still there, but somehow one of a list of conditions which prevent it
being activated has become set.

I'm looking into it.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

In article <mailman.2173.1447351928.7904.bug-gnu-emacs@gnu.org> you wrote:
> On Thu, Nov 12, 2015 at 11:36:55AM -0500, Glenn Morris wrote:
>> Alan Mackenzie wrote:

>> > Thinking about it, this is surely a bug in the documentation.

>> I disagree, since it works as documented in Emacs 24.3.
>> It seems to have been broken since 24.4.

> Thanks for that tip.  The code for it (in font-lock-compile-keywords) is
> still there, but somehow one of a list of conditions which prevent it
> being activated has become set.

> I'm looking into it.

The fix to bug #16247 meant no longer setting syntax-begin-function to a
non-nil value.  This is the condition which used to cause the appropriate
font-lock-keywords form to get added to lisp-font-lock-keywords-1/2.  It
no longer is.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Eli Zaretskii]

> Date: 12 Nov 2015 18:54:24 -0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Alan Mackenzie <acm@muc.de>
> 
> The fix to bug #16247 meant no longer setting syntax-begin-function to a
> non-nil value.  This is the condition which used to cause the appropriate
> font-lock-keywords form to get added to lisp-font-lock-keywords-1/2.  It
> no longer is.

Stefan?

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Dmitry Gutov]

On 11/12/2015 08:54 PM, Alan Mackenzie wrote:

> The fix to bug #16247 meant no longer setting syntax-begin-function to a
> non-nil value.  This is the condition which used to cause the appropriate
> font-lock-keywords form to get added to lisp-font-lock-keywords-1/2.  It
> no longer is.

Looking into this, I'm not sure we still want to highlight them. The 
aforementioned bug, now fixed, mirrored the justifications that we give 
in the manual and the comments for the highlighting of parens in the 0th 
column:

"The convention speeds up many Emacs operations, which would otherwise 
have to scan back to the beginning of the buffer to analyze the syntax 
of the code."

and

;; Try to detect when a string or comment contains something that
;; looks like a defun and would thus confuse font-lock.

We don't have to scan back to the beginning of the buffer, we can use 
syntax-ppss (and it's more reliable with bug#16247 fixed).

font-lock doesn't get confused by something looking like a defun inside 
a docstring (try it; I wasn't able to get it highlight something wrong).

M-x beginning-of-defun does get confused, though. If *that* is problem 
what we want to detect, I think the patch should look like this:

diff --git a/lisp/font-lock.el b/lisp/font-lock.el
index 8ee9f69..eed2766 100644
--- a/lisp/font-lock.el
+++ b/lisp/font-lock.el
@@ -1786,13 +1786,10 @@ font-lock-compile-keywords
  	  (cons t (cons keywords
  			(mapcar #'font-lock-compile-keyword keywords))))
      (if (and (not syntactic-keywords)
-	     (let ((beg-function syntax-begin-function))
-	       (or (eq beg-function 'beginning-of-defun)
-                   (if (symbolp beg-function)
-                       (get beg-function 'font-lock-syntax-paren-check))))
-	     (not beginning-of-defun-function))
+             (not beginning-of-defun-function)
+             open-paren-in-column-0-is-defun-start)
  	;; Try to detect when a string or comment contains something that
-	;; looks like a defun and would thus confuse font-lock.
+	;; looks like a defun and would thus confuse beginning-of-defun.
  	(nconc keywords
  	       `((,(if defun-prompt-regexp
  		       (concat "^\\(?:" defun-prompt-regexp "\\)?\\s(")

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

Hello, Dmitry.

On Mon, May 16, 2016 at 12:50:54AM +0300, Dmitry Gutov wrote:
> On 11/12/2015 08:54 PM, Alan Mackenzie wrote:

> > The fix to bug #16247 meant no longer setting syntax-begin-function to a
> > non-nil value.  This is the condition which used to cause the appropriate
> > font-lock-keywords form to get added to lisp-font-lock-keywords-1/2.  It
> > no longer is.

> Looking into this, I'm not sure we still want to highlight them. The 
> aforementioned bug, now fixed, mirrored the justifications that we give 
> in the manual and the comments for the highlighting of parens in the 0th 
> column:

> "The convention speeds up many Emacs operations, which would otherwise 
> have to scan back to the beginning of the buffer to analyze the syntax 
> of the code."

Note this convention is still active.

> and

> ;; Try to detect when a string or comment contains something that
> ;; looks like a defun and would thus confuse font-lock.

> We don't have to scan back to the beginning of the buffer, we can use 
> syntax-ppss (and it's more reliable with bug#16247 fixed).

Sorry, this isn't true.  The scanning back to BOB is done at the C
level, in function back_comment.  syntax-ppss isn't suitable for use
here (Stefan's view, not merely mine), because syntax-ppss doesn't react
to changes in the syntax table, and suchlike.

> font-lock doesn't get confused by something looking like a defun inside 
> a docstring (try it; I wasn't able to get it highlight something wrong).

You might be getting confused, here.  The scanning back to BOB which is
slow doesn't just happen in font lock; it can (and does) happen
anywhere.  It's just font lock's job to warn the user about this, so
that she can correct it by adding in a backslash, for example.

Things do get confused, for example see bug #22884, where there was an
open paren in column zero in our own C sources.

> M-x beginning-of-defun does get confused, though. If *that* is problem 
> what we want to detect, .....

Not particularly.  We want the user to be warned about things
potentially going wrong in back_comment, and anything which calls it.
The problem we want to fix is the lack of font-lock-warning-face on
these parens in column 0.  Anything beyond that is not for Emacs 25.1.

> .... I think the patch should look like this:

> diff --git a/lisp/font-lock.el b/lisp/font-lock.el
> index 8ee9f69..eed2766 100644
> --- a/lisp/font-lock.el
> +++ b/lisp/font-lock.el
> @@ -1786,13 +1786,10 @@ font-lock-compile-keywords
>   	  (cons t (cons keywords
>   			(mapcar #'font-lock-compile-keyword keywords))))
>       (if (and (not syntactic-keywords)
> -	     (let ((beg-function syntax-begin-function))
> -	       (or (eq beg-function 'beginning-of-defun)
> -                   (if (symbolp beg-function)
> -                       (get beg-function 'font-lock-syntax-paren-check))))
> -	     (not beginning-of-defun-function))
> +             (not beginning-of-defun-function)
> +             open-paren-in-column-0-is-defun-start)

No.  open-paren-in-column-0-is-defun-start is a variable that the user
can change at any time.  We can't make our font-locking dependent upon
what its value was at some time in the past.  If open-paren-... belongs
anywhere, it's in the form just beyond the end of your patch's text.

Do you understand the consequences of taking out the check on
syntax-begin-function?  (I certainly don't.)  It would be good if Stefan
could express a view, here.

>   	;; Try to detect when a string or comment contains something that
> -	;; looks like a defun and would thus confuse font-lock.
> +	;; looks like a defun and would thus confuse beginning-of-defun.

Also no.  It's more general than that.  I think "would thus confuse
Emacs" would be more accurate.

>   	(nconc keywords
>   	       `((,(if defun-prompt-regexp
>   		       (concat "^\\(?:" defun-prompt-regexp "\\)?\\s(")

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Dmitry Gutov]

On 05/16/2016 01:20 PM, Alan Mackenzie wrote:

> Note this convention is still active.

The "convention" may be in place, but the underlying reasons for it  are 
much weaker these days. Any relevant operation can use syntax-ppss.

>> We don't have to scan back to the beginning of the buffer, we can use
>> syntax-ppss (and it's more reliable with bug#16247 fixed).
>
> Sorry, this isn't true.  The scanning back to BOB is done at the C
> level, in function back_comment.

What I wrote is true: font-lock rules can use syntax-ppss, and often do.

> syntax-ppss isn't suitable for use
> here (Stefan's view, not merely mine), because syntax-ppss doesn't react
> to changes in the syntax table, and suchlike.

Here where?

>> font-lock doesn't get confused by something looking like a defun inside
>> a docstring (try it; I wasn't able to get it highlight something wrong).
>
> You might be getting confused, here.

No, I'm not. I'm addressing a comment inside font-lock-compile-keywords, 
which is trying to justify highlighting parens in the first column.

> The scanning back to BOB which is
> slow doesn't just happen in font lock; it can (and does) happen
> anywhere.

Only in certain places, where the programmer didn't think to use the 
cache provided by syntax-ppss.

> It's just font lock's job to warn the user about this, so
> that she can correct it by adding in a backslash, for example.

And it's the job of the programmer to avoid this problem altogether, 
which is not too hard.

> Things do get confused, for example see bug #22884, where there was an
> open paren in column zero in our own C sources.

Even if bug#22884 is somewhat related, it's actually irrelevant is the 
current discussion because c-mode uses a non-default 
beginning-of-defun-function. Which means font-lock-compile-keywords 
won't add highlighting to 0-column parens in c-mode anyway.

It seems the current code was designed with only Lisp modes in mind.

>> M-x beginning-of-defun does get confused, though. If *that* is problem
>> what we want to detect, .....
>
> Not particularly.  We want the user to be warned about things
> potentially going wrong in back_comment, and anything which calls it.

I don't see any reason to believe that the original author of this code 
was concerned with back_comment specifically.

> No.  open-paren-in-column-0-is-defun-start is a variable that the user
> can change at any time.

I don't think it is, or should be, true. The major mode knows better 
whether it can know where a defun starts, or not.

E.g. js-mode and elisp-byte-code-mode set it to nil. If the user changes 
that value in one of these modes, nothing good will happen.

> We can't make our font-locking dependent upon
> what its value was at some time in the past.  If open-paren-... belongs
> anywhere, it's in the form just beyond the end of your patch's text.

I don't think so. I don't mind taking its comparison out altogether, but 
then the predicate will become very simple.

> Do you understand the consequences of taking out the check on
> syntax-begin-function?  (I certainly don't.)  It would be good if Stefan
> could express a view, here.

Point is, there is no way to simply alter the check that it would accept 
the current situation with syntax-begin-function, but still keep it 
meaningful. If we accept the value nil (which it is emacs-lisp-mode 
now), we should accept any syntax-begin-function, I think.

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Andreas Röhler]

On 16.05.2016 15:18, Dmitry Gutov wrote:
> On 05/16/2016 01:20 PM, Alan Mackenzie wrote:
>
>> No.  open-paren-in-column-0-is-defun-start is a variable that the user
>> can change at any time.
>
> I don't think it is, or should be, true. The major mode knows better 
> whether it can know where a defun starts, or not.
>

This open-paren-in-column-0-is-defun-start introduces exercises how to 
jump with crossed legs.
There are some astonishing jumpers around, I see. From the state of art 
of programming it's just a shame.

Emacs will always buggy cherishing such crap.

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

Hello, Dmitry.

On Mon, May 16, 2016 at 04:18:54PM +0300, Dmitry Gutov wrote:
> On 05/16/2016 01:20 PM, Alan Mackenzie wrote:

> > Note this convention is still active.

> The "convention" may be in place, but the underlying reasons for it  are 
> much weaker these days.

The convention is still needed, in particular in CC Mode.  We tried to
do without it for some time, and got complaints (from Martin Rudalics)
about its speed.

> Any relevant operation can use syntax-ppss. 

No, it can't.  Anything which uses back_comment can't.  That includes
scan-lists, backward-list, and so on, which are very widely used,
including in beginning-of-defun.

Or are you proposing to rewrite vast swathes of Emacs, expunging all
(backward) uses of scan-lists, etc.?

> >> We don't have to scan back to the beginning of the buffer, we can use
> >> syntax-ppss (and it's more reliable with bug#16247 fixed).

> > Sorry, this isn't true.  The scanning back to BOB is done at the C
> > level, in function back_comment.

> What I wrote is true: font-lock rules can use syntax-ppss, and often do.

Up to a point, you may be right.  Any time anybody uses
beginning-of-defun, etc., this scanning from BOB may happen.

Also, syntax-ppss will deliver the wrong value if font-lock-syntax-table
is non-nil and syntax-ppss is also used outside of font-lock.  This is
(one of) the problems with syntax-ppss - it ploughs on blindly,
regardless of changes to the syntax table, text-properties, etc.  But it
sort of works most of the time.

> > syntax-ppss isn't suitable for use
> > here (Stefan's view, not merely mine), because syntax-ppss doesn't react
> > to changes in the syntax table, and suchlike.

> Here where?

In back_comment.

[ .... ]

> > The scanning back to BOB which is slow doesn't just happen in font
> > lock; it can (and does) happen anywhere.

> Only in certain places, where the programmer didn't think to use the 
> cache provided by syntax-ppss.

This is simply false.  See above.

> > It's just font lock's job to warn the user about this, so
> > that she can correct it by adding in a backslash, for example.

> And it's the job of the programmer to avoid this problem altogether, 
> which is not too hard.

This is also false.  People have been struggling with the problem for
years, if not decades.

> > Things do get confused, for example see bug #22884, where there was an
> > open paren in column zero in our own C sources.

> Even if bug#22884 is somewhat related, it's actually irrelevant is the 
> current discussion because c-mode uses a non-default 
> beginning-of-defun-function. Which means font-lock-compile-keywords 
> won't add highlighting to 0-column parens in c-mode anyway.

Sadly true.  It ought to, though.  I can't see the connection between a
major mode determining its own BOD, and whether or not it wants parens
in column zero in strings and comments to get warning face.

> It seems the current code was designed with only Lisp modes in mind.

Not at all.  Read the manual.

> >> M-x beginning-of-defun does get confused, though. If *that* is problem
> >> what we want to detect, .....

> > Not particularly.  We want the user to be warned about things
> > potentially going wrong in back_comment, and anything which calls it.

> I don't see any reason to believe that the original author of this code 
> was concerned with back_comment specifically.

No, with things which call it, including scan-lists, beginning-of-defun,
etc.

> > No.  open-paren-in-column-0-is-defun-start is a variable that the user
> > can change at any time.

> I don't think it is, or should be, true. The major mode knows better 
> whether it can know where a defun starts, or not.

open-paren-in-... is a customisable option.  It is up to the user
whether she wants the speed of o-p-i-c-0-i-d-s set at t, or the accuracy
of it set at nil.

> E.g. js-mode and elisp-byte-code-mode set it to nil. If the user changes 
> that value in one of these modes, nothing good will happen.

Set it to nil or bind it to nil?  This may be a misuse of the variable
by these modes.

> > We can't make our font-locking dependent upon
> > what its value was at some time in the past.  If open-paren-... belongs
> > anywhere, it's in the form just beyond the end of your patch's text.

> I don't think so. I don't mind taking its comparison out altogether, but 
> then the predicate will become very simple.

Again, do you understand that comparison, and why all the components of
that `and' form are there?

> > Do you understand the consequences of taking out the check on
> > syntax-begin-function?  (I certainly don't.)  It would be good if Stefan
> > could express a view, here.

> Point is, there is no way to simply alter the check that it would accept 
> the current situation with syntax-begin-function, but still keep it 
> meaningful. If we accept the value nil (which it is emacs-lisp-mode 
> now), we should accept any syntax-begin-function, I think.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Eli Zaretskii]

unblock 24655 by 21871
thanks

> Date: Tue, 17 May 2016 09:02:42 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: 21871@debbugs.gnu.org
> 
> On Mon, May 16, 2016 at 04:18:54PM +0300, Dmitry Gutov wrote:
> > On 05/16/2016 01:20 PM, Alan Mackenzie wrote:
> 
> > > Note this convention is still active.
> 
> > The "convention" may be in place, but the underlying reasons for it  are 
> > much weaker these days.
> 
> The convention is still needed, in particular in CC Mode.  We tried to
> do without it for some time, and got complaints (from Martin Rudalics)
> about its speed.
> 
> > Any relevant operation can use syntax-ppss. 
> 
> No, it can't.  Anything which uses back_comment can't.  That includes
> scan-lists, backward-list, and so on, which are very widely used,
> including in beginning-of-defun.
> 
> Or are you proposing to rewrite vast swathes of Emacs, expunging all
> (backward) uses of scan-lists, etc.?
> 
> > >> We don't have to scan back to the beginning of the buffer, we can use
> > >> syntax-ppss (and it's more reliable with bug#16247 fixed).
> 
> > > Sorry, this isn't true.  The scanning back to BOB is done at the C
> > > level, in function back_comment.
> 
> > What I wrote is true: font-lock rules can use syntax-ppss, and often do.
> 
> Up to a point, you may be right.  Any time anybody uses
> beginning-of-defun, etc., this scanning from BOB may happen.
> 
> Also, syntax-ppss will deliver the wrong value if font-lock-syntax-table
> is non-nil and syntax-ppss is also used outside of font-lock.  This is
> (one of) the problems with syntax-ppss - it ploughs on blindly,
> regardless of changes to the syntax table, text-properties, etc.  But it
> sort of works most of the time.
> 
> > > syntax-ppss isn't suitable for use
> > > here (Stefan's view, not merely mine), because syntax-ppss doesn't react
> > > to changes in the syntax table, and suchlike.
> 
> > Here where?
> 
> In back_comment.
> 
> [ .... ]
> 
> > > The scanning back to BOB which is slow doesn't just happen in font
> > > lock; it can (and does) happen anywhere.
> 
> > Only in certain places, where the programmer didn't think to use the 
> > cache provided by syntax-ppss.
> 
> This is simply false.  See above.
> 
> > > It's just font lock's job to warn the user about this, so
> > > that she can correct it by adding in a backslash, for example.
> 
> > And it's the job of the programmer to avoid this problem altogether, 
> > which is not too hard.
> 
> This is also false.  People have been struggling with the problem for
> years, if not decades.
> 
> > > Things do get confused, for example see bug #22884, where there was an
> > > open paren in column zero in our own C sources.
> 
> > Even if bug#22884 is somewhat related, it's actually irrelevant is the 
> > current discussion because c-mode uses a non-default 
> > beginning-of-defun-function. Which means font-lock-compile-keywords 
> > won't add highlighting to 0-column parens in c-mode anyway.
> 
> Sadly true.  It ought to, though.  I can't see the connection between a
> major mode determining its own BOD, and whether or not it wants parens
> in column zero in strings and comments to get warning face.
> 
> > It seems the current code was designed with only Lisp modes in mind.
> 
> Not at all.  Read the manual.
> 
> > >> M-x beginning-of-defun does get confused, though. If *that* is problem
> > >> what we want to detect, .....
> 
> > > Not particularly.  We want the user to be warned about things
> > > potentially going wrong in back_comment, and anything which calls it.
> 
> > I don't see any reason to believe that the original author of this code 
> > was concerned with back_comment specifically.
> 
> No, with things which call it, including scan-lists, beginning-of-defun,
> etc.
> 
> > > No.  open-paren-in-column-0-is-defun-start is a variable that the user
> > > can change at any time.
> 
> > I don't think it is, or should be, true. The major mode knows better 
> > whether it can know where a defun starts, or not.
> 
> open-paren-in-... is a customisable option.  It is up to the user
> whether she wants the speed of o-p-i-c-0-i-d-s set at t, or the accuracy
> of it set at nil.
> 
> > E.g. js-mode and elisp-byte-code-mode set it to nil. If the user changes 
> > that value in one of these modes, nothing good will happen.
> 
> Set it to nil or bind it to nil?  This may be a misuse of the variable
> by these modes.

More than a year later, it doesn't sound like this is bothering anyone
else, and Stefan still didn't chime in to tell what he thinks.  So I'm
removing the Emacs 26.1 blocking status from this bug.

Thanks.

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 647 bytes --]

Alan Mackenzie <acm@muc.de> writes:

> In the Emacs manual page "Left Margin Paren", it says that:
>
>    To help you catch violations of this convention, Font Lock mode
>    highlights confusing opening delimiters (those that ought to be quoted)
>    in bold red.
>
> , where "this convention" is the convention of not putting opening parens
> in column 0 when they aren't at the beginning of defuns.
>
> In Emacs Lisp Mode, this highlighting isn't done.  It isn't in CC Mode,
> either.

The recent thread in emacs-devel[1] reminded me I wrote a (still
half-baked) patch for this.

[1]: https://lists.gnu.org/r/emacs-devel/2020-04/msg00402.html


[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 3847 bytes --]

From b3794a2a87c6aafc2174162f72f716a5801668a4 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 5 Aug 2017 12:31:02 -0400
Subject: [PATCH] [WIP] Restore highlighting of open parens in column-0
 (Bug#21871)

---
 lisp/emacs-lisp/lisp-mode.el |  1 +
 lisp/font-lock.el            | 38 ++++++++++++++++++++++++--------------
 2 files changed, 25 insertions(+), 14 deletions(-)

diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el
index 3b0f5493ee..50707fd10a 100644
--- a/lisp/emacs-lisp/lisp-mode.el
+++ b/lisp/emacs-lisp/lisp-mode.el
@@ -642,6 +642,7 @@ lisp-mode-variables
   (setq-local imenu-generic-expression lisp-imenu-generic-expression)
   (setq-local multibyte-syntax-as-symbol t)
   ;; (setq-local syntax-begin-function 'beginning-of-defun)  ;;Bug#16247.
+  (setq-local font-lock-syntax-paren-check t) ;; Bug#21871.
   (setq font-lock-defaults
 	`(,(if elisp '(lisp-el-font-lock-keywords
                        lisp-el-font-lock-keywords-1
diff --git a/lisp/font-lock.el b/lisp/font-lock.el
index e0955b74ab..565f4b22b4 100644
--- a/lisp/font-lock.el
+++ b/lisp/font-lock.el
@@ -1779,6 +1779,11 @@ font-lock-fontify-keywords-region
 \f
 ;; Various functions.
 
+(defvar-local font-lock-syntax-paren-check nil
+  "Major modes should set this to non-nil if they rely on the
+`open-paren-in-column-0-is-defun-start' convention for movement
+or fontification.")
+
 (defun font-lock-compile-keywords (keywords &optional syntactic-keywords)
   "Compile KEYWORDS into the form (t KEYWORDS COMPILED...)
 Here each COMPILED is of the form (MATCHER HIGHLIGHT ...) as shown in the
@@ -1798,24 +1803,29 @@ font-lock-compile-keywords
 	  (cons t (cons keywords
 			(mapcar #'font-lock-compile-keyword keywords))))
     (if (and (not syntactic-keywords)
-	     (let ((beg-function (with-no-warnings syntax-begin-function)))
-	       (or (eq beg-function #'beginning-of-defun)
-                   (if (symbolp beg-function)
-                       (get beg-function 'font-lock-syntax-paren-check))))
-	     (not beginning-of-defun-function))
+	     (or (and (let ((beg-function (with-no-warnings syntax-begin-function)))
+                        (or (eq beg-function #'beginning-of-defun)
+                            (if (symbolp beg-function)
+                                (get beg-function 'font-lock-syntax-paren-check))))
+                      (not beginning-of-defun-function))
+                 font-lock-syntax-paren-check))
 	;; Try to detect when a string or comment contains something that
 	;; looks like a defun and would thus confuse font-lock.
 	(nconc keywords
-	       `((,(if defun-prompt-regexp
-		       (concat "^\\(?:" defun-prompt-regexp "\\)?\\s(")
-		     "^\\s(")
-		  (0
-		   (if (memq (get-text-property (match-beginning 0) 'face)
-			     '(font-lock-string-face font-lock-doc-face
-			       font-lock-comment-face))
-		       (list 'face font-lock-warning-face
+               `((,(let ((regexp
+                          (if defun-prompt-regexp
+                              (concat "^\\(?:" defun-prompt-regexp "\\)?\\s(")
+                            "^\\s(")))
+                     (lambda (bound)
+                       (if open-paren-in-column-0-is-defun-start
+                           (re-search-forward regexp bound t))))
+                  (0
+                   (if (memq (get-text-property (match-beginning 0) 'face)
+                             '(font-lock-string-face font-lock-doc-face
+                                                     font-lock-comment-face))
+                       (list 'face font-lock-warning-face
                              'help-echo "Looks like a toplevel defun: escape the parenthesis"))
-		   prepend)))))
+                   prepend)))))
     keywords))
 
 (defun font-lock-compile-keyword (keyword)
-- 
2.11.0

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Stefan Kangas]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Emacs.
>
> In the Emacs manual page "Left Margin Paren", it says that:
>
>    To help you catch violations of this convention, Font Lock mode
>    highlights confusing opening delimiters (those that ought to be quoted)
>    in bold red.
>
> , where "this convention" is the convention of not putting opening parens
> in column 0 when they aren't at the beginning of defuns.
>
> In Emacs Lisp Mode, this highlighting isn't done.  It isn't in CC Mode,
> either.

The above text no longer exists, and in Emacs 27.1 or later, we no
longer treat an unescaped ( in column zero in a docstring as the
beginning of a defun.  (See `(elisp) Documentation Tips', final
paragraph.)

So should this bug be closed, or is there anything more to do here?

> This is either a bug in the code, or a bug in the documentation.  I
> rather tend to the view that this bold red highlighting should be done
> when open-paren-in-column-0-is-defun-start is non-nil.  I think it did,
> at one stage, but searching the emacs-devel archives doesn't reveal what
> happened.

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Alan Mackenzie]

Hello, Stefan.

On Sun, Sep 19, 2021 at 15:14:18 -0700, Stefan Kangas wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Hello, Emacs.

> > In the Emacs manual page "Left Margin Paren", it says that:

> >    To help you catch violations of this convention, Font Lock mode
> >    highlights confusing opening delimiters (those that ought to be quoted)
> >    in bold red.

> > , where "this convention" is the convention of not putting opening parens
> > in column 0 when they aren't at the beginning of defuns.

> > In Emacs Lisp Mode, this highlighting isn't done.  It isn't in CC Mode,
> > either.

> The above text no longer exists, and in Emacs 27.1 or later, we no
> longer treat an unescaped ( in column zero in a docstring as the
> beginning of a defun.  (See `(elisp) Documentation Tips', final
> paragraph.)

> So should this bug be closed, or is there anything more to do here?

I would say it should definitely be closed, the fix that has been
implemented being far and away better than that anticipated by the bug
report.  :-)

> > This is either a bug in the code, or a bug in the documentation.  I
> > rather tend to the view that this bold red highlighting should be done
> > when open-paren-in-column-0-is-defun-start is non-nil.  I think it did,
> > at one stage, but searching the emacs-devel archives doesn't reveal what
> > happened.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#21871: Emacs Lisp Mode (at least): spurious parens in column 0 don't get bold red highlighting.

[Stefan Kangas]

Alan Mackenzie <acm@muc.de> writes:

> I would say it should definitely be closed, the fix that has been
> implemented being far and away better than that anticipated by the bug
> report.  :-)

Excellent!  :-)

I'm therefore closing this bug report.