bug#71469: font-lock does not apply standard faces and their descendants

bug#71469: font-lock does not apply standard faces and their descendants

[Konstantin Kharlamov]

While trying to add a face to a major mode I found that both standard
faces¹ and inherited ones are ignored completely.

In steps below I create a simple major mode and it uses an inherited
face, but face does not get applied. NOTE: if you replace the inherited
face with, for example, `font-lock-constant-face`, it will get
highlighted. I.e. the problem somehow bound to faces.

# Steps to reproduce

1. Create `test.el` file as follows:

     (defface test-face
       '((t (:inherit bold)))
       "Test face.")

     (define-derived-mode my-mode fundamental-mode "My Mode"
       "A minimal mode that highlights 'hello world' text."
       (font-lock-add-keywords nil '(("hello world" 0 test-face)))
         (font-lock-flush))
     (add-to-list 'auto-mode-alist (cons "test.txt" 'my-mode))
     (provide 'my-mode)

   And `test.txt` as follows:
   
     ==> hello world <==

2. Launch Emacs as `emacs -Q -l test.el test.txt`
3. Put a caret over the word `hello` and evaluate M-x describe-char

## Expected

The description buffer mentions that `face` is `test-face`.

## Actual

There is no `face` property at all.

# Additional information

Versions tested: Emacs built from February master and a stable 29.3

The problem actually doesn't seem to be related to `defface`, because
passing `bold` to the `font-lock-add-keywords` had similarly no effect
for some reason. More likely it's related to the "standard faces".

1: https://www.gnu.org/software/emacs/manual/html_node/emacs/Standard-Faces.html

bug#71469: font-lock does not apply standard faces and their descendants

[Eli Zaretskii]

tags 71469 notabug
thanks

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Date: Mon, 10 Jun 2024 14:59:37 +0300
> 
>      (defface test-face
>        '((t (:inherit bold)))
>        "Test face.")
> 
>      (define-derived-mode my-mode fundamental-mode "My Mode"
>        "A minimal mode that highlights 'hello world' text."
>        (font-lock-add-keywords nil '(("hello world" 0 test-face)))

From the ELisp manual:

   Each element of ‘font-lock-keywords’ should have one of these forms:
   [...]
  ‘(MATCHER . FACESPEC)’
       In this kind of element, FACESPEC is an expression whose value
       specifies the face to use for highlighting.  In the simplest case,
       FACESPEC is a Lisp variable (a symbol) whose value is a face name.
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

IOW, there's a difference between a symbol of a variable whose value
is a face name, and that face name itself.

This works for me:

  (defface test-face
    '((t (:inherit bold)))
    "Test face.")
  (defvar test-face 'test-face
    "Face name to use for My Mode.")

  (define-derived-mode my-mode fundamental-mode "My Mode"
    "A minimal mode that highlights 'hello world' text."
    (font-lock-add-keywords nil '(("hello world" 0 test-face)))
    (font-lock-flush))
  (add-to-list 'auto-mode-alist (cons "test.txt" 'my-mode))
  (provide 'my-mode)

bug#71469: [PATCH] Re: bug#71469: font-lock does not apply standard faces and their descendants

[Konstantin Kharlamov]

[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]

On Mon, 2024-06-10 at 15:55 +0300, Eli Zaretskii wrote:
> tags 71469 notabug
> thanks
> 
> > From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> > Date: Mon, 10 Jun 2024 14:59:37 +0300
> > 
> >      (defface test-face
> >        '((t (:inherit bold)))
> >        "Test face.")
> > 
> >      (define-derived-mode my-mode fundamental-mode "My Mode"
> >        "A minimal mode that highlights 'hello world' text."
> >        (font-lock-add-keywords nil '(("hello world" 0 test-face)))
> 
> From the ELisp manual:
> 
>    Each element of ‘font-lock-keywords’ should have one of these
> forms:
>    [...]
>   ‘(MATCHER . FACESPEC)’
>        In this kind of element, FACESPEC is an expression whose value
>        specifies the face to use for highlighting.  In the simplest
> case,
>        FACESPEC is a Lisp variable (a symbol) whose value is a face
> name.
>       
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> IOW, there's a difference between a symbol of a variable whose value
> is a face name, and that face name itself.
> 
> This works for me:
> 
>   (defface test-face
>     '((t (:inherit bold)))
>     "Test face.")
>   (defvar test-face 'test-face
>     "Face name to use for My Mode.")
> 
>   (define-derived-mode my-mode fundamental-mode "My Mode"
>     "A minimal mode that highlights 'hello world' text."
>     (font-lock-add-keywords nil '(("hello world" 0 test-face)))
>     (font-lock-flush))
>   (add-to-list 'auto-mode-alist (cons "test.txt" 'my-mode))
>   (provide 'my-mode)

Ooh, I see, thank you! So using e.g. a `'test-face` also makes it work.

I'm wondering if it would be okay to mention such nuance in the
"standard faces" documentation, such as with the attached patch? I
think it would be really helpful, because the nuance of how it works is
not obvious at all (it would be much easier if none of the faces such
as font-lock-keyword-face, would be defining a variable). I've spent
for about an hour on this trying in different ways to make it work, and
I also think it wasn't the first time I stumbled upon this. Having the
"standard faces" mention that interaction nuance I think could be
helpful for people in the future.

[-- Attachment #2: 1.patch --]
[-- Type: text/x-patch, Size: 1060 bytes --]

From 02811b6400259c0d9b2dedcca76fec771dcd839a Mon Sep 17 00:00:00 2001
From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Date: Mon, 10 Jun 2024 16:34:30 +0300
Subject: [PATCH] Mention interaction of standard faces with font-lock-keywords

* doc/emacs/display.texi (standard faces): mention that these faces do
not produce variables and so can't be passed to font-lock-keywords as
is.
---
 doc/emacs/display.texi | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/doc/emacs/display.texi b/doc/emacs/display.texi
index 8f22e3c88da..bcbe68d21ea 100644
--- a/doc/emacs/display.texi
+++ b/doc/emacs/display.texi
@@ -647,7 +647,9 @@ Standard Faces
 @cindex standard faces
 
   Here are the standard faces for specifying text appearance.  You can
-apply them to specific text when you want the effects they produce.
+apply them to specific text when you want the effects they produce. Note
+that these faces do not define variables, so to pass such face to
+@code{font-lock-keywords} you have to quote it.
 
 @table @code
 @item default
-- 
2.45.2

bug#71469: [PATCH] Re: bug#71469: font-lock does not apply standard faces and their descendants

[Eli Zaretskii]

> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Cc: 71469@debbugs.gnu.org
> Date: Mon, 10 Jun 2024 16:41:53 +0300
> 
> I'm wondering if it would be okay to mention such nuance in the
> "standard faces" documentation, such as with the attached patch? I
> think it would be really helpful, because the nuance of how it works is
> not obvious at all (it would be much easier if none of the faces such
> as font-lock-keyword-face, would be defining a variable). I've spent
> for about an hour on this trying in different ways to make it work, and
> I also think it wasn't the first time I stumbled upon this. Having the
> "standard faces" mention that interaction nuance I think could be
> helpful for people in the future.
> 
> 
> From 02811b6400259c0d9b2dedcca76fec771dcd839a Mon Sep 17 00:00:00 2001
> From: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> Date: Mon, 10 Jun 2024 16:34:30 +0300
> Subject: [PATCH] Mention interaction of standard faces with font-lock-keywords
> 
> * doc/emacs/display.texi (standard faces): mention that these faces do
> not produce variables and so can't be passed to font-lock-keywords as
> is.
> ---
>  doc/emacs/display.texi | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/emacs/display.texi b/doc/emacs/display.texi
> index 8f22e3c88da..bcbe68d21ea 100644
> --- a/doc/emacs/display.texi
> +++ b/doc/emacs/display.texi
> @@ -647,7 +647,9 @@ Standard Faces
>  @cindex standard faces
>  
>    Here are the standard faces for specifying text appearance.  You can
> -apply them to specific text when you want the effects they produce.
> +apply them to specific text when you want the effects they produce. Note
> +that these faces do not define variables, so to pass such face to
> +@code{font-lock-keywords} you have to quote it.

I don't think I agree with this addition, so I added Stefan to this
discussion.

Stefan, any comments?

bug#71469: [PATCH] Re: bug#71469: font-lock does not apply standard faces and their descendants

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> I'm wondering if it would be okay to mention such nuance in the
>> "standard faces" documentation, such as with the attached patch? I
>> think it would be really helpful, because the nuance of how it works is
>> not obvious at all (it would be much easier if none of the faces such
>> as font-lock-keyword-face, would be defining a variable).

No face defines a variable.

Some faces (mostly font-lock faces) have an associated variable whose
content is "the face name" (which is also the variable's name), but it's
not a result of the face itself.  It's a separate explicit definition of
a variable with the same name as the face.

And I agree that it would be simpler if we didn't have those, but
they've been with us since `font-lock.el` in Emacs-19, so it's not easy
to get rid of them.

>>    Here are the standard faces for specifying text appearance.  You can
>> -apply them to specific text when you want the effects they produce.
>> +apply them to specific text when you want the effects they produce. Note
>> +that these faces do not define variables, so to pass such face to
>> +@code{font-lock-keywords} you have to quote it.

You *always* need to quote the face name.  Otherwise you're not
referring to the face but to a variable of the same name (and ideally
we'd want to get rid of the cases where this happens to work, because
the only upside is to save you from typing a quote character, while the
downside is to encourage confusion).


        Stefan

bug#71469: font-lock does not apply standard faces and their descendants

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I'm wondering if it would be okay to mention such nuance in the
>>> "standard faces" documentation, such as with the attached patch? I
>>> think it would be really helpful, because the nuance of how it works is
>>> not obvious at all (it would be much easier if none of the faces such
>>> as font-lock-keyword-face, would be defining a variable).
>
> No face defines a variable.
>
> Some faces (mostly font-lock faces) have an associated variable whose
> content is "the face name" (which is also the variable's name), but it's
> not a result of the face itself.  It's a separate explicit definition of
> a variable with the same name as the face.
>
> And I agree that it would be simpler if we didn't have those, but
> they've been with us since `font-lock.el` in Emacs-19, so it's not easy
> to get rid of them.

We have occasionally declared things obsolete with the understanding
that they'll be deleted much later than the normal "10 years, give or
take".  See the comment in `interactive-p', for example.

The patch below gives me 64 warnings in our tree.  I'm not sure if it's
worth installing or not.

diff --git a/lisp/font-lock.el b/lisp/font-lock.el
index 7b077a826bf..7787af43bc9 100644
--- a/lisp/font-lock.el
+++ b/lisp/font-lock.el
@@ -316,46 +316,74 @@ font-lock-verbose
 ;; need to create variables that specify face names.  Simply using
 ;; faces directly is enough.  Font-lock is not a template to be
 ;; followed in this area.
+(make-obsolete-variable 'font-lock-comment-face
+                        "use 'font-lock-comment-face instead." "31.1")
 (defvar font-lock-comment-face		'font-lock-comment-face
   "Face name to use for comments.")

+(make-obsolete-variable 'font-lock-comment-delimiter-face
+                        "use 'font-lock-comment-delimiter-face
instead." "31.1")
 (defvar font-lock-comment-delimiter-face 'font-lock-comment-delimiter-face
   "Face name to use for comment delimiters.")

+(make-obsolete-variable 'font-lock-string-face
+                        "use 'font-lock-string-face instead." "31.1")
 (defvar font-lock-string-face		'font-lock-string-face
   "Face name to use for strings.")

+(make-obsolete-variable 'font-lock-doc-face
+                        "use 'font-lock-doc-face instead." "31.1")
 (defvar font-lock-doc-face		'font-lock-doc-face
   "Face name to use for documentation.")

+(make-obsolete-variable 'font-lock-doc-markup-face
+                        "use 'font-lock-doc-markup-face instead." "31.1")
 (defvar font-lock-doc-markup-face       'font-lock-doc-markup-face
   "Face name to use for documentation mark-up.")

+(make-obsolete-variable 'font-lock-keyword-face
+                        "use 'font-lock-keyword-face instead." "31.1")
 (defvar font-lock-keyword-face		'font-lock-keyword-face
   "Face name to use for keywords.")

+(make-obsolete-variable 'font-lock-builtin-face
+                        "use 'font-lock-builtin-face instead." "31.1")
 (defvar font-lock-builtin-face		'font-lock-builtin-face
   "Face name to use for builtins.")

+(make-obsolete-variable 'font-lock-function-name-face
+                        "use 'font-lock-function-name-face instead." "31.1")
 (defvar font-lock-function-name-face	'font-lock-function-name-face
   "Face name to use for function names.")

+(make-obsolete-variable 'font-lock-variable-name-face
+                        "use 'font-lock-variable-name-face instead." "31.1")
 (defvar font-lock-variable-name-face	'font-lock-variable-name-face
   "Face name to use for variable names.")

+(make-obsolete-variable 'font-lock-type-face
+                        "use 'font-lock-type-face instead." "31.1")
 (defvar font-lock-type-face		'font-lock-type-face
   "Face name to use for type and class names.")

+(make-obsolete-variable 'font-lock-constant-face
+                        "use 'font-lock-constant-face instead." "31.1")
 (defvar font-lock-constant-face		'font-lock-constant-face
   "Face name to use for constant and label names.")

+(make-obsolete-variable 'font-lock-warning-face
+                        "use 'font-lock-warning-face instead." "31.1")
 (defvar font-lock-warning-face		'font-lock-warning-face
   "Face name to use for things that should stand out.")

+(make-obsolete-variable 'font-lock-negation-char-face
+                        "use 'font-lock-negation-char-face instead." "31.1")
 (defvar font-lock-negation-char-face	'font-lock-negation-char-face
   "Face name to use for easy to overlook negation.
 This can be an \"!\" or the \"n\" in \"ifndef\".")

+(make-obsolete-variable 'font-lock-preprocessor-face
+                        "use 'font-lock-preprocessor-face instead." "31.1")
 (defvar font-lock-preprocessor-face	'font-lock-preprocessor-face
   "Face name to use for preprocessor directives.")

bug#71469: font-lock does not apply standard faces and their descendants

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> We have occasionally declared things obsolete with the understanding
> that they'll be deleted much later than the normal "10 years, give or
> take".  See the comment in `interactive-p', for example.
>
> The patch below gives me 64 warnings in our tree.  I'm not sure if it's
> worth installing or not.

FWIW, it's a +1 from me (assuming we then silence the resulting
warnings, of course).


        Stefan

bug#71469: font-lock does not apply standard faces and their descendants

[Robert Pluim]

>>>>> On Sat, 05 Oct 2024 09:06:24 -0400, Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said:

    >> We have occasionally declared things obsolete with the understanding
    >> that they'll be deleted much later than the normal "10 years, give or
    >> take".  See the comment in `interactive-p', for example.
    >> 
    >> The patch below gives me 64 warnings in our tree.  I'm not sure if it's
    >> worth installing or not.

    Stefan> FWIW, it's a +1 from me (assuming we then silence the resulting
    Stefan> warnings, of course).

emacs-lisp/lisp-mode.el:311:18: Warning: ‘font-lock-warning-face’ is
an obsolete variable (as of 31.1); use ’font-lock-warning-face
instead.

sounds like a great way to cause confusion. How about:

"use value 'font-lock-comment-face directly instead"

or similar?

Robert
-- 

bug#71469: font-lock does not apply standard faces and their descendants

[Konstantin Kharlamov]

On Tue, 2024-10-08 at 14:57 +0200, Robert Pluim wrote:
> > > > > > On Sat, 05 Oct 2024 09:06:24 -0400, Stefan Monnier via "Bug
> > > > > > reports for GNU Emacs, the Swiss army knife of text
> > > > > > editors" <bug-gnu-emacs@gnu.org> said:
> 
>     >> We have occasionally declared things obsolete with the
> understanding
>     >> that they'll be deleted much later than the normal "10 years,
> give or
>     >> take".  See the comment in `interactive-p', for example.
>     >> 
>     >> The patch below gives me 64 warnings in our tree.  I'm not
> sure if it's
>     >> worth installing or not.
> 
>     Stefan> FWIW, it's a +1 from me (assuming we then silence the
> resulting
>     Stefan> warnings, of course).
> 
> emacs-lisp/lisp-mode.el:311:18: Warning: ‘font-lock-warning-face’ is
> an obsolete variable (as of 31.1); use ’font-lock-warning-face
> instead.
> 
> sounds like a great way to cause confusion. How about:
> 
> "use value 'font-lock-comment-face directly instead"

Maybe "pass font-lock-comment-face as a symbol instead"? To understand
what "directly" means one would have to notice the lone singular quote,
which I didn't when I read the suggestion, so I imagine I'd feel
confused had I not know what the warning is about.

bug#71469: font-lock does not apply standard faces and their descendants

[Robert Pluim]

>>>>> On Tue, 08 Oct 2024 16:25:32 +0300, Konstantin Kharlamov <Hi-Angel@yandex.ru> said:
    >> 
    >> "use value 'font-lock-comment-face directly instead"

    Konstantin> Maybe "pass font-lock-comment-face as a symbol instead"? To understand
    Konstantin> what "directly" means one would have to notice the lone singular quote,
    Konstantin> which I didn't when I read the suggestion, so I imagine I'd feel
    Konstantin> confused had I not know what the warning is about.

Then the question becomes "how do I pass font-lock-comment-face as a
symbol", to which the answer is 'font-lock-comment-face, which is in
my suggestion :-)

How about "use quoted value 'font-lock-comment-face directly instead"?

Robert
-- 

bug#71469: font-lock does not apply standard faces and their descendants

[Konstantin Kharlamov]

On Wed, 2024-10-09 at 10:43 +0200, Robert Pluim wrote:
> > > > > > On Tue, 08 Oct 2024 16:25:32 +0300, Konstantin Kharlamov
> > > > > > <Hi-Angel@yandex.ru> said:
>     >> 
>     >> "use value 'font-lock-comment-face directly instead"
> 
>     Konstantin> Maybe "pass font-lock-comment-face as a symbol
> instead"? To understand
>     Konstantin> what "directly" means one would have to notice the
> lone singular quote,
>     Konstantin> which I didn't when I read the suggestion, so I
> imagine I'd feel
>     Konstantin> confused had I not know what the warning is about.
> 
> Then the question becomes "how do I pass font-lock-comment-face as a
> symbol", to which the answer is 'font-lock-comment-face, which is in
> my suggestion :-)

Well, yeah, this implies a user knows what "a lisp symbol" is. I like
your suggestion to include the quote:

> How about "use quoted value 'font-lock-comment-face directly
> instead"?

I'm wondering whether "directly" makes this sentence any more clear or
maybe it's excessive? So like, maybe just "use quoted value 'font-lock-
comment-face instead"? Also, "the value" may be ambiguous here (because
`foo` is a value and `'foo` is a symbol, so we kind end up overloading
the meaning in the sentence) so maybe even make it shorter: "pass
'font-lock-comment-face quoted instead"? (FWIW, english isn't my native
language, Idk if the sentence flow sounds okay here)

bug#71469: font-lock does not apply standard faces and their descendants

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> How about "use quoted value 'font-lock-comment-face directly instead"?

Much better, yes.  I'd even say "symbol" rather than "value" to be
more precise.


        Stefan

bug#71469: font-lock does not apply standard faces and their descendants

[Robert Pluim]

>>>>> On Wed, 09 Oct 2024 08:48:47 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

    >> How about "use quoted value 'font-lock-comment-face directly instead"?
    Stefan> Much better, yes.  I'd even say "symbol" rather than "value" to be
    Stefan> more precise.

Works for me.

Robert
-- 

bug#71469: font-lock does not apply standard faces and their descendants

[Konstantin Kharlamov]

On Wed, 2024-10-09 at 08:48 -0400, Stefan Monnier wrote:
> > How about "use quoted value 'font-lock-comment-face directly
> > instead"?
> 
> Much better, yes.  I'd even say "symbol" rather than "value" to be
> more precise.

Right, same as I mentioned. Do you think my suggestion "pass
'font-lock-comment-face quoted instead" will not be a better wording?
It's shorter.

bug#71469: font-lock does not apply standard faces and their descendants

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > How about "use quoted value 'font-lock-comment-face directly
>> > instead"?
>> 
>> Much better, yes.  I'd even say "symbol" rather than "value" to be
>> more precise.
>
> Right, same as I mentioned. Do you think my suggestion "pass
> 'font-lock-comment-face quoted instead" will not be a better wording?
> It's shorter.

No preference.  We could use a more active form, like

    Quote the font-lock-comment-face symbol instead


- Stefan