problem understanding font-lock-defaults structure

problem understanding font-lock-defaults structure

[Xah]

2008-01-13

i'm having some problem understanding font-lock-defaults.

For example, i have the following text in a buffer:
------------------------

 Sin[x]^2 + Cos[y]^2 = 1

(setq font-lock-defaults
  '(
    (
      (
        ("Sin\\|Cos" . font-lock-function-name-face)
        ("x\\|y" . font-lock-variable-name-face)
        ("π" . font-lock-constant-face)
      ) ; level 1 fontlock
      nil ; no level 2 of fontlock
      nil ; no level 3 of fontlock
    )
;;     t     ; keywords only
;;     nil     ; case matters
;;     nil   ; no highlighting by syntax
;;     nil   ; no hint on syntax begin for highlighting
   )
)

(font-lock-fontify-buffer)
---------------------------------

After i evaluate the lisp code, i expect, the Sin should be syntax
highlighted but is not.

One thing i find elisp manual on this confusing is exactly what is the
structure of the keyword argument, i.e. the first element of the value
for font-lock-defaults. I'm not sure how many level of list is
required, or if the nesting is optional... The other thing i'm not
sure is perhaps my quoting is not correct?

After several variations, i can't get it to work. Any ideas?

Thanks.

  Xah
∑ http://xahlee.org/

☄

Re: problem understanding font-lock-defaults structure

[Nikolaj Schumacher]

Xah <xahlee@gmail.com> wrote:

> 2008-01-13
??

> After i evaluate the lisp code, i expect, the Sin should be syntax
> highlighted but is not.

From the doc: "Defaults for Font Lock mode specified by the major mode."

Setting this variable only works when you define a new major mode.  To
add keywords afterwards, use `font-lock-add-keywords'

> I'm not sure how many level of list is required, or if the nesting is
> optional...

Using multiple keywords is optional, i.e. you may leave out the two nils
and one level of parentheses.  Your nesting and quoting is correct.

You might also want to use "\\<Sin\\>" instead of "Sin", etc.

regards,
Nikolaj Schumacher

Re: problem understanding font-lock-defaults structure

[Xah]

On Oct 8, 2:22 pm, Nikolaj Schumacher <m...@nschum.de> wrote:
> Xah <xah...@gmail.com> wrote:
> > 2008-01-13
>
> ??
>
> > After i evaluate the lisp code, i expect, the Sin should be syntax
> > highlighted but is not.
>
> From the doc: "Defaults for Font Lock mode specified by the major mode."
>
> Setting this variable only works when you define a new major mode.  To
> add keywords afterwards, use `font-lock-add-keywords'
>
> > I'm not sure how many level of list is required, or if the nesting is
> > optional...
>
> Using multiple keywords is optional, i.e. you may leave out the two nils
> and one level of parentheses.  Your nesting and quoting is correct.
>
> You might also want to use "\\<Sin\\>" instead of "Sin", etc.
>
> regards,
> Nikolaj Schumacher

Thanks...

am having a headache with this.

;Why does the following works:

; Sin[x]^2 + Cos[y]^2 = 1
; π^2/6 == Sum[1/x^2,{x,1,∞}]

(setq font-lock-keywords
 (quote
  (
   ("Sin\\|Cos" . font-lock-function-name-face)
   ("π\\|∞" . font-lock-constant-face)
   ("x\\|y" . font-lock-variable-name-face)
  )
 nil
 nil
 )
)

;But in the following, i put my keywords into a var, the value of
myKeywordsLevel1 is just the first element ("Sin\\|Cos" . font-lock-
function-name-face)?

(setq myKeywordsLevel1
 (quote
   ("Sin\\|Cos" . font-lock-function-name-face)
   ("π\\|∞" . font-lock-constant-face)
   ("x\\|y" . font-lock-variable-name-face)
 )
)

(setq font-lock-keywords
 `(
  ,myKeywordsLevel1
 nil
 nil
 )
)

(font-lock-fontify-buffer)

  Xah
∑ http://xahlee.org/

☄

Re: problem understanding font-lock-defaults structure

[Lennart Borgman (gmail)]

Xah wrote:
> On Oct 8, 2:22 pm, Nikolaj Schumacher <m...@nschum.de> wrote:
>> Xah <xah...@gmail.com> wrote:
>>> 2008-01-13
>> ??
>>
>>> After i evaluate the lisp code, i expect, the Sin should be syntax
>>> highlighted but is not.
>> From the doc: "Defaults for Font Lock mode specified by the major mode."
>>
>> Setting this variable only works when you define a new major mode.  To
>> add keywords afterwards, use `font-lock-add-keywords'
>>
>>> I'm not sure how many level of list is required, or if the nesting is
>>> optional...
>> Using multiple keywords is optional, i.e. you may leave out the two nils
>> and one level of parentheses.  Your nesting and quoting is correct.
>>
>> You might also want to use "\\<Sin\\>" instead of "Sin", etc.
>>
>> regards,
>> Nikolaj Schumacher
> 
> Thanks...
> 
> am having a headache with this.
> 
> ;Why does the following works:
> 
> ; Sin[x]^2 + Cos[y]^2 = 1
> ; π^2/6 == Sum[1/x^2,{x,1,∞}]
> 
> (setq font-lock-keywords
>  (quote
>   (
>    ("Sin\\|Cos" . font-lock-function-name-face)
>    ("π\\|∞" . font-lock-constant-face)
>    ("x\\|y" . font-lock-variable-name-face)
>   )
>  nil
>  nil
>  )
> )
> 
> ;But in the following, i put my keywords into a var, the value of
> myKeywordsLevel1 is just the first element ("Sin\\|Cos" . font-lock-
> function-name-face)?
> 
> (setq myKeywordsLevel1
>  (quote
>    ("Sin\\|Cos" . font-lock-function-name-face)
>    ("π\\|∞" . font-lock-constant-face)
>    ("x\\|y" . font-lock-variable-name-face)
>  )
> )

Could it be that the `quote' just takes one argument?

> (setq font-lock-keywords
>  `(
>   ,myKeywordsLevel1
>  nil
>  nil
>  )
> )
> 
> (font-lock-fontify-buffer)
> 
>   Xah
> ∑ http://xahlee.org/
> 
> ☄
> 

Re: problem understanding font-lock-defaults structure

[Kevin Rodgers]

Lennart Borgman (gmail) wrote:
> Xah wrote:
...
>> (setq myKeywordsLevel1
>>  (quote
>>    ("Sin\\|Cos" . font-lock-function-name-face)
>>    ("π\\|∞" . font-lock-constant-face)
>>    ("x\\|y" . font-lock-variable-name-face)
>>  )
>> )
> 
> Could it be that the `quote' just takes one argument?

Indeed.  Is it a bug that quote does not signal "Wrong number of
arguments"?  I know that special forms do not necessarily evaluate their
arguments, but neither do macros and eval/apply manages to make sure
that the number of supplied arguments conforms to a macro's argument
list.

-- 
Kevin Rodgers
Denver, Colorado, USA

Re: problem understanding font-lock-defaults structure

[Xah]

Lennart Borgman (gmail) wrote:
Could it be that the `quote' just takes one argument?

Kevin Rodgers wrote:
> Indeed.  Is it a bug that quote does not signal "Wrong number of
> arguments"?  I know that special forms do not necessarily evaluate their
> arguments, but neither do macros and eval/apply manages to make sure
> that the number of supplied arguments conforms to a macro's argument
> list.

i reported this. They said it's fixed in emacs 23.

http://groups.google.com/group/gnu.emacs.bug/browse_frm/thread/d844d935797779b

  Xah
∑ http://xahlee.org/

☄

Re: problem understanding font-lock-defaults structure

[Nikolaj Schumacher]

Xah <xahlee@gmail.com> wrote:

> (setq myKeywordsLevel1
>  (quote
>    ("Sin\\|Cos" . font-lock-function-name-face)
>    ("π\\|∞" . font-lock-constant-face)
>    ("x\\|y" . font-lock-variable-name-face)
>  )
> )

It needs to be:

(setq myKeywordsLevel1
  (list
    ("Sin\\|Cos" . font-lock-function-name-face)
    ("π\\|∞" . font-lock-constant-face)
    ("x\\|y" . font-lock-variable-name-face)))

or

(setq myKeywordsLevel1
  (quote
    (("Sin\\|Cos" . font-lock-function-name-face)
     ("π\\|∞" . font-lock-constant-face)
     ("x\\|y" . font-lock-variable-name-face))))

> (setq font-lock-keywords
>  `(
>   ,myKeywordsLevel1
>  nil
>  nil
>  )
> )

There's no need to use `, here.  `font-lock-keywords' accepts symbols.

(setq font-lock-keywords '((myKeywordsLevel1 nil nil)))

or

(setq font-lock-keywords '(myKeywordsLevel1)

will work.

regards,
Nikolaj Schumacher

Re: problem understanding font-lock-defaults structure

[Xah]

Lennart Borgman wrote:
> Could it be that the “quote” just takes one argument?

Egads. This costed me few hours.

Am filing a bug report on this...

... i always thought that the syntactical form

'(...)

is just equivalent to

(quote ...)

  Xah
∑ http://xahlee.org/

☄

Re: problem understanding font-lock-defaults structure

[harven]

Xah <xahlee@gmail.com> writes:

> Lennart Borgman wrote:
>> Could it be that the “quote” just takes one argument?
>
> Egads. This costed me few hours.
>
> Am filing a bug report on this...
>
> ... i always thought that the syntactical form
>
> '(...)
>
> is just equivalent to
>
> (quote ...)

From the elisp manual,
...  Thus, the read syntax `'x' is an abbreviation for `(quote x)' ...
So '(...) is equivalent to (quote (...))

Re: problem understanding font-lock-defaults structure

[Nikolaj Schumacher]

harven <harven@free.fr> wrote:

> Xah <xahlee@gmail.com> writes:
>
>> ... i always thought that the syntactical form
>>
>> '(...)
>>
>> is just equivalent to
>>
>> (quote ...)
>
> From the elisp manual,
> ...  Thus, the read syntax `'x' is an abbreviation for `(quote x)' ...
> So '(...) is equivalent to (quote (...))

This is essential, too.  Because otherwise (quote a) could mean either
'a or '(a).


regards,
Nikolaj Schumacher

Re: problem understanding font-lock-defaults structure

[Xah]

(setq myKeywordsLevel1
  (list
    ("Sin\\|Cos" . font-lock-function-name-face)
    ("π\\|∞" . font-lock-constant-face)
    ("x\\|y" . font-lock-variable-name-face)))

Are you sure? It gives (invalid-function "Sin\\|Cos").

> There's no need to use `, here.  `font-lock-keywords' accepts symbols.
> (setq font-lock-keywords '((myKeywordsLevel1 nil nil)))

Thanks.

I tried the following but still not working:

; Sin[x]^2 + Cos[y]^2 = 1
; π^2/6 == Sum[1/x^2,{x,1,∞}]

(setq myKeywordsLevel1
 '(
   ("Sin\\|Cos" . font-lock-function-name-face)
   ("π\\|∞" . font-lock-constant-face)
   ("x\\|y" . font-lock-variable-name-face)
  )
)

(setq font-lock-keywords
 '(
  myKeywordsLevel1
  )
)

(font-lock-fontify-buffer)

... quite frustrated by this.

after eval that form, than i looked at the value of font-lock-keywords
using describe-variable, it says:
«
Its value is
(t
 (myKeywordsLevel1)
 (myKeywordsLevel1
  (0 font-lock-keyword-face)))

Local in buffer untitled<3>;
»

Where did all the extra came from?

I tried various quoting, like “'((...) ...)”, “(quote (list ...))”,
“`( ,...)” in one or the other code block, and also tried a deeper
nested for the value of the value for font-lock-keywords ... (... some
background: am trying to write lsl-mode (yes i know existing ones) and
meanwhile write a simple tutorial about writing mode from the ground
up. I have read the chapters on major mode and font lock mode, also
have taken apart few lang modes. I have a working mode but want
understanding and a simple example.)

Thanks.

  Xah
∑ http://xahlee.org/

☄

Re: problem understanding font-lock-defaults structure

[Nikolaj Schumacher]

Xah <xahlee@gmail.com> wrote:

> (setq myKeywordsLevel1
>   (list
>     ("Sin\\|Cos" . font-lock-function-name-face)
>     ("π\\|∞" . font-lock-constant-face)
>     ("x\\|y" . font-lock-variable-name-face)))
>
> Are you sure? It gives (invalid-function "Sin\\|Cos").

You're right.

(setq myKeywordsLevel1
      (list (cons "Sin\\|Cos" font-lock-function-name-face)
            (cons "π\\|∞" font-lock-constant-face)
            (cons "x\\|y" font-lock-variable-name-face)))

I was looking only at the "(quote" line.

>> There's no need to use `, here.  `font-lock-keywords' accepts symbols.
>> (setq font-lock-keywords '((myKeywordsLevel1 nil nil)))
>
> after eval that form, than i looked at the value of font-lock-keywords
> using describe-variable, it says:

My bad, too.  I meant `font-lock-defaults' -- the variable you were
already using, and we were talking about.  -keywords just crept in from
finger memory.

Here's a complete, working example:

(setq myKeywordsLevel1
 '(("Sin\\|Cos" . font-lock-function-name-face)
   ("π\\|∞" . font-lock-constant-face)
   ("x\\|y" . font-lock-variable-name-face)))

(define-derived-mode foo-mode fundamental-mode
  (setq font-lock-defaults '(myKeywordsLevel1)))

> Its value is
> (t
>  (myKeywordsLevel1)
>  (myKeywordsLevel1
>   (0 font-lock-keyword-face)))
>
> Local in buffer untitled<3>;
> »
>
> Where did all the extra came from?

The list has been "compiled".  The t marks it as such, so it isn't
processed twice.  Due to this kind of voodoo, I avoid touching
`font-lock-keywords'.  I use `font-lock-add-keywords'.



regards,
Nikolaj Schumacher

Re: problem understanding font-lock-defaults structure

[Xah]

On Oct 9, 10:16 am, Nikolaj Schumacher <m...@nschum.de> wrote:

> Here's a complete, working example:
>
> (setq myKeywordsLevel1
>  '(("Sin\\|Cos" . font-lock-function-name-face)
>    ("π\\|∞" . font-lock-constant-face)
>    ("x\\|y" . font-lock-variable-name-face)))
>
> (define-derived-mode foo-mode fundamental-mode
>   (setq font-lock-defaults '(myKeywordsLevel1)))
>
> > Its value is
> > (t
> >  (myKeywordsLevel1)
> >  (myKeywordsLevel1
> >   (0 font-lock-keyword-face)))
>
> > Local in buffer untitled<3>;
> > »
>
> > Where did all the extra came from?
>
> The list has been "compiled".  The t marks it as such, so it isn't
> processed twice.  Due to this kind of voodoo, I avoid touching
> `font-lock-keywords'.  I use `font-lock-add-keywords'.

Thanks a lot. That's helpful and much info in this thread.

possibly begging, is it possible to get this to work with using font-
lock-keywords?

Here's what i have now based on all the info in this thread. I
couldn't see where it went wrong.

; Sin[x]^2 + Cos[y]^2 = 1
; π^2/6 == Sum[1/x^2,{x,1,∞}]

(setq myKeywordsLevel1
 (quote
(
   ("Sin\\|Cos" . font-lock-function-name-face)
   ("π\\|∞" . font-lock-constant-face)
   ("x\\|y" . font-lock-variable-name-face)
)
 )
)

(setq font-lock-keywords
 (
 myKeywordsLevel1
 nil
 nil
 )
)

(font-lock-fontify-buffer)

In the “setq font-lock-keywords”, block, i tried various quoting and
nesting but no go.

  Xah
∑ http://xahlee.org/

☄

Re: problem understanding font-lock-defaults structure

[Nikolaj Schumacher]

Xah <xahlee@gmail.com> wrote:

> (setq font-lock-keywords
>  (
>  myKeywordsLevel1
>  nil
>  nil
>  )
> )
>
> (font-lock-fontify-buffer)
>
> In the “setq font-lock-keywords”, block, i tried various quoting and
> nesting but no go.

The idea of multiple levels of fontification is in a higher abstraction
level than this.  `font-lock-keywords' doesn't support it.

(setq font-lock-keywords myKeywordsLevel1)

regards,
Nikolaj Schumacher

Re: problem understanding font-lock-defaults structure

[Xah]

> (setq font-lock-keywords myKeywordsLevel1)

Thanks!

gosh, something is wrong with the manual. I think the elisp manual is
superb, but for the first time i find the chapter on major-mode and
the font-lock-mode quite sub par in clarity, completeness, and most of
all lacking good content.

http://xahlee.org/elisp/Major-Modes.html
http://xahlee.org/elisp/Font-Lock-Mode.html

the chapter on major-mode doesn't really say exactly how one creates
it. After reading it, one doesn't get any idea what exactly is the
mechanism that creates a major mode. (is it purely a function name
ending in mode??) The bulk of it is a list of conventions. These
conventions are described fuzzily, as if assuming the reader are
already expert of the elisp system. One's not clear which of the item
in the convention section is required for thing to work, which is
required for the mode to work as expected of emacs major modes, and
which are just suggestions on niceties.

the most important part of writing a mode, the syntax highlighting...
the font lock mode section is rather confusing about font-lock-
keywords and font-lock-defaults...

  Xah
∑ http://xahlee.org/

☄

On Thu, Oct 9, 2008 at 4:27 PM, Nikolaj Schumacher <me@nschum.de>
wrote:
Xah <xahlee@gmail.com> wrote:

> (setq font-lock-keywords
>  (
>  myKeywordsLevel1
>  nil
>  nil
>  )
> )
>
> (font-lock-fontify-buffer)
>
> In the “setq font-lock-keywords”, block, i tried various quoting and
> nesting but no go.

The idea of multiple levels of fontification is in a higher
abstraction
level than this.  `font-lock-keywords' doesn't support it.

(setq font-lock-keywords myKeywordsLevel1)

Re: problem understanding font-lock-defaults structure

[Tim X]

Xah <xahlee@gmail.com> writes:

> On Oct 9, 10:16 am, Nikolaj Schumacher <m...@nschum.de> wrote:
>
>> Here's a complete, working example:
>>
>> (setq myKeywordsLevel1
>>  '(("Sin\\|Cos" . font-lock-function-name-face)
>>    ("π\\|∞" . font-lock-constant-face)
>>    ("x\\|y" . font-lock-variable-name-face)))
>>
>> (define-derived-mode foo-mode fundamental-mode
>>   (setq font-lock-defaults '(myKeywordsLevel1)))
>>
>> > Its value is
>> > (t
>> >  (myKeywordsLevel1)
>> >  (myKeywordsLevel1
>> >   (0 font-lock-keyword-face)))
>>
>> > Local in buffer untitled<3>;
>> > »
>>
>> > Where did all the extra came from?
>>
>> The list has been "compiled".  The t marks it as such, so it isn't
>> processed twice.  Due to this kind of voodoo, I avoid touching
>> `font-lock-keywords'.  I use `font-lock-add-keywords'.
>
> Thanks a lot. That's helpful and much info in this thread.
>
> possibly begging, is it possible to get this to work with using font-
> lock-keywords?
>
> Here's what i have now based on all the info in this thread. I
> couldn't see where it went wrong.
>
> ; Sin[x]^2 + Cos[y]^2 = 1
> ; π^2/6 == Sum[1/x^2,{x,1,∞}]
>
> (setq myKeywordsLevel1
>  (quote
> (
>    ("Sin\\|Cos" . font-lock-function-name-face)
>    ("π\\|∞" . font-lock-constant-face)
>    ("x\\|y" . font-lock-variable-name-face)
> )
>  )
> )
>
> (setq font-lock-keywords
>  (
>  myKeywordsLevel1
>  nil
>  nil
>  )
> )
>
> (font-lock-fontify-buffer)
>
> In the “setq font-lock-keywords”, block, i tried various quoting and
> nesting but no go.
>
>   Xah
> ∑ http://xahlee.org/

Have a look at sql.el in the lisp/progmodes directory. It has very clear
examples of how you can do what you want. As the mode supports different
font-locking depending on the type of database your connecting to, it
has a mechanism for setting what keywords are font locked dynamically,
which seems to be close to what you are wanting to do (i.e. assign the
words to be font-locked to a variable and then use that in setting up
font locking). Also, Alex's code
is quite clear and well structured, making it quite easy to follow what
he is doing. 

Tim
☄

-- 
tcross (at) rapttech dot com dot au