How to make font lock work with comments?

How to make font lock work with comments?

[rusi]

Im hacking on an apl mode


 (let ((st (make-syntax-table))
        (comment-char ?\x235d))
:
:
 (modify-syntax-entry comment-char "<" st)

This char shows as follows (Dont know if it shows elsewhere..)

⍝ A comment

When I put point on the char and call describe-char I get

---------------------------------
        character: ⍝ (9053, #o21535, #x235d)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x235D
           syntax: < 	which means: comment
         category: .:Base
         to input: type "{upshoe-jot}" or "{lamp}" or "{comment}" or
"{@}" with apl-ascii
      buffer code: #xE2 #x8D #x9D
        file code: not encodable by coding system iso-latin-1-unix
          display: by this font (glyph code)
    xft:-unknown-unifont-normal-normal-normal-*-12-*-*-*-d-0-
iso10646-1 (#x2359)
------------------------------

In other words emacs sees this as a comment-type char (similar to what
it says for semicolon in elisp buffers

And yet in an elisp buffer the ; to EOL is red
but here it is not.

Any clues?

Re: How to make font lock work with comments?

[Elena]

On Dec 29, 9:50 am, rusi <rustompm...@gmail.com> wrote:
> Im hacking on an apl mode
>
>  (let ((st (make-syntax-table))
>         (comment-char ?\x235d))
> :
> :
>  (modify-syntax-entry comment-char "<" st)
>
> This char shows as follows (Dont know if it shows elsewhere..)
>
> ⍝ A comment
>
> When I put point on the char and call describe-char I get
>
> ---------------------------------
>         character: ⍝ (9053, #o21535, #x235d)
> preferred charset: unicode (Unicode (ISO10646))
>        code point: 0x235D
>            syntax: <         which means: comment
>          category: .:Base
>          to input: type "{upshoe-jot}" or "{lamp}" or "{comment}" or
> "{@}" with apl-ascii
>       buffer code: #xE2 #x8D #x9D
>         file code: not encodable by coding system iso-latin-1-unix
>           display: by this font (glyph code)
>     xft:-unknown-unifont-normal-normal-normal-*-12-*-*-*-d-0-
> iso10646-1 (#x2359)
> ------------------------------
>
> In other words emacs sees this as a comment-type char (similar to what
> it says for semicolon in elisp buffers
>
> And yet in an elisp buffer the ; to EOL is red
> but here it is not.
>
> Any clues?

Syntax and font-locking are two different things and are handled
separately.  Commands look at syntax, humans look at font-locking.

Try typing the string ";" (quotes included) into an Elisp buffer and
then calling `describe-char' on the semicolon: Emacs will tell you the
semicolon is a comment char, yet it will be highlighting it as string.

Check out `font-lock-add-keywords'.

Re: How to make font lock work with comments?

[Tim X]

Elena <egarrulo@gmail.com> writes:

> On Dec 29, 9:50 am, rusi <rustompm...@gmail.com> wrote:
>> Im hacking on an apl mode
>>
>>  (let ((st (make-syntax-table))
>>         (comment-char ?\x235d))
>> :
>> :
>>  (modify-syntax-entry comment-char "<" st)
>>
>> This char shows as follows (Dont know if it shows elsewhere..)
>>
>> ⍝ A comment
>>
>> When I put point on the char and call describe-char I get
>>
>> ---------------------------------
>>         character: ⍝ (9053, #o21535, #x235d)
>> preferred charset: unicode (Unicode (ISO10646))
>>        code point: 0x235D
>>            syntax: <         which means: comment
>>          category: .:Base
>>          to input: type "{upshoe-jot}" or "{lamp}" or "{comment}" or
>> "{@}" with apl-ascii
>>       buffer code: #xE2 #x8D #x9D
>>         file code: not encodable by coding system iso-latin-1-unix
>>           display: by this font (glyph code)
>>     xft:-unknown-unifont-normal-normal-normal-*-12-*-*-*-d-0-
>> iso10646-1 (#x2359)
>> ------------------------------
>>
>> In other words emacs sees this as a comment-type char (similar to what
>> it says for semicolon in elisp buffers
>>
>> And yet in an elisp buffer the ; to EOL is red
>> but here it is not.
>>
>> Any clues?
>
> Syntax and font-locking are two different things and are handled
> separately.  Commands look at syntax, humans look at font-locking.
>
> Try typing the string ";" (quotes included) into an Elisp buffer and
> then calling `describe-char' on the semicolon: Emacs will tell you the
> semicolon is a comment char, yet it will be highlighting it as string.
>
> Check out `font-lock-add-keywords'.

The two are not as seperate as implied. The font-lock system can/does
use the syntax table, primarily for comments and strings. See the
section Syntactic Font Lock in the elisp manual (section 23.6.8).

Also note that the development version of Emacs has some changes in this
area i.e. from the NEWS file

,----
| ** New variable syntax-propertize-function to set syntax-table properties.
| Replaces font-lock-syntactic-keywords which are now obsolete.
| This allows syntax-table properties to be set independently from font-lock:
| just call syntax-propertize to make sure the text is propertized.
| Together with this new variable come a new hook
| syntax-propertize-extend-region-functions, as well as two helper functions:
| syntax-propertize-via-font-lock to reuse old font-lock-syntactic-keywords
| as-is; and syntax-propertize-rules which provides a new way to specify
| syntactic rules.
`----

Tim

-- 
tcross (at) rapttech dot com dot au

Re: How to make font lock work with comments?

[Pascal J. Bourguignon]

rusi <rustompmody@gmail.com> writes:

> Im hacking on an apl mode
> In other words emacs sees this as a comment-type char (similar to what
> it says for semicolon in elisp buffers
>
> And yet in an elisp buffer the ; to EOL is red
> but here it is not.

Font locking is a tricky matter. 

(font-lock-add-keywords nil
    '(("⍝.*"                          0 font-lock-comment-face prepend)
     ("[^\\]\"\\([^\"\\]*\\|\\.\\)\"" 0 font-lock-string-face  prepend)))

The main problem is that it uses regular expressions to find the
"keywords", but you want syntax coloring.  However, there's a way to use
it for syntax coloring, since instead of a regular expression, you can
use a function here, which will have to set the "matched regions".

For example, for an assembler, I wrote a function to parse
(syntactically) an assembler line, and matching the fields.  It is
configured with font-lock-add-keywords as:

 (font-lock-add-keywords 
   nil
   (list
    (list
     (function search-asm7090-fields)
     '(1 font-lock-function-name-face)       ; labels
     '(2 font-lock-keyword-face)             ; operation codes
     '(3 font-lock-reference-face)           ; arguments
     '(4 font-lock-comment-face)             ; comments
     '(5 font-lock-preprocessor-face)        ; ibsys
     '(6 font-lock-type-face)                ; cols 72-80
     )))

Of course, you can also implement buffer-wide syntax analysis, and have
these functions just report the findings.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: How to make font lock work with comments?

[rusi]

On Dec 30, 7:09 pm, "Pascal J. Bourguignon" <p...@informatimago.com>
wrote:
> rusi <rustompm...@gmail.com> writes:
> > Im hacking on an apl mode
> > In other words emacs sees this as a comment-type char (similar to what
> > it says for semicolon in elisp buffers
>
> > And yet in an elisp buffer the ; to EOL is red
> > but here it is not.
>
> Font locking is a tricky matter.
>
> (font-lock-add-keywords nil
>     '(("⍝.*"                          0 font-lock-comment-face prepend)
>      ("[^\\]\"\\([^\"\\]*\\|\\.\\)\"" 0 font-lock-string-face  prepend)))

Thanks Pascal.  I gave this (\x235 is that char) and it works
[Else I get some unicode-related messages when I try ot save the el
file]

(font-lock-add-keywords 'inferior-apl-mode
    '(("\x235d.*"     0 font-lock-comment-face prepend)))

I guess the second line is for strings?
Where do I get all the (typical) faces that should be set up for a
progmode?
Ive been trying to copy stuff from scheme.el -- is there a better
model?

>
> The main problem is that it uses regular expressions to find the
> "keywords", but you want syntax coloring.  However, there's a way to use
> it for syntax coloring, since instead of a regular expression, you can
> use a function here, which will have to set the "matched regions".
>
> For example, for an assembler, I wrote a function to parse
> (syntactically) an assembler line, and matching the fields.  It is
> configured with font-lock-add-keywords as:
>
>  (font-lock-add-keywords
>    nil
>    (list
>     (list
>      (function search-asm7090-fields)
>      '(1 font-lock-function-name-face)       ; labels
>      '(2 font-lock-keyword-face)             ; operation codes
>      '(3 font-lock-reference-face)           ; arguments
>      '(4 font-lock-comment-face)             ; comments
>      '(5 font-lock-preprocessor-face)        ; ibsys
>      '(6 font-lock-type-face)                ; cols 72-80
>      )))


I dont think I really understand this...  In any case I guess I dont
need much parsing (for apl)


>
> Of course, you can also implement buffer-wide syntax analysis, and have
> these functions just report the findings.
>
> --
> __Pascal Bourguignon__                    http://www.informatimago.com/
> A bad day in () is better than a good day in {}.

Re: How to make font lock work with comments?

[Pascal J. Bourguignon]

rusi <rustompmody@gmail.com> writes:

> On Dec 30, 7:09 pm, "Pascal J. Bourguignon" <p...@informatimago.com>
> wrote:
>> rusi <rustompm...@gmail.com> writes:
>> > Im hacking on an apl mode
>> > In other words emacs sees this as a comment-type char (similar to what
>> > it says for semicolon in elisp buffers
>>
>> > And yet in an elisp buffer the ; to EOL is red
>> > but here it is not.
>>
>> Font locking is a tricky matter.
>>
>> (font-lock-add-keywords nil
>>     '(("⍝.*"                          0 font-lock-comment-face prepend)
>>      ("[^\\]\"\\([^\"\\]*\\|\\.\\)\"" 0 font-lock-string-face  prepend)))
>
> Thanks Pascal.  I gave this (\x235 is that char) and it works

\x235d actually.

> [Else I get some unicode-related messages when I try ot save the el
> file]

I save my files encoded in utf-8.  To be sure, you can add on the first
(or second) line a comment containing "-*- coding:utf-8 -*-".
See File Local Variables in emacs documentation.



> (font-lock-add-keywords 'inferior-apl-mode
>     '(("\x235d.*"     0 font-lock-comment-face prepend)))
>
> I guess the second line is for strings?

Yes.  I wanted to avoid strings containing that character to be colorized
as comments.  The order in font-lock-keywords is important.


> Where do I get all the (typical) faces that should be set up for a
> progmode?

One way to find all the existing faces is to type:

M-x customize-face RET TAB

You can use one of the "standard" font-lock-*-face faces, or you can
also create apl specific faces.


> Ive been trying to copy stuff from scheme.el -- is there a better
> model?


>> The main problem is that it uses regular expressions to find the
>> "keywords", but you want syntax coloring.  However, there's a way to use
>> it for syntax coloring, since instead of a regular expression, you can
>> use a function here, which will have to set the "matched regions".
>>
>> For example, for an assembler, I wrote a function to parse
>> (syntactically) an assembler line, and matching the fields.  It is
>> configured with font-lock-add-keywords as:
>>
>>  (font-lock-add-keywords
>>    nil
>>    (list
>>     (list
>>      (function search-asm7090-fields)
>>      '(1 font-lock-function-name-face)       ; labels
>>      '(2 font-lock-keyword-face)             ; operation codes
>>      '(3 font-lock-reference-face)           ; arguments
>>      '(4 font-lock-comment-face)             ; comments
>>      '(5 font-lock-preprocessor-face)        ; ibsys
>>      '(6 font-lock-type-face)                ; cols 72-80
>>      )))
>
>
> I dont think I really understand this...  

Check the documentation of the function font-lock-add-keywords:
      C-h f font-lock-add-keywords RET
and the variable font-lock-keywords:
      C-h v font-lock-keywords RET

The numbers refers to the regular expression groups.

For example, if you have a regexpp that matches three different parts,
identified by three regexp groups, you can colorize them differently
with:

(font-lock-add-keywords nil
    '("\\(procedure\\) +\\([A-Za-z0-9]+\\) *(\\(.*\\));"
       (1 font-lock-keyword-face)
       (2 font-lock-function-name-face)
       (3 font-lock-type-face)))


> In any case I guess I dont need much parsing (for apl)

Right.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: How to make font lock work with comments?

[Stefan Monnier]

> Font locking is a tricky matter. 

> (font-lock-add-keywords nil
>     '(("⍝.*"                          0 font-lock-comment-face prepend)
>      ("[^\\]\"\\([^\"\\]*\\|\\.\\)\"" 0 font-lock-string-face  prepend)))

Bad idea: font-lock normally uses syntax-tables for comments and
strings, so unless you *really* can't make it work with syntax-table,
you shouldn't do it via font-lock-keywords.


        Stefan

Re: How to make font lock work with comments?

[Stefan Monnier]

> Im hacking on an apl mode
>  (let ((st (make-syntax-table))
>         (comment-char ?\x235d))
> :
> :
>  (modify-syntax-entry comment-char "<" st)

> In other words emacs sees this as a comment-type char (similar to what
> it says for semicolon in elisp buffers

> And yet in an elisp buffer the ; to EOL is red
> but here it is not.

> Any clues?

How do you setup font-lock-defaults?  I.e. is the KEYWORDS-ONLY entry
set to t by any chance?


        Stefan