Re: Font-lock of comments using comment tokens, does it work?

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> Also, as Emacs maintainer I have enough experience/knowledge to fix
>> most users's problems, but if I do that I'll just end up with more
>> users with new problems to fix.  So instead I'm better off trying to
>> train them so they can fix their problems themselves and even help me
>> improve Emacs.
>>> I've tried a dozen different permutations of the regexp and none of
>>> them produces the desired result.
>> What have you tried?  What/where were the undesired results?

> ("[a-zA-Z0-9_]\\(! \\) " (1 "_")))

IIUC you want all "!" that are surrounded by spaces to be treated as
comment starters.  And you've marked "!" as a comment starter by default
(i.e. in the mode's syntax-table), so you need to mark all "!" which are
not surrounded by spaces as being not-comment-starters.

The above regexp does part of the work, but only does it for those "!"
which are preceded by a latin letter or a number and are followed by
a space.  E.g. it will fail on those "!" which don't have a space afterwards.

> ("\\(!\\)[a-zA-Z0-9_]" (1 "_")))

This one will fail on those "!" which are followed with a letter that's
neither a space nor a latin letter nor a number.  And it will fail on
those "!" which are followed by a space but are not preceded by a space.

To me, the translation into regexp of «all "!" which are not surrounded
by spaces» would look like "[^ ]![^ ]".  Have you tried something like
that?  Of course, it'll still probably require more tweaking because
I suspect that «all "!" which are not surrounded by spaces» is not
actually a precise description of all cases that matter.  E.g. I suspect
that if the "!" is preceded by a newline (i.e. is at the beginning of
a line) it should still be considered a comment starter.  Same thing if
it's preceded by a TAB.  Also it's likely that " !! " would also start
a comment, so "followed by a space" is too strict as well.  But then,
I don't know if " !!a" would be treated as starting a comment.
IOW, maybe you'll want something like "[^ \n\t]\\(!+\\)[^
\t\n]" instead.

One more thing: if "! as a normal char" is more common than "! as
a comment starter", it might be worthwhile to take the opposite approach
and define the syntax of "!" in the mode's syntax-table as being "_" and
then in syntax-propertize-function mark those "!" which start a comment
as having syntax "<".

Yet another thing: if you have trouble catching all cases with a single
regexp, you can use more rules, as in

   (syntax-propertize-rules
    ("[a-zA-Z0-9_]\\(! \\) " (1 "_"))
    ("\\(!\\)[a-zA-Z0-9_]" (1 "_")))
    

        Stefan