Re: Font-lock of comments using comment tokens, does it work?

[<tomas@tuxteam.de>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Jun 05, 2015 at 05:29:36AM +0200, Björn Lindqvist wrote:
> 2015-06-05 0:11 GMT+02:00 Stefan Monnier <monnier@iro.umontreal.ca>:

[...]

> No. I want two strings, FOO and BAR (or ! doesn't matter, same
> principle) to start comments iff they are separate tokens.

Sorry to jump in the middle. I've been lurking in case I could help
(and to learn about font-lock).

Björn: you are assuming that everyone knows what's a "token" to you.
And you are assuming that everyone has the time to read and grasp
all your examples, first time. I for one don't know what your tokens
are. To put one extreme example, to C, the string 'a+b' are three
tokens, '++a' are two; for Lisp, the first and the second example are
both just *one* token.

Given that, you can't expect Stefan to even come near a regular expression
useful to you, since what they are doing is exactly *separate tokens*.

> > To me, the translation into regexp of «all "!" which are not surrounded
> > by spaces» would look like "[^ ]![^ ]".  Have you tried something like
> > that?
> 
> That turns the comment face of if the ! is in the middle, but not if
> it prefixes or suffixes the token. abcFOO is wrongly interpreted as a
> comment starter.

You mean when FOO is at the end of the line? Then no character would be
there and the second '[^ ] wouldn't match?

That's what Stefan said, you'll have to tweak this. Use an alternative '\|',
something like "\(^\|[^ ]\)!\([^ ]\|$\)" (i.e. match at beginning-of-line-
or-space, then "!" then space-or-end-of-line. You can use "\(?: ... \)
if you want non-capturing groups. Watch out for those backslashes: you
want to double them when writing them as an Elisp string [1].

> > Also it's likely that " !! " would also start
> > a comment [...]

> No. In "!!" and "!!a" the comment token is not separate, so no comment.

See? we are all guessing at what your tokens are. Designing the
nitty-gritties of your regexps and testing them can only be your
work, because we'd be all fighting phantoms.

> > Yet another thing: if you have trouble catching all cases with a single
> > regexp, you can use more rules, as in
> >
> >    (syntax-propertize-rules
> >     ("[a-zA-Z0-9_]\\(! \\) " (1 "_"))
> >     ("\\(!\\)[a-zA-Z0-9_]" (1 "_")))
> 
> It still messes up the comment font-locking. BTW I've noticed that if
> the regexp is "test\\(!\\)" emacs correctly does not use comment face
> on "test!". But if it is "\\(!\\)test" then "!test" is still seen as a
> comment. That is inconsistent with what you have explained and the
> elisp manual. So I think it is a bug.

I don't understand you. What is this "test", where does it come from
and what is it doing *in* the regular expression? (and in which one:
in syntax-propertize rules, as in Stefan's example above, or somewhere
else?

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlVxR0wACgkQBcgs9XrR2kZXwwCdGe35a21eiwMqPmJ/xYmnjd5H
RK8AniLaRZ3iKcGU1ah3uAPeJNpERABf
=vTni
-----END PGP SIGNATURE-----