Regexp search: syntax class for comment starter (\s<) does not work consistently in all modes

Regexp search: syntax class for comment starter (\s<) does not work consistently in all modes

[Janne Nykopp]

Hello!

Seems that regexp search for comment starter syntax class, \s<,
doesn't work out of the box in C, C++, Java, javascript (neither for
// or /* comment styles), or pascal ((*), ada or sql (--) modes, for
instance.

It does work out of the box in e.g. lisp (;) - but not for the
two-character comment delimiter (#|), sh (#), fortran (C and !),
postscript ps-mode (%), DOS batch bat-mode (:: and REM), HTML mode
(<!--) ...

I find this strange, because e.g. bat-mode's or html-mode's syntax
specifications (C-h s) don't say anything about comments. On the other
hand, C and the other modes have syntax specifications filled with
detailed comment syntax information. Hence, I'd expect it to be
vice-versa, that \s< wouldn't work in bat-mode but would in c-mode.

I tested with freshly compiled emacs-24.5, 25.3 and 26.3, launching
them with emacs -q -Q. They all seem to behave identically.

I'm wondering whether it's a known and accepted issue that comment
starter syntax class regexp search doesn't quite work in all modes?

Best regards,
Janne Nykopp

Re: Regexp search: syntax class for comment starter (\s<) does not work consistently in all modes

[Stefan Monnier]

> Seems that regexp search for comment starter syntax class, \s<,
> doesn't work out of the box in ...

It doesn't do what you think: it looks for characters with the < syntax.
These chars are recognized as starting a comment (tho it depends where
they appear, e.g. within a string), but they're not the only kinds of
chars that can start a comment.

If you want to look for comments, you're probably better off using
something like comment-search-forward (or comment-start-skip).


        Stefan

Re: Regexp search: syntax class for comment starter (\s<) does not work consistently in all modes

[Janne Nykopp]

On Fri, Sep 27, 2019 at 05:32:43PM -0400, Stefan Monnier wrote:
> > Seems that regexp search for comment starter syntax class, \s<,
> > doesn't work out of the box in ...
> 
> It doesn't do what you think: it looks for characters with the < syntax.
> These chars are recognized as starting a comment (tho it depends where
> they appear, e.g. within a string), but they're not the only kinds of
> chars that can start a comment.

I see. I was hoping it would automatically "expand" to a regexp
containing the possible comment start character sequences in the
particular mode the buffer is in, regardless of the context (inside a
string, or not - by the way, at least in lisp-mode, \s< finds ";"
inside a string as well).

I'm still pretty confused how bat-mode, for example, works as the
syntax table doesn't contain < syntax. It does define a comment-start
variable, however. I took a quick look at search.c and regex.c, but I
guess it takes a while of digging around the innards of emacs to get
some understanding. Sounds like an interesting challenge. But perhaps
later, as:

> If you want to look for comments, you're probably better off using
> something like comment-search-forward (or comment-start-skip).

Right, I'll take a look at these next, it sounds they're good matches
for my current use case.

Thank you!

Best regards,
Janne Nykopp 

Re: Regexp search: syntax class for comment starter (\s<) does not work consistently in all modes

[Stefan Monnier]

> by the way, at least in lisp-mode, \s< finds ";"
> inside a string as well).

Indeed, as I said, \s< only denotes those characters which the syntax
tables says "can" start a comment.  Whether they do depends on where
they are.

> I'm still pretty confused how bat-mode, for example, works as the
> syntax table doesn't contain < syntax.

There is no single char which starts a comment in bat-mode, since the
comment delimiter is composed of 3 characters ("rem") and only in some
places (e.g. "remember" doesn't count as a comment, even though it
starts with "rem").

Syntax-tables can handle 2-char comment delimiters (which covers
languages like C), and for more complex cases such as bat-mode's you
have to use `syntax-propertize-function` as does
`bat--syntax-propertize` to apply a `<` syntax to those `r` characters
which are part of the magic "rem" sequence.

You might like to read the syntax-tables chapter in the Elisp manual.


        Stefan