Combining syntax comment sequences (yaml-mode)

[Vasilij Schneidermann <v.schneidermann@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

Hello,

I have reached out for maintainership of yaml-mode recently and am
currently fixing a number of bugs that have been reported on its issue
tracker.  One of these bugs is rather subtle as it demonstrates that its
syntax table isn't quite correct:

YAML does use the "#" character to indicate a comment, like many other
popular languages.  You'd typically encode this in a syntax table as
follows:

(modify-syntax-entry ?# "<" yaml-mode-syntax-table)
(modify-syntax-entry ?\n ">" yaml-mode-syntax-table)

While this covers most cases one would run into, it doesn't conform to
the YAML specification[1]:  A comment is either a token separated by
whitespace from other tokens or is on its own line.  This rule allows
you to use something like "foo#bar" as a token without "bar" getting
interpreted as a comment.  However with the code above "bar" will get
fontified as if it were a comment.

To highlight the former kind of comments correctly, the following works:

(modify-syntax-entry ?\s ". 1" yaml-mode-syntax-table)
(modify-syntax-entry ?# ". 2" yaml-mode-syntax-table)
(modify-syntax-entry ?\n ">" yaml-mode-syntax-table)

To do the same for the latter kind of comments:

(modify-syntax-entry ?\n "> 1" yaml-mode-syntax-table)
(modify-syntax-entry ?# ". 2" yaml-mode-syntax-table)

I cannot find a way to combine both code snippets though.  The obvious
approach yields incorrect results:

(modify-syntax-entry ?\s ". 1" yaml-mode-syntax-table)
(modify-syntax-entry ?# ". 2" yaml-mode-syntax-table)
(modify-syntax-entry ?\n "> 1" yaml-mode-syntax-table)

Is there some other way to combine multiple comment syntaxes?  I find
the existing documentation rather confusing and couldn't find any
tooling for debugging syntax table problems (other than staring hard at
them and having an incomplete mental model of how they work).

I've attached a test file for verifying the results.  yaml-mode itself
can be found on Github[2].

[1]: http://www.yaml.org/spec/1.2/spec.html#id2780069
[2]: https://github.com/yoshiki/yaml-mode

[-- Attachment #2: test.yaml --]
[-- Type: text/plain, Size: 33 bytes --]

foo: bar#baz
# qux quux
qux #bar