unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Combining syntax comment sequences (yaml-mode)
@ 2015-12-23 10:47 Vasilij Schneidermann
  2015-12-23 11:01 ` Andreas Schwab
  0 siblings, 1 reply; 5+ messages in thread
From: Vasilij Schneidermann @ 2015-12-23 10:47 UTC (permalink / raw)
  To: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

Hello,

I have reached out for maintainership of yaml-mode recently and am
currently fixing a number of bugs that have been reported on its issue
tracker.  One of these bugs is rather subtle as it demonstrates that its
syntax table isn't quite correct:

YAML does use the "#" character to indicate a comment, like many other
popular languages.  You'd typically encode this in a syntax table as
follows:

(modify-syntax-entry ?# "<" yaml-mode-syntax-table)
(modify-syntax-entry ?\n ">" yaml-mode-syntax-table)

While this covers most cases one would run into, it doesn't conform to
the YAML specification[1]:  A comment is either a token separated by
whitespace from other tokens or is on its own line.  This rule allows
you to use something like "foo#bar" as a token without "bar" getting
interpreted as a comment.  However with the code above "bar" will get
fontified as if it were a comment.

To highlight the former kind of comments correctly, the following works:

(modify-syntax-entry ?\s ". 1" yaml-mode-syntax-table)
(modify-syntax-entry ?# ". 2" yaml-mode-syntax-table)
(modify-syntax-entry ?\n ">" yaml-mode-syntax-table)

To do the same for the latter kind of comments:

(modify-syntax-entry ?\n "> 1" yaml-mode-syntax-table)
(modify-syntax-entry ?# ". 2" yaml-mode-syntax-table)

I cannot find a way to combine both code snippets though.  The obvious
approach yields incorrect results:

(modify-syntax-entry ?\s ". 1" yaml-mode-syntax-table)
(modify-syntax-entry ?# ". 2" yaml-mode-syntax-table)
(modify-syntax-entry ?\n "> 1" yaml-mode-syntax-table)

Is there some other way to combine multiple comment syntaxes?  I find
the existing documentation rather confusing and couldn't find any
tooling for debugging syntax table problems (other than staring hard at
them and having an incomplete mental model of how they work).

I've attached a test file for verifying the results.  yaml-mode itself
can be found on Github[2].

[1]: http://www.yaml.org/spec/1.2/spec.html#id2780069
[2]: https://github.com/yoshiki/yaml-mode

[-- Attachment #2: test.yaml --]
[-- Type: text/plain, Size: 33 bytes --]

foo: bar#baz
# qux quux
qux #bar

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Combining syntax comment sequences (yaml-mode)
  2015-12-23 10:47 Combining syntax comment sequences (yaml-mode) Vasilij Schneidermann
@ 2015-12-23 11:01 ` Andreas Schwab
  2015-12-23 18:28   ` Vasilij Schneidermann
  2015-12-23 21:56   ` Vasilij Schneidermann
  0 siblings, 2 replies; 5+ messages in thread
From: Andreas Schwab @ 2015-12-23 11:01 UTC (permalink / raw)
  To: Vasilij Schneidermann; +Cc: emacs-devel

Vasilij Schneidermann <v.schneidermann@gmail.com> writes:

> While this covers most cases one would run into, it doesn't conform to
> the YAML specification[1]:  A comment is either a token separated by
> whitespace from other tokens or is on its own line.  This rule allows
> you to use something like "foo#bar" as a token without "bar" getting
> interpreted as a comment.  However with the code above "bar" will get
> fontified as if it were a comment.

Very similar to bourne shell syntax, so look at sh-script for hints,
especially sh-syntax-propertize-function.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Combining syntax comment sequences (yaml-mode)
  2015-12-23 11:01 ` Andreas Schwab
@ 2015-12-23 18:28   ` Vasilij Schneidermann
  2015-12-23 18:46     ` Dmitry Gutov
  2015-12-23 21:56   ` Vasilij Schneidermann
  1 sibling, 1 reply; 5+ messages in thread
From: Vasilij Schneidermann @ 2015-12-23 18:28 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: emacs-devel

> Very similar to bourne shell syntax, so look at sh-script for hints,
> especially sh-syntax-propertize-function.

Thanks, I've assumed the only alternative was writing extra
fontification and movement functions.  If I understand correctly, this
is a helper function that applies extra syntax-table properties and can
therefore go beyond what a syntax table is able to do?



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Combining syntax comment sequences (yaml-mode)
  2015-12-23 18:28   ` Vasilij Schneidermann
@ 2015-12-23 18:46     ` Dmitry Gutov
  0 siblings, 0 replies; 5+ messages in thread
From: Dmitry Gutov @ 2015-12-23 18:46 UTC (permalink / raw)
  To: Vasilij Schneidermann, Andreas Schwab; +Cc: emacs-devel

On 12/23/2015 08:28 PM, Vasilij Schneidermann wrote:
> If I understand correctly, this
> is a helper function that applies extra syntax-table properties and can
> therefore go beyond what a syntax table is able to do?

That's right. So in that function, you can check whether each particular 
hash character is preceded by space or bol.



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Combining syntax comment sequences (yaml-mode)
  2015-12-23 11:01 ` Andreas Schwab
  2015-12-23 18:28   ` Vasilij Schneidermann
@ 2015-12-23 21:56   ` Vasilij Schneidermann
  1 sibling, 0 replies; 5+ messages in thread
From: Vasilij Schneidermann @ 2015-12-23 21:56 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: emacs-devel

With the given hint I was able to write up a helper function adding
`syntax-table` properties:

(defvar yaml-mode-syntax-propertize-function
  (syntax-propertize-rules
   ("^#" (0 "<"))
   ("[ \t]+#" (0 "<"))))

While this looks much neater than my previous approach, it doesn't
nearly function as well.  After loading up the file, not a single
comment is highlighted correctly.  The only thing that triggers
refontification is deleting(!) text close to the comment.  Inserting
text(!!) before or after the comment disables the highlighting again.

This is inacceptable behaviour and since I'm not in the mood for
figuring out how font-lock works, I'll wait and see whether someone's
got an idea how to combine the syntax entries presented in the original
post.



^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2015-12-23 21:56 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-23 10:47 Combining syntax comment sequences (yaml-mode) Vasilij Schneidermann
2015-12-23 11:01 ` Andreas Schwab
2015-12-23 18:28   ` Vasilij Schneidermann
2015-12-23 18:46     ` Dmitry Gutov
2015-12-23 21:56   ` Vasilij Schneidermann

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).