How does one set up a syntax table for (* and //?

How does one set up a syntax table for (* and //?

[Clément Pit--Claudel]

Hi Emacs devel,

As far as I can tell from the docs, the following syntax table should recognize // … line comments and nestable (* … *) comments properly:

    (defvar example-syntax-table
      (let ((table (make-syntax-table)))
        (modify-syntax-entry ?*  ". 23" table)
        (modify-syntax-entry ?/  ". 12c" table)
        (modify-syntax-entry ?\n  "> c" table)
        (modify-syntax-entry ?\( "()1n" table)
        (modify-syntax-entry ?\) ")(4n" table)
        table))

Yet it does not:

    (with-current-buffer (get-buffer-create "test")
      (erase-buffer)
      (prog-mode)
      (set-syntax-table example-syntax-table)
      (insert "// line comment\n")
      (insert "regular text\n")
      (insert "(* nested (* comments *) *)\n")
      (insert "regular text\n")
      (insert "(// another line comment,\nwhich incorrectly extends past the newline?\n")
      (pop-to-buffer (current-buffer)))

In this example, the last line (“which incorrectly extends past the newline?”), is considered to be a comment, because the sequence "(/" is treated as a multiline comment opener.

How can I work around this? Did I miss something in the manual?

Thanks!
Clément.

Re: How does one set up a syntax table for (* and //?

[Noam Postavsky]

On Wed, Jan 11, 2017 at 10:44 AM, Clément Pit--Claudel
<clement.pit@gmail.com> wrote:
>
> As far as I can tell from the docs, the following syntax table should recognize // … line comments and nestable (* … *) comments properly:
>
>     (defvar example-syntax-table
>       (let ((table (make-syntax-table)))
>         (modify-syntax-entry ?*  ". 23" table)
>         (modify-syntax-entry ?/  ". 12c" table)
>         (modify-syntax-entry ?\n  "> c" table)
>         (modify-syntax-entry ?\( "()1n" table)
>         (modify-syntax-entry ?\) ")(4n" table)
>         table))
[...]
> In this example, the last line (“which incorrectly extends past the newline?”), is considered to be a comment, because the sequence "(/" is treated as a multiline comment opener.

You don't want "a" style comment chars combining with "c" style
comment chars. Unfortunately, Emacs doesn't implement that. I think
you'll have to implement these kind of comments with a
syntax-propertize-function.

Re: How does one set up a syntax table for (* and //?

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1510 bytes --]

On 2017-01-11 20:19, Noam Postavsky wrote:
> On Wed, Jan 11, 2017 at 10:44 AM, Clément Pit--Claudel
> <clement.pit@gmail.com> wrote:
>>
>> As far as I can tell from the docs, the following syntax table should recognize // … line comments and nestable (* … *) comments properly:
>>
>>     (defvar example-syntax-table
>>       (let ((table (make-syntax-table)))
>>         (modify-syntax-entry ?*  ". 23" table)
>>         (modify-syntax-entry ?/  ". 12c" table)
>>         (modify-syntax-entry ?\n  "> c" table)
>>         (modify-syntax-entry ?\( "()1n" table)
>>         (modify-syntax-entry ?\) ")(4n" table)
>>         table))
> [...]
>> In this example, the last line (“which incorrectly extends past the newline?”), is considered to be a comment, because the sequence "(/" is treated as a multiline comment opener.
> 
> You don't want "a" style comment chars combining with "c" style
> comment chars. Unfortunately, Emacs doesn't implement that. I think
> you'll have to implement these kind of comments with a
> syntax-propertize-function.

Yeah, that's what I was afraid of.

Is there code that depends on this (a mixing with c)? If not, could we consider changing the semantics of "c" instead? This mixing isn't clear from the documentation, AFAICT.

I guess such a change would break the hypothetical situation in which one would need to be able to say that a character is an opener for both "a" and "c", but a closer for only "c", though.

Cheers,
Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: How does one set up a syntax table for (* and //?

[Stefan Monnier]

> Is there code that depends on this (a mixing with c)? If not, could we
> consider changing the semantics of "c" instead? This mixing isn't clear from
> the documentation, AFAICT.

`c` is a fairly recent introduction (introduced for opascal.el, FWIW),
so maybe it's not too late to change its semantics.


        Stefan

Re: How does one set up a syntax table for (* and //?

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 473 bytes --]

On 2017-01-12 08:55, Stefan Monnier wrote:
>> Is there code that depends on this (a mixing with c)? If not, could we
>> consider changing the semantics of "c" instead? This mixing isn't clear from
>> the documentation, AFAICT.
> 
> `c` is a fairly recent introduction (introduced for opascal.el, FWIW),
> so maybe it's not too late to change its semantics.

That would be nice :) In the meantime, I implemented Noam's suggestion in the attached patch.

Clément.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Use-a-syntax-propertize-function-to-properly-handle-.patch --]
[-- Type: text/x-diff; name="0001-Use-a-syntax-propertize-function-to-properly-handle-.patch", Size: 2733 bytes --]

From 3a9be64827bbed8e34d38803b5c44d8d4f6cd688 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Thu, 12 Jan 2017 20:26:04 -0500
Subject: [PATCH] Use a syntax-propertize-function to properly handle '(//'
 comments

Fixes #42.
---
 fstar-mode.el | 30 +++++++++++++++++++++++-------
 1 file changed, 23 insertions(+), 7 deletions(-)

diff --git a/fstar-mode.el b/fstar-mode.el
index b9d1899..8582f9e 100755
--- a/fstar-mode.el
+++ b/fstar-mode.el
@@ -423,15 +423,30 @@ If MUST-FIND-TYPE is nil, the :type part is not necessary."
     ;; Comments and strings
     (modify-syntax-entry ?\\ "\\" table)
     (modify-syntax-entry ?\" "\"" table)
-    (modify-syntax-entry ?*  ". 23" table)
-    (modify-syntax-entry ?/  ". 12b" table)
-    (modify-syntax-entry ?\n  "> b" table)
-    (modify-syntax-entry ?\^m "> b" table)
-    (modify-syntax-entry ?\( "()1n" table)
-    (modify-syntax-entry ?\) ")(4n" table)
+    ;; ‘/’ is handled by a `syntax-propertize-function'.  For background on this
+    ;; see http://lists.gnu.org/archive/html/emacs-devel/2017-01/msg00144.html.
+    ;; The comment enders are left here, since they don't match the ‘(*’ openers.
+    ;; (modify-syntax-entry ?/ ". 12c" table)
+    (modify-syntax-entry ?\n  ">" table)
+    (modify-syntax-entry ?\^m ">" table)
+    (modify-syntax-entry ?\( "()1nb" table)
+    (modify-syntax-entry ?*  ". 23b" table)
+    (modify-syntax-entry ?\) ")(4nb" table)
     table)
   "Syntax table for F*.")
 
+(defconst fstar-mode-syntax-propertize-function
+  (let ((opener-1 (string-to-syntax ". 1"))
+        (opener-2 (string-to-syntax ". 2")))
+    (syntax-propertize-rules
+     ("//" (0 (let* ((pt (match-beginning 0))
+                     (state (syntax-ppss pt)))
+                (goto-char (match-end 0)) ;; syntax-ppss adjusts point
+                (unless (or (nth 3 state) (nth 4 state))
+                  (put-text-property pt (+ pt 1) 'syntax-table opener-1)
+                  (put-text-property (+ pt 1) (+ pt 2) 'syntax-table opener-2)
+                  (ignore (goto-char (point-at-eol))))))))))
+
 ;;; Mode map
 
 (defvar fstar-mode-map
@@ -1228,7 +1243,8 @@ into blocks; process it as one large block instead."
   (setq-local comment-continue   " *")
   (setq-local comment-end        "*)")
   (setq-local comment-start-skip "\\(//+\\|(\\*+\\)[ \t]*")
-  (setq-local font-lock-syntactic-face-function #'fstar-syntactic-face-function))
+  (setq-local font-lock-syntactic-face-function #'fstar-syntactic-face-function)
+  (setq-local syntax-propertize-function fstar-mode-syntax-propertize-function))
 
 ;;; Main mode
 
-- 
2.7.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]