syntax-propertize-function for multiline strings

syntax-propertize-function for multiline strings

[Robin Neatherway]

Hi,

I'm trying to improve the syntax highlighting for fsharp-mode [1]. F#
allow strings to be specified over multiple lines [2]. I'm trying to
define verbatim strings, which are preceded with '@' and in which
backslash is should not be treated as an escape character (and "" is
interpreted as "). The syntax highlighting hasn't been updated to use
syntax-propertize-function, so I am trying to use that. I've defined a
rule with a regexp, but with a string that ends with \" the
highlighting breaks if I insert newlines inside the verbatim string. I
think this is because the parsing is being restarted at the newline
and so the string is treated normally, with the backslash escaping the
double quote. I don't know what the recommended technique is to handle
this though.

A minimal working example follows. I define a major mode "test-mode"
that highlights just "//.*" style comments and give an example file.
To reproduce open the example file and switch to "test-mode". Then
insert newlines after the first line. The comment highlighting will
turn to string highlighting. Then insert a space at the end of the
first line. The highlighting will again be correct.

Thanks for any help,
Robin


(setq test-syntax-table
      (let ((synTable (make-syntax-table)))
        (modify-syntax-entry ?\/ ". 12" synTable)
        (modify-syntax-entry ?\n "> " synTable)
        synTable))

(defconst test-syntax-propertize-function
  (syntax-propertize-rules
   ("@\\(\"\\)\\(:?\"\"\\|[^\"]\\)*\\(\"\\)"
    (1 "\"") (2 "\""))))

(define-derived-mode test-mode prog-mode "test"
  :syntax-table test-syntax-table

  (setq font-lock-mode t)
  (setq syntax-propertize-function test-syntax-propertize-function)
  (setq mode-name "test")
  )

Example file:
let mystring = @"hello
   \"
// This is a comment

[1] https://github.com/fsharp/fsharpbinding/tree/master/emacs
[2] http://msdn.microsoft.com/en-us/library/dd323829.aspx

Re: syntax-propertize-function for multiline strings

[Stefan Monnier]

> (defconst test-syntax-propertize-function
>   (syntax-propertize-rules
>    ("@\\(\"\\)\\(:?\"\"\\|[^\"]\\)*\\(\"\\)"
>     (1 "\"") (2 "\""))))

Usually, the best way to deal with it is along the following lines
(guaranteed 100% untested):

 (defun test-syntax-propertize-function (start end)
   (goto-char start)
   (test-syntax-verbatim-string end)
   (funcall (syntax-propertize-rules
             ("\\(@\\)\"" (1 (prog1 "|" (test-syntax-verbatim-string end)))))
            start end))

 (defun test-syntax-verbatim-string (end)
   (when (eq t (nth 3 (syntax-ppss)))
     (while
       (when (re-search-forward "\"\"?" end 'move)
         (if (> (- (match-end 0) (match-beginning 0)) 1)
             t ;; Skip this "" and keep looking further.
           (put-text-property (match-beginning 0) (match-end 0)
                              'syntax-table (string-to-syntax "|"))
           nil)))))

-- Stefan