Extending lua-mode syntax table (report from help-gnu-emacs)

Extending lua-mode syntax table (report from help-gnu-emacs)

[Niels Aan de Brugh]

Hi,

First of all I'm sorry if this is the second time you see this e-mail.
I posted it to the help-gnu-emacs list a week ago, but so far I've not
received a reply. Perhaps I was posting to the wrong list.

A week ago I decided to try and improve a little on the handling of
long strings in lua-mode (and possibly long comments after that).
Unfortunately I'm new to Emacs and I'm not sure whether I'm on the
right track. Needless to say I didn't succeed in my goals and I'm
seeking guru-help.

Some context: Long strings in Lua are started by [[ and closed by ]].
Between the brackets the programmer can type zero or more = signs to
make the string delimiter unique.

Reuben Thomas already made a regexp that works fine:

  "\\(?:^\\|[^[-]\\)\\(\\[\\(=*\\)\\[\\(?:.\\|\n\\)*?\\]\\2\\]\\)"

I can use that regexp to highlight the string (font-lock-string-face),
but I'd rather convince Emacs that this text element belongs to the
string class. Lua-mode uses parse-partial-sexp to detect strings. That
information is in turn used to determine indentation, etc.

The first of two questions. I'm trying to assign the right character
class like so:

(defun lua-mode ()
 ;; ....
   (set (make-local-variable 'font-lock-defaults)
                       '(lua-font-lock-keywords nil nil ((?_ . "w"))))


(defvar lua-font-lock-keywords
 (eval-when-compile
   (list
   ;; .....
    ;; Long strings.
    '("\\(?:^\\|[^[-]\\)\\(\\[\\(=*\\)\\[\\(?:.\\|\n\\)*?\\]\\2\\]\\)"
      (1 "\""))

I see no visual impact on the rendering of long strings (i.e. they
don't use the same highlighting as regular strings). Am I doing this
wrong? Is there a way to debug this kind of thing?

Second, the function that determines whether something is a string
looks like this (not my code):

(defun lua-syntax-status ()
 "Returns the syntactic status of the character after the point."
 (parse-partial-sexp (save-excursion (beginning-of-line) (point))
                     (point)))

(defun lua-string-p ()
 "Returns true if the point is in a string."
 (elt (lua-syntax-status) 3))

As you can see, the parse-partial-sexp function gets a range of text
from the start of the line until the point. The long strings can span
multiple lines (in fact, that's the whole point). So I'm worried that
the parse-partial-sexp doesn't have enough context to work with. Is
this true? If so, is there a work-around (other than parsing from the
start of the buffer), or do I need to implement something clever and
specific?

Any help would be greatly appreciated.

Regards,
Niels

Re: Extending lua-mode syntax table (report from help-gnu-emacs)

[Stefan Monnier]

> (defun lua-mode ()
>  ;; ....
>    (set (make-local-variable 'font-lock-defaults)
>                        '(lua-font-lock-keywords nil nil ((?_ . "w"))))


> (defvar lua-font-lock-keywords
>  (eval-when-compile
>    (list
>    ;; .....
>     ;; Long strings.
>     '("\\(?:^\\|[^[-]\\)\\(\\[\\(=*\\)\\[\\(?:.\\|\n\\)*?\\]\\2\\]\\)"
>       (1 "\""))

You want to use font-lock-syntactic-keywords.  E.g. (warning: 100%
guaranteed untested code ahead):

   (defun lua-mode ()
    ;; ....
    (set (make-local-variable 'font-lock-defaults)
         '(lua-font-lock-keywords nil nil ((?_ . "w")) nil
           (font-lock-syntactic-keywords . lua-font-lock-syntactic-keywords)))
  
  
   (defvar lua-font-lock-syntactic-keywords
    (eval-when-compile
      (list
      ;; .....
       ;; Long strings.
       '("\\(\\[\\)\\[\\|]\\(]\\)"
         (1 (if (null (nth 8 (syntax-ppss))) "\"") nil lax)
         (2 (if (nth 8 (syntax-ppss)) "\"") nil lax)))))

> (defun lua-syntax-status ()
>  "Returns the syntactic status of the character after the point."
>  (parse-partial-sexp (save-excursion (beginning-of-line) (point))
>                      (point)))

(save-excursion (beginning-of-line) (point)) can be replaced by
(line-beginning-position).

> (defun lua-string-p ()
>  "Returns true if the point is in a string."
>  (elt (lua-syntax-status) 3))

> As you can see, the parse-partial-sexp function gets a range of text
> from the start of the line until the point. The long strings can span
> multiple lines (in fact, that's the whole point). So I'm worried that
> the parse-partial-sexp doesn't have enough context to work with. Is
> this true? 

Yes.

> If so, is there a work-around (other than parsing from the start of
> the buffer), or do I need to implement something clever and specific?

You'll probably want to use (syntax-ppss) as a replacement for
(lua-syntax-status), which does exactly the same thing except that it
parses the whole buffer, but uses caching internally to try and make it
still fast enough.


        Stefan