[ELPA] New package: lua-ts-mode

[ELPA] New package: lua-ts-mode

[john muhl]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

I’d like to submit ‘lua-ts-mode’ for inclusion. I’m new to all of this
so suggestions for improvements or anything else are very welcome.

There is already ‘lua-mode’ in NonGNU ELPA but this one is copyright FSF
and uses tree-sitter to provide it’s functionality so I hope the
duplication of effort is not a problem.

Thanks.

URL: https://git.sr.ht/~johnmuhl/lua-ts-mode

;;; Commentary:

This package provides ‘lua-ts-mode’ which is a major mode for
editing Lua files that uses Tree Sitter to parse the language.

This package is compatible with and tested against the grammar
for Lua found at https://github.com/MunifTanjim/tree-sitter-lua

With Git, a C compiler and linker in PATH you can install it by
running:

    M-x treesit-install-language-grammar RET lua RET y
    https://github.com/MunifTanjim/tree-sitter-lua RET RET RET RET RET

To automatically enable it when you open a Lua file add the following
to your ‘user-init-file’:

    (add-to-list 'auto-mode-alist '("\\.lua\\'" . lua-ts-mode))


[-- Attachment #2: lua-ts-mode.el --]
[-- Type: text/plain, Size: 11520 bytes --]

;;; lua-ts-mode.el --- tree sitter support for Lua  -*- lexical-binding: t; -*-

;; Copyright (C) 2023 Free Software Foundation, Inc.

;; Author: John Muhl <jm@pub.pink>
;; Maintainer: John Muhl <jm@pub.pink>
;; Created: June 27, 2023
;; Version: 1.0
;; Keywords: lua languages tree-sitter
;; URL: https://git.sr.ht/~johnmuhl/lua-ts-mode
;; Package-Requires: ((emacs "29.1"))

;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.
;;
;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; This package provides ‘lua-ts-mode’ which is a major mode for
;; editing Lua files that uses Tree Sitter to parse the language.
;;
;; This package is compatible with and tested against the grammar
;; for Lua found at https://github.com/MunifTanjim/tree-sitter-lua
;;
;; With Git, a C compiler and linker in PATH you can install it by
;; running:
;;
;;     M-x treesit-install-language-grammar RET lua RET y
;;     https://github.com/MunifTanjim/tree-sitter-lua RET RET RET RET RET
;;
;; To automatically enable it when you open a Lua file add the following
;; to your ‘user-init-file’:
;;
;;     (add-to-list 'auto-mode-alist '("\\.lua\\'" . lua-ts-mode))

;;; News

;; Version 1.0 <2023-07-11 Tue>
;; Supports font-lock, indentation, navigation, imenu,
;; which-function-mode and outline-minor-mode.

;;; Code:

(require 'treesit)
(eval-when-compile (require 'rx))

(declare-function treesit-node-child-by-field-name "treesit.c")
(declare-function treesit-node-type "treesit.c")
(declare-function treesit-parser-create "treesit.c")
(declare-function treesit-search-subtree "treesit.c")

(defcustom lua-ts-mode-indent-offset 4
  "Number of spaces for each indentation step in `lua-ts-mode'."
  :type 'integer
  :safe 'integerp
  :group 'lua)

(defvar lua-ts-mode--builtins
  '("assert" "collectgarbage" "coroutine" "debug" "dofile"
    "error" "getmetatable" "io" "ipairs" "load" "loadfile"
    "math" "next" "os" "package" "pairs" "pcall" "print"
    "rawequal" "rawget" "rawlen" "rawset" "require" "select"
    "setmetatable" "string" "table" "tonumber" "tostring"
    "type" "utf8" "warn" "xpcall")
  "Lua built-in functions for tree-sitter font-locking.")

(defvar lua-ts-mode--syntax-table
  (let ((table (make-syntax-table)))
    (modify-syntax-entry ?+  "."    table)
    (modify-syntax-entry ?-  ". 12" table)
    (modify-syntax-entry ?=  "."    table)
    (modify-syntax-entry ?%  "."    table)
    (modify-syntax-entry ?^  "."    table)
    (modify-syntax-entry ?~  "."    table)
    (modify-syntax-entry ?<  "."    table)
    (modify-syntax-entry ?>  "."    table)
    (modify-syntax-entry ?/  "."    table)
    (modify-syntax-entry ?*  "."    table)
    (modify-syntax-entry ?\n ">"    table)
    (modify-syntax-entry ?\' "\""   table)
    (modify-syntax-entry ?\" "\""   table)
    table)
  "Syntax table for `lua-ts-mode'.")

(defvar lua-ts-mode--font-lock-settings
  (treesit-font-lock-rules
   :language 'lua
   :feature 'bracket
   '(["(" ")" "[" "]" "{" "}"] @font-lock-bracket-face)

   :language 'lua
   :feature 'builtin
   `(((identifier) @font-lock-builtin-face
      (:match ,(rx-to-string
                `(seq bol (or ,@lua-ts-mode--builtins) eol))
              @font-lock-builtin-face)))

   :language 'lua
   :feature 'delimiter
   '(["," ";"] @font-lock-delimiter-face)

   :language 'lua
   :feature 'escape
   '((escape_sequence) @font-lock-escape-face)

   :language 'lua
   :feature 'function
   '((function_call name: (identifier) @font-lock-function-call-face)
     (function_call
      name: (method_index_expression
             method: (identifier) @font-lock-function-call-face))
     (function_call
	  name: (dot_index_expression
             table: (identifier) @font-lock-function-call-face)))

   :language 'lua
   :feature 'constant
   '((variable_list
      attribute: (attribute (["<" ">"] (identifier))))
     @font-lock-constant-face)

   :language 'lua
   :feature 'operator
   '(["and" "not" "or" "+" "-" "*" "/" "%" "^"
      "#" "==" "~=" "<=" ">=" "<" ">" "=" "&"
      "~" "|" "<<" ">>" "//" ".."]
     @font-lock-operator-face
     (vararg_expression) @font-lock-operator-face)

   :language 'lua
   :feature 'property
   '((field name: (identifier) @font-lock-property-name-face)
     (dot_index_expression
      field: (identifier) @font-lock-property-use-face))

   :language 'lua
   :feature 'punctuation
   '(["." ":"] @font-lock-punctuation-face)

   :language 'lua
   :feature 'variable
   '((function_call
      arguments: (arguments (identifier))
      @font-lock-variable-use-face)
     (function_call
      name: (method_index_expression
             table: (identifier) @font-lock-variable-use-face))
     (goto_statement (identifier) @font-lock-variable-use-face))

   :language 'lua
   :feature 'assignment
   '((variable_list (identifier) @font-lock-variable-name-face))

   :language 'lua
   :feature 'number
   '((number) @font-lock-number-face)

   :language 'lua
   :feature 'keyword
   '((break_statement) @font-lock-keyword-face
     (true) @font-lock-constant-face
     (false) @font-lock-constant-face
     (nil) @font-lock-constant-face
     ["and" "do" "else" "elseif" "end" "for" "function"
      "goto" "if" "in" "local" "not" "or" "repeat"
      "return" "then" "until" "while"]
     @font-lock-keyword-face)

   :language 'lua
   :feature 'string
   '((string) @font-lock-string-face)

   :language 'lua
   :feature 'comment
   '((comment) @font-lock-comment-face
     (hash_bang_line) @font-lock-comment-face)

   :language 'lua
   :feature 'definition
   '((function_declaration
      name: (identifier) @font-lock-function-name-face)
     (parameters
      name: (identifier) @font-lock-variable-name-face)
     (label_statement) @font-lock-variable-name-face)

   :language 'lua
   :feature 'error
   :override t
   '((ERROR) @font-lock-warning-face))
  "Tree-sitter font-lock settings for `lua-ts-mode'.")

(defvar lua-ts-mode--indent-rules
  `((lua
     ((parent-is "chunk") column-0 0)
     ((node-is "comment_end") column-0 0)
     ((parent-is "block") parent-bol 0)
     ((node-is "}") parent-bol 0)
     ((node-is ")") parent-bol 0)
     ((node-is "else_statement") parent-bol 0)
     ((node-is "elseif_statement") parent-bol 0)
     ((node-is "end") parent-bol 0)
     ((node-is "until") parent-bol 0)
     ((parent-is "for_statement") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "function_declaration") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "function_definition") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "if_statement") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "else_statement") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "repeat_statement") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "while_statement") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "table_constructor") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "arguments") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "parameters") parent-bol lua-ts-mode-indent-offset)
     ((parent-is "ERROR") no-indent 0))))

(defun lua-ts-mode--defun-name (node)
  "Return the defun name of NODE.
Return nil if there is no name or if NODE is not a defun node."
  (pcase (treesit-node-type node)
    ((or "function_declaration" "function_definition")
     (treesit-node-text
      (treesit-node-child-by-field-name node "name") t))
    ("variable_declaration"
     (let ((child (treesit-node-child-by-field-name node "name")))
       (if child (treesit-node-text child t)
         (treesit-node-text
          (treesit-node-child-by-field-name
           (treesit-search-subtree node "assignment_statement" nil nil 1)
           "name")
          t))))
    ("field"
     (and (treesit-search-subtree node "function_definition" nil nil 1)
          (treesit-node-text
           (treesit-node-child-by-field-name node "name") t)))))

;;;###autoload
(define-derived-mode lua-ts-mode prog-mode "Lua"
  "Major mode for editing Lua, powered by tree-sitter."
  :group 'lua
  :syntax-table lua-ts-mode--syntax-table

  (when (treesit-ready-p 'lua)
    (treesit-parser-create 'lua)

    (setq-local treesit-defun-prefer-top-level t)

    ;; Comments.
    (setq-local comment-start "--")
    (setq-local comment-start-skip (rx "--" (* (syntax whitespace))))
    (setq-local comment-end "")

    ;; Font-lock.
    (setq-local treesit-font-lock-settings lua-ts-mode--font-lock-settings)
    (setq-local treesit-font-lock-feature-list
                '((comment definition)
                  (builtin keyword string)
                  (assignment constant number)
                  (bracket
                   delimiter
                   escape
                   function
                   operator
                   property
                   punctuation
                   variable)))

    ;; Indent.
    (setq-local treesit-simple-indent-rules lua-ts-mode--indent-rules)

    ;; Navigation.
    (setq-local treesit-defun-name-function #'lua-ts-mode--defun-name)
    (setq-local treesit-defun-type-regexp
                (regexp-opt '("function_declaration"
                              "function_definition")))
    (setq-local treesit-sentence-type-regexp
                (regexp-opt '("do_statement"
                              "while_statement"
                              "repeat_statement"
                              "if_statement"
                              "for_statement"
                              "variable_declaration")))
    (setq-local treesit-sexp-type-regexp
                (regexp-opt '("arguments"
                              "comment"
                              "string"
                              "table_constructor")))

    ;; Imenu.
    (setq-local treesit-simple-imenu-settings
                `(("Variable" "\\`variable_declaration\\'" nil nil)
                  ("Function" ,(rx bos (or "function_declaration"
                                           "function_definition"
                                           "field")
                                   eos)
                   nil nil)))

    ;; Which-function.
    (setq-local which-func-functions (treesit-defun-at-point))

    ;; Outline.
    (setq-local outline-regexp
                (regexp-opt '("do" "for" "function" "local"
                              "if" "repeat" "while" "--[[")))

    (treesit-major-mode-setup)))

(if (treesit-ready-p 'lua)
    (add-to-list 'auto-mode-alist '("\\.lua\\'" . lua-ts-mode)))

(provide 'lua-ts-mode)

;;; lua-ts-mode.el ends here

;; Local Variables:
;; indent-tabs-mode: nil
;; sentence-end-double-space: t
;; End:

Re: [ELPA] New package: lua-ts-mode

[Theodor Thornhill]

On 26 July 2023 18:49:20 CEST, john muhl <jm@pub.pink> wrote:
>I’d like to submit ‘lua-ts-mode’ for inclusion. I’m new to all of this
>so suggestions for improvements or anything else are very welcome.
>

Nice, thanks!

I'll take a look at the code a bit later, but is anything preventing us from adding this in-tree?

Theo

>There is already ‘lua-mode’ in NonGNU ELPA but this one is copyright FSF
>and uses tree-sitter to provide it’s functionality so I hope the
>duplication of effort is not a problem.
>
>Thanks.
>
>URL: https://git.sr.ht/~johnmuhl/lua-ts-mode
>
>;;; Commentary:
>
>This package provides ‘lua-ts-mode’ which is a major mode for
>editing Lua files that uses Tree Sitter to parse the language.
>
>This package is compatible with and tested against the grammar
>for Lua found at https://github.com/MunifTanjim/tree-sitter-lua
>
>With Git, a C compiler and linker in PATH you can install it by
>running:
>
>    M-x treesit-install-language-grammar RET lua RET y
>    https://github.com/MunifTanjim/tree-sitter-lua RET RET RET RET RET
>
>To automatically enable it when you open a Lua file add the following
>to your ‘user-init-file’:
>
>    (add-to-list 'auto-mode-alist '("\\.lua\\'" . lua-ts-mode))
>

Re: [ELPA] New package: lua-ts-mode

[Eli Zaretskii]

> Date: Wed, 26 Jul 2023 19:42:16 +0200
> From: Theodor Thornhill <theo@thornhill.no>
> 
> On 26 July 2023 18:49:20 CEST, john muhl <jm@pub.pink> wrote:
> >I’d like to submit ‘lua-ts-mode’ for inclusion. I’m new to all of this
> >so suggestions for improvements or anything else are very welcome.
> >
> 
> Nice, thanks!
> 
> I'll take a look at the code a bit later, but is anything preventing us from adding this in-tree?

My thoughts exactly.

And another question: does this mode work with the other Lua grammar
library I know about, https://github.com/Azganoth/tree-sitter-lua?  If
not, would it be possible to make it work with both?

Re: [ELPA] New package: lua-ts-mode

[john muhl]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 26 Jul 2023 19:42:16 +0200
>> From: Theodor Thornhill <theo@thornhill.no>
>>
>> I'll take a look at the code a bit later, but is anything preventing
>> us from adding this in-tree?
>
> My thoughts exactly.

Sounds good. I’ll prepare a patch for Emacs.

> And another question: does this mode work with the other Lua grammar
> library I know about, https://github.com/Azganoth/tree-sitter-lua?

It works in the sense of nothing going horribly wrong if you try to use
it. You can still edit Lua files but due to differences in the trees and
node names the experience is somewhat degraded.

> If not, would it be possible to make it work with both?

Without having tried more than just swapping out the tree-sitter-lua.so
files it looks like it would be possible without much hassle.

I don’t think it’s worth any hassle though. The one I used is the same
as used by Neovim[1] and is already packaged for Alpine[2] and NixOS[3]
and is in the queue for Debian[4]. I’ll see about getting it added to
GNU Guix (which already has a number of other grammars).

Is working with both a prerequisite for inclusion or is there some other
reason to support them both? I’m not opposed but it doesn’t seem like an
important thing to do at this time.

1: https://github.com/nvim-treesitter/nvim-treesitter/blob/6aababfca4e3d72cb1e8fbbdac4115a5f88107d0/lua/nvim-treesitter/parsers.lua#L900
2: https://pkgs.alpinelinux.org/package/edge/community/x86/tree-sitter-lua
3: https://github.com/NixOS/nixpkgs/blob/nixos-23.05/pkgs/development/tools/parsing/tree-sitter/grammars/tree-sitter-lua.json
4: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1034075

Re: [ELPA] New package: lua-ts-mode

[Eli Zaretskii]

> From: john muhl <jm@pub.pink>
> Cc: Theodor Thornhill <theo@thornhill.no>, emacs-devel@gnu.org
> Date: Thu, 27 Jul 2023 08:34:45 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > And another question: does this mode work with the other Lua grammar
> > library I know about, https://github.com/Azganoth/tree-sitter-lua?
> 
> It works in the sense of nothing going horribly wrong if you try to use
> it. You can still edit Lua files but due to differences in the trees and
> node names the experience is somewhat degraded.
> 
> > If not, would it be possible to make it work with both?
> 
> Without having tried more than just swapping out the tree-sitter-lua.so
> files it looks like it would be possible without much hassle.
> 
> I don’t think it’s worth any hassle though. The one I used is the same
> as used by Neovim[1] and is already packaged for Alpine[2] and NixOS[3]
> and is in the queue for Debian[4]. I’ll see about getting it added to
> GNU Guix (which already has a number of other grammars).
> 
> Is working with both a prerequisite for inclusion or is there some other
> reason to support them both?

It isn't a prerequisite, no.  It just would be nice to be able to
support both.

Re: [ELPA] New package: lua-ts-mode

[john muhl]

Eli Zaretskii <eliz@gnu.org> writes:

> It isn't a prerequisite, no.  It just would be nice to be able to
> support both.

Fair enough. I gave it a closer look and don’t think it’s possible to
support even on its own. Try opening a file containing:

  print(1)

Enable treesit-explore-mode and see:

  (call
   function: (variable name: (identifier))
   arguments: (ERROR ( (number) ))      ; That ERROR node is erroneous
   (argument_list (string)))

Those stray ERRORs are pervasive and all kinds of code end up with them
in unexpected places. There may be other issues but this one is enough
to make it unusable. I don’t think this is fixable on the Emacs side.

Another problem is that in an empty buffer Emacs freezes when you start
typing. The Azganoth grammar and t-e-m must be enabled for it to freeze.
This might be fixable in Emacs but wouldn’t change the problem with
misplaced ERROR nodes.

If someone fixes the problems with the grammar or shows where I’ve gone
wrong I’m happy to revisit.

Re: [ELPA] New package: lua-ts-mode

[Eli Zaretskii]

> From: john muhl <jm@pub.pink>
> Cc: theo@thornhill.no, emacs-devel@gnu.org
> Date: Thu, 27 Jul 2023 16:52:47 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It isn't a prerequisite, no.  It just would be nice to be able to
> > support both.
> 
> Fair enough. I gave it a closer look and don’t think it’s possible to
> support even on its own. Try opening a file containing:
> 
>   print(1)
> 
> Enable treesit-explore-mode and see:
> 
>   (call
>    function: (variable name: (identifier))
>    arguments: (ERROR ( (number) ))      ; That ERROR node is erroneous
>    (argument_list (string)))
> 
> Those stray ERRORs are pervasive and all kinds of code end up with them
> in unexpected places. There may be other issues but this one is enough
> to make it unusable. I don’t think this is fixable on the Emacs side.
> 
> Another problem is that in an empty buffer Emacs freezes when you start
> typing. The Azganoth grammar and t-e-m must be enabled for it to freeze.
> This might be fixable in Emacs but wouldn’t change the problem with
> misplaced ERROR nodes.

Thanks, if that grammar is buggy to the degree that we cannot use it
in Emacs, then we don't need to support it, of course.