ruby-ts-mode.el -- first draft

ruby-ts-mode.el -- first draft

[Perry Smith]

[-- Attachment #1.1: Type: text/plain, Size: 17505 bytes --]

Ruby is a versatile language and I fear that I may have missed wide swaths of its features.  So, here is the first pass.  I hope folks can play with it and find the bugs.

Tree sitter is so versatile that for fortification, it is practically endless the features you could add.

I have a git repository here: https://github.com/pedz/ruby-ts-mode

And here inline is the file:

;;; This is currently a work in progress.  My intent is to release it
;;; with whatever copyright notice Free Software Foundation,
;;; Inc. wants.

;; Author     : Perry Smith <pedz@easesoftware.com>
;; Created    : December 2022
;; Keywords   : ruby languages tree-sitter

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Code:

(require 'treesit)

(declare-function treesit-parser-create "treesit.c")

(defcustom ruby-ts-mode-indent-offset 2
  "Number of spaces for each indentation step in `ruby-ts-mode'."
  :version "29.1"
  :type 'integer
  :safe 'integerp
  :group 'ruby)

(defcustom ruby-ts-indent-tabs-mode nil
  "Indentation can insert tabs in Ruby TS mode if this is non-nil."
  :version "29.1"
  :type 'boolean
  :safe 'booleanp
  :group 'ruby)

(defcustom ruby-ts-mode-indent-style 'base
  "Style used for indentation.

The selected style could be one of Ruby.  If one of the supplied
styles doesn't suffice a function could be set instead.  This
function is expected return a list that follows the form of
`treesit-simple-indent-rules'."
  :version "29.1"
  :type '(choice (symbol :tag "Base" 'base)
                 (function :tag "A function for user customized style" ignore))
  :group 'ruby)

(defcustom ruby-ts-mode-indent-style 'gnu
  "Style used for indentation.

Currently can only be set to BASE.  If one of the supplied styles
doesn't suffice a function could be set instead.  This function
is expected return a list that follows the form of
`treesit-simple-indent-rules'."
  :version "29.1"
  :type '(choice (symbol :tag "Base" 'base)
                 (function :tag "A function for user customized style" ignore))
  :group 'ruby)

(defface ruby-ts-mode--constant-assignment-face
  '((((class grayscale) (background light)) :foreground "DimGray" :slant italic)
    (((class grayscale) (background dark))  :foreground "LightGray" :slant italic)
    (((class color) (min-colors 88) (background light)) :foreground "VioletRed4")
    (((class color) (min-colors 88) (background dark))  :foreground "plum2")
    (((class color) (min-colors 16) (background light)) :foreground "RosyBrown")
    (((class color) (min-colors 16) (background dark))  :foreground "LightSalmon")
    (((class color) (min-colors 8)) :foreground "green")
    (t :slant italic))
  "Font Lock mode face used in ruby-ts-mode to highlight assignments to constants."
  :group 'font-lock-faces)

(defface ruby-ts-mode--assignment-face
  '((((class grayscale) (background light)) :foreground "DimGray" :slant italic)
    (((class grayscale) (background dark))  :foreground "LightGray" :slant italic)
    (((class color) (min-colors 88) (background light)) :foreground "VioletRed4")
    (((class color) (min-colors 88) (background dark))  :foreground "coral1")
    (((class color) (min-colors 16) (background light)) :foreground "RosyBrown")
    (((class color) (min-colors 16) (background dark))  :foreground "LightSalmon")
    (((class color) (min-colors 8)) :foreground "green")
    (t :slant italic))
  "Font Lock mode face used in ruby-ts-mode to hightlight assignments."
  :group 'font-lock-faces)

(defvar ruby-ts-mode--syntax-table
  (let ((table (make-syntax-table)))
    ;; Mostly stolen from ruby-mode but enh-ruby-mode also added ??
    (modify-syntax-entry ?#  "<"  table)
    (modify-syntax-entry ?$  "'"  table)
    (modify-syntax-entry ?%  "."  table)
    (modify-syntax-entry ?&  "."  table)
    (modify-syntax-entry ?*  "."  table)
    (modify-syntax-entry ?+  "."  table)
    (modify-syntax-entry ?-  "."  table)
    (modify-syntax-entry ?/  "."  table)
    (modify-syntax-entry ?:  "'"  table)
    (modify-syntax-entry ?<  "."  table)
    (modify-syntax-entry ?=  "."  table)
    (modify-syntax-entry ?>  "."  table)
    (modify-syntax-entry ??  "_"  table)
    (modify-syntax-entry ?@  "'"  table)
    (modify-syntax-entry ?\" "\"" table)
    (modify-syntax-entry ?\' "\"" table)
    (modify-syntax-entry ?\( "()" table)
    (modify-syntax-entry ?\) ")(" table)
    (modify-syntax-entry ?\; "."  table)
    (modify-syntax-entry ?\[ "(]" table)
    (modify-syntax-entry ?\\ "\\" table)
    (modify-syntax-entry ?\] ")[" table)
    (modify-syntax-entry ?\` "\"" table)
    (modify-syntax-entry ?\n ">"  table)
    (modify-syntax-entry ?\{ "(}" table)
    (modify-syntax-entry ?\} "){" table)
    (modify-syntax-entry ?_  "_"  table)
    (modify-syntax-entry ?|  "."  table)
    table)
  "Syntax table used by ‘ruby-ts-mode’ buffers.")

(defvar ruby-ts-mode--operators-arithmetic
  '("+" "-" "*" "/" "%" "**")
  "Ruby arithmetic operators for tree-sitter font-locking.")

;; treesit-query-validate doesn't like these:
;; "eql?" "equal?"
(defvar ruby-ts-mode--operators-comparison
  '("==" "!=" ">" "<" ">=" "<=" "<=>" "===")
  "Ruby comparison operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-assignment
  '("=" "+=" "-=" "*=" "/=" "%=" "**=")
  "Ruby assignment operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-bitwise
  '("&" "|" "^" "~" "<<" ">>")
  "Ruby bitwise operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-logical
  '("!" "&&" "and" "not" "or" "||")
  "Ruby logical operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-ternary
  '("?" ":")
  "Ruby ternary operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-range
  '(".." "...")
  "Ruby range operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-defined
  '("defined?")
  "Ruby defined? operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators-dot-colon
  '("." "::")
  "Ruby dot and double colon operators for tree-sitter font-locking.")

(defvar ruby-ts-mode--operators
  (append ruby-ts-mode--operators-arithmetic
          ruby-ts-mode--operators-comparison
          ruby-ts-mode--operators-assignment
          ruby-ts-mode--operators-bitwise
          ruby-ts-mode--operators-logical
          ruby-ts-mode--operators-ternary
          ruby-ts-mode--operators-range
          ruby-ts-mode--operators-defined
          ruby-ts-mode--operators-dot-colon)
  "Ruby operators for tree-sitter font-locking.")

;; doc/keywords.rdoc in the Ruby git repository considers these to be
;; reserved keywords.  If these keywords are added to the list, it
;; causes the font-lock to stop working.
;;
;; "__ENCODING__" "__FILE__" "__LINE__" "false" "self" "super" "true"
;;
;; "nil" (which does not exhibit this issue) is also considered a
;; keyword but I removed it and added it as a constant.
;;
(defun ruby-ts-mode--keywords (language)
  "Ruby keywords for tree-sitter font-locking.
Currently LANGUAGE is ignored but shoule be set to `ruby'."
  (let ((common-keywords
         '("BEGIN" "END" "alias" "and" "begin" "break" "case" "class"
           "def" "defined?" "do" "else" "elsif" "end" "ensure" "for"
           "if" "in" "module" "next" "not" "or" "redo" "rescue"
           "retry" "return" "then" "undef" "unless" "until" "when"
           "while" "yield")))
    common-keywords))

;; Ideas of what could be added:
;;   1. The regular expressions start, end, and content could be font
;;      locked.  Ditto for the command strings `foo`.  The symbols
;;      inside a %s, %i, and %I could be given the "symbol" font.
;;      etc.
(defun ruby-ts-mode--font-lock-settings (language)
  "Tree-sitter font-lock settings.
Currently LANGUAGE is ignored but should be set to `ruby'."
  (treesit-font-lock-rules
   :language language
   :feature 'comment
   `((comment) @font-lock-comment-face
     (comment) @contextual)

   :language language
   :feature 'keyword
   `([,@(ruby-ts-mode--keywords language)] @font-lock-keyword-face)

   :language language
   :feature 'constant
   `((true) @font-lock-constant-face
     (false) @font-lock-constant-face
     (nil) @font-lock-constant-face
     (self) @font-lock-constant-face
     (super)  @font-lock-constant-face)

   ;; Before 'operator so (unary) works.  (I didn't want to try
   ;; :override)
   :language language
   :feature 'literal
   `((unary ["+" "-"] [(integer) (rational) (float) (complex)]) @font-lock-number-face
     (simple_symbol) @font-lock-number-face
     (delimited_symbol) @font-lock-number-face
     (integer) @font-lock-number-face
     (float) @font-lock-number-face
     (complex) @font-lock-number-face
     (rational) @font-lock-number-face)

   :language language
   :feature 'operator
   `("!" @font-lock-negation-char-face
     [,@ruby-ts-mode--operators] @font-lock-operator-face)

   :language language
   :feature 'string
   `((string) @font-lock-string-face
     (string_content) @font-lock-string-face)

   :language language
   :feature 'type
   `((constant) @font-lock-type-face)

   :language language
   :feature 'assignment
   '((assignment
      left: (identifier) @ruby-ts-mode--assignment-face)
     (assignment
      left: (left_assignment_list (identifier) @ruby-ts-mode--assignment-face))
     (operator_assignment
      left: (identifier) @ruby-ts-mode--assignment-face))

   ;; Constant and scoped constant assignment (declaration)
   ;; Must be enabled explicitly
   :language language
   :feature 'constant-assignment
   :override t
   `((assignment
      left: (constant) @ruby-ts-mode--constant-assignment-face)
     (assignment
      left: (scope_resolution name: (constant) @ruby-ts-mode--constant-assignment-face)))

   :language language
   :feature 'function
   '((call
      method: (identifier) @font-lock-function-name-face)
     (method
      name: (identifier) @font-lock-function-name-face))

   :language language
   :feature 'variable
   '((identifier) @font-lock-variable-name-face)

   :language language
   :feature 'error
   '((ERROR) @font-lock-warning-face)

   :feature 'escape-sequence
   :language language
   :override t
   '((escape_sequence) @font-lock-escape-face)

   :language language
   :feature 'bracket
   '((["(" ")" "[" "]" "{" "}"]) @font-lock-bracket-face)
   )
  )

(defun ruby-ts-mode--indent-styles (language)
  "Indent rules supported by `ruby-ts-mode'.
Currently LANGUAGE is ignored but should be set to `ruby'"
  (let ((common
         `(
           ;; Slam all top level nodes to the left margin
           ((parent-is "program") parent 0)

           ((node-is ")") parent 0)
           ((node-is "end") grand-parent 0)

           ;; method parameters with and without '('
           ((query "(method_parameters \"(\" _ @indent)") first-sibling 1)
           ((parent-is "method_parameters") first-sibling 0)


           ((node-is "body_statement") parent ruby-ts-mode-indent-offset)
           ((parent-is "body_statement") first-sibling 0)
           ((parent-is "binary") first-sibling 0)

           ;; "when" list spread across multiple lines
           ((n-p-gp "pattern" "when" "case") (nth-sibling 1) 0)
           ((n-p-gp nil "then" "when") grand-parent ruby-ts-mode-indent-offset)

           ;; if / unless unless expressions
           ((node-is "else") parent-bol 0)
           ((node-is "elsif") parent-bol 0)
           ((node-is "when")  parent-bol 0)
           ((parent-is "then") parent-bol ruby-ts-mode-indent-offset)
           ((parent-is "else") parent-bol ruby-ts-mode-indent-offset)
           ((parent-is "elsif") parent-bol ruby-ts-mode-indent-offset)

           ;; for, while, until loops
           ((parent-is "do") grand-parent ruby-ts-mode-indent-offset)

           ;; Assignment of hash and array
           ((n-p-gp "}" "hash" "assignment") grand-parent 0)
           ((n-p-gp "pair" "hash" "assignment") grand-parent ruby-ts-mode-indent-offset)
           ((n-p-gp "]" "array" "assignment") grand-parent 0)
           ((n-p-gp ".*" "array" "assignment") grand-parent ruby-ts-mode-indent-offset)

           ;; hash and array other than assignments
           ((node-is "}") first-sibling 0)
           ((parent-is "hash") first-sibling 1)
           ((node-is "]") first-sibling 0)
           ((parent-is "array") first-sibling 1)

           ;; method call arguments with and without '('
           ((query "(argument_list \"(\" _ @indent)") first-sibling 1)
           ((parent-is "argument_list") first-sibling 0)

           )))
    `((base ,@common))))

(defun ruby-ts-mode--class-or-module-p (node)
  "Predicate returns turthy if NODE is a class or module"
  (string-match-p "class\\|module" (treesit-node-type node)))

(defun ruby-ts-mode--get-name (node)
  "Returns the text of the `name' field of NODE"
  (treesit-node-text (treesit-node-child-by-field-name node "name")))

(defun ruby-ts-mode--full-name (node)
  "Returns the fully qualified name of NODE"
  (let* ((name (get-name node))
         (delimiter "#"))
    (while (setq node (treesit-parent-until node #'ruby-ts-mode--class-or-module-p))
      (setq name (concat (get-name node) delimiter name))
      (setq delimiter "::"))
    name))

(defun ruby-ts-mode--imenu-helper (node)
  "Helper for `ruby-ts-mode--imenu' converting a treesit sparse tree
  into a list of imenu ( name . pos ) nodes"
  (let* ((ts-node (car node))
         (subtrees (mapcan #'ruby-ts-mode--imenu-helper (cdr node)))
         (name (when ts-node
                 (ruby-ts-mode--full-name ts-node)))
         (marker (when ts-node
                   (set-marker (make-marker)
                               (treesit-node-start ts-node)))))
    (cond
     ((or (null ts-node) (null name)) subtrees)
     ;; Don't include the anonymous "class" and "module" nodes
     ((string-match-p "(\"\\(class\\|module\\)\")"
                      (treesit-node-string ts-node))
      nil)
     (subtrees
      `((,name ,(cons name marker) ,@subtrees)))
     (t
      `((,name . ,marker))))))

;; For now, this is going to work like ruby-mode and return a list of
;; class, modules, def (methods), and alias.  It is likely that this
;; can be rigged to be easily extended.
(defun ruby-ts-mode--imenu ()
  "Return Imenu alist for the current buffer."
  (let* ((root (treesit-buffer-root-node))
         (nodes (treesit-induce-sparse-tree root "^\\(method\\|alias\\|class\\|module\\)$")))
    (ruby-ts-mode--imenu-helper nodes)))

(defun ruby-ts-mode--set-indent-style (language)
  "Helper function to set the indentation style.
Currently LANGUAGE is ignored but should be set to `ruby'."
  (let ((style
         (if (functionp ruby-ts-mode-indent-style)
             (funcall ruby-ts-mode-indent-style)
           (pcase ruby-ts-mode-indent-style
             ('base (alist-get 'base (ruby-ts-mode--indent-styles language)))))))
    `((,language ,@style))))

(define-derived-mode ruby-ts-base-mode prog-mode "Ruby"
  "Major mode for editing Ruby, powered by tree-sitter."
  :syntax-table ruby-ts-mode--syntax-table

  ;; Navigation.
  (setq-local treesit-defun-type-regexp
              (regexp-opt '("method"
                            "singleton_method")))

  ;; AFAIK, Ruby can not nest methods
  (setq-local treesit-defun-prefer-top-level nil)

  ;; Imenu.
  (setq-local imenu-create-index-function #'ruby-ts-mode--imenu)

  ;; seems like this could be defined when I know more how tree sitter
  ;; works.
  (setq-local which-func-functions nil)

  (setq-local treesit-font-lock-feature-list
              '(( comment definition)
                ( keyword preprocessor string type)
                ( assignment constant escape-sequence label literal property )
                ( bracket delimiter error function operator variable)))
  )

(define-derived-mode ruby-ts-mode ruby-ts-base-mode "Ruby"
  "Major mode for editing Ruby, powered by tree-sitter."
  :group 'ruby

  (unless (treesit-ready-p 'ruby)
    (error "Tree-sitter for Ruby isn't available"))

  (treesit-parser-create 'ruby)

  ;; Comments.
  (setq-local comment-start "# ")
  (setq-local comment-end "")
  (setq-local comment-start-skip "#+ *")

  (setq indent-tabs-mode ruby-ts-indent-tabs-mode)

  (setq-local treesit-simple-indent-rules
              (ruby-ts-mode--set-indent-style 'ruby))

  ;; Font-lock.
  (setq-local treesit-font-lock-settings (ruby-ts-mode--font-lock-settings 'ruby))

  (treesit-major-mode-setup))

;; end of ruby-ts-mode.el


[-- Attachment #1.2: Type: text/html, Size: 26981 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: ruby-ts-mode.el -- first draft

[Stefan Monnier]

> (defcustom ruby-ts-indent-tabs-mode nil
>   "Indentation can insert tabs in Ruby TS mode if this is non-nil."
>   :version "29.1"
>   :type 'boolean
>   :safe 'booleanp
>   :group 'ruby)

I'd discourage this practice of duplicating a standard buffer-local
variable <foo> with a <mode>-<foo> config var.

If `add-hook` is too much trouble for Emacs users, then we should devise
a generic way for the users to customize global vars like
`indent-tabs-mode` per-mode without forcing every major mode author to
do it by hand.

> (defcustom ruby-ts-mode-indent-style 'base

I think this var is defined twice in your file.  Did you byte-compile it?
I recommend `flymake-mode`.


        Stefan

Re: ruby-ts-mode.el -- first draft

[Eli Zaretskii]

> From: Perry Smith <pedz@easesoftware.com>
> Date: Sat, 10 Dec 2022 21:51:10 -0600
> 
> I have a git repository here: https://github.com/pedz/ruby-ts-mode
> 
> And here inline is the file:

Thanks.

What is the status of your copyright assignment?  We need it to accept
such a substantial contribution.  If you didn't yet start your legal
paperwork, would you like to start it now?

Re: ruby-ts-mode.el -- first draft

[Theodor Thornhill]

Perry Smith <pedz@easesoftware.com> writes:

> Ruby is a versatile language and I fear that I may have missed wide swaths of its features.  So, here is the first pass.  I hope folks can play with it and find the bugs.
>
> Tree sitter is so versatile that for fortification, it is practically endless the features you could add.
>
> I have a git repository here: https://github.com/pedz/ruby-ts-mode
>
> And here inline is the file:
>

[...]

> (defcustom ruby-ts-mode-indent-style 'base
>   "Style used for indentation.
>
> The selected style could be one of Ruby.  If one of the supplied
> styles doesn't suffice a function could be set instead.  This
> function is expected return a list that follows the form of
> `treesit-simple-indent-rules'."
>   :version "29.1"
>   :type '(choice (symbol :tag "Base" 'base)
>                  (function :tag "A function for user customized style" ignore))
>   :group 'ruby)

I believe we decided against using this indent style technique unless we
had specific styles to show.  A user could just:

(add-hook 'ruby-mode-hook
          (lambda ()
           (setq treesit-simple-indent-rules
                 my-personal-ruby-indent-rules)))

to override the current default anyway.


>
> (defcustom ruby-ts-mode-indent-style 'gnu
>   "Style used for indentation.
>
> Currently can only be set to BASE.  If one of the supplied styles
> doesn't suffice a function could be set instead.  This function
> is expected return a list that follows the form of
> `treesit-simple-indent-rules'."
>   :version "29.1"
>   :type '(choice (symbol :tag "Base" 'base)
>                  (function :tag "A function for user customized style" ignore))
>   :group 'ruby)

This can be removed.

>
> (defface ruby-ts-mode--constant-assignment-face
>   '((((class grayscale) (background light)) :foreground "DimGray" :slant italic)
>     (((class grayscale) (background dark))  :foreground "LightGray" :slant italic)
>     (((class color) (min-colors 88) (background light)) :foreground "VioletRed4")
>     (((class color) (min-colors 88) (background dark))  :foreground "plum2")
>     (((class color) (min-colors 16) (background light)) :foreground "RosyBrown")
>     (((class color) (min-colors 16) (background dark))  :foreground "LightSalmon")
>     (((class color) (min-colors 8)) :foreground "green")
>     (t :slant italic))
>   "Font Lock mode face used in ruby-ts-mode to highlight assignments to constants."
>   :group 'font-lock-faces)
>
> (defface ruby-ts-mode--assignment-face
>   '((((class grayscale) (background light)) :foreground "DimGray" :slant italic)
>     (((class grayscale) (background dark))  :foreground "LightGray" :slant italic)
>     (((class color) (min-colors 88) (background light)) :foreground "VioletRed4")
>     (((class color) (min-colors 88) (background dark))  :foreground "coral1")
>     (((class color) (min-colors 16) (background light)) :foreground "RosyBrown")
>     (((class color) (min-colors 16) (background dark))  :foreground "LightSalmon")
>     (((class color) (min-colors 8)) :foreground "green")
>     (t :slant italic))
>   "Font Lock mode face used in ruby-ts-mode to hightlight assignments."
>   :group 'font-lock-faces)

Are you sure we need these very specific faces?  Can't we reuse any of
the provided ones?

> ;; doc/keywords.rdoc in the Ruby git repository considers these to be
> ;; reserved keywords.  If these keywords are added to the list, it
> ;; causes the font-lock to stop working.
> ;;
> ;; "__ENCODING__" "__FILE__" "__LINE__" "false" "self" "super" "true"
> ;;
> ;; "nil" (which does not exhibit this issue) is also considered a
> ;; keyword but I removed it and added it as a constant.
> ;;
> (defun ruby-ts-mode--keywords (language)
>   "Ruby keywords for tree-sitter font-locking.
> Currently LANGUAGE is ignored but shoule be set to `ruby'."
>   (let ((common-keywords
>          '("BEGIN" "END" "alias" "and" "begin" "break" "case" "class"
>            "def" "defined?" "do" "else" "elsif" "end" "ensure" "for"
>            "if" "in" "module" "next" "not" "or" "redo" "rescue"
>            "retry" "return" "then" "undef" "unless" "until" "when"
>            "while" "yield")))
>     common-keywords))
>
> ;; Ideas of what could be added:
> ;;   1. The regular expressions start, end, and content could be font
> ;;      locked.  Ditto for the command strings `foo`.  The symbols
> ;;      inside a %s, %i, and %I could be given the "symbol" font.
> ;;      etc.
> (defun ruby-ts-mode--font-lock-settings (language)
>   "Tree-sitter font-lock settings.
> Currently LANGUAGE is ignored but should be set to `ruby'."
>   (treesit-font-lock-rules
>    :language language
>    :feature 'comment
>    `((comment) @font-lock-comment-face
>      (comment) @contextual)
>
>    :language language
>    :feature 'keyword
>    `([,@(ruby-ts-mode--keywords language)] @font-lock-keyword-face)
>
>    :language language
>    :feature 'constant
>    `((true) @font-lock-constant-face
>      (false) @font-lock-constant-face
>      (nil) @font-lock-constant-face
>      (self) @font-lock-constant-face
>      (super)  @font-lock-constant-face)
>
>    ;; Before 'operator so (unary) works.  (I didn't want to try
>    ;; :override)
>    :language language
>    :feature 'literal
>    `((unary ["+" "-"] [(integer) (rational) (float) (complex)]) @font-lock-number-face
>      (simple_symbol) @font-lock-number-face
>      (delimited_symbol) @font-lock-number-face
>      (integer) @font-lock-number-face
>      (float) @font-lock-number-face
>      (complex) @font-lock-number-face
>      (rational) @font-lock-number-face)
>
>    :language language
>    :feature 'operator
>    `("!" @font-lock-negation-char-face
>      [,@ruby-ts-mode--operators] @font-lock-operator-face)
>
>    :language language
>    :feature 'string
>    `((string) @font-lock-string-face
>      (string_content) @font-lock-string-face)
>
>    :language language
>    :feature 'type
>    `((constant) @font-lock-type-face)
>
>    :language language
>    :feature 'assignment
>    '((assignment
>       left: (identifier) @ruby-ts-mode--assignment-face)
>      (assignment
>       left: (left_assignment_list (identifier) @ruby-ts-mode--assignment-face))
>      (operator_assignment
>       left: (identifier) @ruby-ts-mode--assignment-face))
>
>    ;; Constant and scoped constant assignment (declaration)
>    ;; Must be enabled explicitly
>    :language language
>    :feature 'constant-assignment
>    :override t
>    `((assignment
>       left: (constant) @ruby-ts-mode--constant-assignment-face)
>      (assignment
>       left: (scope_resolution name: (constant) @ruby-ts-mode--constant-assignment-face)))
>
>    :language language
>    :feature 'function
>    '((call
>       method: (identifier) @font-lock-function-name-face)
>      (method
>       name: (identifier) @font-lock-function-name-face))
>
>    :language language
>    :feature 'variable
>    '((identifier) @font-lock-variable-name-face)
>
>    :language language
>    :feature 'error
>    '((ERROR) @font-lock-warning-face)
>
>    :feature 'escape-sequence
>    :language language
>    :override t
>    '((escape_sequence) @font-lock-escape-face)
>
>    :language language
>    :feature 'bracket
>    '((["(" ")" "[" "]" "{" "}"]) @font-lock-bracket-face)
>    )
>   )

Tuck these end-parens up together with the third to last line.

>
> (defun ruby-ts-mode--indent-styles (language)
>   "Indent rules supported by `ruby-ts-mode'.
> Currently LANGUAGE is ignored but should be set to `ruby'"
>   (let ((common
>          `(
>            ;; Slam all top level nodes to the left margin
>            ((parent-is "program") parent 0)
>
>            ((node-is ")") parent 0)
>            ((node-is "end") grand-parent 0)
>
>            ;; method parameters with and without '('
>            ((query "(method_parameters \"(\" _ @indent)") first-sibling 1)
>            ((parent-is "method_parameters") first-sibling 0)
>
>
>            ((node-is "body_statement") parent ruby-ts-mode-indent-offset)
>            ((parent-is "body_statement") first-sibling 0)
>            ((parent-is "binary") first-sibling 0)
>
>            ;; "when" list spread across multiple lines
>            ((n-p-gp "pattern" "when" "case") (nth-sibling 1) 0)
>            ((n-p-gp nil "then" "when") grand-parent ruby-ts-mode-indent-offset)
>
>            ;; if / unless unless expressions
>            ((node-is "else") parent-bol 0)
>            ((node-is "elsif") parent-bol 0)
>            ((node-is "when")  parent-bol 0)
>            ((parent-is "then") parent-bol ruby-ts-mode-indent-offset)
>            ((parent-is "else") parent-bol ruby-ts-mode-indent-offset)
>            ((parent-is "elsif") parent-bol ruby-ts-mode-indent-offset)
>
>            ;; for, while, until loops
>            ((parent-is "do") grand-parent ruby-ts-mode-indent-offset)
>
>            ;; Assignment of hash and array
>            ((n-p-gp "}" "hash" "assignment") grand-parent 0)
>            ((n-p-gp "pair" "hash" "assignment") grand-parent ruby-ts-mode-indent-offset)
>            ((n-p-gp "]" "array" "assignment") grand-parent 0)
>            ((n-p-gp ".*" "array" "assignment") grand-parent ruby-ts-mode-indent-offset)
>
>            ;; hash and array other than assignments
>            ((node-is "}") first-sibling 0)
>            ((parent-is "hash") first-sibling 1)
>            ((node-is "]") first-sibling 0)
>            ((parent-is "array") first-sibling 1)
>
>            ;; method call arguments with and without '('
>            ((query "(argument_list \"(\" _ @indent)") first-sibling 1)
>            ((parent-is "argument_list") first-sibling 0)
>
>            )))
>     `((base ,@common))))

Just return the common when the indent style is removed :)

>
> (defun ruby-ts-mode--class-or-module-p (node)
>   "Predicate returns turthy if NODE is a class or module"
>   (string-match-p "class\\|module" (treesit-node-type node)))
>
> (defun ruby-ts-mode--get-name (node)
>   "Returns the text of the `name' field of NODE"
>   (treesit-node-text (treesit-node-child-by-field-name node "name")))
>
> (defun ruby-ts-mode--full-name (node)
>   "Returns the fully qualified name of NODE"
>   (let* ((name (get-name node))
>          (delimiter "#"))
>     (while (setq node (treesit-parent-until node #'ruby-ts-mode--class-or-module-p))
>       (setq name (concat (get-name node) delimiter name))
>       (setq delimiter "::"))
>     name))
>
> (defun ruby-ts-mode--imenu-helper (node)
>   "Helper for `ruby-ts-mode--imenu' converting a treesit sparse tree
>   into a list of imenu ( name . pos ) nodes"
>   (let* ((ts-node (car node))
>          (subtrees (mapcan #'ruby-ts-mode--imenu-helper (cdr node)))
>          (name (when ts-node
>                  (ruby-ts-mode--full-name ts-node)))
>          (marker (when ts-node
>                    (set-marker (make-marker)
>                                (treesit-node-start ts-node)))))
>     (cond
>      ((or (null ts-node) (null name)) subtrees)
>      ;; Don't include the anonymous "class" and "module" nodes
>      ((string-match-p "(\"\\(class\\|module\\)\")"
>                       (treesit-node-string ts-node))
>       nil)
>      (subtrees
>       `((,name ,(cons name marker) ,@subtrees)))
>      (t
>       `((,name . ,marker))))))
>
> ;; For now, this is going to work like ruby-mode and return a list of
> ;; class, modules, def (methods), and alias.  It is likely that this
> ;; can be rigged to be easily extended.
> (defun ruby-ts-mode--imenu ()
>   "Return Imenu alist for the current buffer."
>   (let* ((root (treesit-buffer-root-node))
>          (nodes (treesit-induce-sparse-tree root "^\\(method\\|alias\\|class\\|module\\)$")))
>     (ruby-ts-mode--imenu-helper nodes)))
>

Are you sure we don't want more granularity than this?  Why is
everything in the same regexp?

> (defun ruby-ts-mode--set-indent-style (language)
>   "Helper function to set the indentation style.
> Currently LANGUAGE is ignored but should be set to `ruby'."
>   (let ((style
>          (if (functionp ruby-ts-mode-indent-style)
>              (funcall ruby-ts-mode-indent-style)
>            (pcase ruby-ts-mode-indent-style
>              ('base (alist-get 'base (ruby-ts-mode--indent-styles language)))))))
>     `((,language ,@style))))
>

Remove this when indent style is removed.

> (define-derived-mode ruby-ts-base-mode prog-mode "Ruby"
>   "Major mode for editing Ruby, powered by tree-sitter."
>   :syntax-table ruby-ts-mode--syntax-table
>
>   ;; Navigation.
>   (setq-local treesit-defun-type-regexp
>               (regexp-opt '("method"
>                             "singleton_method")))
>
>   ;; AFAIK, Ruby can not nest methods
>   (setq-local treesit-defun-prefer-top-level nil)
>
>   ;; Imenu.
>   (setq-local imenu-create-index-function #'ruby-ts-mode--imenu)
>
>   ;; seems like this could be defined when I know more how tree sitter
>   ;; works.
>   (setq-local which-func-functions nil)
>
>   (setq-local treesit-font-lock-feature-list
>               '(( comment definition)
>                 ( keyword preprocessor string type)
>                 ( assignment constant escape-sequence label literal property )
>                 ( bracket delimiter error function operator variable)))
>   )
>
> (define-derived-mode ruby-ts-mode ruby-ts-base-mode "Ruby"
>   "Major mode for editing Ruby, powered by tree-sitter."
>   :group 'ruby
>
>   (unless (treesit-ready-p 'ruby)
>     (error "Tree-sitter for Ruby isn't available"))
>
>   (treesit-parser-create 'ruby)
>
>   ;; Comments.
>   (setq-local comment-start "# ")
>   (setq-local comment-end "")
>   (setq-local comment-start-skip "#+ *")
>
>   (setq indent-tabs-mode ruby-ts-indent-tabs-mode)
>
>   (setq-local treesit-simple-indent-rules
>               (ruby-ts-mode--set-indent-style 'ruby))
>
>   ;; Font-lock.
>   (setq-local treesit-font-lock-settings (ruby-ts-mode--font-lock-settings 'ruby))
>
>   (treesit-major-mode-setup))
>
> ;; end of ruby-ts-mode.el


Also when this is ready, also add an entry to the NEWS file along with
an update to the build script in 'admin/notes/tree-sitter/build-module/'
so that we can get the ruby language installed easily!

Thanks for your effort!

Theo

Re: ruby-ts-mode.el -- first draft

[Perry Smith]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]



> On Dec 11, 2022, at 01:55, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Perry Smith <pedz@easesoftware.com>
>> Date: Sat, 10 Dec 2022 21:51:10 -0600
>> 
>> I have a git repository here: https://github.com/pedz/ruby-ts-mode
>> 
>> And here inline is the file:
> 
> Thanks.
> 
> What is the status of your copyright assignment?  We need it to accept
> such a substantial contribution.  If you didn't yet start your legal
> paperwork, would you like to start it now?

Yes. Please


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: ruby-ts-mode.el -- first draft

[Eli Zaretskii]

> From: Perry Smith <pedz@easesoftware.com>
> Date: Sun, 11 Dec 2022 07:58:54 -0600
> Cc: emacs-devel@gnu.org
> 
> > What is the status of your copyright assignment?  We need it to accept
> > such a substantial contribution.  If you didn't yet start your legal
> > paperwork, would you like to start it now?
> 
> Yes. Please

Form sent off-list.

Re: ruby-ts-mode.el -- first draft

[Theodor Thornhill]

(added back in emacs-devel)

On 11 December 2022 16:10:54 CET, Perry Smith <pedz@easesoftware.com> wrote:
>On Dec 11, 2022, at 02:22, Theodor Thornhill <theo@thornhill.no> wrote:
>> 
>>> (defcustom ruby-ts-mode-indent-style 'base
>>>  "Style used for indentation.
>>> 
>> 
>> I believe we decided against using this indent style technique unless we
>> had specific styles to show.  A user could just:
>> 
>> (add-hook 'ruby-mode-hook
>>          (lambda ()
>>           (setq treesit-simple-indent-rules
>>                 my-personal-ruby-indent-rules)))
>> 
>> to override the current default anyway.
>
>I don’t disagree but here are some Ruby specific thoughts and why I choose to have a “base” style.
>
>The “lint” in the Ruby community is called Rubocop and it is rather strongly opinionated out of the box but you can adjust it to your liking.  I predict someone will write a set of rules that 100% recreates Rubocop’s out of the box settings.  I think a lot of users would like to have an easy way to enable those “rules” which I consider rather strict.  And, at the same time, I think there will be a lot of users would like a more relax set of rules.  So I am expecting two sets of rules to be developed.
>

Why not create the two sets now, rather than later? I think we should try to make as good modes as possible so the need for external packages vanishes. If we are not good enough people will just create the modes. Maybe rubocop could be plugged into the Emacs tree-sitter integration?Luckily rubocops author loves Emacs, so they might offer some useful feedback (added Bozhidar Batsov to cc).

>Of course, these rules could be off in another package that users can easily load. … problem solved.
>
>The other thought I had while writing it is to have various switches that could be turned off and on.  This concept is implemented in the font lock side of the house but isn’t implemented on the indent side.  For example, the way I format assignment of a variable to an array or hash I could see having three different styles (I won’t elaborate here).  So rather a choice of global styles, there could be a set of customizable variables that cherry pick particular sets of indent rules.
>

Yes I agree, but if you have a particular design in mind I'm sure other modes can benefit from that as well. Maybe that should be an addition to treesit.el for all to use?


>Another option would be to put the various sets of rules in separate variables and then the final set the user could cherry pick the desired set of rules.
>
>The system is incredibly beautiful and versatile and I haven’t seen all the discussions on how to manage and use the versatility that Emacs now has.  I view myself at being rather bad when faced with these types of decisions and choices.

I say bring forward something so we have something to discuss :-)

>
>>>  "Font Lock mode face used in ruby-ts-mode to hightlight assignments."
>>>  :group 'font-lock-faces)
>> 
>> Are you sure we need these very specific faces?  Can't we reuse any of
>> the provided ones?
>
>The font lock features can be turned on and off.  I wanted the face to be different from the other faces if the user decides to turn on the feature.
>
>>>    (ruby-ts-mode--imenu-helper nodes)))
>>> 
>> 
>> Are you sure we don't want more granularity than this?  Why is
>> everything in the same regexp?
>
>I didn’t even know imenu existed two days ago :-)  The current implementation appears to produce the same list of items that ruby-mode does and styles it like c-ts-mode does.
>
>Perry
>

Sure! I have no strong opinions, but you could look at how java-ts-mode does it for a slightly different approach.

Theo

Re: ruby-ts-mode.el -- first draft

[Perry Smith]

[-- Attachment #1.1: Type: text/plain, Size: 2425 bytes --]


> On Dec 11, 2022, at 10:26, Theodor Thornhill <theo@thornhill.no> wrote:
> 
>> The “lint” in the Ruby community is called Rubocop and it is rather strongly opinionated out of the box but you can adjust it to your liking.  I predict someone will write a set of rules that 100% recreates Rubocop’s out of the box settings.  I think a lot of users would like to have an easy way to enable those “rules” which I consider rather strict.  And, at the same time, I think there will be a lot of users would like a more relax set of rules.  So I am expecting two sets of rules to be developed.
>> 
> 
> Why not create the two sets now, rather than later? I think we should try to make as good modes as possible so the need for external packages vanishes. If we are not good enough people will just create the modes. Maybe rubocop could be plugged into the Emacs tree-sitter integration?Luckily rubocops author loves Emacs, so they might offer some useful feedback (added Bozhidar Batsov to cc).

This is why I called it a first draft.  I’m hoping others start to play with it and provide feedback.  And I plan to fire up Rubocop (or perhaps contact its author as well) and try to implement the Rubocop mode.

>> Another option would be to put the various sets of rules in separate variables and then the final set the user could cherry pick the desired set of rules.
>> 
>> The system is incredibly beautiful and versatile and I haven’t seen all the discussions on how to manage and use the versatility that Emacs now has.  I view myself at being rather bad when faced with these types of decisions and choices.
> 
> I say bring forward something so we have something to discuss :-)

I was just about to do this.

I want to play around with treesit coupled with align.  Wouldn’t it be super slick if

new_hash = {
  long_ugly_name: 9,
  frog: 12,
  daft: 92,
  egg: 99
}

was automagically formatted as:

new_hash = {
  long_ugly_name:  9,
  frog:           12,
  daft:           92,
  egg:            99
}

Well… I am sure some will say “Yea!!!” while others will say “OMG! NO!!!”.  And, clearly how something like that is styled has a lot of alternatives and choices.

But I’m not even sure it is even possible in the practical sense.  I wanted to play with it and see how hard it is as well as ask the list for advice / hints on how to approach this.

Perry


[-- Attachment #1.2: Type: text/html, Size: 5929 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: ruby-ts-mode.el -- first draft

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 3082 bytes --]

RuboCop's author here. :-)

I'd encourage the new mode to support all most common indentation styles, otherwise gaining traction would be hard given how versatile the current ruby-mode is. I think it'd be really nice for the sake of easy migration between the modes to maintain exactly the same default indentation settings. 

If I recall correctly out-of-the-box the current ruby-mode will generate no RuboCop layout-related warnings and I'd like us to keep this going forward.

On Sun, Dec 11, 2022, at 6:42 PM, Perry Smith wrote:
> 
>> On Dec 11, 2022, at 10:26, Theodor Thornhill <theo@thornhill.no> wrote:
>> 
>>> The “lint” in the Ruby community is called Rubocop and it is rather strongly opinionated out of the box but you can adjust it to your liking.  I predict someone will write a set of rules that 100% recreates Rubocop’s out of the box settings.  I think a lot of users would like to have an easy way to enable those “rules” which I consider rather strict.  And, at the same time, I think there will be a lot of users would like a more relax set of rules.  So I am expecting two sets of rules to be developed.
>> 
>> Why not create the two sets now, rather than later? I think we should try to make as good modes as possible so the need for external packages vanishes. If we are not good enough people will just create the modes. Maybe rubocop could be plugged into the Emacs tree-sitter integration?Luckily rubocops author loves Emacs, so they might offer some useful feedback (added Bozhidar Batsov to cc).
> 
> This is why I called it a first draft.  I’m hoping others start to play with it and provide feedback.  And I plan to fire up Rubocop (or perhaps contact its author as well) and try to implement the Rubocop mode.
> 
>>> Another option would be to put the various sets of rules in separate variables and then the final set the user could cherry pick the desired set of rules.
>>> 
>>> The system is incredibly beautiful and versatile and I haven’t seen all the discussions on how to manage and use the versatility that Emacs now has.  I view myself at being rather bad when faced with these types of decisions and choices.
>> 
>> I say bring forward something so we have something to discuss :-)
> I was just about to do this.
> 
> I want to play around with treesit coupled with align.  Wouldn’t it be super slick if
> 
>> new_hash = {
>>   long_ugly_name: 9,
>>   frog: 12,
>>   daft: 92,
>>   egg: 99
>> }
> 
> was automagically formatted as:
> 
>> new_hash = {
>>   long_ugly_name:  9,
>>   frog:           12,
>>   daft:           92,
>>   egg:            99
>> }
> 
> Well… I am sure some will say “Yea!!!” while others will say “OMG! NO!!!”.  And, clearly how something like that is styled has a lot of alternatives and choices.
> 
> But I’m not even sure it is even possible in the practical sense.  I wanted to play with it and see how hard it is as well as ask the list for advice / hints on how to approach this.
> 
> Perry
> 
> 
> *Attachments:*
>  * signature.asc

[-- Attachment #2: Type: text/html, Size: 7706 bytes --]