Re: toml-ts-mode: first draft

["Jostein Kjønigsen" <jostein@secure.kjonigsen.net>]

[-- Attachment #1.1: Type: text/plain, Size: 1499 bytes --]

On 12.12.2022 00:01, Yuan Fu wrote:
> Looks great!
>
> +  (unless (treesit-ready-p 'toml)
> +    (error "Tree-sitter for TOML isn't avilable”))
>
> Treesit-ready-p already raises a warning when something goes wrong. So you want to either pass the QUIET argument or let it do the barking ;-)
>
> Yuan
Hey everyone. A quick update.

I've updated the code to follow the above-mentioned (when 
(treesit-ready-p ...)) convention.

Bigger news perhaps is that I've also updated it to support imenu.

The imenu-implementation is limited to tables/sections/array-elements 
only, and has no accessor for direct values. That said it still works 
pretty good and the code should be fully understandable.

Said simply, the patch attached to this email contains all fixes and 
features requested by everyone so far.

I consider it more than good enough for everyday use at this point.

I'm obviously biased, but I have to say this version fares quite 
favourable when doing an objective comparison to toml-mode as found in 
MELPA:

  * better and more consistent fontification
  * better support for various quoting formats
  * better support for dotted values
  * better support for inline tables
  * it highlights syntax-errors
  * it actually has imenu support (limited to headers/tables only, now
    direct value access)

Pretty much every obvious bug I see in the MELPA-version is fixed in 
this one.

So... Any other requests I should look into before we start considering 
merging this? :)

--
Jostein

[-- Attachment #1.2: Type: text/html, Size: 2084 bytes --]

[-- Attachment #2: 0002-Introduce-support-for-TOML-config-format.patch --]
[-- Type: text/x-patch, Size: 6935 bytes --]

From f073159d6cc19aafda2cc18debe0e1b890bee715 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jostein=20Kj=C3=B8nigsen?= <jostein@kjonigsen.net>
Date: Sun, 11 Dec 2022 13:05:29 +0100
Subject: [PATCH 2/5] Introduce support for TOML config-format

This commit introduces support for the semi-popular TOML
config-format[1] through a new major-mode: toml-ts-mode.

I've read through the full spec[2], and from what I can see this
major-mode should provide correct syntax-highligting for every sort of
config-declaration which adheres to the specification.

Besides that it also adds support for imenu and basic tree-sitter
based navigation.

[1] https://toml.io/en/
[2] https://toml.io/en/v1.0.0
---
 admin/notes/tree-sitter/build-module/batch.sh |   1 +
 lisp/textmodes/toml-ts-mode.el                | 152 ++++++++++++++++++
 2 files changed, 153 insertions(+)
 create mode 100644 lisp/textmodes/toml-ts-mode.el

diff --git a/admin/notes/tree-sitter/build-module/batch.sh b/admin/notes/tree-sitter/build-module/batch.sh
index 6dce000caa6..e955bb6bcf5 100755
--- a/admin/notes/tree-sitter/build-module/batch.sh
+++ b/admin/notes/tree-sitter/build-module/batch.sh
@@ -14,6 +14,7 @@ languages=
     'rust'
     'typescript'
     'tsx'
+    'toml'
 )
 
 for language in "${languages[@]}"
diff --git a/lisp/textmodes/toml-ts-mode.el b/lisp/textmodes/toml-ts-mode.el
new file mode 100644
index 00000000000..322f63a3170
--- /dev/null
+++ b/lisp/textmodes/toml-ts-mode.el
@@ -0,0 +1,152 @@
+;;; toml-ts-mode.el --- tree-sitter support for TOML  -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2022 Free Software Foundation, Inc.
+
+;; Author     : Jostein Kjønigsen <jostein@kjonigsen.net>
+;; Maintainer : Jostein Kjønigsen <jostein@kjonigsen.net>
+;; Created    : December 2022
+;; Keywords   : toml languages tree-sitter
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+;;
+
+;;; Code:
+
+(require 'treesit)
+
+(declare-function treesit-parser-create "treesit.c")
+(declare-function treesit-induce-sparse-tree "treesit.c")
+(declare-function treesit-node-start "treesit.c")
+(declare-function treesit-node-child-by-field-name "treesit.c")
+
+(defvar toml-ts-mode--font-lock-settings
+      (treesit-font-lock-rules
+       :language 'toml
+       :feature 'constant
+       '((boolean) @font-lock-constant-face)
+       :language 'toml
+       :feature 'delimiter
+       '((["="]) @font-lock-delimiter-face)
+       :language 'toml
+       :feature 'number
+       '((integer) @font-lock-number-face
+         (float) @font-lock-number-face
+         (local_date) @font-lock-number-face
+         (local_date_time) @font-lock-number-face
+         (local_time) @font-lock-number-face)
+       :language 'toml
+       :feature 'string
+       '((string) @font-lock-string-face)
+       :language 'toml
+       :feature 'escape-sequence
+       :override t
+       '((escape_sequence) @font-lock-escape-face)
+       :language 'toml
+       :feature 'pair
+       :override t ; Needed for overriding string face on keys.
+       '((comment) @font-lock-comment-face
+         (bare_key) @font-lock-property-face
+         (quoted_key) @font-lock-property-face
+         (table ("[" @font-lock-bracket-face
+                 (_) @font-lock-type-face
+                 "]" @font-lock-bracket-face))
+         (table_array_element ("[[" @font-lock-bracket-face
+                               (_) @font-lock-type-face
+                               "]]" @font-lock-bracket-face))
+         (table (quoted_key) @font-lock-type-face)
+         (table (dotted_key (quoted_key)) @font-lock-type-face))
+       :language 'toml
+       :feature 'error
+       :override t
+       '((ERROR) @font-lock-warning-face))
+      "Font-lock settings for TOML.")
+
+(defun toml-ts-mode--get-table-name (node)
+  "Obtains the header-name for the associated tree-sitter `NODE'."
+  (if node
+      (treesit-node-text
+       (car (cdr (treesit-node-children node))))
+    "Root table"))
+
+(defun toml-ts-mode--imenu-1 (node)
+  "Helper for `toml-ts-mode--imenu'.
+Find string representation for NODE and set marker, then recurse
+the subtrees."
+  (let* ((ts-node (car node))
+         (subtrees (mapcan #'toml-ts-mode--imenu-1 (cdr node)))
+         (name (toml-ts-mode--get-table-name ts-node))
+         (marker (when ts-node
+                   (set-marker (make-marker)
+                               (treesit-node-start ts-node)))))
+    (cond
+     ((null ts-node) subtrees)
+     (subtrees
+      `((,name ,(cons name marker) ,@subtrees)))
+     (t
+      `((,name . ,marker))))))
+
+(defun toml-ts-mode--imenu ()
+  "Return Imenu alist for the current buffer."
+  (let* ((node (treesit-buffer-root-node))
+         (table-tree (treesit-induce-sparse-tree
+                      node "^table$" nil 1000))
+         (table-array-tree (treesit-induce-sparse-tree
+                            node "^table_array_element$" nil 1000))
+         (table-index (toml-ts-mode--imenu-1 table-tree))
+         (table-array-index (toml-ts-mode--imenu-1 table-array-tree)))
+    ;;(setq global-toml-node (treesit-buffer-root-node))
+    (append
+     (when table-index `(("Headers" . ,table-index)))
+     (when table-array-index `(("Arrays" . ,table-array-index))))))
+
+
+;;;###autoload
+(add-to-list 'auto-mode-alist '("\\.toml\\'" . toml-ts-mode))
+
+;;;###autoload
+(define-derived-mode toml-ts-mode text-mode "TOML"
+  "Major mode for editing TOML, powered by tree-sitter."
+  :group 'toml-mode
+
+  (when (treesit-ready-p 'toml)
+    (treesit-parser-create 'toml)
+
+    ;; Comments
+    (setq-local comment-start "# ")
+    (setq-local commend-end "")
+
+    ;; Navigation.
+    (setq-local treesit-defun-type-regexp
+                (rx (or "table" "table_array_element")))
+
+    ;; Font-lock.
+    (setq-local treesit-font-lock-settings toml-ts-mode--font-lock-settings)
+    (setq-local treesit-font-lock-feature-list
+                '((constant number pair string)
+                  (escape-sequence)
+                  (delimiter error)))
+
+    ;; Imenu.
+    (setq-local imenu-create-index-function #'toml-ts-mode--imenu)
+    (setq-local which-func-functions nil) ;; Piggyback on imenu
+
+    (treesit-major-mode-setup)))
+
+(provide 'toml-ts-mode)
+
+;;; toml-ts-mode.el ends here
-- 
2.37.2