From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.bugs Subject: bug#68993: treesitter support for forward-sexp-default-function Date: Sun, 11 Feb 2024 17:42:34 -0800 Message-ID: <89817820-6928-43AD-B4BB-0AB48B1808FB@gmail.com> References: <86y1bv6jsm.fsf@mail.linkov.net> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5633"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 68993@debbugs.gnu.org To: Juri Linkov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Feb 12 02:44:16 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rZLMl-0001E4-P8 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 12 Feb 2024 02:44:15 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rZLMP-0001TS-C1; Sun, 11 Feb 2024 20:43:53 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rZLMK-0001Sg-7t for bug-gnu-emacs@gnu.org; Sun, 11 Feb 2024 20:43:50 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rZLMI-0008VI-E2 for bug-gnu-emacs@gnu.org; Sun, 11 Feb 2024 20:43:47 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rZLMY-0007aT-Rx for bug-gnu-emacs@gnu.org; Sun, 11 Feb 2024 20:44:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 12 Feb 2024 01:44:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 68993 X-GNU-PR-Package: emacs Original-Received: via spool by 68993-submit@debbugs.gnu.org id=B68993.170770219128984 (code B ref 68993); Mon, 12 Feb 2024 01:44:02 +0000 Original-Received: (at 68993) by debbugs.gnu.org; 12 Feb 2024 01:43:11 +0000 Original-Received: from localhost ([127.0.0.1]:40174 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rZLLj-0007XP-8j for submit@debbugs.gnu.org; Sun, 11 Feb 2024 20:43:11 -0500 Original-Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]:45497) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rZLLg-0007X1-Ri for 68993@debbugs.gnu.org; Sun, 11 Feb 2024 20:43:09 -0500 Original-Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-5dbcfa0eb5dso2603600a12.3 for <68993@debbugs.gnu.org>; Sun, 11 Feb 2024 17:42:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707702166; x=1708306966; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=cVApWHj9VPTxFK6EtiWYkYGDqYGfsIh8KrB111NF+Ug=; b=XslPIXzO6UWPR0oK1lzku+V5Wl+3U4Z9ZFgwYJBay5eWOetxb1sNejJbTsFbFko0an kP52ILO94s552r1EeKD+Mxvcug2bkvAK06JBz/4OVVliz82qXWF9zXrXrzr0hudTRa0E 6YXQSSlSNcDLQJqCiGOupcLTLGzbixoiUbAsFXFZOku8rYCrz3rMwbydHvG4/4Jf1xI3 1GhnlRB8VR0rFa8yOyRlPT6jQDuLueMoOkPg0hvgTlq5puQHWFX/iz1T+THcqAe3oX5W FMqkIl8+TeP1zJ2ThpbC1Yn/sT8gpIOimH2X5qZSNT0mmr0OfsOeqYgW/DWIGvWhRvPf x4aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707702166; x=1708306966; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cVApWHj9VPTxFK6EtiWYkYGDqYGfsIh8KrB111NF+Ug=; b=qf/BCvSO6HftFZqEGrJAlcTw0vYhLjn4MJv8Xq3FLqQTIofFpGKVStuoP92W13icS5 eL5xBcLU6fCp93RyQq0RxC3NGECRhiVw61v4BF8PiEGRrHinZFOfzeKmz+de43EPVwIB 9GrC+SawA5lBasUMixvPPt6Nn583cKs8hQ57v5BgGLylHcVeTMH9jQfAapkroDrKh8dn aok9H7vu517OhMPM/J+VHbZST1XSrs/AbCszUuQYhjUsDiv8j2iI/KqiEasSLo2hFsKi 3FMZUaV5QhEH13GzD9qhVhibaWlZbgQ+KTY8QmcgLpzw1wMgYA5RWSphP5Ms1nS5f1bh hK8g== X-Gm-Message-State: AOJu0YzcyyTs5hFWkvCKQem+uBuannkF227l0Rxev02T6VlsErIJVEtQ AwP7u2XCsN+0Wtc8CNgL5KsqKYnfwW1+KWnD/ORBifXYrLxT1auU X-Google-Smtp-Source: AGHT+IFUMFqmp0XmRAGsK6zkmB+fddcZZrSRkG3B9t6vhLx2pOetds9VuQdL70f+gHn53Bhpme8t5w== X-Received: by 2002:aa7:888e:0:b0:6e0:91c4:be1c with SMTP id z14-20020aa7888e000000b006e091c4be1cmr6357719pfe.9.1707702166382; Sun, 11 Feb 2024 17:42:46 -0800 (PST) Original-Received: from smtpclient.apple (172-117-161-177.res.spectrum.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id o17-20020a637e51000000b005cd78f13608sm5664783pgn.13.2024.02.11.17.42.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Feb 2024 17:42:45 -0800 (PST) In-Reply-To: <86y1bv6jsm.fsf@mail.linkov.net> X-Mailer: Apple Mail (2.3731.700.6) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:279883 Archived-At: > On Feb 8, 2024, at 9:38 AM, Juri Linkov wrote: >=20 > 'treesit-forward-sentence' supports the node type 'text', > and for matching nodes it uses the fallback to > 'forward-sentence-default-function'. >=20 > This patch does exactly the same for 'treesit-forward-sexp': > for nodes that match a new node type 'comment', > it uses the fallback to the new function > 'forward-sexp-default-function'. >=20 > diff --git a/lisp/treesit.el b/lisp/treesit.el > index 82b2f97b4a5..284c4915f3a 100644 > --- a/lisp/treesit.el > +++ b/lisp/treesit.el > @@ -2137,7 +2137,10 @@ treesit-forward-sexp > (interactive "^p") > (let ((arg (or arg 1)) > (pred (or treesit-sexp-type-regexp 'sexp))) > - (or (if (> arg 0) > + (or (when (treesit-node-match-p (treesit-node-at (point)) = 'comment t) > + (funcall #'forward-sexp-default-function arg) > + t) > + (if (> arg 0) > (treesit-end-of-thing pred (abs arg) 'restricted) > (treesit-beginning-of-thing pred (abs arg) 'restricted)) > ;; If we couldn't move, we should signal an error and report > diff --git a/lisp/emacs-lisp/lisp.el b/lisp/emacs-lisp/lisp.el > index 4b722b4e9a7..d3c3bf55de3 100644 > --- a/lisp/emacs-lisp/lisp.el > +++ b/lisp/emacs-lisp/lisp.el > @@ -45,7 +45,12 @@ parens-require-spaces > :type 'boolean > :group 'lisp) >=20 > -(defvar forward-sexp-function nil > +(defun forward-sexp-default-function (&optional arg) > + "Default function for `forward-sexp-function'." > + (goto-char (or (scan-sexps (point) arg) (buffer-end arg))) > + (if (< arg 0) (backward-prefix-chars))) > + > +(defvar forward-sexp-function #'forward-sexp-default-function > ;; FIXME: > ;; - for some uses, we may want a "sexp-only" version, which only > ;; jumps over a well-formed sexp, rather than some dwimish thing > @@ -74,10 +79,7 @@ forward-sexp > "No next sexp" > "No previous sexp")))) > (or arg (setq arg 1)) > - (if forward-sexp-function > - (funcall forward-sexp-function arg) > - (goto-char (or (scan-sexps (point) arg) (buffer-end arg))) > - (if (< arg 0) (backward-prefix-chars))))) > + (funcall forward-sexp-function arg))) >=20 > (defun backward-sexp (&optional arg interactive) > "Move backward across one balanced expression (sexp). >=20 > Maybe the node type 'comment' is not the best name, > but it was intended to allow using the default function > to be able to move with 'M-C-f' in the comments and strings > there tree-sitter has no information. >=20 > It makes sense to support the default movement with 'M-C-f' > in the comments and strings of all ts modes. The second patch > shows how this could be achieved by adding the default > 'comment' match to 'treesit-thing-settings' of all modes. > Or maybe this should be customizable? I think treesit-thing-settings is something we want to left for major = mode=E2=80=99s to set. They=E2=80=99ll need to define other =E2=80=9Cthing= s=E2=80=9D in treesit-thing-settings anyway. Sure, it=E2=80=99s nice if = we can set a few definitions automatically, but I don=E2=80=99t think = the gain is worth that much; OTOH, it=E2=80=99s nice to have clear = boundaries, and minimizes the possibility of confusion. >=20 > diff --git a/lisp/treesit.el b/lisp/treesit.el > index 82b2f97b4a5..284c4915f3a 100644 > --- a/lisp/treesit.el > +++ b/lisp/treesit.el > @@ -3054,6 +3057,18 @@ treesit-major-mode-setup > (setq-local outline-search-function #'treesit-outline-search > outline-level #'treesit-outline-level)) >=20 > + (dolist (parser (treesit-parser-list)) > + (let ((language (treesit-parser-language parser)) > + (comment (regexp-opt '("comment" "string" = "string_content")))) > + (unless (treesit-thing-defined-p 'comment language) > + (if-let ((l (alist-get language treesit-thing-settings))) > + (progn > + (setf (alist-get 'comment l) (list comment)) > + (setf (alist-get language treesit-thing-settings) l)) > + (setq-local treesit-thing-settings > + (append `((,language (comment ,comment))) > + treesit-thing-settings)))))) > + > ;; Remove existing local parsers. > (dolist (ov (overlays-in (point-min) (point-max))) > (when-let ((parser (overlay-get ov 'treesit-parser))) >=20 > The third patch demonstrates how it's possible to close bug#67036 > that was impossible to fix without more general changes in treesit.el. >=20 > The problem is that e.g. Ruby parser to such text: >=20 > hash[:key] >=20 > produces such syntax tree: >=20 > (element_reference object: (identifier) [ (simple_symbol) ]) >=20 > so when point is on [ then 'M-C-f' can't move to ]. >=20 > This is fixed now by the third patch: >=20 > diff --git a/lisp/progmodes/ruby-ts-mode.el = b/lisp/progmodes/ruby-ts-mode.el > index 598eaa461ff..4d0ae2e9303 100644 > --- a/lisp/progmodes/ruby-ts-mode.el > +++ b/lisp/progmodes/ruby-ts-mode.el > @@ -1170,7 +1170,20 @@ ruby-ts-mode > "global_variable" > ) > eol) > - #'ruby-ts--sexp-p))))) > + #'ruby-ts--sexp-p)) > + (comment ,(lambda (node) > + (or (member (treesit-node-type node) > + '("comment" = "string_content")) > + (and (member (treesit-node-text = node) > + '("[" "]")) > + (equal (treesit-node-type > + (treesit-node-parent = node)) > + "element_reference")) > + (and (member (treesit-node-text = node) > + '("#{" "}")) > + (equal (treesit-node-type > + (treesit-node-parent = node)) > + "interpolation")))))))) >=20 > ;; AFAIK, Ruby can not nest methods > (setq-local treesit-defun-prefer-top-level nil) IIUC, this doesn=E2=80=99t look like a good idea: you don=E2=80=99t want = to mark something that=E2=80=99s not comment as comment. In the future, = other packages will start using these thing definitions, and I=E2=80=99m = sure you don=E2=80=99t want them consider regular code as comments. For the specific problem you described, maybe the change made in #68899 = can help? Yuan=