bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Basil L. Contovounesios]

Severity: minor

0. emacs -Q
1. (let (x) `(,@x))
2. M-s _ x

This highlights the bound x, but not the spliced x.  The same happens
with 'M-s .' or the equivalent 'C-M-s'.

This is because lisp--mode-syntax-table gives 'prefix' syntax to ?, and
'symbol' syntax to ?@.  Is it possible to give 'prefix' syntax to ",@"
as a whole, and in general?

[This behaviour exists since at least as far back as Emacs 24.5.1.
 I wouldn't be surprised if this has been reported before,
 but my searches came up dry.]

Thanks,

-- 
Basil

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2020-11-02 built on thunk
Repository revision: 95f7a2835a79c1f12b5dc86230405e8040910c72
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux bullseye/sid

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O2 -march=native' --config-cache
 --prefix=/home/blc/.local --with-x-toolkit=lucid
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT
LIBOTF ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
LIBSYSTEMD JSON PDUMPER LCMS2

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml easymenu mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
misearch multi-isearch tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite charscript charprop case-table epa-hook jka-cmpr-hook help
simple abbrev obarray cl-preloaded nadvice button loaddefs faces
cus-face macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 52709 5287)
 (symbols 48 6786 1)
 (strings 32 19027 1309)
 (string-bytes 1 616874)
 (vectors 16 12242)
 (vector-slots 8 168612 9829)
 (floats 8 23 45)
 (intervals 56 231 0)
 (buffers 992 10))

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Lars Ingebrigtsen]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> 0. emacs -Q
> 1. (let (x) `(,@x))
> 2. M-s _ x
>
> This highlights the bound x, but not the spliced x.  The same happens
> with 'M-s .' or the equivalent 'C-M-s'.
>
> This is because lisp--mode-syntax-table gives 'prefix' syntax to ?, and
> 'symbol' syntax to ?@.  Is it possible to give 'prefix' syntax to ",@"
> as a whole, and in general?

(This is still present in Emacs 29.)

I'm not sure whether that is possible or not -- perhaps Stefan knows;
added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> 0. emacs -Q
>> 1. (let (x) `(,@x))
>> 2. M-s _ x
>>
>> This highlights the bound x, but not the spliced x.  The same happens
>> with 'M-s .' or the equivalent 'C-M-s'.
>>
>> This is because lisp--mode-syntax-table gives 'prefix' syntax to ?, and
>> 'symbol' syntax to ?@.  Is it possible to give 'prefix' syntax to ",@"
>> as a whole, and in general?
>
> (This is still present in Emacs 29.)
>
> I'm not sure whether that is possible or not -- perhaps Stefan knows;
> added to the CCs.

`@x` is a valid symbol, which is why `?@` is given symbol syntax.
If we want to fix the above search we need to use
`syntax-propertize-function` to change the syntax of those `@` that
follow a comma.


        Stefan

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> `@x` is a valid symbol, which is why `?@` is given symbol syntax.
> If we want to fix the above search we need to use
> `syntax-propertize-function` to change the syntax of those `@` that
> follow a comma.

The following seems to do the trick -- does this look OK to you?

diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index 77bf3f1ed1..210270bc67 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -245,6 +245,9 @@ elisp-mode-syntax-propertize
       ;; Empty symbol.
       ("##" (0 (unless (nth 8 (syntax-ppss))
                  (string-to-syntax "_"))))
+      ;; Give ,@ a prefix syntax.
+      (",@" (0 (unless (ppss-comment-or-string-start (syntax-ppss))
+                 (string-to-syntax "'"))))
       ;; Unicode character names.  (The longest name is 88 characters
       ;; long.)
       ("\\?\\\\N{[-A-Za-z0-9 ]\\{,100\\}}"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> The following seems to do the trick -- does this look OK to you?

Fine by me,


        Stefan

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The following seems to do the trick -- does this look OK to you?
>
> Fine by me,

OK; now pushed to Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Lars Ingebrigtsen]

This led to edebug tests hanging, so I've reverted the change for now
and am reopening this bug report.  The offending change was
d003848b5e3ad2dfbe84cc62b99776fdc6734325.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

A tentative fix has been pushed to master. I didn't bother with a NEWS entry but will be happy to add one if someone thinks it would be useful.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Eli Zaretskii]

> Cc: Basil Contovounesios <contovob@tcd.ie>, 44418@debbugs.gnu.org,
>  Stefan Monnier <monnier@iro.umontreal.ca>
> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Wed, 21 Jun 2023 17:59:44 +0200
> 
> A tentative fix has been pushed to master. I didn't bother with a NEWS entry but will be happy to add one if someone thinks it would be useful.

If this changes user-facing behavior or behavior of public functions,
I think we must call this out in NEWS, under "incompatible changes".

Thanks.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

> If this changes user-facing behavior or behavior of public functions,
> I think we must call this out in NEWS, under "incompatible changes".

This is just a bug fix, not a change to documented or otherwise intended behaviour.
Maybe announcing it NEWS would make some people happy to know that the bug has finally been fixed.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Thu, 22 Jun 2023 10:50:18 +0200
> Cc: larsi@gnus.org,
>  contovob@tcd.ie,
>  44418@debbugs.gnu.org,
>  monnier@iro.umontreal.ca
> 
> > If this changes user-facing behavior or behavior of public functions,
> > I think we must call this out in NEWS, under "incompatible changes".
> 
> This is just a bug fix, not a change to documented or otherwise intended behaviour.

Whether it was intended is not necessarily relevant.  It could be
known long-standing behavior that became a de-facto standard.  Which
is why I think we should call it out.

If you are unwilling to mention this in NEWS, please describe the
change and I will add to NEWS what I think needs to be added.

TIA

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

> Whether it was intended is not necessarily relevant.  It could be
> known long-standing behavior that became a de-facto standard.  Which
> is why I think we should call it out.

That's fine with me -- now done. Closing bug.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Mattias Engdegård [2023-06-21 17:59 +0200] wrote:

> A tentative fix has been pushed to master.

Thanks.  Too bad the same can't be done for tree-sitter capture names
such as @font-lock-foo-face or @my--fontify-foo :).

-- 
Basil

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

23 juni 2023 kl. 13.58 skrev Basil Contovounesios <contovob@tcd.ie>:

> Too bad the same can't be done for tree-sitter capture names
> such as @font-lock-foo-face or @my--fontify-foo :).

Yes, I think that's a bit unfortunate. Maybe we can introduce (@ NAME) as alternative syntax?

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 422 bytes --]

> Maybe we can introduce (@ NAME) as alternative syntax?

Here is a proof-of-concept (missing tests and documentation), but it works and I think it's a definite improvement.

Maybe we can get a blessing from Yuan Fu. To recap, this permits the syntax (@ symbol-name) as an alternative to @symbol-name in treesit queries because that allows symbol searching to match the symbol-name, and this can be really helpful.


[-- Attachment #2: treesit-alt-capture-construct.diff --]
[-- Type: application/octet-stream, Size: 1824 bytes --]

diff --git a/src/treesit.c b/src/treesit.c
index 87aa1eeb377..64570094347 100644
--- a/src/treesit.c
+++ b/src/treesit.c
@@ -2358,6 +2358,7 @@ DEFUN ("treesit-pattern-expand",
     (TYPE PATTERN...)
     [PATTERN...]
     FIELD-NAME:
+    (@ CAPTURE-NAME)
     @CAPTURE-NAME
     (_)
     _
@@ -2380,18 +2381,27 @@ DEFUN ("treesit-pattern-expand",
     return Vtreesit_str_pound_match;
   if (BASE_EQ (pattern, QCpred))
     return Vtreesit_str_pound_pred;
-  Lisp_Object opening_delimeter
-    = VECTORP (pattern)
-      ? Vtreesit_str_open_bracket : Vtreesit_str_open_paren;
-  Lisp_Object closing_delimiter
-    = VECTORP (pattern)
-      ? Vtreesit_str_close_bracket : Vtreesit_str_close_paren;
+
+  /* (@ X) -> @Y where X -> Y */
+  if (CONSP (pattern) && BASE_EQ (XCAR (pattern), Qat)
+      && CONSP (XCDR (pattern)) && NILP (XCDR (XCDR (pattern))))
+    return concat2 (build_string ("@"),
+		    Ftreesit_pattern_expand (XCAR (XCDR (pattern))));
+
   if (VECTORP (pattern) || CONSP (pattern))
+    {
+      Lisp_Object opening_delimeter
+	= VECTORP (pattern)
+	? Vtreesit_str_open_bracket : Vtreesit_str_open_paren;
+      Lisp_Object closing_delimiter
+	= VECTORP (pattern)
+	? Vtreesit_str_close_bracket : Vtreesit_str_close_paren;
     return concat3 (opening_delimeter,
 		    Fmapconcat (Qtreesit_pattern_expand,
 				pattern,
 				Vtreesit_str_space),
 		    closing_delimiter);
+    }
   if (STRINGP (pattern))
     return treesit_query_string_string (pattern);
 
@@ -2414,6 +2424,7 @@ DEFUN ("treesit-query-expand",
     (TYPE PATTERN...)
     [PATTERN...]
     FIELD-NAME:
+    (@ CAPTURE-NAME)
     @CAPTURE-NAME
     (_)
     _
@@ -4101,4 +4112,6 @@ syms_of_treesit (void)
   defsubr (&Streesit_subtree_stat);
 #endif /* HAVE_TREE_SITTER */
   defsubr (&Streesit_available_p);
+
+  DEFSYM (Qat, "@");
 }

[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Yuan Fu]

Mattias Engdegård <mattias.engdegard@gmail.com> writes:

>> Maybe we can introduce (@ NAME) as alternative syntax?
>
> Here is a proof-of-concept (missing tests and documentation), but it works and I think it's a definite improvement.
>
> Maybe we can get a blessing from Yuan Fu. To recap, this permits the
> syntax (@ symbol-name) as an alternative to @symbol-name in treesit
> queries because that allows symbol searching to match the symbol-name,
> and this can be really helpful.

I want to point out that a) this will add complexity and make the syntax
harder to read and learn (however minuscule the impact might be); b) is
not enough by itself to make isearch symbol work with capture names:
whoever write the code must _use_ this syntax; and c) I don’t think
isearch symbol itself is popular/important enough to justify the change.

Yuan

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Eli Zaretskii]

> Cc: contovob@tcd.ie, 44418@debbugs.gnu.org, larsi@gnus.org,
>  monnier@iro.umontreal.ca
> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 23 Jun 2023 15:39:20 -0700
> 
> 
> Mattias Engdegård <mattias.engdegard@gmail.com> writes:
> 
> >> Maybe we can introduce (@ NAME) as alternative syntax?
> >
> > Here is a proof-of-concept (missing tests and documentation), but it works and I think it's a definite improvement.
> >
> > Maybe we can get a blessing from Yuan Fu. To recap, this permits the
> > syntax (@ symbol-name) as an alternative to @symbol-name in treesit
> > queries because that allows symbol searching to match the symbol-name,
> > and this can be really helpful.
> 
> I want to point out that a) this will add complexity and make the syntax
> harder to read and learn (however minuscule the impact might be); b) is
> not enough by itself to make isearch symbol work with capture names:
> whoever write the code must _use_ this syntax; and c) I don’t think
> isearch symbol itself is popular/important enough to justify the change.

Given what Yuan says, I think we should drop this for now.  Let's
collect some real-world use experience, once Emacs 29 is released,
before planning any such changes.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

24 juni 2023 kl. 00.39 skrev Yuan Fu <casouri@gmail.com>:

> I want to point out that a) this will add complexity and make the syntax
> harder to read and learn (however minuscule the impact might be); b) is
> not enough by itself to make isearch symbol work with capture names:
> whoever write the code must _use_ this syntax; and c) I don’t think
> isearch symbol itself is popular/important enough to justify the change.

Well, you are the treesit maintainer and get to decide, but perhaps I could try to sway your opinion. First note that the change is very small with no increase in complexity -- most treesit users would never bother to learn that there is an alternative @SYMBOL notation in addition to the standard (@ SYMBOL).

Furthermore, the @SYMBOL notation is quite un-Lisp-like and alien to Lisp programmers who definitely tend to make use of symbol search, which is why this bug (about the ,@ prefix) was so annoying: it not only prevented us from effectively finding all occurrences of a symbol, but deprived us of a common way to check that the spelling of name is correct and corresponds to its definition. A pity if we now introduce the same kind of bug again, unforced.

I did a quick translation of some treesit patterns in various language modes from @SYMBOL to (@ SYMBOL) and the result was no less readable. Try it yourself.

Your decision to expose treesit queries as Elisp S-expressions was fortunate, as can be seen by the fact that no Emacs editing mode seems to use the other (string) syntax, and the modes make frequent use of backquote forms to interpolate Lisp values into patterns.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Mattias Engdegård [2023-06-24 12:12 +0200] wrote:

> Furthermore, the @SYMBOL notation is quite un-Lisp-like and alien to Lisp
> programmers who definitely tend to make use of symbol search, which is why this
> bug (about the ,@ prefix) was so annoying: it not only prevented us from
> effectively finding all occurrences of a symbol, but deprived us of a common way
> to check that the spelling of name is correct and corresponds to its
> definition. A pity if we now introduce the same kind of bug again, unforced.

FWIW, as a workaround I'm trying to use 'M-s M-.' in place of 'M-s .'.
I may eventually be compelled to customise
isearch-forward-thing-at-point with a custom 'thing' specifically for
@-prefixed Elisp symbols.

-- 
Basil

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Yuan Fu]

> On Jun 24, 2023, at 3:12 AM, Mattias Engdegård <mattias.engdegard@gmail.com> wrote:
> 
> 24 juni 2023 kl. 00.39 skrev Yuan Fu <casouri@gmail.com>:
> 
>> I want to point out that a) this will add complexity and make the syntax
>> harder to read and learn (however minuscule the impact might be); b) is
>> not enough by itself to make isearch symbol work with capture names:
>> whoever write the code must _use_ this syntax; and c) I don’t think
>> isearch symbol itself is popular/important enough to justify the change.
> 
> Well, you are the treesit maintainer and get to decide, but perhaps I could try to sway your opinion. First note that the change is very small with no increase in complexity -- most treesit users would never bother to learn that there is an alternative @SYMBOL notation in addition to the standard (@ SYMBOL).
> 
> Furthermore, the @SYMBOL notation is quite un-Lisp-like and alien to Lisp programmers who definitely tend to make use of symbol search, which is why this bug (about the ,@ prefix) was so annoying: it not only prevented us from effectively finding all occurrences of a symbol, but deprived us of a common way to check that the spelling of name is correct and corresponds to its definition. A pity if we now introduce the same kind of bug again, unforced.
> 
> I did a quick translation of some treesit patterns in various language modes from @SYMBOL to (@ SYMBOL) and the result was no less readable. Try it yourself.
> 
> Your decision to expose treesit queries as Elisp S-expressions was fortunate, as can be seen by the fact that no Emacs editing mode seems to use the other (string) syntax, and the modes make frequent use of backquote forms to interpolate Lisp values into patterns.

Sorry for taking so long, I never decided against other people for important things in Emacs, and I certainly don’t want to make a wrong decision and regret it later, so I struggled a bit and kept procrastinating 😃

I agree that it’s easy to figure out what does the detached @ syntax mean. I mainly just don’t like having two ways to do the same thing.

Since I don’t use search symbol myself, I can’t grasp the importance of it. You apparently care about it enough to go this far, and I’m willing to take your word for it.

You mentioned sexp queries, which reminded me that using the detached syntax would allow people to interpolate capture names with quasi quotes. That’s a plus for the detached syntax.

Ultimately I’m still on the fence for this. But if you still want to add it, please go ahead :-)

Yuan

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Eli Zaretskii]

> Cc: Basil Contovounesios <contovob@tcd.ie>, 44418@debbugs.gnu.org,
>  Lars Ingebrigtsen <larsi@gnus.org>, monnier@iro.umontreal.ca
> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 8 Jul 2023 16:48:57 -0700
> 
> I agree that it’s easy to figure out what does the detached @ syntax mean. I mainly just don’t like having two ways to do the same thing.
> 
> Since I don’t use search symbol myself, I can’t grasp the importance of it. You apparently care about it enough to go this far, and I’m willing to take your word for it.
> 
> You mentioned sexp queries, which reminded me that using the detached syntax would allow people to interpolate capture names with quasi quotes. That’s a plus for the detached syntax.
> 
> Ultimately I’m still on the fence for this. But if you still want to add it, please go ahead :-)

I'd like to hear Stefan's opinion on this, before we make such a
change (if we make it).

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> I agree that it’s easy to figure out what does the detached @ syntax
>> mean. I mainly just don’t like having two ways to do the same thing.
>> 
>> Since I don’t use search symbol myself, I can’t grasp the importance of
>> it. You apparently care about it enough to go this far, and I’m willing to
>> take your word for it.
>> 
>> You mentioned sexp queries, which reminded me that using the detached
>> syntax would allow people to interpolate capture names with quasi
>> quotes. That’s a plus for the detached syntax.
>> 
>> Ultimately I’m still on the fence for this. But if you still want to add it, please go ahead :-)
>
> I'd like to hear Stefan's opinion on this, before we make such a
> change (if we make it).

Yuan knows this area much better than I do, and I fully trust his
judgment on this.

The only I'd point out is that if we introduce a (@ ...) syntax here
we should try and phase out the old @... syntax, i.e. emit an
obsolescence warning when we bump into such code.


        Stefan

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Dmitry Gutov]

On 09/07/2023 15:36, Stefan Monnier via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> The only I'd point out is that if we introduce a (@ ...) syntax here
> we should try and phase out the old @... syntax, i.e. emit an
> obsolescence warning when we bump into such code.

IMO that's indeed the biggest problem with the proposal: if we could 
swap @ in the current syntax for something else, that might be ideal.

But as proposed (and at this stage of Emacs 29 development) we'll likely 
carry two different syntaxes for a number of years, and only one of them 
will show up in symbol searches (such as xref-find-references or isearch 
for "\\_<foo"). Having these methods work in some cases can discourage 
the users from trying other methods, or simply searching for input 
prefixed with "@".

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Mattias Engdegård]

9 juli 2023 kl. 01.48 skrev Yuan Fu <casouri@gmail.com>:

> Sorry for taking so long, I never decided against other people for important things in Emacs, and I certainly don’t want to make a wrong decision and regret it later, so I struggled a bit and kept procrastinating

No, I'm the one to apologise here: in no way did I intend to heap pressure on you.

The native tree-sitter query syntax is actually not using S-expressions at all -- the parsing is completely ad-hoc, and the query data is constructed directly during parsing.

But it looks very much like S-expressions and that's the problem. If the syntax had been less Lisp-like then we would have been able to choose our own S-expressions freely and all would be well, but now it's a bit too close so that it's really tempting to use something with almost identical printed appearance. I think you have done a fine job under the circumstances.

> Since I don’t use search symbol myself, I can’t grasp the importance of it. You apparently care about it enough to go this far, and I’m willing to take your word for it.

It probably isn't much of a problem in practice. I just thought that it would be a pity to forego the opportunity to deal with it from the start, but I definitely won't complain if you keep it the way it is.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Mattias Engdegård [2023-07-12 16:46 +0200] wrote:

> But it looks very much like S-expressions and that's the problem. If the syntax
> had been less Lisp-like then we would have been able to choose our own
> S-expressions freely and all would be well, but now it's a bit too close so that
> it's really tempting to use something with almost identical printed
> appearance. I think you have done a fine job under the circumstances.

Agreed.

>> Since I don’t use search symbol myself, I can’t grasp the importance of
>> it. You apparently care about it enough to go this far, and I’m willing to
>> take your word for it.
>
> It probably isn't much of a problem in practice. I just thought that it would be
> a pity to forego the opportunity to deal with it from the start, but I
> definitely won't complain if you keep it the way it is.

And agreed.  I wouldn't be too confident straying far from the familiar
upstream syntax either.

FWIW, another result of the @-prefix capture syntax in the case of font
lock is that it is a bit harder to detect invalid capture names, such as
function names that are no longer defined:

  :feature 'foo
  :language 'bar
  '((foo @my-baz))

Where my-baz may or may not be defined as a function, and if not is
silently ignored.

There's probably more than one way to warn about such cases in
treesit.el (or catch them with testing), but my current downstream
workaround is:

  (defun my--@ (name) (intern (concat "@" (symbol-name name))))

  (defvar my--font-lock-rules
    (let ((baz (my--@ #'my-baz)))
      `( :feature foo
         :language bar
         ((foo ,baz)))))

Sadly I couldn't find a convenient compile-time way of achieving the
same (the byte-compiler doesn't see the arguments to inline functions,
and my-baz isn't known to be defined at macroexpansion time).

-- 
Basil

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Dmitry Gutov]

On 12/07/2023 18:26, Basil Contovounesios via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Where my-baz may or may not be defined as a function, and if not is
> silently ignored.

@captures can also be face names directly, not always functions.

bug#44418: 28.0.50; Spliced variable not matched as symbol in isearch

[Basil Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov [2023-07-18 04:54 +0300] wrote:

> On 12/07/2023 18:26, Basil Contovounesios via Bug reports for GNU Emacs, the
> Swiss army knife of text editors wrote:
>> Where my-baz may or may not be defined as a function, and if not is
>> silently ignored.
>
> @captures can also be face names directly, not always functions.

Of course, that's why I said:

> it is a bit harder to detect invalid capture names, such as
                                                      ^^^^^^^
> function names that are no longer defined

-- 
Basil