bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eric Gillespie]

1. emacs -Q Cargo.toml
2. emacs -Q Dockerfile

Expected results:
Files opened without fanfare, as emacs always faithfully did for me
these last 25 years.

Actual results:
In both cases, a *Warnings* buffer pops up noisily occupying half the
screen and showing a bright red emoji saying:

	Warning (treesit): Cannot activate tree-sitter, because
	tree-sitter library is not compiled with Emacs

In GNU Emacs 29.0.60 (build 5, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0) of 2023-01-03 built on dogato
Repository revision: da69f116bfc37d28942ca6d35eaff978e6805bac
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: openSUSE Leap 15.4

Configured using:
 'configure -C --prefix=/home/epg/.local --disable-silent-rules
 --without-libsystemd --without-xft --without-m17n-flt
 --without-toolkit-scroll-bars --without-gpm --without-dbus
 --without-gsettings --without-selinux --without-compress-install
 --with-x CC=gcc-11'

Configured features:
CAIRO FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ JPEG JSON LIBXML2 MODULES
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF WEBP
X11 XDBE XIM XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Dockerfile

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
dockerfile-ts-mode vc-git diff-mode easy-mmode vc-dispatcher warnings
icons toml-ts-mode treesit cl-seq cl-loaddefs cl-lib rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads inotify
dynamic-setting font-render-setting cairo x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 45439 9825)
 (symbols 48 5990 0)
 (strings 32 17025 1792)
 (string-bytes 1 480690)
 (vectors 16 11015)
 (vector-slots 8 167209 16484)
 (floats 8 28 39)
 (intervals 56 302 0)
 (buffers 976 13))

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

tags 60559 notabug wontfix
thanks

> From: Eric Gillespie <epg@pretzelnet.org>
> Date: Wed, 04 Jan 2023 08:48:43 -0600
> 
> 1. emacs -Q Cargo.toml
> 2. emacs -Q Dockerfile
> 
> Expected results:
> Files opened without fanfare, as emacs always faithfully did for me
> these last 25 years.
> 
> Actual results:
> In both cases, a *Warnings* buffer pops up noisily occupying half the
> screen and showing a bright red emoji saying:
> 
> 	Warning (treesit): Cannot activate tree-sitter, because
> 	tree-sitter library is not compiled with Emacs

This is not a bug.  Emacs 29 comes with major modes for TOML files and
for Dockerfiles, and those new major modes require that you build
Emacs with the tree-sitter library (and install the corresponding
parser grammar libraries).  If you don't want to do that, you can
instead customize auto-mode-alist to make Emacs use Fundamental mode
(or any other mode you like) for these two file types.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>
> This is not a bug.  Emacs 29 comes with major modes for TOML files and
> for Dockerfiles, and those new major modes require that you build
> Emacs with the tree-sitter library (and install the corresponding
> parser grammar libraries).  If you don't want to do that, you can
> instead customize auto-mode-alist to make Emacs use Fundamental mode
> (or any other mode you like) for these two file types.

Isn't this another manifestation of what's being discussed in bug#60511?
I also think that treesit-ready-p should not emit a warning by default,
it's too noisy and confusing for people that don't know about
Tree-sitter and consequently use a build of Emacs 29 without Tree-sitter
enabled.

Also, it's strange that, even if Tree-sitter is not configured at all, a
major mode whose description mentions Tree-sitter is apparently loaded.
The user will naturally doubt if the major mode works at all or not.
IMHO, Tree-sitter-only modes should not change the major mode if
treesit-ready-p returns nil.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Daniel Martín <mardani29@yahoo.es>
> Cc: Eric Gillespie <epg@pretzelnet.org>,  60559@debbugs.gnu.org
> Date: Wed, 04 Jan 2023 20:40:24 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >
> > This is not a bug.  Emacs 29 comes with major modes for TOML files and
> > for Dockerfiles, and those new major modes require that you build
> > Emacs with the tree-sitter library (and install the corresponding
> > parser grammar libraries).  If you don't want to do that, you can
> > instead customize auto-mode-alist to make Emacs use Fundamental mode
> > (or any other mode you like) for these two file types.
> 
> Isn't this another manifestation of what's being discussed in bug#60511?
> I also think that treesit-ready-p should not emit a warning by default,
> it's too noisy and confusing for people that don't know about
> Tree-sitter and consequently use a build of Emacs 29 without Tree-sitter
> enabled.

That is one use case.  The other, no less important one, is when the
user thinks his/her Emacs is built with tree-sitter and has the
relevant stuff installed, whereas the reality is different.  Silently
doing nothing in that case is hardly TRT.

So if we want to solve this, we need to find a solution that fits both
of these use cases.

> Also, it's strange that, even if Tree-sitter is not configured at all, a
> major mode whose description mentions Tree-sitter is apparently loaded.

I don't find it strange at all.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Martín <mardani29@yahoo.es>
>> Cc: Eric Gillespie <epg@pretzelnet.org>,  60559@debbugs.gnu.org
>> Date: Wed, 04 Jan 2023 20:40:24 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >
>> > This is not a bug.  Emacs 29 comes with major modes for TOML files and
>> > for Dockerfiles, and those new major modes require that you build
>> > Emacs with the tree-sitter library (and install the corresponding
>> > parser grammar libraries).  If you don't want to do that, you can
>> > instead customize auto-mode-alist to make Emacs use Fundamental mode
>> > (or any other mode you like) for these two file types.
>> 
>> Isn't this another manifestation of what's being discussed in bug#60511?
>> I also think that treesit-ready-p should not emit a warning by default,
>> it's too noisy and confusing for people that don't know about
>> Tree-sitter and consequently use a build of Emacs 29 without Tree-sitter
>> enabled.
>
> That is one use case.  The other, no less important one, is when the
> user thinks his/her Emacs is built with tree-sitter and has the
> relevant stuff installed, whereas the reality is different.  Silently
> doing nothing in that case is hardly TRT.
>
> So if we want to solve this, we need to find a solution that fits both
> of these use cases.
>
>> Also, it's strange that, even if Tree-sitter is not configured at all, a
>> major mode whose description mentions Tree-sitter is apparently loaded.
>
> I don't find it strange at all.

Slightly unrelated to this bug.

I think we will have to find a way to bundle certain tree-sitter
grammars with Emacs.  On Android, system policy prohibits programs from
loading dynamic libraries that are not installed by the system in a read
only location upon installation of the application package itself, so
asking users to build those grammars will not work at all.

Android users typically have no C compiler installed either.

Thanks.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> Cc: 60559@debbugs.gnu.org, epg@pretzelnet.org
> Date: Wed, 04 Jan 2023 22:03:12 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Daniel Martín <mardani29@yahoo.es>
> > Cc: Eric Gillespie <epg@pretzelnet.org>,  60559@debbugs.gnu.org
> > Date: Wed, 04 Jan 2023 20:40:24 +0100
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > >
> > > This is not a bug.  Emacs 29 comes with major modes for TOML files and
> > > for Dockerfiles, and those new major modes require that you build
> > > Emacs with the tree-sitter library (and install the corresponding
> > > parser grammar libraries).  If you don't want to do that, you can
> > > instead customize auto-mode-alist to make Emacs use Fundamental mode
> > > (or any other mode you like) for these two file types.
> > 
> > Isn't this another manifestation of what's being discussed in bug#60511?
> > I also think that treesit-ready-p should not emit a warning by default,
> > it's too noisy and confusing for people that don't know about
> > Tree-sitter and consequently use a build of Emacs 29 without Tree-sitter
> > enabled.
> 
> That is one use case.  The other, no less important one, is when the
> user thinks his/her Emacs is built with tree-sitter and has the
> relevant stuff installed, whereas the reality is different.  Silently
> doing nothing in that case is hardly TRT.
> 
> So if we want to solve this, we need to find a solution that fits both
> of these use cases.

So here's a suggestion for such a solution: we make all the
*-ts-mode's optional.  That is, we don't add any of them to
auto-mode-alist unless the file *-ts-mode.el is loaded, and we
document them all in NEWS and the user manual as optional.  users who
want them will have to manually activate them.  This way, the original
use case that started this bug report is automatically solved, and the
other use case, where the user intends to activate one of these modes,
is also served by showing the warning, which in that case is perfectly
justified: the user asked for something that we cannot do, so we warn
him/her.

This is a retreat of sorts, but I think it strikes a better balance
wrt user expectations, assuming not everyone will build with
tree-sitter.

Any comments or objections?  If not, I will do this in a couple of
days on the emacs-29 branch.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> So here's a suggestion for such a solution: we make all the
> *-ts-mode's optional.  That is, we don't add any of them to
> auto-mode-alist unless the file *-ts-mode.el is loaded

What if they are auto-loaded?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Fu Yuan <casouri@gmail.com>, Stefan Monnier <monnier@iro.umontreal.ca>,
>  Lars Ingebrigtsen <larsi@gnus.org>, Theodor Thornhill <theo@thornhill.no>,
>  Jostein Kjønigsen <jostein@secure.kjonigsen.net>,
>  60559@debbugs.gnu.org,
>  epg@pretzelnet.org, mardani29@yahoo.es
> Date: Thu, 05 Jan 2023 07:06:24 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So here's a suggestion for such a solution: we make all the
> > *-ts-mode's optional.  That is, we don't add any of them to
> > auto-mode-alist unless the file *-ts-mode.el is loaded
> 
> What if they are auto-loaded?

Autoloaded how?  IOW, what variable or function is autoloaded and
causes these modes to be automatically loaded?  And where do we have
or could have such code?

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> What if they are auto-loaded?
>
> Autoloaded how?  IOW, what variable or function is autoloaded and
> causes these modes to be automatically loaded?  And where do we have
> or could have such code?

Auto-completion.

1. emacs -Q
2. M-: (featurep 'go-ts-mode)  ; => nil
3. <F1> f go-ts- <TAB> mode <RET>
4. M-:  (featurep 'go-ts-mode)  ; => t

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: casouri@gmail.com, monnier@iro.umontreal.ca, larsi@gnus.org,
>  theo@thornhill.no, jostein@secure.kjonigsen.net, 60559@debbugs.gnu.org,
>  epg@pretzelnet.org, mardani29@yahoo.es
> Date: Thu, 05 Jan 2023 08:00:09 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> What if they are auto-loaded?
> >
> > Autoloaded how?  IOW, what variable or function is autoloaded and
> > causes these modes to be automatically loaded?  And where do we have
> > or could have such code?
> 
> Auto-completion.
> 
> 1. emacs -Q
> 2. M-: (featurep 'go-ts-mode)  ; => nil
> 3. <F1> f go-ts- <TAB> mode <RET>
> 4. M-:  (featurep 'go-ts-mode)  ; => t

I cannot reproduce this with toml-ts-mode on the emacs-29 branch.
go-ts-mode is still the default for Go files, but toml-ts-mode is not
the default for TOML files.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Auto-completion.
>> 
>> 1. emacs -Q
>> 2. M-: (featurep 'go-ts-mode)  ; => nil
>> 3. <F1> f go-ts- <TAB> mode <RET>
>> 4. M-:  (featurep 'go-ts-mode)  ; => t
>
> I cannot reproduce this with toml-ts-mode on the emacs-29 branch.
> go-ts-mode is still the default for Go files, but toml-ts-mode is not
> the default for TOML files.

I just checked, and I cannot reproduce on emacs-29 branch with both
go-ts-mode and toml-ts-mode. <TAB> in (3) simply don't trigger
completion (and thus no auto-loading). Presumably because auto-loading
on completion is not yet available.

The above steps only work on master.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> The above steps only work on master.

That said, is the approach when require produce side effects a good
approach? I have been told in another thread that side effects during
loading should be avoided if possible - they may create surprises to
users.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: casouri@gmail.com, monnier@iro.umontreal.ca, larsi@gnus.org,
>  theo@thornhill.no, jostein@secure.kjonigsen.net, 60559@debbugs.gnu.org,
>  epg@pretzelnet.org, mardani29@yahoo.es
> Date: Thu, 05 Jan 2023 08:42:31 +0000
> 
> Ihor Radchenko <yantar92@posteo.net> writes:
> 
> > The above steps only work on master.
> 
> That said, is the approach when require produce side effects a good
> approach? I have been told in another thread that side effects during
> loading should be avoided if possible - they may create surprises to
> users.

Yes, this "should be avoided if possible".  So if someone shows a way
of doing this some other way, but without adding any additional
complexity wrt what the user should do to activate these modes, I'm
all ears.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: casouri@gmail.com, monnier@iro.umontreal.ca, larsi@gnus.org,
>  theo@thornhill.no, jostein@secure.kjonigsen.net, 60559@debbugs.gnu.org,
>  epg@pretzelnet.org, mardani29@yahoo.es
> Date: Thu, 05 Jan 2023 08:40:33 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Auto-completion.
> >> 
> >> 1. emacs -Q
> >> 2. M-: (featurep 'go-ts-mode)  ; => nil
> >> 3. <F1> f go-ts- <TAB> mode <RET>
> >> 4. M-:  (featurep 'go-ts-mode)  ; => t
> >
> > I cannot reproduce this with toml-ts-mode on the emacs-29 branch.
> > go-ts-mode is still the default for Go files, but toml-ts-mode is not
> > the default for TOML files.
> 
> I just checked, and I cannot reproduce on emacs-29 branch with both
> go-ts-mode and toml-ts-mode. <TAB> in (3) simply don't trigger
> completion (and thus no auto-loading). Presumably because auto-loading
> on completion is not yet available.
> 
> The above steps only work on master.

Which change on master has this effect?  I cannot find anything in
NEWS.

Moreover, doing the above on master with c-ts-mode and toml-ts-mode I
cannot reproduce the behavior you describe.  Only go-ts-mode does
reproduce it.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> The above steps only work on master.
>
> Which change on master has this effect?  I cannot find anything in
> NEWS.

I am referring to `help-enable-autoload'.
I think there were some fixes or improvement to `help-enable-autoload'
behaviour recently.

> Moreover, doing the above on master with c-ts-mode and toml-ts-mode I
> cannot reproduce the behavior you describe.  Only go-ts-mode does
> reproduce it.

I confirm. I don't know why. AFAIU, `help-enable-autoload' should
trigger autoloading in all three cases according to its docstring.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Jostein Kjønigsen]

On 05.01.2023 07:51, Eli Zaretskii wrote:
>
> So here's a suggestion for such a solution: we make all the
> *-ts-mode's optional.  That is, we don't add any of them to
> auto-mode-alist unless the file *-ts-mode.el is loaded

So if I understand this correctly, the change is simply that people 
wanting to use these modes now needs to do two things instead of one:

1. build with tree-sitter support (already a requirement)

2. will have to (require) the modes they want to use in their init.el 
(the change)

Have I understood things correctly? If so, that sounds perfectly 
reasonable to me.

--
Jostein

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> Date: Thu, 5 Jan 2023 08:57:05 +0100
> Cc: 60559@debbugs.gnu.org, mardani29@yahoo.es, epg@pretzelnet.org
> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
> 
> On 05.01.2023 07:51, Eli Zaretskii wrote:
> >
> > So here's a suggestion for such a solution: we make all the
> > *-ts-mode's optional.  That is, we don't add any of them to
> > auto-mode-alist unless the file *-ts-mode.el is loaded
> 
> So if I understand this correctly, the change is simply that people 
> wanting to use these modes now needs to do two things instead of one:
> 
> 1. build with tree-sitter support (already a requirement)
> 
> 2. will have to (require) the modes they want to use in their init.el 
> (the change)
> 
> Have I understood things correctly?

Yes, that's the idea.  We could also make the mode command be
autoloaded, so just "M-x foo-ts-mode RET" the first time will load the
mode, thus avoiding the need for require in the init files.  I guess
there will be users which will prefer one or the other.

> If so, that sounds perfectly reasonable to me.

Thanks.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Jostein Kjønigsen]

On 05.01.2023 09:12, Eli Zaretskii wrote:
>> Date: Thu, 5 Jan 2023 08:57:05 +0100
>> Cc: 60559@debbugs.gnu.org, mardani29@yahoo.es, epg@pretzelnet.org
>> From: Jostein Kjønigsen <jostein@secure.kjonigsen.net>
>>
>>
>> So if I understand this correctly, the change is simply that people
>> wanting to use these modes now needs to do two things instead of one:
>>
>> 1. build with tree-sitter support (already a requirement)
>>
>> 2. will have to (require) the modes they want to use in their init.el
>> (the change)
>>
>> Have I understood things correctly?
> Yes, that's the idea.  We could also make the mode command be
> autoloaded, so just "M-x foo-ts-mode RET" the first time will load the
> mode, thus avoiding the need for require in the init files.  I guess
> there will be users which will prefer one or the other.
>
If we assume that distros are going to be building with 
treesitter-support sooner rather than later, I guess that would be the 
absolutely smoothest way we can have for users to have this working OOB, 
without causing issues for users/distros building without such support.

I definitely support such an approach if it's doable without too much 
effort or risk.

--

Jostein

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Gregory Heytings]

>
> So here's a suggestion for such a solution: we make all the *-ts-mode's 
> optional.  That is, we don't add any of them to auto-mode-alist unless 
> the file *-ts-mode.el is loaded, and we document them all in NEWS and 
> the user manual as optional.  users who want them will have to manually 
> activate them.  This way, the original use case that started this bug 
> report is automatically solved, and the other use case, where the user 
> intends to activate one of these modes, is also served by showing the 
> warning, which in that case is perfectly justified: the user asked for 
> something that we cannot do, so we warn him/her.
>

I suggest a slightly different solution: add a global minor mode, say 
'ts-modes'.  Users would only have to add (ts-modes 1) in their init 
files, and the logic would be:

(treesit-available-p) ts-modes
nil                   nil      => do nothing
nil                   t        => warning
t                     nil      => use available tree-sitter modes
t                     t        => use available tree-sitter modes

This seems more future-proof: otherwise users who want to user the 
available tree-sitter modes would have to require all such modes.  ATM 
there are only 19, but Emacs 30 could very well have twice or thrice that 
number of tree-sitter modes.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Gregory Heytings]

>
> I suggest a slightly different solution: add a global minor mode, say 
> 'ts-modes'.  Users would only have to add (ts-modes 1) in their init 
> files, and the logic would be:
>
> (treesit-available-p) ts-modes
> nil                   nil      => do nothing
> nil                   t        => warning
> t                     nil      => use available tree-sitter modes
> t                     t        => use available tree-sitter modes
>
> This seems more future-proof: otherwise users who want to user the 
> available tree-sitter modes would have to require all such modes.  ATM 
> there are only 19, but Emacs 30 could very well have twice or thrice 
> that number of tree-sitter modes.
>

And perhaps we could even add a third value for ts-modes, say "no".  In 
that case tree-sitter modes would not be used even if they are available. 
Users may prefer, for one reason or another, to not use them even if they 
are available because their system administrator installed them.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> Date: Thu, 05 Jan 2023 08:48:11 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: Fu Yuan <casouri@gmail.com>, 60559@debbugs.gnu.org, 
>     Theodor Thornhill <theo@thornhill.no>, mardani29@yahoo.es, 
>     Jostein Kjønigsen <jostein@secure.kjonigsen.net>, 
>     Stefan Monnier <monnier@iro.umontreal.ca>, 
>     Lars Ingebrigtsen <larsi@gnus.org>, epg@pretzelnet.org
> 
> > I suggest a slightly different solution: add a global minor mode, say 
> > 'ts-modes'.  Users would only have to add (ts-modes 1) in their init 
> > files, and the logic would be:
> >
> > (treesit-available-p) ts-modes
> > nil                   nil      => do nothing
> > nil                   t        => warning
> > t                     nil      => use available tree-sitter modes
> > t                     t        => use available tree-sitter modes
> >
> > This seems more future-proof: otherwise users who want to user the 
> > available tree-sitter modes would have to require all such modes.  ATM 
> > there are only 19, but Emacs 30 could very well have twice or thrice 
> > that number of tree-sitter modes.
> 
> And perhaps we could even add a third value for ts-modes, say "no".  In 
> that case tree-sitter modes would not be used even if they are available. 
> Users may prefer, for one reason or another, to not use them even if they 
> are available because their system administrator installed them.

We discussed a minor mode approach, and discovered it doesn't work
well enough, because different modes need different considerations.

Moreover, silently doing something depending on whether
treesit-available-p returns nil is not the best UX, as I explained
up-thread.  Actually loading a mode that the user wants is much more
natural and easy.

P.S. Please disregard the previous email about customizing a list: I
was confused when I wrote it.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> Date: Thu, 05 Jan 2023 08:44:30 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: Fu Yuan <casouri@gmail.com>, Stefan Monnier <monnier@iro.umontreal.ca>, 
>     Lars Ingebrigtsen <larsi@gnus.org>, Theodor Thornhill <theo@thornhill.no>, 
>     Jostein Kjønigsen <jostein@secure.kjonigsen.net>, 
>     60559@debbugs.gnu.org, epg@pretzelnet.org, mardani29@yahoo.es
> 
> I suggest a slightly different solution: add a global minor mode, say 
> 'ts-modes'.  Users would only have to add (ts-modes 1) in their init 
> files, and the logic would be:
> 
> (treesit-available-p) ts-modes
> nil                   nil      => do nothing
> nil                   t        => warning
> t                     nil      => use available tree-sitter modes
> t                     t        => use available tree-sitter modes
> 
> This seems more future-proof: otherwise users who want to user the 
> available tree-sitter modes would have to require all such modes.  ATM 
> there are only 19, but Emacs 30 could very well have twice or thrice that 
> number of tree-sitter modes.

Thanks.  This was already considered.  I don't like asking users to
customize a list as a means of activating an optional mode.  I also
don't think we have enough information for committing ourselves to
such a solution at this stage.

So I'd prefer not to do this on the release branch.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Dmitry Gutov]

On 05/01/2023 08:51, Eli Zaretskii wrote:
> So here's a suggestion for such a solution: we make all the
> *-ts-mode's optional.  That is, we don't add any of them to
> auto-mode-alist unless the file *-ts-mode.el is loaded, and we
> document them all in NEWS and the user manual as optional.  users who
> want them will have to manually activate them.  This way, the original
> use case that started this bug report is automatically solved, and the
> other use case, where the user intends to activate one of these modes,
> is also served by showing the warning, which in that case is perfectly
> justified: the user asked for something that we cannot do, so we warn
> him/her.

Personally, I'd rather the users be made aware that they can install 
such-and-such grammars and enjoy good support for the file types they 
just visited.

I understand that it can be a bother for people who are used to just 
editing such files using fundamental-mode, but they could similarly 
complain if we added a major mode (for a given file type) that is 
functional but still different from the "last 25 years".

So I agree with your original assessment that it's not a bug.

What we could do here, though, to reduce the aggravation:

- Move the warning to *Messages*. It will be slightly less noticeable, 
but won't occupy "half the screen" either.

and/or

- Add a new defcustom which would allow people to silence the 
tree-sitter related warnings altogether. tree-sitter major modes without 
available grammars work more or less like fundamental-mode.

Or we can just recommend such users edit their auto-mode-alist to 
associate fundamental-mode with those file types. That only takes a few 
minutes.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> So here's a suggestion for such a solution: we make all the
> *-ts-mode's optional.  That is, we don't add any of them to
> auto-mode-alist unless the file *-ts-mode.el is loaded, and we
> document them all in NEWS and the user manual as optional.  users who
> want them will have to manually activate them.  This way, the original
> use case that started this bug report is automatically solved, and the
> other use case, where the user intends to activate one of these modes,
> is also served by showing the warning, which in that case is perfectly
> justified: the user asked for something that we cannot do, so we warn
> him/her.
>
> This is a retreat of sorts, but I think it strikes a better balance
> wrt user expectations, assuming not everyone will build with
> tree-sitter.
>
> Any comments or objections?  If not, I will do this in a couple of
> days on the emacs-29 branch.

Sounds good to me :-)

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Stephen Berman]

On Wed, 04 Jan 2023 20:36:15 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

> tags 60559 notabug wontfix
> thanks
>
>> From: Eric Gillespie <epg@pretzelnet.org>
>> Date: Wed, 04 Jan 2023 08:48:43 -0600
>>
>> 1. emacs -Q Cargo.toml
>> 2. emacs -Q Dockerfile
>>
>> Expected results:
>> Files opened without fanfare, as emacs always faithfully did for me
>> these last 25 years.
>>
>> Actual results:
>> In both cases, a *Warnings* buffer pops up noisily occupying half the
>> screen and showing a bright red emoji saying:
>>
>> 	Warning (treesit): Cannot activate tree-sitter, because
>> 	tree-sitter library is not compiled with Emacs
>
> This is not a bug.  Emacs 29 comes with major modes for TOML files and
> for Dockerfiles, and those new major modes require that you build
> Emacs with the tree-sitter library (and install the corresponding
> parser grammar libraries).  If you don't want to do that, you can
> instead customize auto-mode-alist to make Emacs use Fundamental mode
> (or any other mode you like) for these two file types.

Emacs 28 already has conf-toml-mode for TOML files, so shouldn't
toml-ts-mode be handled like e.g. c-ts-mode instead of being added to
auto-mode-alist? (The default value of that variable in Emacs 29 still
contains an entry for conf-toml-mode, lower down than the entry for
toml-ts-mode.)

Steve Berman

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: Eric Gillespie <epg@pretzelnet.org>,  60559@debbugs.gnu.org
> Date: Wed, 04 Jan 2023 20:59:04 +0100
> 
> >> 	Warning (treesit): Cannot activate tree-sitter, because
> >> 	tree-sitter library is not compiled with Emacs
> >
> > This is not a bug.  Emacs 29 comes with major modes for TOML files and
> > for Dockerfiles, and those new major modes require that you build
> > Emacs with the tree-sitter library (and install the corresponding
> > parser grammar libraries).  If you don't want to do that, you can
> > instead customize auto-mode-alist to make Emacs use Fundamental mode
> > (or any other mode you like) for these two file types.
> 
> Emacs 28 already has conf-toml-mode for TOML files, so shouldn't
> toml-ts-mode be handled like e.g. c-ts-mode instead of being added to
> auto-mode-alist? (The default value of that variable in Emacs 29 still
> contains an entry for conf-toml-mode, lower down than the entry for
> toml-ts-mode.)

We could indeed do that for TOML files, but I believe the problem
raised here is more general.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Stephen Berman]

On Wed, 04 Jan 2023 22:10:47 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: Eric Gillespie <epg@pretzelnet.org>,  60559@debbugs.gnu.org
>> Date: Wed, 04 Jan 2023 20:59:04 +0100
>>
>> >> 	Warning (treesit): Cannot activate tree-sitter, because
>> >> 	tree-sitter library is not compiled with Emacs
>> >
>> > This is not a bug.  Emacs 29 comes with major modes for TOML files and
>> > for Dockerfiles, and those new major modes require that you build
>> > Emacs with the tree-sitter library (and install the corresponding
>> > parser grammar libraries).  If you don't want to do that, you can
>> > instead customize auto-mode-alist to make Emacs use Fundamental mode
>> > (or any other mode you like) for these two file types.
>>
>> Emacs 28 already has conf-toml-mode for TOML files, so shouldn't
>> toml-ts-mode be handled like e.g. c-ts-mode instead of being added to
>> auto-mode-alist? (The default value of that variable in Emacs 29 still
>> contains an entry for conf-toml-mode, lower down than the entry for
>> toml-ts-mode.)
>
> We could indeed do that for TOML files, but I believe the problem
> raised here is more general.

That may be, but the TOML case appears to be a regression, since
visiting a TOML file in Emacs 28 displays various faces but visiting a
TOML file in Emacs 29 without having the tree-sitter TOML grammar
library installed displays only the default face.  But if fixing the
more general problem in Emacs 29 also fixes this apparent regression,
then of course that's better than an ad hoc fix.

Steve Berman

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: epg@pretzelnet.org,  60559@debbugs.gnu.org
> Date: Wed, 04 Jan 2023 21:33:38 +0100
> 
> On Wed, 04 Jan 2023 22:10:47 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
> 
> >> From: Stephen Berman <stephen.berman@gmx.net>
> >> Cc: Eric Gillespie <epg@pretzelnet.org>,  60559@debbugs.gnu.org
> >> Date: Wed, 04 Jan 2023 20:59:04 +0100
> >>
> >> >> 	Warning (treesit): Cannot activate tree-sitter, because
> >> >> 	tree-sitter library is not compiled with Emacs
> >> >
> >> > This is not a bug.  Emacs 29 comes with major modes for TOML files and
> >> > for Dockerfiles, and those new major modes require that you build
> >> > Emacs with the tree-sitter library (and install the corresponding
> >> > parser grammar libraries).  If you don't want to do that, you can
> >> > instead customize auto-mode-alist to make Emacs use Fundamental mode
> >> > (or any other mode you like) for these two file types.
> >>
> >> Emacs 28 already has conf-toml-mode for TOML files, so shouldn't
> >> toml-ts-mode be handled like e.g. c-ts-mode instead of being added to
> >> auto-mode-alist? (The default value of that variable in Emacs 29 still
> >> contains an entry for conf-toml-mode, lower down than the entry for
> >> toml-ts-mode.)
> >
> > We could indeed do that for TOML files, but I believe the problem
> > raised here is more general.
> 
> That may be, but the TOML case appears to be a regression, since
> visiting a TOML file in Emacs 28 displays various faces but visiting a
> TOML file in Emacs 29 without having the tree-sitter TOML grammar
> library installed displays only the default face.

I've now made toml-ts-mode an optional one, leaving conf-toml-mode as
the default.

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Juri Linkov]

>> >> Emacs 28 already has conf-toml-mode for TOML files, so shouldn't
>> >> toml-ts-mode be handled like e.g. c-ts-mode instead of being added to
>> >> auto-mode-alist? (The default value of that variable in Emacs 29 still
>> >> contains an entry for conf-toml-mode, lower down than the entry for
>> >> toml-ts-mode.)
>> >
>> > We could indeed do that for TOML files, but I believe the problem
>> > raised here is more general.
>>
>> That may be, but the TOML case appears to be a regression, since
>> visiting a TOML file in Emacs 28 displays various faces but visiting a
>> TOML file in Emacs 29 without having the tree-sitter TOML grammar
>> library installed displays only the default face.
>
> I've now made toml-ts-mode an optional one, leaving conf-toml-mode as
> the default.

So now non-ts users are satisfied at the expense of ts users.

Why not to do the right thing:

(add-to-list 'auto-mode-alist `("\\.toml\\'" .
  ,(if (treesit-ready-p 'toml) 'toml-ts-mode 'conf-toml-mode)))

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Stephen Berman <stephen.berman@gmx.net>,  60559@debbugs.gnu.org,
>   epg@pretzelnet.org
> Date: Thu, 05 Jan 2023 09:46:48 +0200
> 
> >> >> Emacs 28 already has conf-toml-mode for TOML files, so shouldn't
> >> >> toml-ts-mode be handled like e.g. c-ts-mode instead of being added to
> >> >> auto-mode-alist? (The default value of that variable in Emacs 29 still
> >> >> contains an entry for conf-toml-mode, lower down than the entry for
> >> >> toml-ts-mode.)
> >> >
> >> > We could indeed do that for TOML files, but I believe the problem
> >> > raised here is more general.
> >>
> >> That may be, but the TOML case appears to be a regression, since
> >> visiting a TOML file in Emacs 28 displays various faces but visiting a
> >> TOML file in Emacs 29 without having the tree-sitter TOML grammar
> >> library installed displays only the default face.
> >
> > I've now made toml-ts-mode an optional one, leaving conf-toml-mode as
> > the default.
> 
> So now non-ts users are satisfied at the expense of ts users.

The cost for the latter is simple: require toml-ts-mode in the init
file.

> Why not to do the right thing:
> 
> (add-to-list 'auto-mode-alist `("\\.toml\\'" .
>   ,(if (treesit-ready-p 'toml) 'toml-ts-mode 'conf-toml-mode)))

Because I don't want to risk any breakage by last-minute changes.  Do
we want to release Emacs 29 soon, or do we want to keep developing it
for another year?

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eric Gillespie]

Eli Zaretskii <eliz@gnu.org> writes:

> thanks

You're welcome.

> This is not a bug.

Okay.

> Because I don't want to risk any breakage by last-minute changes.  Do
> we want to release Emacs 29 soon, or do we want to keep developing it
> for another year?

It seems to me the safest approach for a release is the
conservative approach.  The more gradual a change, the less
likely an embarassing release or rushed follow-on patch release.

All this discussion has been interesting, but it seems to me it
is mostly focused on a world where not only tree-sitter is
common, but all those grammar libraries are as well!

I think very few emacs users in 2023 live in that world.

If emacs was configured with tree-sitter, it seems productive to
warn the user when tree-sitter grammars are missing.  It seems
likely that user intended to have tree-sitter.

When emacs is NOT configured with tree-sitter, it seems
counter-productive to warn about missing tree-sitter.

I even pass --without-tree-sitter to configure now.  It seems
particularly surprising to me that I explicitly tell emacs "don't
use tree-sitter" and then it immediately starts complaining to me
that it doesn't have tree-sitter.

Thanks everyone!

bug#60559: 29.0.60; "Cannot activate tree-sitter" spam

[Eli Zaretskii]

> From: Eric Gillespie <epg@pretzelnet.org>
> Date: Thu, 05 Jan 2023 08:59:00 -0600
> Cc: 60559@debbugs.gnu.org, stephen.berman@gmx.net,
>  Juri Linkov <juri@linkov.net>
> 
> All this discussion has been interesting, but it seems to me it
> is mostly focused on a world where not only tree-sitter is
> common, but all those grammar libraries are as well!
> 
> I think very few emacs users in 2023 live in that world.
> 
> If emacs was configured with tree-sitter, it seems productive to
> warn the user when tree-sitter grammars are missing.  It seems
> likely that user intended to have tree-sitter.
> 
> When emacs is NOT configured with tree-sitter, it seems
> counter-productive to warn about missing tree-sitter.
> 
> I even pass --without-tree-sitter to configure now.  It seems
> particularly surprising to me that I explicitly tell emacs "don't
> use tree-sitter" and then it immediately starts complaining to me
> that it doesn't have tree-sitter.

This annoyance should now be fixed on the emacs-29 branch, and I'm
therefore closing this bug (and the two others that were merged to
it).