Cutoff date for adding ruby-ts-mode?

Cutoff date for adding ruby-ts-mode?

[Dmitry Gutov]

Hi all and Eli in particular,

How close are we to "hard freeze" date of Emacs 29, after which no new 
tree-sitter modes should be added?

I'm told the copyright papers might be signed next week.

The code is largely functional: font-lock just needs minor updates; the 
indentation has more problems, but it's still probably better to have 
the new mode in Emacs 29, rather than not.

Also regarding indentation, Ruby community uses a bunch of different 
styles, and it was my impression (could be wrong, though) that Perry 
wanted to at least have ruby-ts-mode be able to use a different style 
than what is currently ruby-mode's default. To that end, he implemented 
a couple of user options.

In bug#60186, I also implement a bunch of options that allow similar 
flexibility to the users of "plain" ruby-mode. I believe rather than 
have different incompatible options, it would be better to unify them 
between the modes, at least where the capabilities match, to use the 
same options.

So... if there is still time to get ruby-ts-mode in (and give it a 
little polish), it might be a good idea to get the patch for bug#60186 
in first. Alternatively, it could come with non-customizable indentation 
(options would be added later), or it could have a set of customization 
points which we'll promptly deprecate right after (ones that will become 
redundant).

Or we defer all that and release both new options in ruby-mode and 
ruby-ts-mode from master to GNU ELPA.

Re: Cutoff date for adding ruby-ts-mode?

[Eli Zaretskii]

> Date: Thu, 29 Dec 2022 23:39:37 +0200
> Cc: Perry Smith <pedz@easesoftware.com>
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> Hi all and Eli in particular,
> 
> How close are we to "hard freeze" date of Emacs 29, after which no new 
> tree-sitter modes should be added?

You have a day or two, and I hope this should be enough for adding new
features.  (If they have still some rough edges, that could be fixed
later on.)

> I'm told the copyright papers might be signed next week.
> 
> The code is largely functional: font-lock just needs minor updates; the 
> indentation has more problems, but it's still probably better to have 
> the new mode in Emacs 29, rather than not.

So I suggest you install that ASAP.

> Also regarding indentation, Ruby community uses a bunch of different 
> styles, and it was my impression (could be wrong, though) that Perry 
> wanted to at least have ruby-ts-mode be able to use a different style 
> than what is currently ruby-mode's default. To that end, he implemented 
> a couple of user options.
> 
> In bug#60186, I also implement a bunch of options that allow similar 
> flexibility to the users of "plain" ruby-mode. I believe rather than 
> have different incompatible options, it would be better to unify them 
> between the modes, at least where the capabilities match, to use the 
> same options.

I agree that having compatible options makes sense, provided that
changes in ruby-mode that affect existing/default indentation style
and font-lock are relatively safe.

> So... if there is still time to get ruby-ts-mode in (and give it a 
> little polish), it might be a good idea to get the patch for bug#60186 
> in first. Alternatively, it could come with non-customizable indentation 
> (options would be added later), or it could have a set of customization 
> points which we'll promptly deprecate right after (ones that will become 
> redundant).
> 
> Or we defer all that and release both new options in ruby-mode and 
> ruby-ts-mode from master to GNU ELPA.

It's up to you.  If the above-mentioned method suits you, we can have
that on the release branch.

Re: Cutoff date for adding ruby-ts-mode?

[Dmitry Gutov]

On 30/12/2022 10:16, Eli Zaretskii wrote:
>> Date: Thu, 29 Dec 2022 23:39:37 +0200
>> Cc: Perry Smith <pedz@easesoftware.com>
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> Hi all and Eli in particular,
>>
>> How close are we to "hard freeze" date of Emacs 29, after which no new
>> tree-sitter modes should be added?
> 
> You have a day or two, and I hope this should be enough for adding new
> features.  (If they have still some rough edges, that could be fixed
> later on.)

Thanks, I'll try to verify now that everything is good with the pieces 
involved and get on with that.

>> I'm told the copyright papers might be signed next week.
>>
>> The code is largely functional: font-lock just needs minor updates; the
>> indentation has more problems, but it's still probably better to have
>> the new mode in Emacs 29, rather than not.
> 
> So I suggest you install that ASAP.

All right.

>> Also regarding indentation, Ruby community uses a bunch of different
>> styles, and it was my impression (could be wrong, though) that Perry
>> wanted to at least have ruby-ts-mode be able to use a different style
>> than what is currently ruby-mode's default. To that end, he implemented
>> a couple of user options.
>>
>> In bug#60186, I also implement a bunch of options that allow similar
>> flexibility to the users of "plain" ruby-mode. I believe rather than
>> have different incompatible options, it would be better to unify them
>> between the modes, at least where the capabilities match, to use the
>> same options.
> 
> I agree that having compatible options makes sense, provided that
> changes in ruby-mode that affect existing/default indentation style
> and font-lock are relatively safe.

The latest revision of the patch seems particularly safe to me (like 
with previous option, no substantially different code is running as long 
as all the options are at their default values). But I'm waiting to hear 
back from Aaron regarding his experience. It has extra regression tests, 
though.

>> So... if there is still time to get ruby-ts-mode in (and give it a
>> little polish), it might be a good idea to get the patch for bug#60186
>> in first. Alternatively, it could come with non-customizable indentation
>> (options would be added later), or it could have a set of customization
>> points which we'll promptly deprecate right after (ones that will become
>> redundant).
>>
>> Or we defer all that and release both new options in ruby-mode and
>> ruby-ts-mode from master to GNU ELPA.
> 
> It's up to you.  If the above-mentioned method suits you, we can have
> that on the release branch.

All right. This was more of a "yes" than I expected. With apologies, let 
me try to push a little further.

Speaking of the method, how do you feel about renaming a file?

If ruby-ts-mode is going to live in the same file as ruby-mode, perhaps 
the more neutral file name ruby.el would be proper? This would be in 
line with the scheme used by js.el and python.el.

This is not an idle lets-decide-it-later question because I want to have 
some later releases of these modes in GNU ELPA, and we'll want to have 
the built-in packages properly shadowed when someone installs a new 
version. (Perhaps Stefan has an advice here, so Cc'd).

I see several (require 'ruby-mode)-s in third-party packages. There are 
not too many of them, I could just go around and ask everyone to replace 
that with (or (require 'ruby nil t) (require 'ruby-mode)). Or we could 
have a compatibility stub in Emacs 29 where ruby-mode.el just calls 
"(require 'ruby) (provide 'ruby-mode)".

We don't often rename files also because of 'git log' problems, but we 
have a decent (and simple enough) plan to improve that in bug#55871.

Alternatively, we break from the common scheme and put the modes in 
separate files, one depending on the other. Then they'll have to be 
separate GNU ELPA packages (I think), and we'll need to synchronously 
bump version and required-version in them every time when making 
interrelated changes in the future.

Re: Cutoff date for adding ruby-ts-mode?

[Eli Zaretskii]

> Date: Fri, 30 Dec 2022 17:32:47 +0200
> Cc: emacs-devel@gnu.org, pedz@easesoftware.com,
>  Stefan Monnier <monnier@IRO.UMontreal.CA>
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> Speaking of the method, how do you feel about renaming a file?
> 
> If ruby-ts-mode is going to live in the same file as ruby-mode, perhaps 
> the more neutral file name ruby.el would be proper? This would be in 
> line with the scheme used by js.el and python.el.

I'd like to avoid any more decisions related to *-ts-mode's, let alone
binding ones.  I'd like to leave that for the future, when we
(hopefully) will have a clearer picture of how best to mix
tree-sitter-supported modes with the traditional ones.

So renaming now doesn't sound like a good idea to me.

> This is not an idle lets-decide-it-later question because I want to have 
> some later releases of these modes in GNU ELPA, and we'll want to have 
> the built-in packages properly shadowed when someone installs a new 
> version. (Perhaps Stefan has an advice here, so Cc'd).
> 
> I see several (require 'ruby-mode)-s in third-party packages. There are 
> not too many of them, I could just go around and ask everyone to replace 
> that with (or (require 'ruby nil t) (require 'ruby-mode)). Or we could 
> have a compatibility stub in Emacs 29 where ruby-mode.el just calls 
> "(require 'ruby) (provide 'ruby-mode)".
> 
> We don't often rename files also because of 'git log' problems, but we 
> have a decent (and simple enough) plan to improve that in bug#55871.
> 
> Alternatively, we break from the common scheme and put the modes in 
> separate files, one depending on the other. Then they'll have to be 
> separate GNU ELPA packages (I think), and we'll need to synchronously 
> bump version and required-version in them every time when making 
> interrelated changes in the future.

I'd prefer this alternative for now (modulo the separate ELPA packages
part, which I leave for you to decide: they could also be a single
package from my POV).  If nothing else, having separate files is
similar to what at least some other *-ts-mode's do.  When we revisit
these issues in the future, we'll be able to make the decisions about
all of them.

Thanks.

Re: Cutoff date for adding ruby-ts-mode?

[Dmitry Gutov]

On 30/12/2022 18:59, Eli Zaretskii wrote:
>> Date: Fri, 30 Dec 2022 17:32:47 +0200
>> Cc: emacs-devel@gnu.org, pedz@easesoftware.com,
>>   Stefan Monnier <monnier@IRO.UMontreal.CA>
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> Speaking of the method, how do you feel about renaming a file?
>>
>> If ruby-ts-mode is going to live in the same file as ruby-mode, perhaps
>> the more neutral file name ruby.el would be proper? This would be in
>> line with the scheme used by js.el and python.el.
> 
> I'd like to avoid any more decisions related to *-ts-mode's, let alone
> binding ones.  I'd like to leave that for the future, when we
> (hopefully) will have a clearer picture of how best to mix
> tree-sitter-supported modes with the traditional ones.
> 
> So renaming now doesn't sound like a good idea to me.

I like to defer decisions too, but this comes with consequences. E.g. if 
we release ELPA packages with certain names, and later change the names, 
people who already installed the previous names will stop getting 
updates, but won't know to switch to the new names. Unless they read 
about it on Reddit or etc.

So it's something to do as rarely as possible.

>> This is not an idle lets-decide-it-later question because I want to have
>> some later releases of these modes in GNU ELPA, and we'll want to have
>> the built-in packages properly shadowed when someone installs a new
>> version. (Perhaps Stefan has an advice here, so Cc'd).
>>
>> I see several (require 'ruby-mode)-s in third-party packages. There are
>> not too many of them, I could just go around and ask everyone to replace
>> that with (or (require 'ruby nil t) (require 'ruby-mode)). Or we could
>> have a compatibility stub in Emacs 29 where ruby-mode.el just calls
>> "(require 'ruby) (provide 'ruby-mode)".
>>
>> We don't often rename files also because of 'git log' problems, but we
>> have a decent (and simple enough) plan to improve that in bug#55871.
>>
>> Alternatively, we break from the common scheme and put the modes in
>> separate files, one depending on the other. Then they'll have to be
>> separate GNU ELPA packages (I think), and we'll need to synchronously
>> bump version and required-version in them every time when making
>> interrelated changes in the future.
> 
> I'd prefer this alternative for now (modulo the separate ELPA packages
> part, which I leave for you to decide: they could also be a single
> package from my POV).  If nothing else, having separate files is
> similar to what at least some other *-ts-mode's do.  When we revisit
> these issues in the future, we'll be able to make the decisions about
> all of them.

Okay.

I wonder what's going to happen if we release Emacs 29 with ruby-mode 
and ruby-ts-mode in separate files, and then later add a package called 
'ruby' to ELPA with ruby.el containing both modes.

Will the autoloads file from an ELPA package always take priority over 
the built-in autoloads? If yes, then we don't have to do anything now. 
Hopefully that's not undefined behavior.

Alternatively, it would be possible to release two files as one ELPA 
package, but it would seem to require a third file: one matching the 
package name, ruby.el.

Re: Cutoff date for adding ruby-ts-mode?

[Eli Zaretskii]

> Date: Fri, 30 Dec 2022 23:42:50 +0200
> Cc: emacs-devel@gnu.org, pedz@easesoftware.com, monnier@IRO.UMontreal.CA
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> >> Alternatively, we break from the common scheme and put the modes in
> >> separate files, one depending on the other. Then they'll have to be
> >> separate GNU ELPA packages (I think), and we'll need to synchronously
> >> bump version and required-version in them every time when making
> >> interrelated changes in the future.
> > 
> > I'd prefer this alternative for now (modulo the separate ELPA packages
> > part, which I leave for you to decide: they could also be a single
> > package from my POV).  If nothing else, having separate files is
> > similar to what at least some other *-ts-mode's do.  When we revisit
> > these issues in the future, we'll be able to make the decisions about
> > all of them.
> 
> Okay.
> 
> I wonder what's going to happen if we release Emacs 29 with ruby-mode 
> and ruby-ts-mode in separate files, and then later add a package called 
> 'ruby' to ELPA with ruby.el containing both modes.

The ELPA package could also include 2 files, one for each mode.

> Will the autoloads file from an ELPA package always take priority over 
> the built-in autoloads? If yes, then we don't have to do anything now. 
> Hopefully that's not undefined behavior.
> 
> Alternatively, it would be possible to release two files as one ELPA 
> package, but it would seem to require a third file: one matching the 
> package name, ruby.el.

I'll leave this part of the issue to Stefan and others who know more
than I do about ELPA packaging.

Re: Cutoff date for adding ruby-ts-mode?

[Stefan Kangas]

Dmitry Gutov <dgutov@yandex.ru> writes:

> I like to defer decisions too, but this comes with consequences. E.g. if
> we release ELPA packages with certain names, and later change the names,
> people who already installed the previous names will stop getting
> updates, but won't know to switch to the new names. Unless they read
> about it on Reddit or etc.

We could make the old packages into transitional packages, that are
basically empty except for a dependency on the new package.  That way,
they'll get the new package installed automatically.  This could
include any necessary compatibility aliases too.

Re: Cutoff date for adding ruby-ts-mode?

[Dmitry Gutov]

On 30/12/2022 10:16, Eli Zaretskii wrote:
>> Date: Thu, 29 Dec 2022 23:39:37 +0200
>> Cc: Perry Smith<pedz@easesoftware.com>
>> From: Dmitry Gutov<dgutov@yandex.ru>
>>
>> Hi all and Eli in particular,
>>
>> How close are we to "hard freeze" date of Emacs 29, after which no new
>> tree-sitter modes should be added?
> You have a day or two, and I hope this should be enough for adding new
> features.  (If they have still some rough edges, that could be fixed
> later on.)
> 
>> I'm told the copyright papers might be signed next week.
>>
>> The code is largely functional: font-lock just needs minor updates; the
>> indentation has more problems, but it's still probably better to have
>> the new mode in Emacs 29, rather than not.
> So I suggest you install that ASAP.
> 

Sorry for the delay, ruby-ts-mode is now in emacs-29.

Some rough edges around indentation remain (to be addressed in the 
coming weeks), but the mode should be useful enough now.

Re: Cutoff date for adding ruby-ts-mode?

[Eli Zaretskii]

> Date: Mon, 2 Jan 2023 03:35:04 +0200
> Cc: emacs-devel@gnu.org, pedz@easesoftware.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> Sorry for the delay, ruby-ts-mode is now in emacs-29.

Thanks.

> Some rough edges around indentation remain (to be addressed in the 
> coming weeks), but the mode should be useful enough now.

Good plan.

Re: Cutoff date for adding ruby-ts-mode?

[Juri Linkov]

>> Sorry for the delay, ruby-ts-mode is now in emacs-29.
>
> Thanks.
>
>> Some rough edges around indentation remain (to be addressed in the
>> coming weeks), but the mode should be useful enough now.
>
> Good plan.

I confirm that everything in ruby-ts-mode works great.
Thanks Perry and Dmitry.  One question is how to share
configuration between ruby-mode and ruby-ts-mode.

I guess since only one of modes makes sense within the
same session then the same hook could be run only
for one of these modes, selecting one on them
with `treesit-available-p' (note that `treesit-ready-p'
can't be used until treesit is loaded):

  (add-hook (if (treesit-available-p) 'ruby-ts-mode-hook 'ruby-mode-hook)
            (lambda ()

              (when (treesit-available-p)
                ;; Copied from `ruby-mode' for `ruby-ts-mode':
                (setq-local outline-regexp (concat "^\\s *"
                                   (regexp-opt '("class" "module" "def"))
                                   "\\_>"))
                (setq-local outline-level (lambda () (1+ (/ (current-indentation)
                                              ruby-indent-level))))
                (add-hook 'flymake-diagnostic-functions #'ruby-flymake-auto nil 'local))

              ;; Other common settings
              ;; ...
  ))

Also the problem is that the default settings seen above
are not shared between modes.

Re: Cutoff date for adding ruby-ts-mode?

[Dmitry Gutov]

On 02/01/2023 20:44, Juri Linkov wrote:
>>> Sorry for the delay, ruby-ts-mode is now in emacs-29.
>>
>> Thanks.
>>
>>> Some rough edges around indentation remain (to be addressed in the
>>> coming weeks), but the mode should be useful enough now.
>>
>> Good plan.
> 
> I confirm that everything in ruby-ts-mode works great.
> Thanks Perry and Dmitry.  One question is how to share
> configuration between ruby-mode and ruby-ts-mode.
> 
> I guess since only one of modes makes sense within the
> same session then the same hook could be run only
> for one of these modes, selecting one on them
> with `treesit-available-p' (note that `treesit-ready-p'
> can't be used until treesit is loaded):
> 
>    (add-hook (if (treesit-available-p) 'ruby-ts-mode-hook 'ruby-mode-hook)
>              (lambda ()
> 
>                (when (treesit-available-p)
>                  ;; Copied from `ruby-mode' for `ruby-ts-mode':
>                  (setq-local outline-regexp (concat "^\\s *"
>                                     (regexp-opt '("class" "module" "def"))
>                                     "\\_>"))
>                  (setq-local outline-level (lambda () (1+ (/ (current-indentation)
>                                                ruby-indent-level))))
>                  (add-hook 'flymake-diagnostic-functions #'ruby-flymake-auto nil 'local))
> 
>                ;; Other common settings
>                ;; ...
>    ))
> 
> Also the problem is that the default settings seen above
> are not shared between modes.

Good question, thanks. I've pushed a change in 6dd3e352f44 that moves 
the common code to a common base mode to derive from.

Those include the settings you quote above, as well as the setup for 
electric-indent-functions and prettify-symbols-alist.

And to answer the question itself, I guess you can use 
ruby-base-mode-hook now.