My resignation from Emacs development

My resignation from Emacs development

[Alan Mackenzie]

Hello, Emacs.

I'm resigning my position as Emacs contributor.

The immediate reason is that, as maintainer of CC Mode, CC Mode's
symbols, its names, were taken by Emacs and used for other purposes
without informing me, much less consulting me.  That makes my position as
CC Mode maintainer here untenable.

Eli Zaretskii and I have had extensive discussions, both in public and in
private email, over the last week or so, but we have been unable to reach
any satisfactory compromise solution.

Names are important.  They have power.  To take somebody's/somthing's
name and misuse it is an exercise of aggression.  Try using "Emacs" or
even "free software" to mean something different, and see just how
quickly you would hear back from Richard Stallman.  This misuse of CC
Mode's "trademarks", the symbols `c-mode', `c++-mode', and perhaps
`c-or-c++-mode', is just such an act of aggression.

These symbols have been appropriated by Emacs to mean "the current
preferred mode for C", etc., rather than C Mode, C++ Mode etc.  In
certain circumstances, in particular, in Local Variables: sections and
auto-mode-alist, there is now no longer any way unambiguously to specify
C Mode or C++ Mode.  Up till recently ("\\.myc\\'" . c-mode) in
auto-mode-alist meant C Mode, and would have had the effect of
auto-loading CC Mode, if needed, and running C Mode.

The change took place in the commit for bug#69191 "New var
`major-mode-remap-defaults`, for packages".  It sounds so innocent, but
is an extremely bad solution for whatever problem (unspecified in the
commit message) it was intended to solve.  A major mode using it changes
the interfaces of other libraries in an uncontrolled way.  This is not
good software engineering.

This bug was raised and committed by Stefan Monnier.  Despite the fact
that the bug fix directly impinged upon CC Mode, and there was even a
change to cc-mode.el in the patch, he failed even to inform me.  The only
two modes substantially affected by this change were ruby-mode and CC
Mode, and it is clear that Dmitry Gutov, maintainer of ruby-mode, was
aware of the change.  Had I known of this proposal, I would certainly
have objected to it.  Stefan is intelligent enough to have realised this,
and maybe his avoidance of open discussion was motivated by this.

Bug#69191 was a big change.  In Emacs, we have a convention whereby big
changes are discussed openly on emacs-devel and a consensus reached
before the change is made.  Stefan Monnier has regularly violated this
convention, possibly believing that his ideas for Emacs are so good as to
be beyond question.  Any attempt to question his ideas is likely to be
met by evasive non-answers, if any response at all is forthcoming.  I
could give several paragraphs worth of justification for these
assertions, but I think everybody here knows I am right.

In Emacs there is also a convention of treating eachother with respect on
the mailing lists.  Sadly this convention is superficial, and seems only
to mean things like not using swear words.  The truly contemptuous
communication style, this evasive non-answering, seems to be regarded as
acceptable.  I suggest that this change.

Stefan's habit of making big changes in Emacs without seeking consensus
is at the heart of why I am resigning.  These changes have caused Emacs a
lot of damage over the years and have caused other contributors,
including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
character.  On the one hand, he's a very capable hacker, and is always
ready to help others with technical questions.  On the other hand, as
mentioned, he is contemptuous of the Emacs conventions, and unlike
Richard and Eli, does not have the gift of knowing what the Right Thing
is.

I strongly recommend that Stefan somehow be reigned in and required to
observe Emacs's conventions about open discussion and courteous
communication.  As I mentioned, his violations of these are at the core
of why I feel unable to continue contributing to Emacs.

I will shortly be unsubscribing from emacs-devel.  I intend to carry on
maintaining stand alone CC Mode, and I'm prepared to deal with any CC
Mode issues which arise in Emacs.  Please post these to
bug-cc-mode@gnu.org.

It just remains to say that my respect for Eli and the other maintainers
remains undiminished, and that I wish all of them and the Emacs project
all success in the future.

--
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Eli Zaretskii]

> Date: Wed, 20 Nov 2024 15:13:18 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> 
> Hello, Emacs.
> 
> I'm resigning my position as Emacs contributor.

I regret very much this decision of yours, and urge you to reconsider.
Your many-year contributions to Emacs in general and to CC Mode in
particular are greatly appreciated and will be sorely missed if you
decide to go with this decision.

> Stefan's habit of making big changes in Emacs without seeking consensus
> is at the heart of why I am resigning.  These changes have caused Emacs a
> lot of damage over the years and have caused other contributors,
> including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
> character.  On the one hand, he's a very capable hacker, and is always
> ready to help others with technical questions.  On the other hand, as
> mentioned, he is contemptuous of the Emacs conventions, and unlike
> Richard and Eli, does not have the gift of knowing what the Right Thing
> is.

I must say that I disagree with this assessment of what Stefan did in
that case, and don't find anything unbecoming in his behavior, neither
in general nor in that particular case.  Yes, that change should have
been discussed more thoroughly; no, Stefan didn't do anything that
doesn't happen here every other day, and certainly didn't have any
malicious intentions when he installed that change.

> It just remains to say that my respect for Eli and the other maintainers
> remains undiminished, and that I wish all of them and the Emacs project
> all success in the future.

And the same to you.  But please do reconsider.

My resignation from Emacs development

[Christopher Dimech]

The claim that using free software or its associated names constitutes
aggression is fundamentally flawed. Aggression involves hostile actions
meant to cause harm, and using names in a way not intended by their
original authors is neither violent nor malicious.

One of the key principles of free software is that software should be
modifiable, and free to use in any context.  Restricting how names are
used run counter to the ethos to empower users and developers, not to
limit or control their language or expressions.

Although the approach should be reconsidered, there should be some
thoughtful conversation among the community.  Avoiding tones of contempt
or disregard for the foundations laid by previous contributors.

For instance, I agree with you that ("\\.myc\\'" . c-mode) in auto-mode-alist
should mean C Mode.  Although C Mode would mean the emacs preferred mode.
Still, your mode name can be changed for those who want to apply an alternative
mode.  Changing the mode should be a straightforward thing.



> Sent: Thursday, November 21, 2024 at 3:13 AM
> From: "Alan Mackenzie" <acm@muc.de>
> To: emacs-devel@gnu.org
> Subject: My resignation from Emacs development
>
>
> Hello, Emacs.
>
> I'm resigning my position as Emacs contributor.
>
> The immediate reason is that, as maintainer of CC Mode, CC Mode's
> symbols, its names, were taken by Emacs and used for other purposes
> without informing me, much less consulting me.  That makes my position as
> CC Mode maintainer here untenable.
>
> Eli Zaretskii and I have had extensive discussions, both in public and in
> private email, over the last week or so, but we have been unable to reach
> any satisfactory compromise solution.
>
> Names are important.  They have power.  To take somebody's/somthing's
> name and misuse it is an exercise of aggression.  Try using "Emacs" or
> even "free software" to mean something different, and see just how
> quickly you would hear back from Richard Stallman.  This misuse of CC
> Mode's "trademarks", the symbols `c-mode', `c++-mode', and perhaps
> `c-or-c++-mode', is just such an act of aggression.
>
> These symbols have been appropriated by Emacs to mean "the current
> preferred mode for C", etc., rather than C Mode, C++ Mode etc.  In
> certain circumstances, in particular, in Local Variables: sections and
> auto-mode-alist, there is now no longer any way unambiguously to specify
> C Mode or C++ Mode.  Up till recently ("\\.myc\\'" . c-mode) in
> auto-mode-alist meant C Mode, and would have had the effect of
> auto-loading CC Mode, if needed, and running C Mode.
>
> The change took place in the commit for bug#69191 "New var
> `major-mode-remap-defaults`, for packages".  It sounds so innocent, but
> is an extremely bad solution for whatever problem (unspecified in the
> commit message) it was intended to solve.  A major mode using it changes
> the interfaces of other libraries in an uncontrolled way.  This is not
> good software engineering.
>
> This bug was raised and committed by Stefan Monnier.  Despite the fact
> that the bug fix directly impinged upon CC Mode, and there was even a
> change to cc-mode.el in the patch, he failed even to inform me.  The only
> two modes substantially affected by this change were ruby-mode and CC
> Mode, and it is clear that Dmitry Gutov, maintainer of ruby-mode, was
> aware of the change.  Had I known of this proposal, I would certainly
> have objected to it.  Stefan is intelligent enough to have realised this,
> and maybe his avoidance of open discussion was motivated by this.
>
> Bug#69191 was a big change.  In Emacs, we have a convention whereby big
> changes are discussed openly on emacs-devel and a consensus reached
> before the change is made.  Stefan Monnier has regularly violated this
> convention, possibly believing that his ideas for Emacs are so good as to
> be beyond question.  Any attempt to question his ideas is likely to be
> met by evasive non-answers, if any response at all is forthcoming.  I
> could give several paragraphs worth of justification for these
> assertions, but I think everybody here knows I am right.
>
> In Emacs there is also a convention of treating eachother with respect on
> the mailing lists.  Sadly this convention is superficial, and seems only
> to mean things like not using swear words.  The truly contemptuous
> communication style, this evasive non-answering, seems to be regarded as
> acceptable.  I suggest that this change.
>
> Stefan's habit of making big changes in Emacs without seeking consensus
> is at the heart of why I am resigning.  These changes have caused Emacs a
> lot of damage over the years and have caused other contributors,
> including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
> character.  On the one hand, he's a very capable hacker, and is always
> ready to help others with technical questions.  On the other hand, as
> mentioned, he is contemptuous of the Emacs conventions, and unlike
> Richard and Eli, does not have the gift of knowing what the Right Thing
> is.
>
> I strongly recommend that Stefan somehow be reigned in and required to
> observe Emacs's conventions about open discussion and courteous
> communication.  As I mentioned, his violations of these are at the core
> of why I feel unable to continue contributing to Emacs.
>
> I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> Mode issues which arise in Emacs.  Please post these to
> bug-cc-mode@gnu.org.
>
> It just remains to say that my respect for Eli and the other maintainers
> remains undiminished, and that I wish all of them and the Emacs project
> all success in the future.
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>
>

Re: My resignation from Emacs development

[Gerd Möllmann]

Christopher Dimech <dimech@gmx.com> writes:

> One of the key principles of free software is that software should be
> modifiable, and free to use in any context.  Restricting how names are
> used run counter to the ethos to empower users and developers, not to
> limit or control their language or expressions.

Seriously?

My resignation from Emacs development

[Christopher Dimech]

> Sent: Thursday, November 21, 2024 at 6:22 PM
> From: "Gerd Möllmann" <gerd.moellmann@gmail.com>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: "Alan Mackenzie" <acm@muc.de>, emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> Christopher Dimech <dimech@gmx.com> writes:
> 
> > One of the key principles of free software is that software should be
> > modifiable, and free to use in any context.  Restricting how names are
> > used run counter to the ethos to empower users and developers, not to
> > limit or control their language or expressions.
> 
> Seriously?

The whole thing is about freedom, not erecting some new bureaucracy to police 
how people write their projects.

Re: My resignation from Emacs development

[Gerd Möllmann]

Christopher Dimech <dimech@gmx.com> writes:

>> Sent: Thursday, November 21, 2024 at 6:22 PM
>> From: "Gerd Möllmann" <gerd.moellmann@gmail.com>
>> To: "Christopher Dimech" <dimech@gmx.com>
>> Cc: "Alan Mackenzie" <acm@muc.de>, emacs-devel@gnu.org
>> Subject: Re: My resignation from Emacs development
>>
>> Christopher Dimech <dimech@gmx.com> writes:
>> 
>> > One of the key principles of free software is that software should be
>> > modifiable, and free to use in any context.  Restricting how names are
>> > used run counter to the ethos to empower users and developers, not to
>> > limit or control their language or expressions.
>> 
>> Seriously?
>
> The whole thing is about freedom, not erecting some new bureaucracy to police 
> how people write their projects.

With a bit of work, I think one could make a Monty Python sketch from
that.

"I want to use that function name."
"What?"
"I want to use that function name. I have the right to."
"But it's already used for 20 years. No reasonable man would..."
"Or woman..."
"Where was I?"
"I think you were finished."
"I have the right to use that function name! I'm born free!"
"From now on, I want you all to call me Loretta"

:-)

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Alan Mackenzie <acm@muc.de>,  emacs-devel@gnu.org
> Date: Thu, 21 Nov 2024 12:23:36 +0100
> 
> Christopher Dimech <dimech@gmx.com> writes:
> 
> >> Sent: Thursday, November 21, 2024 at 6:22 PM
> >> From: "Gerd Möllmann" <gerd.moellmann@gmail.com>
> >> To: "Christopher Dimech" <dimech@gmx.com>
> >> Cc: "Alan Mackenzie" <acm@muc.de>, emacs-devel@gnu.org
> >> Subject: Re: My resignation from Emacs development
> >>
> >> Christopher Dimech <dimech@gmx.com> writes:
> >> 
> >> > One of the key principles of free software is that software should be
> >> > modifiable, and free to use in any context.  Restricting how names are
> >> > used run counter to the ethos to empower users and developers, not to
> >> > limit or control their language or expressions.
> >> 
> >> Seriously?
> >
> > The whole thing is about freedom, not erecting some new bureaucracy to police 
> > how people write their projects.
> 
> With a bit of work, I think one could make a Monty Python sketch from
> that.
> 
> "I want to use that function name."
> "What?"
> "I want to use that function name. I have the right to."
> "But it's already used for 20 years. No reasonable man would..."
> "Or woman..."
> "Where was I?"
> "I think you were finished."
> "I have the right to use that function name! I'm born free!"
> "From now on, I want you all to call me Loretta"
> 
> :-)

This is beginning to be off-topic on this list.  Please either wrap up
this sub-thread, or take it to emacs-tangents@gnu.org.

TIA

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Christopher.

On Wed, Nov 20, 2024 at 17:23:20 +0100, Christopher Dimech wrote:

> The claim that using free software or its associated names constitutes
> aggression is fundamentally flawed. Aggression involves hostile actions
> meant to cause harm, and using names in a way not intended by their
> original authors is neither violent nor malicious.

There are forms of aggression which don't use fists or guns.

> One of the key principles of free software is that software should be
> modifiable, and free to use in any context.  Restricting how names are
> used run counter to the ethos to empower users and developers, not to
> limit or control their language or expressions.

Fine.  I put it to you that if somebody were to take the name
dimech@gmx.com and prevent it connecting up with your inbox, you would
be somewhat unhappy.

> Although the approach should be reconsidered, there should be some
> thoughtful conversation among the community.  Avoiding tones of contempt
> or disregard for the foundations laid by previous contributors.

How very considerate and reasonable of you.  The time for "thoughtful
conversation" around the current matter is long past.  You should
perhaps address your comments to those who bypass and evade "thoughtful
conversation" at the appropriate time.

> For instance, I agree with you that ("\\.myc\\'" . c-mode) in auto-mode-alist
> should mean C Mode.  Although C Mode would mean the emacs preferred mode.
> Still, your mode name can be changed for those who want to apply an alternative
> mode.  Changing the mode should be a straightforward thing.

C Mode has been called that for a long time, possibly longer than you
have been called Christopher Dimech.  As far as I'm concerned, it's
going to keep its name.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Christopher Dimech]

> Sent: Thursday, November 21, 2024 at 10:29 PM
> From: "Alan Mackenzie" <acm@muc.de>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> Hello, Christopher.
>
> On Wed, Nov 20, 2024 at 17:23:20 +0100, Christopher Dimech wrote:
>
> > The claim that using free software or its associated names constitutes
> > aggression is fundamentally flawed. Aggression involves hostile actions
> > meant to cause harm, and using names in a way not intended by their
> > original authors is neither violent nor malicious.
>
> There are forms of aggression which don't use fists or guns.

It is not unusual for people to do so.  Nobody did any harm.  Some
frustration perhaps.

> > One of the key principles of free software is that software should be
> > modifiable, and free to use in any context.  Restricting how names are
> > used run counter to the ethos to empower users and developers, not to
> > limit or control their language or expressions.
>
> Fine.  I put it to you that if somebody were to take the name
> dimech@gmx.com and prevent it connecting up with your inbox, you would
> be somewhat unhappy.

Emacs is not my inbox.  There are many things I need to get accustomed to.
The final decision has always been with the emacs maintainers.  Perhaps
you could become an emacs maintainer than keeping it non-gnu.

> > Although the approach should be reconsidered, there should be some
> > thoughtful conversation among the community.  Avoiding tones of contempt
> > or disregard for the foundations laid by previous contributors.
>
> How very considerate and reasonable of you.  The time for "thoughtful
> conversation" around the current matter is long past.  You should
> perhaps address your comments to those who bypass and evade "thoughtful
> conversation" at the appropriate time.

Right.

> > For instance, I agree with you that ("\\.myc\\'" . c-mode) in auto-mode-alist
> > should mean C Mode.  Although C Mode would mean the emacs preferred mode.
> > Still, your mode name can be changed for those who want to apply an alternative
> > mode.  Changing the mode should be a straightforward thing.
>
> C Mode has been called that for a long time, possibly longer than you
> have been called Christopher Dimech.  As far as I'm concerned, it's
> going to keep its name.

Didn't you think something general as C Mode could produce conflicts with
an built-in emacs mode?  It is customary to include a package name with
other code.

> --
> Alan Mackenzie (Nuremberg, Germany).
>

Re: My resignation from Emacs development

[Alfred M. Szmidt]

   I'm resigning my position as Emacs contributor.

:-( I hope you reconsider, and that Eli, Stefan and the rest of the
Emacs maintainers find a better middle ground -- CC Mode is amazing --
I too think that "c-mode" should mean CC Mode and nothing else.  This
just all smells of the debacle of pcase...

Some _other_ mechanism to pick between CC Mode and c-ts-mode ... or
whatever should exist, while respecting others namespaces.

If one was to introduce a dired-ts .. and M-x dired "magically"
decides between one or the other, people would be just as angry.

Re: My resignation from Emacs development

[tomas]

[-- Attachment #1: Type: text/plain, Size: 217 bytes --]

On Wed, Nov 20, 2024 at 03:13:18PM +0000, Alan Mackenzie wrote:
> 
> Hello, Emacs.
> 
> I'm resigning my position as Emacs contributor.

[...]

I, for one, will (would?) miss you dearly :-(

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: My resignation from Emacs development

[Dmitry Gutov]

Hi Alan,

On 20/11/2024 17:13, Alan Mackenzie wrote:

> I'm resigning my position as Emacs contributor.

I would be sorry to see you leave.

> This bug was raised and committed by Stefan Monnier.  Despite the fact
> that the bug fix directly impinged upon CC Mode, and there was even a
> change to cc-mode.el in the patch, he failed even to inform me.  The only
> two modes substantially affected by this change were ruby-mode and CC
> Mode, and it is clear that Dmitry Gutov, maintainer of ruby-mode, was
> aware of the change.

To clarify on this: I've been made aware of the change, just like other 
contributors, from reading the bug#69191 submission. And from my POV it 
didn't make things worse, globally - but reshaped existing problems. And 
it did improve some things - like ones that I had myself submitted a 
proposal previously (https://debbugs.gnu.org/68246#283), which was 
collectively rejected.

To be fair, I have less of a reason to take it personally due to less 
focus on particular major mode(s), and less years of tenure as well.

Speaking of other solutions, maybe you'll want to check out the patch in 
the nearby thread: 
https://lists.gnu.org/archive/html/emacs-devel/2024-11/msg00515.html

That scheme could make major-mode-remap-defaults unnecessary for 
c-ts-mode, or in any case remove the need for the corresponding 
overrides in CC Mode. I'm not sure what migration path should be 
selected, though.

Best,
Dmitry

Re: My resignation from Emacs development

[Stefan Kangas]

Hi Alan,

Alan Mackenzie <acm@muc.de> writes:

> The immediate reason is that, as maintainer of CC Mode, CC Mode's
> symbols, its names, were taken by Emacs and used for other purposes
> without informing me, much less consulting me.  That makes my position as
> CC Mode maintainer here untenable.

That is highly regrettable.  You are a valued member of our team, and
it's sad to see you go.

> These symbols have been appropriated by Emacs to mean "the current
> preferred mode for C", etc., rather than C Mode, C++ Mode etc.  In
> certain circumstances, in particular, in Local Variables: sections and
> auto-mode-alist, there is now no longer any way unambiguously to specify
> C Mode or C++ Mode.  Up till recently ("\\.myc\\'" . c-mode) in
> auto-mode-alist meant C Mode, and would have had the effect of
> auto-loading CC Mode, if needed, and running C Mode.

From my point of view, we are still in early days when it comes to the
new tree-sitter modes.  For starters, we do not recommend them by
default, and some language modes are also not yet ready for prime-time.
I'm not even sure that a majority of distros ship the feature in a
useful form yet, but I didn't really check.

AFAIU, the purpose of `major-mode-remap-alist` is to provide a mechanism
to respect what users want.  Where there is disagrement, it concerns the
technical details of how to best achieve that, and to which extent we
should set things up automatically based on indicators such as the user
actions "running a mode", "loading a file", or "running a command".

But the feature has teething problems.  My understanding was that we
agreed in Bug#74339 that the situation in Emacs 30 is already better
than in Emacs 29, and that we will continue working on this in Emacs 31.

For example, it has been suggested that we should replace the automatic
setting of `major-mode-remap-defaults` with an entirely new command like
`foo-ts-mode-prefer`, that would be used as the canonical indication
that a user wants to use the tree-sitter mode everywhere.  There surely
exist other options that we could evaluate also.

For this reason, I hope that there is still room to reconsider your
decision to resign.

> Stefan's habit of making big changes in Emacs without seeking consensus
> is at the heart of why I am resigning.  These changes have caused Emacs a
> lot of damage over the years and have caused other contributors,
> including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
> character.  On the one hand, he's a very capable hacker, and is always
> ready to help others with technical questions.  On the other hand, as
> mentioned, he is contemptuous of the Emacs conventions, and unlike
> Richard and Eli, does not have the gift of knowing what the Right Thing
> is.

This is where I have to disagree quite strongly.  I find the charges
directed at Stefan Monnier both unfair and one-sided.  I fail to see
which of his actions or words that could possibly warrant such a
negative interpretation, or that would justify assuming any ill intent.

I have to agree with Eli.  Although it would, in hindsight, certainly
have been better to discuss these particular changes in more detail in
advance, I don't see that he has done anything very unusual or different
from what most other core contributors do on a routine basis.

I also do not appreciate where it veers into ad-hominem, such as talking
about Stefan M's character, etc.  That is strictly off-topic here, as
you well know, and does not reach the usual high level of standard that
one would expect from one of your posts.

Can we please all remember that we share the same goal here; that we all
want to help advance Emacs and free software?

> I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> Mode issues which arise in Emacs.  Please post these to
> bug-cc-mode@gnu.org.
>
> It just remains to say that my respect for Eli and the other maintainers
> remains undiminished, and that I wish all of them and the Emacs project
> all success in the future.

Thanks for continuing to maintain CC-mode, and likewise.

I hope that you will seriously consider the idea to reverse your
decision to quit Emacs development.  It would be much better if we could
find a way where we can all continue working together.  I'd suggest
giving the idea at least a couple of days to fully consider, though I'll
of course respect your decision either way.

Meanwhile, if there is anything I can do to help improve things, please
feel free to reach out.  Thanks again for all your work on Emacs.

Tree-sitter maturity (was: My resignation from Emacs development)

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

On Wed, 20 Nov 2024, Stefan Kangas wrote:

> From my point of view, we are still in early days when it comes to the
> new tree-sitter modes.  For starters, we do not recommend them by
> default, and some language modes are also not yet ready for prime-time.
> I'm not even sure that a majority of distros ship the feature in a
> useful form yet, but I didn't really check.

It depends on what you mean by useful.  In Fedora, for example, Emacs is built with Tree-sitter, but each user has to (ask Emacs to) download and compile each parser as they go along.

If any Fedora packagers read this and would like to help with packaging the parsers used by Emacs, that would be welcome.  The tracking bug is https://bugzilla.redhat.com/show_bug.cgi?id=2258924

It’s also worth noting that Tree-sitter itself is somewhat immature; the developers say that until it reaches version 1.0, we should be wary of potentially unannounced incompatible changes (although they are trying harder to avoid this, over time).

-- 
Peter Oliver

Re: Tree-sitter maturity (was: My resignation from Emacs development)

[Stefan Kangas]

Peter Oliver <p.d.oliver@mavit.org.uk> writes:

> On Wed, 20 Nov 2024, Stefan Kangas wrote:
>
>> From my point of view, we are still in early days when it comes to
>> the new tree-sitter modes.  For starters, we do not recommend them by
>> default, and some language modes are also not yet ready for
>> prime-time.  I'm not even sure that a majority of distros ship the
>> feature in a useful form yet, but I didn't really check.
>
> It depends on what you mean by useful.  In Fedora, for example, Emacs
> is built with Tree-sitter, but each user has to (ask Emacs to)
> download and compile each parser as they go along.

Thanks.  I meant that one cannot yet use them after simply installing
the "emacs" package, without taking additional steps.

> If any Fedora packagers read this and would like to help with
> packaging the parsers used by Emacs, that would be welcome.  The
> tracking bug is https://bugzilla.redhat.com/show_bug.cgi?id=2258924

Thanks for working on this.  Packaging Tree-sitter seems somewhat
challenging, but getting it done is essential if we want to see more
widespread use of Tree-sitter in our user base.

> It’s also worth noting that Tree-sitter itself is somewhat immature;
> the developers say that until it reaches version 1.0, we should be
> wary of potentially unannounced incompatible changes (although they
> are trying harder to avoid this, over time).

Re: Tree-sitter maturity

[Björn Bidar]

Peter Oliver <p.d.oliver@mavit.org.uk> writes:

> On Wed, 20 Nov 2024, Stefan Kangas wrote:
>
>> From my point of view, we are still in early days when it comes to the
>> new tree-sitter modes.  For starters, we do not recommend them by
>> default, and some language modes are also not yet ready for prime-time.
>> I'm not even sure that a majority of distros ship the feature in a
>> useful form yet, but I didn't really check.
>
> It depends on what you mean by useful.  In Fedora, for example, Emacs is built with Tree-sitter, but each user has to (ask Emacs to) download and compile each parser as they go along.
>
> If any Fedora packagers read this and would like to help with
> packaging the parsers used by Emacs, that would be welcome.  The
> tracking bug is https://bugzilla.redhat.com/show_bug.cgi?id=2258924

Feel free to take a look into the RPM packaging I created for SUSE.
There should be a way to collaborate on that if anyone is interested. [1]

Using that packaging all the user has to do is to install the required
grammar they want.

> It’s also worth noting that Tree-sitter itself is somewhat immature; the developers say that until it reaches version 1.0, we should be wary of potentially unannounced incompatible changes (although they are trying harder to avoid this, over time).


[1] https://build.opensuse.org/package/show/editors/tree-sitter

Re: Tree-sitter maturity

[Yuan Fu]

> On Nov 23, 2024, at 6:10 PM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
> 
> Peter Oliver <p.d.oliver@mavit.org.uk> writes:
> 
>> On Wed, 20 Nov 2024, Stefan Kangas wrote:
>> 
>>> From my point of view, we are still in early days when it comes to the
>>> new tree-sitter modes.  For starters, we do not recommend them by
>>> default, and some language modes are also not yet ready for prime-time.
>>> I'm not even sure that a majority of distros ship the feature in a
>>> useful form yet, but I didn't really check.
>> 
>> It depends on what you mean by useful.  In Fedora, for example, Emacs is built with Tree-sitter, but each user has to (ask Emacs to) download and compile each parser as they go along.
>> 
>> If any Fedora packagers read this and would like to help with
>> packaging the parsers used by Emacs, that would be welcome.  The
>> tracking bug is https://bugzilla.redhat.com/show_bug.cgi?id=2258924
> 
> Feel free to take a look into the RPM packaging I created for SUSE.
> There should be a way to collaborate on that if anyone is interested. [1]
> 
> Using that packaging all the user has to do is to install the required
> grammar they want.
> 
>> It’s also worth noting that Tree-sitter itself is somewhat immature; the developers say that until it reaches version 1.0, we should be wary of potentially unannounced incompatible changes (although they are trying harder to avoid this, over time).
> 
> 
> [1] https://build.opensuse.org/package/show/editors/tree-sitter

I wonder if we can formalize a way for tree-sitter major modes to state the compatible version of language grammar it uses. Maybe a package.el cookies, or a variable that set, or even just comments in the beginning of the file.

Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.

I especially want built-in major modes to give a version, so that packagers can package Emacs with the right version of tree-sitter grammar. I know Eli has problems with pinning a grammar version for builtin modes before, but I wonder what’s he’s stance now?

Yuan

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Tue, 17 Dec 2024 14:11:51 -0800
> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  Emacs Devel <emacs-devel@gnu.org>,
>  Eli Zaretskii <eliz@gnu.org>
> 
> >> It’s also worth noting that Tree-sitter itself is somewhat immature; the developers say that until it reaches version 1.0, we should be wary of potentially unannounced incompatible changes (although they are trying harder to avoid this, over time).
> > 
> > 
> > [1] https://build.opensuse.org/package/show/editors/tree-sitter
> 
> I wonder if we can formalize a way for tree-sitter major modes to state the compatible version of language grammar it uses. Maybe a package.el cookies, or a variable that set, or even just comments in the beginning of the file.
> 
> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
> 
> I especially want built-in major modes to give a version, so that packagers can package Emacs with the right version of tree-sitter grammar. I know Eli has problems with pinning a grammar version for builtin modes before, but I wonder what’s he’s stance now?

What's changed?

Many language grammars don't make official releases and thus don't
have versions.  Moreover, AFAIK there's no API to determine the
version of the grammar library we load.  So how can we manage such
version-pinning in a way that (a) is up-to-date, and (b) doesn't
preclude people from using a grammar library due to false negatives?

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>> Stefan Kangas <stefankangas@gmail.com>,
>> Emacs Devel <emacs-devel@gnu.org>,
>> Eli Zaretskii <eliz@gnu.org>
>> 
>>>> It’s also worth noting that Tree-sitter itself is somewhat immature; the developers say that until it reaches version 1.0, we should be wary of potentially unannounced incompatible changes (although they are trying harder to avoid this, over time).
>>> 
>>> 
>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter
>> 
>> I wonder if we can formalize a way for tree-sitter major modes to state the compatible version of language grammar it uses. Maybe a package.el cookies, or a variable that set, or even just comments in the beginning of the file.
>> 
>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>> 
>> I especially want built-in major modes to give a version, so that packagers can package Emacs with the right version of tree-sitter grammar. I know Eli has problems with pinning a grammar version for builtin modes before, but I wonder what’s he’s stance now?
> 
> What's changed?

People are starting to package tree-sitter and tree-sitter grammars. If Emacs can be packaged with the right grammars, then tree-sitter modes will work out-of-the-box.

> 
> Many language grammars don't make official releases and thus don't
> have versions.  Moreover, AFAIK there's no API to determine the
> version of the grammar library we load.  So how can we manage such
> version-pinning in a way that (a) is up-to-date, and (b) doesn't
> preclude people from using a grammar library due to false negatives?

I’m talking about a softer pin. We’re basically providing a “known to work” version. This way packagers can package Emacs with a known-to-work version of grammar, so the builtin modes work out-of-the-box. This doesn’t prevent people from using a newer version and sending us a bug report, and we still try our best to make the major modes work with the newest grammar.

If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?

Yuan

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 18 Dec 2024 17:40:39 -0800
> Cc: Björn Bidar <bjorn.bidar@thaodan.de>,
>  Peter Oliver <p.d.oliver@mavit.org.uk>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  emacs-devel@gnu.org
> 
> 
> 
> > On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> I especially want built-in major modes to give a version, so that packagers can package Emacs with the right version of tree-sitter grammar. I know Eli has problems with pinning a grammar version for builtin modes before, but I wonder what’s he’s stance now?
> > 
> > What's changed?
> 
> People are starting to package tree-sitter and tree-sitter grammars. If Emacs can be packaged with the right grammars, then tree-sitter modes will work out-of-the-box.
> 
> > 
> > Many language grammars don't make official releases and thus don't
> > have versions.  Moreover, AFAIK there's no API to determine the
> > version of the grammar library we load.  So how can we manage such
> > version-pinning in a way that (a) is up-to-date, and (b) doesn't
> > preclude people from using a grammar library due to false negatives?
> 
> I’m talking about a softer pin. We’re basically providing a “known to work” version. This way packagers can package Emacs with a known-to-work version of grammar, so the builtin modes work out-of-the-box. This doesn’t prevent people from using a newer version and sending us a bug report, and we still try our best to make the major modes work with the newest grammar.
> 
> If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?

If you are suggesting to have the known version in some comment, and
we don't have to guarantee that it's always up-to-date (this should be
stated in the comment), then I don't object.  As long as users know
they should take that with a grain of salt, I'm okay with it.

Re: Tree-sitter maturity

[Björn Bidar]

Yuan Fu <casouri@gmail.com> writes:

>> On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>>> From: Yuan Fu <casouri@gmail.com>
>>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>>> Stefan Kangas <stefankangas@gmail.com>,
>>> Emacs Devel <emacs-devel@gnu.org>,
>>> Eli Zaretskii <eliz@gnu.org>
>>> 
>>>>> It’s also worth noting that Tree-sitter itself is somewhat
>>>> immature; the developers say that until it reaches version 1.0, we
>>>> should be wary of potentially unannounced incompatible changes
>>>> (although they are trying harder to avoid this, over time).
>>>> 
>>>> 
>>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter
>>> 
>>> I wonder if we can formalize a way for tree-sitter major modes to
>>> state the compatible version of language grammar it uses. Maybe a
>>> package.el cookies, or a variable that set, or even just comments
>>> in the beginning of the file.
>>> 
>>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>>> 
>>> I especially want built-in major modes to give a version, so that
>>> packagers can package Emacs with the right version of tree-sitter
>>> grammar. I know Eli has problems with pinning a grammar version for
>>> builtin modes before, but I wonder what’s he’s stance now?
>> 
>> What's changed?
>
> People are starting to package tree-sitter and tree-sitter
> grammars. If Emacs can be packaged with the right grammars, then
> tree-sitter modes will work out-of-the-box.

Please don't. That would require nodejs to build Emacs bundled with
these grammars. These grammar packages are also not just used with
Emacs.

Grammars are very easy to package once the infrastructure to reuse the
packaging automation in the package manager is there. Don't try to
reinvent that IMHO. If you must generated and build the parser implement
a bindings.gyp parser so you can automate the compilation process
independently of the grammar.

For reference here's my implementation of it in python:
https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitter/files/tree-sitter-target.py?expand=1

>> 
>> Many language grammars don't make official releases and thus don't
>> have versions.  Moreover, AFAIK there's no API to determine the
>> version of the grammar library we load.  So how can we manage such
>> version-pinning in a way that (a) is up-to-date, and (b) doesn't
>> preclude people from using a grammar library due to false negatives?
>
> I’m talking about a softer pin. We’re basically providing a “known to
> work” version. This way packagers can package Emacs with a
> known-to-work version of grammar, so the builtin modes work
> out-of-the-box. This doesn’t prevent people from using a newer version
> and sending us a bug report, and we still try our best to make the
> major modes work with the newest grammar.
>
> If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?

That doesn't make sense as the versions numbers are arbitrary, e.g. not
always does the version number relate the changes to grammar but also to
the in-tree dependencies in the repository packaging the
language-grammar bindings which have nothing todo with the parser.

What matters much more is the tree-sitter version which is more related
to Emacs itself rather than the particular version of the grammar.

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 20, 2024, at 1:13 AM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>>> On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri@gmail.com>
>>>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>>>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>>>> Stefan Kangas <stefankangas@gmail.com>,
>>>> Emacs Devel <emacs-devel@gnu.org>,
>>>> Eli Zaretskii <eliz@gnu.org>
>>>> 
>>>>>> It’s also worth noting that Tree-sitter itself is somewhat
>>>>> immature; the developers say that until it reaches version 1.0, we
>>>>> should be wary of potentially unannounced incompatible changes
>>>>> (although they are trying harder to avoid this, over time).
>>>>> 
>>>>> 
>>>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter
>>>> 
>>>> I wonder if we can formalize a way for tree-sitter major modes to
>>>> state the compatible version of language grammar it uses. Maybe a
>>>> package.el cookies, or a variable that set, or even just comments
>>>> in the beginning of the file.
>>>> 
>>>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>>>> 
>>>> I especially want built-in major modes to give a version, so that
>>>> packagers can package Emacs with the right version of tree-sitter
>>>> grammar. I know Eli has problems with pinning a grammar version for
>>>> builtin modes before, but I wonder what’s he’s stance now?
>>> 
>>> What's changed?
>> 
>> People are starting to package tree-sitter and tree-sitter
>> grammars. If Emacs can be packaged with the right grammars, then
>> tree-sitter modes will work out-of-the-box.
> 
> Please don't. That would require nodejs to build Emacs bundled with
> these grammars. These grammar packages are also not just used with
> Emacs.
> 
> Grammars are very easy to package once the infrastructure to reuse the
> packaging automation in the package manager is there. Don't try to
> reinvent that IMHO. If you must generated and build the parser implement
> a bindings.gyp parser so you can automate the compilation process
> independently of the grammar.

There might be some misunderstanding. We don’t want to build the grammars as part of building Emacs. Ideally building the grammars are the package managers job. We just want to list the versions of grammars that are known to work with the major modes, so packagers have an easier time to package Emacs with the right version of grammars.

> 
> For reference here's my implementation of it in python:
> https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitter/files/tree-sitter-target.py?expand=1
> 
>>> 
>>> Many language grammars don't make official releases and thus don't
>>> have versions.  Moreover, AFAIK there's no API to determine the
>>> version of the grammar library we load.  So how can we manage such
>>> version-pinning in a way that (a) is up-to-date, and (b) doesn't
>>> preclude people from using a grammar library due to false negatives?
>> 
>> I’m talking about a softer pin. We’re basically providing a “known to
>> work” version. This way packagers can package Emacs with a
>> known-to-work version of grammar, so the builtin modes work
>> out-of-the-box. This doesn’t prevent people from using a newer version
>> and sending us a bug report, and we still try our best to make the
>> major modes work with the newest grammar.
>> 
>> If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?
> 
> That doesn't make sense as the versions numbers are arbitrary, e.g. not
> always does the version number relate the changes to grammar but also to
> the in-tree dependencies in the repository packaging the
> language-grammar bindings which have nothing todo with the parser.

Sure, let’s call it snapshot then. I just want to make sure when packagers package Emacs with tree-sitter grammars, the grammar works with Emacs’s major mode.

> 
> What matters much more is the tree-sitter version which is more related
> to Emacs itself rather than the particular version of the grammar.

The tree-sitter library version is up to the packagers right? As long as it satisfies Emacs’ requirements and is compatible with the bundled grammars.

Yuan

Re: Tree-sitter maturity

[Björn Bidar]

Yuan Fu <casouri@gmail.com> writes:

>> On Dec 20, 2024, at 1:13 AM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
>> 
>> Yuan Fu <casouri@gmail.com> writes:
>> 
>>>> On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> 
>>>>> From: Yuan Fu <casouri@gmail.com>
>>>>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>>>>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>>>>> Stefan Kangas <stefankangas@gmail.com>,
>>>>> Emacs Devel <emacs-devel@gnu.org>,
>>>>> Eli Zaretskii <eliz@gnu.org>
>>>>> 
>>>>>>> It’s also worth noting that Tree-sitter itself is somewhat
>>>>>> immature; the developers say that until it reaches version 1.0, we
>>>>>> should be wary of potentially unannounced incompatible changes
>>>>>> (although they are trying harder to avoid this, over time).
>>>>>> 
>>>>>> 
>>>>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter
>>>>> 
>>>>> I wonder if we can formalize a way for tree-sitter major modes to
>>>>> state the compatible version of language grammar it uses. Maybe a
>>>>> package.el cookies, or a variable that set, or even just comments
>>>>> in the beginning of the file.
>>>>> 
>>>>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>>>>> 
>>>>> I especially want built-in major modes to give a version, so that
>>>>> packagers can package Emacs with the right version of tree-sitter
>>>>> grammar. I know Eli has problems with pinning a grammar version for
>>>>> builtin modes before, but I wonder what’s he’s stance now?
>>>> 
>>>> What's changed?
>>> 
>>> People are starting to package tree-sitter and tree-sitter
>>> grammars. If Emacs can be packaged with the right grammars, then
>>> tree-sitter modes will work out-of-the-box.
>> 
>> Please don't. That would require nodejs to build Emacs bundled with
>> these grammars. These grammar packages are also not just used with
>> Emacs.
>> 
>> Grammars are very easy to package once the infrastructure to reuse the
>> packaging automation in the package manager is there. Don't try to
>> reinvent that IMHO. If you must generated and build the parser implement
>> a bindings.gyp parser so you can automate the compilation process
>> independently of the grammar.
>
> There might be some misunderstanding. We don’t want to build the
> grammars as part of building Emacs. Ideally building the grammars are
> the package managers job. We just want to list the versions of
> grammars that are known to work with the major modes, so packagers
> have an easier time to package Emacs with the right version of
> grammars.

Ah ok now I understand. I don't think that would work.

>> 
>> For reference here's my implementation of it in python:
>> https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitter/files/tree-sitter-target.py?expand=1
>> 
>>>> 
>>>> Many language grammars don't make official releases and thus don't
>>>> have versions.  Moreover, AFAIK there's no API to determine the
>>>> version of the grammar library we load.  So how can we manage such
>>>> version-pinning in a way that (a) is up-to-date, and (b) doesn't
>>>> preclude people from using a grammar library due to false negatives?
>>> 
>>> I’m talking about a softer pin. We’re basically providing a “known to
>>> work” version. This way packagers can package Emacs with a
>>> known-to-work version of grammar, so the builtin modes work
>>> out-of-the-box. This doesn’t prevent people from using a newer version
>>> and sending us a bug report, and we still try our best to make the
>>> major modes work with the newest grammar.
>>> 
>>> If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?
>> 
>> That doesn't make sense as the versions numbers are arbitrary, e.g. not
>> always does the version number relate the changes to grammar but also to
>> the in-tree dependencies in the repository packaging the
>> language-grammar bindings which have nothing todo with the parser.
>
> Sure, let’s call it snapshot then. I just want to make sure when
> packagers package Emacs with tree-sitter grammars, the grammar works
> with Emacs’s major mode.

The point was that now matter what you call the development of grammars
is more or less fluent. Maybe there are some more mature grammar but
those should be the minority.
But lets just assume for a second it would be possible to freeze or
recommend the supported grammar versions. The development of grammars is
to fast for that, especially for builtin modes.

>> 
>> What matters much more is the tree-sitter version which is more related
>> to Emacs itself rather than the particular version of the grammar.
>
> The tree-sitter library version is up to the packagers right? As long as it satisfies Emacs’ requirements and is compatible with the bundled grammars.

Do mean bundled or recommended grammars? Grammars bundled would be again
grammars included within the Emacs sources which is a different thing
from what I you were saying further above.

Yes the tree-sitter version is up to the package or respectively the
distribution.
The only issue that existed regarding was that tree-sitter once broke
the ABI without bumping the sover but that's fixed now or was fixed when
Emacs correctly rebuilt once a dependency of it changed.

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 22, 2024, at 4:43 PM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>>> On Dec 20, 2024, at 1:13 AM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
>>> 
>>> Yuan Fu <casouri@gmail.com> writes:
>>> 
>>>>> On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>> 
>>>>>> From: Yuan Fu <casouri@gmail.com>
>>>>>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>>>>>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>>>>>> Stefan Kangas <stefankangas@gmail.com>,
>>>>>> Emacs Devel <emacs-devel@gnu.org>,
>>>>>> Eli Zaretskii <eliz@gnu.org>
>>>>>> 
>>>>>>>> It’s also worth noting that Tree-sitter itself is somewhat
>>>>>>> immature; the developers say that until it reaches version 1.0, we
>>>>>>> should be wary of potentially unannounced incompatible changes
>>>>>>> (although they are trying harder to avoid this, over time).
>>>>>>> 
>>>>>>> 
>>>>>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter
>>>>>> 
>>>>>> I wonder if we can formalize a way for tree-sitter major modes to
>>>>>> state the compatible version of language grammar it uses. Maybe a
>>>>>> package.el cookies, or a variable that set, or even just comments
>>>>>> in the beginning of the file.
>>>>>> 
>>>>>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>>>>>> 
>>>>>> I especially want built-in major modes to give a version, so that
>>>>>> packagers can package Emacs with the right version of tree-sitter
>>>>>> grammar. I know Eli has problems with pinning a grammar version for
>>>>>> builtin modes before, but I wonder what’s he’s stance now?
>>>>> 
>>>>> What's changed?
>>>> 
>>>> People are starting to package tree-sitter and tree-sitter
>>>> grammars. If Emacs can be packaged with the right grammars, then
>>>> tree-sitter modes will work out-of-the-box.
>>> 
>>> Please don't. That would require nodejs to build Emacs bundled with
>>> these grammars. These grammar packages are also not just used with
>>> Emacs.
>>> 
>>> Grammars are very easy to package once the infrastructure to reuse the
>>> packaging automation in the package manager is there. Don't try to
>>> reinvent that IMHO. If you must generated and build the parser implement
>>> a bindings.gyp parser so you can automate the compilation process
>>> independently of the grammar.
>> 
>> There might be some misunderstanding. We don’t want to build the
>> grammars as part of building Emacs. Ideally building the grammars are
>> the package managers job. We just want to list the versions of
>> grammars that are known to work with the major modes, so packagers
>> have an easier time to package Emacs with the right version of
>> grammars.
> 
> Ah ok now I understand. I don't think that would work.
> 
>>> 
>>> For reference here's my implementation of it in python:
>>> https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitter/files/tree-sitter-target.py?expand=1
>>> 
>>>>> 
>>>>> Many language grammars don't make official releases and thus don't
>>>>> have versions.  Moreover, AFAIK there's no API to determine the
>>>>> version of the grammar library we load.  So how can we manage such
>>>>> version-pinning in a way that (a) is up-to-date, and (b) doesn't
>>>>> preclude people from using a grammar library due to false negatives?
>>>> 
>>>> I’m talking about a softer pin. We’re basically providing a “known to
>>>> work” version. This way packagers can package Emacs with a
>>>> known-to-work version of grammar, so the builtin modes work
>>>> out-of-the-box. This doesn’t prevent people from using a newer version
>>>> and sending us a bug report, and we still try our best to make the
>>>> major modes work with the newest grammar.
>>>> 
>>>> If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?
>>> 
>>> That doesn't make sense as the versions numbers are arbitrary, e.g. not
>>> always does the version number relate the changes to grammar but also to
>>> the in-tree dependencies in the repository packaging the
>>> language-grammar bindings which have nothing todo with the parser.
>> 
>> Sure, let’s call it snapshot then. I just want to make sure when
>> packagers package Emacs with tree-sitter grammars, the grammar works
>> with Emacs’s major mode.
> 
> The point was that now matter what you call the development of grammars
> is more or less fluent. Maybe there are some more mature grammar but
> those should be the minority.
> But lets just assume for a second it would be possible to freeze or
> recommend the supported grammar versions. The development of grammars is
> to fast for that, especially for builtin modes.
> 
>>> 
>>> What matters much more is the tree-sitter version which is more related
>>> to Emacs itself rather than the particular version of the grammar.
>> 
>> The tree-sitter library version is up to the packagers right? As long as it satisfies Emacs’ requirements and is compatible with the bundled grammars.
> 
> Do mean bundled or recommended grammars? Grammars bundled would be again
> grammars included within the Emacs sources which is a different thing
> from what I you were saying further above.

Recommended. So packagers control the version of both tree-sitter lib and grammars. Emacs will recommend version or commit hash of grammars, and packagers will provide Emacs with the grammars that work with the builtin major modes.

> 
> Yes the tree-sitter version is up to the package or respectively the
> distribution.
> The only issue that existed regarding was that tree-sitter once broke
> the ABI without bumping the sover but that's fixed now or was fixed when
> Emacs correctly rebuilt once a dependency of it changed.

Yeah, hopefully they’ll be more careful in the future.

Yuan

Re: Tree-sitter maturity

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Integrating the treesitter grammars smoothly is a good thing to do in
principle, and I don't have much to say about the details assuming it
doesn't lead users to use nonfree software.  But there is one big
design issue that is important:

Making Emacs _integrate_ with other tools (as distinguished from
invoking them at build time) is problematical and calls for careful
judgment.

Ideally, users should be able install the grammars separately, perhaps
using their `make install' or `apt', and Emacs would only use them
where they normally get installed.  That is ideal because it is not
very integrated -- it preserves modularity.  In particular, it assures
that the details of how they get installed are not a direct concern of
Emacs maintenance.

Please forgive this small waste of time, if the point seems obvious to
all of you.  I couldn't tell that from the deeply mested messages I
saw.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 23 Dec 2024 23:52:43 -0500
> 
> Ideally, users should be able install the grammars separately, perhaps
> using their `make install' or `apt', and Emacs would only use them
> where they normally get installed.  That is ideal because it is not
> very integrated -- it preserves modularity.  In particular, it assures
> that the details of how they get installed are not a direct concern of
> Emacs maintenance.

Emacs needs to be built with the tree-sitter library to support the
modes based on it, and the grammar library needs to be installed.  If
Emacs was built without tree-sitter, or if the grammar required for a
mode is not installed, the mode will display a warning to that effect.

Re: Tree-sitter maturity

[Xiyue Deng]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Richard Stallman <rms@gnu.org>
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 23 Dec 2024 23:52:43 -0500
>> 
>> Ideally, users should be able install the grammars separately, perhaps
>> using their `make install' or `apt', and Emacs would only use them
>> where they normally get installed.  That is ideal because it is not
>> very integrated -- it preserves modularity.  In particular, it assures
>> that the details of how they get installed are not a direct concern of
>> Emacs maintenance.
>
> Emacs needs to be built with the tree-sitter library to support the
> modes based on it, and the grammar library needs to be installed.  If
> Emacs was built without tree-sitter, or if the grammar required for a
> mode is not installed, the mode will display a warning to that effect.
>

Not sure whether this has been mentioned: there is a "treesit-auto"[1]
addon on melpa that can detect missing treesitter grammars and handle
their installation automatically.  It would be great if something
similar can be integrated into core, and if possible, grammar version
handling and compatibility with distribution supplied grammar would be
good to have.

[1] https://github.com/renzmann/treesit-auto

-- 
Regards,
Xiyue Deng

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: Tree-sitter maturity

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Not sure whether this has been mentioned: there is a "treesit-auto"[1]
  > addon on melpa that can detect missing treesitter grammars and handle
  > their installation automatically.  It would be great if something
  > similar can be integrated into core, and if possible, grammar version
  > handling and compatibility with distribution supplied grammar would be
  > good to have.

If we add something like this to Emacs, there is an issue we need to
take care about: to make carefully sure that it does not install
any nonfree grammars.  I don't know how those grammars are released,
ir by whom, or how much they care about free software.  We can't
take for granted that they do.

Perhaps we could check automatically that the grammar found is properly
licenses, and disregard any grammars that are not free.

By contrast, if grammars are going to be packaged and released for
distros, and chosen for installation by users, then it is the user's
responsibility, not Emacs's responsibility, to reject the nonfree ones
(and the GNU/Linux distro might insist on that).

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Tree-sitter maturity

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Not sure whether this has been mentioned: there is a "treesit-auto"[1]
>   > addon on melpa that can detect missing treesitter grammars and handle
>   > their installation automatically.  It would be great if something
>   > similar can be integrated into core, and if possible, grammar version
>   > handling and compatibility with distribution supplied grammar would be
>   > good to have.
>
> If we add something like this to Emacs, there is an issue we need to
> take care about: to make carefully sure that it does not install
> any nonfree grammars.  I don't know how those grammars are released,
> ir by whom, or how much they care about free software.  We can't
> take for granted that they do.
>
> Perhaps we could check automatically that the grammar found is properly
> licenses, and disregard any grammars that are not free.
>
> By contrast, if grammars are going to be packaged and released for
> distros, and chosen for installation by users, then it is the user's
> responsibility, not Emacs's responsibility, to reject the nonfree ones
> (and the GNU/Linux distro might insist on that).

It might take a while for that to happen, which is why I still believe
it would be better if tree-sitter major modes would populate
`treesit-language-source-alist' on their own, and point to the specific
checkouts that the major mode developer tested their implementation
against.  That way users are assured that when using the command
`treesit-install-language-grammar', that their grammar is both stable
and free.

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
> Date: Fri, 27 Dec 2024 10:54:29 +0000
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> > If we add something like this to Emacs, there is an issue we need to
> > take care about: to make carefully sure that it does not install
> > any nonfree grammars.  I don't know how those grammars are released,
> > ir by whom, or how much they care about free software.  We can't
> > take for granted that they do.
> >
> > Perhaps we could check automatically that the grammar found is properly
> > licenses, and disregard any grammars that are not free.
> >
> > By contrast, if grammars are going to be packaged and released for
> > distros, and chosen for installation by users, then it is the user's
> > responsibility, not Emacs's responsibility, to reject the nonfree ones
> > (and the GNU/Linux distro might insist on that).
> 
> It might take a while for that to happen, which is why I still believe
> it would be better if tree-sitter major modes would populate
> `treesit-language-source-alist' on their own, and point to the specific
> checkouts that the major mode developer tested their implementation
> against.

We could have done that, but there's no way we could keep the value of
treesit-language-source-alist up-to-date, because the grammar
libraries put out new versions much more frequently than Emacs
releases, especially if you consider libraries that have no official
versions at all (in which case we can only point to some revision in
their repository).

The question that bothers me is how useful is it to have
treesit-language-source-alist that is outdated?  What do we expect the
users to do with such an outdated value?

Re: Tree-sitter maturity

[Daniel Colascione]

On December 27, 2024 7:40:19 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>> 
>> Richard Stallman <rms@gnu.org> writes:
>> 
>> > If we add something like this to Emacs, there is an issue we need to
>> > take care about: to make carefully sure that it does not install
>> > any nonfree grammars.  I don't know how those grammars are released,
>> > ir by whom, or how much they care about free software.  We can't
>> > take for granted that they do.
>> >
>> > Perhaps we could check automatically that the grammar found is properly
>> > licenses, and disregard any grammars that are not free.
>> >
>> > By contrast, if grammars are going to be packaged and released for
>> > distros, and chosen for installation by users, then it is the user's
>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>> > (and the GNU/Linux distro might insist on that).
>> 
>> It might take a while for that to happen, which is why I still believe
>> it would be better if tree-sitter major modes would populate
>> `treesit-language-source-alist' on their own, and point to the specific
>> checkouts that the major mode developer tested their implementation
>> against.
>
>We could have done that, but there's no way we could keep the value of
>treesit-language-source-alist up-to-date, because the grammar
>libraries put out new versions much more frequently than Emacs
>releases, especially if you consider libraries that have no official
>versions at all (in which case we can only point to some revision in
>their repository).
>
>The question that bothers me is how useful is it to have
>treesit-language-source-alist that is outdated?  What do we expect the
>users to do with such an outdated value?
>

Why not just vendor all the grammars with the Emacs modes that use them?

Re: Tree-sitter maturity

[Philip Kaludercic]

Daniel Colascione <dancol@dancol.org> writes:

> On December 27, 2024 7:40:19 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>>> 
>>> Richard Stallman <rms@gnu.org> writes:
>>> 
>>> > If we add something like this to Emacs, there is an issue we need to
>>> > take care about: to make carefully sure that it does not install
>>> > any nonfree grammars.  I don't know how those grammars are released,
>>> > ir by whom, or how much they care about free software.  We can't
>>> > take for granted that they do.
>>> >
>>> > Perhaps we could check automatically that the grammar found is properly
>>> > licenses, and disregard any grammars that are not free.
>>> >
>>> > By contrast, if grammars are going to be packaged and released for
>>> > distros, and chosen for installation by users, then it is the user's
>>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>>> > (and the GNU/Linux distro might insist on that).
>>> 
>>> It might take a while for that to happen, which is why I still believe
>>> it would be better if tree-sitter major modes would populate
>>> `treesit-language-source-alist' on their own, and point to the specific
>>> checkouts that the major mode developer tested their implementation
>>> against.
>>
>>We could have done that, but there's no way we could keep the value of
>>treesit-language-source-alist up-to-date, because the grammar
>>libraries put out new versions much more frequently than Emacs
>>releases, especially if you consider libraries that have no official
>>versions at all (in which case we can only point to some revision in
>>their repository).
>>
>>The question that bothers me is how useful is it to have
>>treesit-language-source-alist that is outdated?  What do we expect the
>>users to do with such an outdated value?
>>
>
> Why not just vendor all the grammars with the Emacs modes that use them?

I am guessing part of the reason is that TS grammars are not fun to
build.  IIRC they are specified in a Javascript DSL (that used to
require node.js but AFAIU works with other implementations as well),
that a program written in Rust translates to C code.  So do we vendor
the DSL and depend on the TreeSitter toolchain or do we vender the
generated code?

Re: Tree-sitter maturity

[Daniel Colascione]

On December 27, 2024 9:19:12 AM EST, Philip Kaludercic <philipk@posteo.net> wrote:
>Daniel Colascione <dancol@dancol.org> writes:
>
>> On December 27, 2024 7:40:19 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> From: Philip Kaludercic <philipk@posteo.net>
>>>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>>>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>>>> 
>>>> Richard Stallman <rms@gnu.org> writes:
>>>> 
>>>> > If we add something like this to Emacs, there is an issue we need to
>>>> > take care about: to make carefully sure that it does not install
>>>> > any nonfree grammars.  I don't know how those grammars are released,
>>>> > ir by whom, or how much they care about free software.  We can't
>>>> > take for granted that they do.
>>>> >
>>>> > Perhaps we could check automatically that the grammar found is properly
>>>> > licenses, and disregard any grammars that are not free.
>>>> >
>>>> > By contrast, if grammars are going to be packaged and released for
>>>> > distros, and chosen for installation by users, then it is the user's
>>>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>>>> > (and the GNU/Linux distro might insist on that).
>>>> 
>>>> It might take a while for that to happen, which is why I still believe
>>>> it would be better if tree-sitter major modes would populate
>>>> `treesit-language-source-alist' on their own, and point to the specific
>>>> checkouts that the major mode developer tested their implementation
>>>> against.
>>>
>>>We could have done that, but there's no way we could keep the value of
>>>treesit-language-source-alist up-to-date, because the grammar
>>>libraries put out new versions much more frequently than Emacs
>>>releases, especially if you consider libraries that have no official
>>>versions at all (in which case we can only point to some revision in
>>>their repository).
>>>
>>>The question that bothers me is how useful is it to have
>>>treesit-language-source-alist that is outdated?  What do we expect the
>>>users to do with such an outdated value?
>>>
>>
>> Why not just vendor all the grammars with the Emacs modes that use them?
>
>I am guessing part of the reason is that TS grammars are not fun to
>build.  IIRC they are specified in a Javascript DSL (that used to
>require node.js but AFAIU works with other implementations as well),
>that a program written in Rust translates to C code.  So do we vendor
>the DSL and depend on the TreeSitter toolchain or do we vender the
>generated code?

It's a shame there's no way to write TS grammars in plain elisp. I figure vendoring both the source and the generated code would be best, as it'd allow building Emacs anywhere but still make it convenient on systems with needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As with any scheme involving checking in generated outputs, the source and output can get out of sync, but I think there are build time guardrails we can build to make sure it doesn't happen.

Re: Tree-sitter maturity

[Philip Kaludercic]

Daniel Colascione <dancol@dancol.org> writes:

> On December 27, 2024 9:19:12 AM EST, Philip Kaludercic <philipk@posteo.net> wrote:
>>Daniel Colascione <dancol@dancol.org> writes:
>>
>>> On December 27, 2024 7:40:19 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>> From: Philip Kaludercic <philipk@posteo.net>
>>>>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>>>>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>>>>> 
>>>>> Richard Stallman <rms@gnu.org> writes:
>>>>> 
>>>>> > If we add something like this to Emacs, there is an issue we need to
>>>>> > take care about: to make carefully sure that it does not install
>>>>> > any nonfree grammars.  I don't know how those grammars are released,
>>>>> > ir by whom, or how much they care about free software.  We can't
>>>>> > take for granted that they do.
>>>>> >
>>>>> > Perhaps we could check automatically that the grammar found is properly
>>>>> > licenses, and disregard any grammars that are not free.
>>>>> >
>>>>> > By contrast, if grammars are going to be packaged and released for
>>>>> > distros, and chosen for installation by users, then it is the user's
>>>>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>>>>> > (and the GNU/Linux distro might insist on that).
>>>>> 
>>>>> It might take a while for that to happen, which is why I still believe
>>>>> it would be better if tree-sitter major modes would populate
>>>>> `treesit-language-source-alist' on their own, and point to the specific
>>>>> checkouts that the major mode developer tested their implementation
>>>>> against.
>>>>
>>>>We could have done that, but there's no way we could keep the value of
>>>>treesit-language-source-alist up-to-date, because the grammar
>>>>libraries put out new versions much more frequently than Emacs
>>>>releases, especially if you consider libraries that have no official
>>>>versions at all (in which case we can only point to some revision in
>>>>their repository).
>>>>
>>>>The question that bothers me is how useful is it to have
>>>>treesit-language-source-alist that is outdated?  What do we expect the
>>>>users to do with such an outdated value?
>>>>
>>>
>>> Why not just vendor all the grammars with the Emacs modes that use them?
>>
>>I am guessing part of the reason is that TS grammars are not fun to
>>build.  IIRC they are specified in a Javascript DSL (that used to
>>require node.js but AFAIU works with other implementations as well),
>>that a program written in Rust translates to C code.  So do we vendor
>>the DSL and depend on the TreeSitter toolchain or do we vender the
>>generated code?
>
> It's a shame there's no way to write TS grammars in plain elisp. I
> figure vendoring both the source and the generated code would be best,
> as it'd allow building Emacs anywhere but still make it convenient on
> systems with needed tools (JS runtime, Rust, etc.) to update and
> modify the grammar. As with any scheme involving checking in generated
> outputs, the source and output can get out of sync, but I think there
> are build time guardrails we can build to make sure it doesn't happen.

Writing the grammar in Elisp would require both a new toolchain and the
effort of rewriting all the existing grammars in Elisp.  My
understanding of the benefit that TS intends to provide, is that the
manpower invested into writing grammars that deal with all the
edge-cases which traditional regexp/heuristic parsing had difficulties
with.

There is also the general point of helping to realise software freedom,
where a -ts-mode makes it much more difficult (though of course not
impossible) to adjust a grammar.  Wasn't there some complication when
trying to reload a grammar?  The additional dependencies and the
indirect effect of changes compared with Elisp is something we should be
concerned about when trying to maintain "the spirit of Emacs" (which of
course means different things to different people).

Vendoring might help to reproduce builds if that turns out to be a big
issue, but I am not a fan of the additional hurdles in making use of the
source code.  Does anyone know of alternative, less invested
build-chain the re-uses the libtree-sitter.so library.

Re: Tree-sitter maturity

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Daniel Colascione <dancol@dancol.org> writes:
>
>> On December 27, 2024 9:19:12 AM EST, Philip Kaludercic <philipk@posteo.net> wrote:
>>>Daniel Colascione <dancol@dancol.org> writes:
>>>
>>>> On December 27, 2024 7:40:19 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>>> From: Philip Kaludercic <philipk@posteo.net>
>>>>>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>>>>>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>>>>>> 
>>>>>> Richard Stallman <rms@gnu.org> writes:
>>>>>> 
>>>>>> > If we add something like this to Emacs, there is an issue we need to
>>>>>> > take care about: to make carefully sure that it does not install
>>>>>> > any nonfree grammars.  I don't know how those grammars are released,
>>>>>> > ir by whom, or how much they care about free software.  We can't
>>>>>> > take for granted that they do.
>>>>>> >
>>>>>> > Perhaps we could check automatically that the grammar found is properly
>>>>>> > licenses, and disregard any grammars that are not free.
>>>>>> >
>>>>>> > By contrast, if grammars are going to be packaged and released for
>>>>>> > distros, and chosen for installation by users, then it is the user's
>>>>>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>>>>>> > (and the GNU/Linux distro might insist on that).
>>>>>> 
>>>>>> It might take a while for that to happen, which is why I still believe
>>>>>> it would be better if tree-sitter major modes would populate
>>>>>> `treesit-language-source-alist' on their own, and point to the specific
>>>>>> checkouts that the major mode developer tested their implementation
>>>>>> against.
>>>>>
>>>>>We could have done that, but there's no way we could keep the value of
>>>>>treesit-language-source-alist up-to-date, because the grammar
>>>>>libraries put out new versions much more frequently than Emacs
>>>>>releases, especially if you consider libraries that have no official
>>>>>versions at all (in which case we can only point to some revision in
>>>>>their repository).
>>>>>
>>>>>The question that bothers me is how useful is it to have
>>>>>treesit-language-source-alist that is outdated?  What do we expect the
>>>>>users to do with such an outdated value?
>>>>>
>>>>
>>>> Why not just vendor all the grammars with the Emacs modes that use them?
>>>
>>>I am guessing part of the reason is that TS grammars are not fun to
>>>build.  IIRC they are specified in a Javascript DSL (that used to
>>>require node.js but AFAIU works with other implementations as well),
>>>that a program written in Rust translates to C code.  So do we vendor
>>>the DSL and depend on the TreeSitter toolchain or do we vender the
>>>generated code?
>>
>> It's a shame there's no way to write TS grammars in plain elisp. I
>> figure vendoring both the source and the generated code would be best,
>> as it'd allow building Emacs anywhere but still make it convenient on
>> systems with needed tools (JS runtime, Rust, etc.) to update and
>> modify the grammar. As with any scheme involving checking in generated
>> outputs, the source and output can get out of sync, but I think there
>> are build time guardrails we can build to make sure it doesn't happen.
>
> Writing the grammar in Elisp would require both a new toolchain and the
> effort of rewriting all the existing grammars in Elisp.  My
> understanding of the benefit that TS intends to provide, is that the
> manpower invested into writing grammars that deal with all the
> edge-cases which traditional regexp/heuristic parsing had difficulties
> with.
>
> There is also the general point of helping to realise software freedom,
> where a -ts-mode makes it much more difficult (though of course not
> impossible) to adjust a grammar.  Wasn't there some complication when
> trying to reload a grammar?  The additional dependencies and the
> indirect effect of changes compared with Elisp is something we should be
> concerned about when trying to maintain "the spirit of Emacs" (which of
> course means different things to different people).
>
> Vendoring might help to reproduce builds if that turns out to be a big
> issue, but I am not a fan of the additional hurdles in making use of the
> source code.  Does anyone know of alternative, less invested
> build-chain the re-uses the libtree-sitter.so library.

Oh, and another point I have been reminded of while writing this: The
recent addition of more and more -ts-modes without "regular" -modes has
been slightly concerning.  While I understand that re-implementing a
"lua-mode" or "php-mode" from scratch is not an effort one wants to
impose on anymore, simpler files such as dockerfile-mode or go-mod-mode
/without/ Tree Sitter would be a nice thing to have for people on
systems without Tree Sitter, or without the ability to download and
build code from GitHub (e.g. missing internet access, without Git/GCC,
without the necessary development libraries).  Even if the experience
were degraded, just re-using the keywords to provide some basic
highlighting would be a nice fallback.

Re: Tree-sitter maturity

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Oh, and another point I have been reminded of while writing this: The
  > recent addition of more and more -ts-modes without "regular" -modes has
  > been slightly concerning.  While I understand that re-implementing a
  > "lua-mode" or "php-mode" from scratch is not an effort one wants to
  > impose on anymore,

This is not clean, and might be a real problem.  What should happen
when someone who does not use tree-sitter visits a Lua file or a PHP
file?  What happens in those cases now?

In general, Emacs should not have a FOO-ts-mode without a correspnding
FOO-mode.  Otherwise users will get surprised.  I'm not talking about
_how_ they work, just that the commands should exist.

It might be simple to modify lua-ts-mode so that it can be directed to
disable tree-sitter, perhaps by a global variable -- and then define
lua-mode to call lua-ts-mode after specifying that.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Tree-sitter maturity

[Daniel Colascione]

On December 28, 2024 11:19:45 PM EST, Richard Stallman <rms@gnu.org> wrote:
>[[[ To any NSA and FBI agents reading my email: please consider    ]]]
>[[[ whether defending the US Constitution against all enemies,     ]]]
>[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>  > Oh, and another point I have been reminded of while writing this: The
>  > recent addition of more and more -ts-modes without "regular" -modes has
>  > been slightly concerning.  While I understand that re-implementing a
>  > "lua-mode" or "php-mode" from scratch is not an effort one wants to
>  > impose on anymore,
>
>This is not clean, and might be a real problem.  What should happen
>when someone who does not use tree-sitter visits a Lua file or a PHP
>file?  What happens in those cases now?
>
>In general, Emacs should not have a FOO-ts-mode without a correspnding
>FOO-mode.  Otherwise users will get surprised.  I'm not talking about
>_how_ they work, just that the commands should exist.

Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.

>
>It might be simple to modify lua-ts-mode so that it can be directed to
>disable tree-sitter, perhaps by a global variable -- and then define
>lua-mode to call lua-ts-mode after specifying that.
>


But the mode wouldn't do anything then.

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Sat, 28 Dec 2024 23:23:55 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: emacs-devel@gnu.org, manphiz@gmail.com
> 
> >In general, Emacs should not have a FOO-ts-mode without a correspnding
> >FOO-mode.  Otherwise users will get surprised.  I'm not talking about
> >_how_ they work, just that the commands should exist.
> 
> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.

We started with this concept of adding tree-sitter based modes to
auto-mode-alist by default, but found that people who don't have the
grammar installed didn't appreciate seeing the warnings about the
missing grammars.  So Emacs 29 made these modes optional, activated
only by an explicit user action.  Emacs 30 still does that.

We are currently discussing how to improve this (see the thread Re:
Turning on/off tree-sitter modes, which seems to have stalled lately).
But until the grammar libraries are ubiquitous, and we can rely on
them being present on most systems, I think we will still need some
user say-so before enabling tree-sitter based modes.

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 2:44:19 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Sat, 28 Dec 2024 23:23:55 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: emacs-devel@gnu.org, manphiz@gmail.com
>> 
>> >In general, Emacs should not have a FOO-ts-mode without a correspnding
>> >FOO-mode.  Otherwise users will get surprised.  I'm not talking about
>> >_how_ they work, just that the commands should exist.
>> 
>> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.
>
>We started with this concept of adding tree-sitter based modes to
>auto-mode-alist by default, but found that people who don't have the
>grammar installed didn't appreciate seeing the warnings about the
>missing grammars.  So Emacs 29 made these modes optional, activated
>only by an explicit user action.  Emacs 30 still does that.
>
>We are currently discussing how to improve this (see the thread Re:
>Turning on/off tree-sitter modes, which seems to have stalled lately).
>But until the grammar libraries are ubiquitous, and we can rely on
>them being present on most systems, I think we will still need some
>user say-so before enabling tree-sitter based modes.

Wouldn't vendoring the grammars, and maybe even tree sitter itself, silence the complaints about the warnings? Tree sitter is pure algorithmic code. It doesn't have any particular platform dependencies. Why not simplify the whole system and make it a mandatory (and optionally bundled) dependency so that the show cognitive load of having to consider non-TS environments is just deleted?

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Sun, 29 Dec 2024 03:01:44 -0500
> From: Daniel Colascione <dancol@dancol.org>
> 
> >> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.
> >
> >We started with this concept of adding tree-sitter based modes to
> >auto-mode-alist by default, but found that people who don't have the
> >grammar installed didn't appreciate seeing the warnings about the
> >missing grammars.  So Emacs 29 made these modes optional, activated
> >only by an explicit user action.  Emacs 30 still does that.
> >
> >We are currently discussing how to improve this (see the thread Re:
> >Turning on/off tree-sitter modes, which seems to have stalled lately).
> >But until the grammar libraries are ubiquitous, and we can rely on
> >them being present on most systems, I think we will still need some
> >user say-so before enabling tree-sitter based modes.
> 
> Wouldn't vendoring the grammars, and maybe even tree sitter itself, silence the complaints about the warnings? Tree sitter is pure algorithmic code. It doesn't have any particular platform dependencies. Why not simplify the whole system and make it a mandatory (and optionally bundled) dependency so that the show cognitive load of having to consider non-TS environments is just deleted?

First, the tree-sitter library itself is optional, so Emacs could be
built without it.  Or are you suggesting to import the library as well
into Emacs?  If we don't import the library, making it a mandatory
dependency is not TRT, IMO, because some users don't need the modes
supported by tree-sitter, so forcing them to install the library that
is not really useful to them is not right.  We never do anything like
that with any other external libraries.  GMP is special, but even for
it we added our own "mini-gmp".

Next, importing the grammar libraries into Emacs is not a simple
matter, either.  Their sources are in JavaScript, so if we want to let
users produce modified grammars (as we do with everything we have in
the release tarballs), they will need to have Node.js etc. installed,
which will become a prerequisite.  And there are other complications,
like the need to sync regularly with their upstream repositories.
Moreover, there's no precedent for doing this, if you exclude lwlib
and oldXMenu (which are different, since they are not developed
outside Emacs).

So I, for one, am not very happy to add this to our maintenance
burden.  It might make things easier for some (but see below), but it
doesn't come for free.

I also don't understand the fuss, really.  Compiling a grammar library
after cloning the repository takes seconds, so why do we have to do
all this on behalf of the users if the users can do it so easily, even
if distros don't?  E.g., I have on my system almost 70 grammar
libraries, which I regularly update and build with a small number of
simple Makefiles -- how hard can that be for anyone who is interested
in these modes?  Why does it have to be _our_ responsibility, any more
than, say, Grep or Findutils -- which are also heavily used by Emacs?
Or even the image libraries?  Why shouldn't this be the job of the
distros?  The upstream project doesn't have to think about packaging,
it's the job of the distros.

Can you describe why the current situation means "cognitive load" on
you personally?  Because I don't think I see the problem, honestly.

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 29, 2024, at 12:41 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> Date: Sun, 29 Dec 2024 03:01:44 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> 
>>>> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.
>>> 
>>> We started with this concept of adding tree-sitter based modes to
>>> auto-mode-alist by default, but found that people who don't have the
>>> grammar installed didn't appreciate seeing the warnings about the
>>> missing grammars.  So Emacs 29 made these modes optional, activated
>>> only by an explicit user action.  Emacs 30 still does that.
>>> 
>>> We are currently discussing how to improve this (see the thread Re:
>>> Turning on/off tree-sitter modes, which seems to have stalled lately).
>>> But until the grammar libraries are ubiquitous, and we can rely on
>>> them being present on most systems, I think we will still need some
>>> user say-so before enabling tree-sitter based modes.
>> 
>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, silence the complaints about the warnings? Tree sitter is pure algorithmic code. It doesn't have any particular platform dependencies. Why not simplify the whole system and make it a mandatory (and optionally bundled) dependency so that the show cognitive load of having to consider non-TS environments is just deleted?
> 
> First, the tree-sitter library itself is optional, so Emacs could be
> built without it.  Or are you suggesting to import the library as well
> into Emacs?  If we don't import the library, making it a mandatory
> dependency is not TRT, IMO, because some users don't need the modes
> supported by tree-sitter, so forcing them to install the library that
> is not really useful to them is not right.  We never do anything like
> that with any other external libraries.  GMP is special, but even for
> it we added our own "mini-gmp".
> 
> Next, importing the grammar libraries into Emacs is not a simple
> matter, either.  Their sources are in JavaScript, so if we want to let
> users produce modified grammars (as we do with everything we have in
> the release tarballs), they will need to have Node.js etc. installed,
> which will become a prerequisite.  And there are other complications,
> like the need to sync regularly with their upstream repositories.
> Moreover, there's no precedent for doing this, if you exclude lwlib
> and oldXMenu (which are different, since they are not developed
> outside Emacs).
> 
> So I, for one, am not very happy to add this to our maintenance
> burden.  It might make things easier for some (but see below), but it
> doesn't come for free.
> 
> I also don't understand the fuss, really.  Compiling a grammar library
> after cloning the repository takes seconds, so why do we have to do
> all this on behalf of the users if the users can do it so easily, even
> if distros don't?  E.g., I have on my system almost 70 grammar
> libraries, which I regularly update and build with a small number of
> simple Makefiles -- how hard can that be for anyone who is interested
> in these modes?  Why does it have to be _our_ responsibility, any more
> than, say, Grep or Findutils -- which are also heavily used by Emacs?
> Or even the image libraries?  Why shouldn't this be the job of the
> distros?  The upstream project doesn't have to think about packaging,
> it's the job of the distros.

Also, distros are picking up on packaging tree-sitter grammars, so I’m hopeful of a future where Emacs is packaged with tree-sitter grammars for the builtin major modes. And AFAIK packagers very much dislike editors bundling tree-sitter library and grammars themselves.

Bundling grammars has another complication which is tree-sitter library-grammar compatibility. If we were to bundle grammars, we must also bundle tree-sitter library, lest we risk to encounter a tree-sitter library provided by the system that’s incompatible with the bundled grammars. And bundling the tree-sitter library is obviously undesirable.

Yuan

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 3:59:50 AM EST, Yuan Fu <casouri@gmail.com> wrote:
>
>
>> On Dec 29, 2024, at 12:41 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>>> Date: Sun, 29 Dec 2024 03:01:44 -0500
>>> From: Daniel Colascione <dancol@dancol.org>
>>> 
>>>>> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.
>>>> 
>>>> We started with this concept of adding tree-sitter based modes to
>>>> auto-mode-alist by default, but found that people who don't have the
>>>> grammar installed didn't appreciate seeing the warnings about the
>>>> missing grammars.  So Emacs 29 made these modes optional, activated
>>>> only by an explicit user action.  Emacs 30 still does that.
>>>> 
>>>> We are currently discussing how to improve this (see the thread Re:
>>>> Turning on/off tree-sitter modes, which seems to have stalled lately).
>>>> But until the grammar libraries are ubiquitous, and we can rely on
>>>> them being present on most systems, I think we will still need some
>>>> user say-so before enabling tree-sitter based modes.
>>> 
>>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, silence the complaints about the warnings? Tree sitter is pure algorithmic code. It doesn't have any particular platform dependencies. Why not simplify the whole system and make it a mandatory (and optionally bundled) dependency so that the show cognitive load of having to consider non-TS environments is just deleted?
>> 
>> First, the tree-sitter library itself is optional, so Emacs could be
>> built without it.  Or are you suggesting to import the library as well
>> into Emacs?  If we don't import the library, making it a mandatory
>> dependency is not TRT, IMO, because some users don't need the modes
>> supported by tree-sitter, so forcing them to install the library that
>> is not really useful to them is not right.  We never do anything like
>> that with any other external libraries.  GMP is special, but even for
>> it we added our own "mini-gmp".
>> 
>> Next, importing the grammar libraries into Emacs is not a simple
>> matter, either.  Their sources are in JavaScript, so if we want to let
>> users produce modified grammars (as we do with everything we have in
>> the release tarballs), they will need to have Node.js etc. installed,
>> which will become a prerequisite.  And there are other complications,
>> like the need to sync regularly with their upstream repositories.
>> Moreover, there's no precedent for doing this, if you exclude lwlib
>> and oldXMenu (which are different, since they are not developed
>> outside Emacs).
>> 
>> So I, for one, am not very happy to add this to our maintenance
>> burden.  It might make things easier for some (but see below), but it
>> doesn't come for free.
>> 
>> I also don't understand the fuss, really.  Compiling a grammar library
>> after cloning the repository takes seconds, so why do we have to do
>> all this on behalf of the users if the users can do it so easily, even
>> if distros don't?  E.g., I have on my system almost 70 grammar
>> libraries, which I regularly update and build with a small number of
>> simple Makefiles -- how hard can that be for anyone who is interested
>> in these modes?  Why does it have to be _our_ responsibility, any more
>> than, say, Grep or Findutils -- which are also heavily used by Emacs?
>> Or even the image libraries?  Why shouldn't this be the job of the
>> distros?  The upstream project doesn't have to think about packaging,
>> it's the job of the distros.
>
>Also, distros are picking up on packaging tree-sitter grammars, so I’m hopeful of a future where Emacs is packaged with tree-sitter grammars for the builtin major modes. And AFAIK packagers very much dislike editors bundling tree-sitter library and grammars themselves.
>
>Bundling grammars has another complication which is tree-sitter library-grammar compatibility. If we were to bundle grammars, we must also bundle tree-sitter library, lest we risk to encounter a tree-sitter library provided by the system that’s incompatible with the bundled grammars. And bundling the tree-sitter library is obviously undesirable.
>
>Yuan

The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such. ISTM the only way to guarantee compatibility is to vendor the whole stack.

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Sun, 29 Dec 2024 04:14:37 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: emacs-devel@gnu.org
> 
> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such. ISTM the only way to guarantee compatibility is to vendor the whole stack.

I hope that, as the use of tree-sitter and the grammars increases, the
distros will work with the grammar developers so that the latter get
their act together and start producing more dependable releases with
clear versioning and library compatibilities.

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 4:24:11 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Sun, 29 Dec 2024 04:14:37 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: emacs-devel@gnu.org
>> 
>> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such. ISTM the only way to guarantee compatibility is to vendor the whole stack.
>
>I hope that, as the use of tree-sitter and the grammars increases, the
>distros will work with the grammar developers so that the latter get
>their act together and start producing more dependable releases with
>clear versioning and library compatibilities.

Is hope a strategy?

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Sun, 29 Dec 2024 05:01:23 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: casouri@gmail.com, emacs-devel@gnu.org
> 
> 
> 
> On December 29, 2024 4:24:11 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
> >> Date: Sun, 29 Dec 2024 04:14:37 -0500
> >> From: Daniel Colascione <dancol@dancol.org>
> >> CC: emacs-devel@gnu.org
> >> 
> >> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such. ISTM the only way to guarantee compatibility is to vendor the whole stack.
> >
> >I hope that, as the use of tree-sitter and the grammars increases, the
> >distros will work with the grammar developers so that the latter get
> >their act together and start producing more dependable releases with
> >clear versioning and library compatibilities.
> 
> Is hope a strategy?

No, just hope.

Re: Tree-sitter maturity

[Daniel Colascione]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 29 Dec 2024 05:01:23 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: casouri@gmail.com, emacs-devel@gnu.org
>> 
>> 
>> 
>> On December 29, 2024 4:24:11 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> Date: Sun, 29 Dec 2024 04:14:37 -0500
>> >> From: Daniel Colascione <dancol@dancol.org>
>> >> CC: emacs-devel@gnu.org
>> >> 
>> >> The grammars don't make any backwards compatibility
>> >> guarantees. There have been multiple Emacs bugs arising from
>> >> grammars unilaterally changing terminal names and such. ISTM the
>> >> only way to guarantee compatibility is to vendor the whole stack.
>> >
>> >I hope that, as the use of tree-sitter and the grammars increases, the
>> >distros will work with the grammar developers so that the latter get
>> >their act together and start producing more dependable releases with
>> >clear versioning and library compatibilities.
>> 
>> Is hope a strategy?
>
> No, just hope.

I think grammars and are too tightly coupled to AST consumers to ever
have stable interfaces.  If the "API" of the system is the precise set
of terminals and non-terminals a grammar recognizes in a stream of text,
then, when you enlarge the recognized language, you break old consumers,
who have no idea what to do with AST nodes that didn't even exist when
they were written. Besides: augmenting a grammar with support for a new
language feature sometimes requires changing the parse trees for even
old code!

That's why I come down so strongly in favor of bundling grammars with
code that consumes them.  If you do so, then at worst, your combined
grammar-and-Emacs-mode might not recognize new language features, but
it's not going to suddenly choke on language features that
haven't changed!

Any kind of stable language-analysis library would have to work at a
higher level than tree-sitter.  Such a library would have to provide
interfaces for highlighting, indentation, completion, and so on: these
concepts, unlike AST node identity, can be stable across
language revisions.  It'd look a lot like LSP.

(AFAIK, there's no general-purpose implementation of this concept.
LSP has some support for highlighting, but it's meant as a helper for
built-in highlighting, not a total replacement.)

Tree sitter is great, but I'd much rather treat it more like flex and
bison than as some externally-shipped thing.  Nobody objects to a
program having a .y file and generating the .c at build time.
Nobody insists that we dynamically link against some separately-shipped
Bison parser output.  We don't have, e.g. GCC link against some kind of
libcparser.so and expect the latter to provide a long-term stable
interface.  GCC just does its own parsing, and there's no expectation
that GCC's internal parser present a stable interface to a different
part of the same GCC program.

It's because of this dynamic, BTW, that I really want c-ts-mode *not* to
provide indentation customization based on raw tree-sitter AST queries
but instead emulate cc-mode's higher level customization.

Re: Tree-sitter maturity

[tomas]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Sun, Dec 29, 2024 at 04:14:37AM -0500, Daniel Colascione wrote:

[...]

> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such. ISTM the only way to guarantee compatibility is to vendor the whole stack.

Since you asked in another thread: you just gave me
another strong reason to steer clear of TS.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 29, 2024, at 1:14 AM, Daniel Colascione <dancol@dancol.org> wrote:
> 
> 
> 
> On December 29, 2024 3:59:50 AM EST, Yuan Fu <casouri@gmail.com> wrote:
>> 
>> 
>>> On Dec 29, 2024, at 12:41 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> Date: Sun, 29 Dec 2024 03:01:44 -0500
>>>> From: Daniel Colascione <dancol@dancol.org>
>>>> 
>>>>>> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.
>>>>> 
>>>>> We started with this concept of adding tree-sitter based modes to
>>>>> auto-mode-alist by default, but found that people who don't have the
>>>>> grammar installed didn't appreciate seeing the warnings about the
>>>>> missing grammars.  So Emacs 29 made these modes optional, activated
>>>>> only by an explicit user action.  Emacs 30 still does that.
>>>>> 
>>>>> We are currently discussing how to improve this (see the thread Re:
>>>>> Turning on/off tree-sitter modes, which seems to have stalled lately).
>>>>> But until the grammar libraries are ubiquitous, and we can rely on
>>>>> them being present on most systems, I think we will still need some
>>>>> user say-so before enabling tree-sitter based modes.
>>>> 
>>>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, silence the complaints about the warnings? Tree sitter is pure algorithmic code. It doesn't have any particular platform dependencies. Why not simplify the whole system and make it a mandatory (and optionally bundled) dependency so that the show cognitive load of having to consider non-TS environments is just deleted?
>>> 
>>> First, the tree-sitter library itself is optional, so Emacs could be
>>> built without it.  Or are you suggesting to import the library as well
>>> into Emacs?  If we don't import the library, making it a mandatory
>>> dependency is not TRT, IMO, because some users don't need the modes
>>> supported by tree-sitter, so forcing them to install the library that
>>> is not really useful to them is not right.  We never do anything like
>>> that with any other external libraries.  GMP is special, but even for
>>> it we added our own "mini-gmp".
>>> 
>>> Next, importing the grammar libraries into Emacs is not a simple
>>> matter, either.  Their sources are in JavaScript, so if we want to let
>>> users produce modified grammars (as we do with everything we have in
>>> the release tarballs), they will need to have Node.js etc. installed,
>>> which will become a prerequisite.  And there are other complications,
>>> like the need to sync regularly with their upstream repositories.
>>> Moreover, there's no precedent for doing this, if you exclude lwlib
>>> and oldXMenu (which are different, since they are not developed
>>> outside Emacs).
>>> 
>>> So I, for one, am not very happy to add this to our maintenance
>>> burden.  It might make things easier for some (but see below), but it
>>> doesn't come for free.
>>> 
>>> I also don't understand the fuss, really.  Compiling a grammar library
>>> after cloning the repository takes seconds, so why do we have to do
>>> all this on behalf of the users if the users can do it so easily, even
>>> if distros don't?  E.g., I have on my system almost 70 grammar
>>> libraries, which I regularly update and build with a small number of
>>> simple Makefiles -- how hard can that be for anyone who is interested
>>> in these modes?  Why does it have to be _our_ responsibility, any more
>>> than, say, Grep or Findutils -- which are also heavily used by Emacs?
>>> Or even the image libraries?  Why shouldn't this be the job of the
>>> distros?  The upstream project doesn't have to think about packaging,
>>> it's the job of the distros.
>> 
>> Also, distros are picking up on packaging tree-sitter grammars, so I’m hopeful of a future where Emacs is packaged with tree-sitter grammars for the builtin major modes. And AFAIK packagers very much dislike editors bundling tree-sitter library and grammars themselves.
>> 
>> Bundling grammars has another complication which is tree-sitter library-grammar compatibility. If we were to bundle grammars, we must also bundle tree-sitter library, lest we risk to encounter a tree-sitter library provided by the system that’s incompatible with the bundled grammars. And bundling the tree-sitter library is obviously undesirable.
>> 
>> Yuan
> 
> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such.

They don’t have to be backward compatible. Any change to the grammar is probably backward-incompatible simply due to the nature of grammars. The situation will be better once package managers start packing grammars with Emacs. I’m also working on making Emacs explicitly state what versions of grammars are compatible, so people can choose the right grammar version to install.

> ISTM the only way to guarantee compatibility is to vendor the whole stack.

Vendoring the whole stack is one way, yes, but why do you think letting package managers to package Emacs with the right grammars wouldn’t work? 

Yuan

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 5:21:21 AM EST, Yuan Fu <casouri@gmail.com> wrote:
>
>
>> On Dec 29, 2024, at 1:14 AM, Daniel Colascione <dancol@dancol.org> wrote:
>> 
>> 
>> 
>> On December 29, 2024 3:59:50 AM EST, Yuan Fu <casouri@gmail.com> wrote:
>>> 
>>> 
>>>> On Dec 29, 2024, at 12:41 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> 
>>>>> Date: Sun, 29 Dec 2024 03:01:44 -0500
>>>>> From: Daniel Colascione <dancol@dancol.org>
>>>>> 
>>>>>>> Enforcing this policy will just mean that Emacs doesn't support *at all* some languages out of the box and will put even more wind in the sails of soft forks like Doom. Tree sitter language descriptions are free software. There's no reason not to rely on them.
>>>>>> 
>>>>>> We started with this concept of adding tree-sitter based modes to
>>>>>> auto-mode-alist by default, but found that people who don't have the
>>>>>> grammar installed didn't appreciate seeing the warnings about the
>>>>>> missing grammars.  So Emacs 29 made these modes optional, activated
>>>>>> only by an explicit user action.  Emacs 30 still does that.
>>>>>> 
>>>>>> We are currently discussing how to improve this (see the thread Re:
>>>>>> Turning on/off tree-sitter modes, which seems to have stalled lately).
>>>>>> But until the grammar libraries are ubiquitous, and we can rely on
>>>>>> them being present on most systems, I think we will still need some
>>>>>> user say-so before enabling tree-sitter based modes.
>>>>> 
>>>>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, silence the complaints about the warnings? Tree sitter is pure algorithmic code. It doesn't have any particular platform dependencies. Why not simplify the whole system and make it a mandatory (and optionally bundled) dependency so that the show cognitive load of having to consider non-TS environments is just deleted?
>>>> 
>>>> First, the tree-sitter library itself is optional, so Emacs could be
>>>> built without it.  Or are you suggesting to import the library as well
>>>> into Emacs?  If we don't import the library, making it a mandatory
>>>> dependency is not TRT, IMO, because some users don't need the modes
>>>> supported by tree-sitter, so forcing them to install the library that
>>>> is not really useful to them is not right.  We never do anything like
>>>> that with any other external libraries.  GMP is special, but even for
>>>> it we added our own "mini-gmp".
>>>> 
>>>> Next, importing the grammar libraries into Emacs is not a simple
>>>> matter, either.  Their sources are in JavaScript, so if we want to let
>>>> users produce modified grammars (as we do with everything we have in
>>>> the release tarballs), they will need to have Node.js etc. installed,
>>>> which will become a prerequisite.  And there are other complications,
>>>> like the need to sync regularly with their upstream repositories.
>>>> Moreover, there's no precedent for doing this, if you exclude lwlib
>>>> and oldXMenu (which are different, since they are not developed
>>>> outside Emacs).
>>>> 
>>>> So I, for one, am not very happy to add this to our maintenance
>>>> burden.  It might make things easier for some (but see below), but it
>>>> doesn't come for free.
>>>> 
>>>> I also don't understand the fuss, really.  Compiling a grammar library
>>>> after cloning the repository takes seconds, so why do we have to do
>>>> all this on behalf of the users if the users can do it so easily, even
>>>> if distros don't?  E.g., I have on my system almost 70 grammar
>>>> libraries, which I regularly update and build with a small number of
>>>> simple Makefiles -- how hard can that be for anyone who is interested
>>>> in these modes?  Why does it have to be _our_ responsibility, any more
>>>> than, say, Grep or Findutils -- which are also heavily used by Emacs?
>>>> Or even the image libraries?  Why shouldn't this be the job of the
>>>> distros?  The upstream project doesn't have to think about packaging,
>>>> it's the job of the distros.
>>> 
>>> Also, distros are picking up on packaging tree-sitter grammars, so I’m hopeful of a future where Emacs is packaged with tree-sitter grammars for the builtin major modes. And AFAIK packagers very much dislike editors bundling tree-sitter library and grammars themselves.
>>> 
>>> Bundling grammars has another complication which is tree-sitter library-grammar compatibility. If we were to bundle grammars, we must also bundle tree-sitter library, lest we risk to encounter a tree-sitter library provided by the system that’s incompatible with the bundled grammars. And bundling the tree-sitter library is obviously undesirable.
>>> 
>>> Yuan
>> 
>> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such.
>
>They don’t have to be backward compatible. Any change to the grammar is probably backward-incompatible simply due to the nature of grammars. The situation will be better once package managers start packing grammars with Emacs. I’m also working on making Emacs explicitly state what versions of grammars are compatible, so people can choose the right grammar version to install.
>
>> ISTM the only way to guarantee compatibility is to vendor the whole stack.
>
>Vendoring the whole stack is one way, yes, but why do you think letting package managers to package Emacs with the right grammars wouldn’t work? 

I'm thinking of single-versioning issues. If there's a Debian package for, e.g. the C++ grammar, and different grammar versions are incompatible, and two programs want that grammar, each at a different version, there's no package system that will make both programs happy.

Just vendoring the stack keeps things simple, reproducible, and compatible. The grammars are small enough that I don't see much of a downside in vendoring them either.

Re: Tree-sitter maturity

[Dmitry Gutov]

On 29/12/2024 11:14, Daniel Colascione wrote:
> The grammars don't make any backwards compatibility guarantees. There have been multiple Emacs bugs arising from grammars unilaterally changing terminal names and such. ISTM the only way to guarantee compatibility is to vendor the whole stack.

Keeping the list of grammar repos' urls and the "last known working" 
commit hashes is a way to achieve the same (with 99.9% certainty) 
without the associated burden of maintaining the vendored code and the 
commit history churn.

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org, manphiz@gmail.com
> Date: Sat, 28 Dec 2024 23:19:45 -0500
> 
>   > Oh, and another point I have been reminded of while writing this: The
>   > recent addition of more and more -ts-modes without "regular" -modes has
>   > been slightly concerning.  While I understand that re-implementing a
>   > "lua-mode" or "php-mode" from scratch is not an effort one wants to
>   > impose on anymore,
> 
> This is not clean, and might be a real problem.  What should happen
> when someone who does not use tree-sitter visits a Lua file or a PHP
> file?  What happens in those cases now?

Nothing happens.  These modes are currently all optional, so the users
currently must load the mode manually (or in their init files) to have
auto-mode-alist modified to affect visiting the corresponding files.
For example, lua-ts-mode.el has this at top-level:

  (when (treesit-ready-p 'lua)
    (add-to-list 'auto-mode-alist '("\\.lua\\'" . lua-ts-mode))
    (add-to-list 'interpreter-mode-alist '("\\<lua\\(?:jit\\)?" . lua-ts-mode)))

So to activate this, the user needs to load the mode, Emacs should be
compiled with tree-sitter support, and the grammar library should be
installed; otherwise auto-mode-alist will not be modified, and
visiting Lua files will use Fundamental mode.

> In general, Emacs should not have a FOO-ts-mode without a correspnding
> FOO-mode.

I disagree.  I don't object having non-ts modes, of course, but see no
reason to mandate such modes when a tree-sitter-based mode exists.  We
made those modes optional as above for that very reason: to make sure
they are activated only if the user says so and only if the necessary
prerequisites are fulfilled.

> Otherwise users will get surprised.

We made those modes optional to avoid surprising users.

> It might be simple to modify lua-ts-mode so that it can be directed to
> disable tree-sitter, perhaps by a global variable -- and then define
> lua-mode to call lua-ts-mode after specifying that.

The last part was exactly what prompted Alan Mackenzie to resign.  It
sounds like you now agree with what the rest of us think: that
interpreting lua-mode as meaning invoke lua-ts-mode is fine?

Re: Tree-sitter maturity

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 1451 bytes --]

On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:

>
>
> It's a shame there's no way to write TS grammars in plain elisp. I figure
> vendoring both the source and the generated code would be best, as it'd
> allow building Emacs anywhere but still make it convenient on systems with
> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
> with any scheme involving checking in generated outputs, the source and
> output can get out of sync, but I think there are build time guardrails we
> can build to make sure it doesn't happen.
>

I looked into this last year.  The tree-sitter library provides a parsing
engine that references a fairly standard LR type parsing table in binary
form.  I got stuck in adding a generic primitive functionality for reading
and writing arbitrary binary data structures based on a data description
DSL, since I wouldn't want to tie the interpreter core to the data
structures of an external, dynamically-loadable library.  But, I wasn't
sure such an extension would be accepted into emacs, as I am not an expert
on the possible security implications.

Other than that, emacs already has the code for calculating (LA)LR parsing
tables in the semantic packages.  The tree-sitter grammar compiler may have
additional logic for providing multiple starting symbols, but the parsing
engine should still function with a classic parsing table.

Lynn

[-- Attachment #2: Type: text/html, Size: 2180 bytes --]

Re: Tree-sitter maturity

[Daniel Colascione]

Lynn Winebarger <owinebar@gmail.com> writes:

> On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:
>
>>
>>
>> It's a shame there's no way to write TS grammars in plain elisp. I figure
>> vendoring both the source and the generated code would be best, as it'd
>> allow building Emacs anywhere but still make it convenient on systems with
>> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
>> with any scheme involving checking in generated outputs, the source and
>> output can get out of sync, but I think there are build time guardrails we
>> can build to make sure it doesn't happen.
>>
>
> I looked into this last year.  The tree-sitter library provides a parsing
> engine that references a fairly standard LR type parsing table in binary
> form.  I got stuck in adding a generic primitive functionality for reading
> and writing arbitrary binary data structures based on a data description
> DSL, since I wouldn't want to tie the interpreter core to the data
> structures of an external, dynamically-loadable library.  But, I wasn't
> sure such an extension would be accepted into emacs, as I am not an expert
> on the possible security implications.
>
> Other than that, emacs already has the code for calculating (LA)LR parsing
> tables in the semantic packages.  The tree-sitter grammar compiler may have
> additional logic for providing multiple starting symbols, but the parsing
> engine should still function with a classic parsing table.

Thanks.  Such an approach would let us treat tree-sitter grammars a lot
more like font-lock-keywords, and I think for some modes, that'd be a
good option.  (Of course, SHTDI.)

Tree sitter, as wonderful as it is, strikes me as a bit of a Rube
Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-)

Do you happen to know whether the subset of Rust that gccrs recognizes
is sufficient to compile the tree sitter grammar compiler?  If so, we
could in principle combine gccrs with a bare-bones embedded JS
interpreter like https://duckjs.org/ to produce a mechanism that would
let us customize and rebuild tree sitter grammars as easily as we do
elisp files, even on obscure platforms like DJGPP.

Some Emacs modes could ship with .js grammars sourced from upstream
editor-neutral projects.  Other modes might just build tree sitter parse
tables in elisp using something vaguely like SMIE syntax.  Both styles
of mode would be customizable by end users, and we'd (because, I'm a
broken record, vendor vendor vendor) we'd maintain compatibility without
mysterious AST-change-related breakages.

Re: Tree-sitter maturity

[Björn Bidar]

Daniel Colascione <dancol@dancol.org> writes:

> Lynn Winebarger <owinebar@gmail.com> writes:
>
>> On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:
>>
>>>
>>>
>>> It's a shame there's no way to write TS grammars in plain elisp. I figure
>>> vendoring both the source and the generated code would be best, as it'd
>>> allow building Emacs anywhere but still make it convenient on systems with
>>> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
>>> with any scheme involving checking in generated outputs, the source and
>>> output can get out of sync, but I think there are build time guardrails we
>>> can build to make sure it doesn't happen.
>>>
>>
>> I looked into this last year.  The tree-sitter library provides a parsing
>> engine that references a fairly standard LR type parsing table in binary
>> form.  I got stuck in adding a generic primitive functionality for reading
>> and writing arbitrary binary data structures based on a data description
>> DSL, since I wouldn't want to tie the interpreter core to the data
>> structures of an external, dynamically-loadable library.  But, I wasn't
>> sure such an extension would be accepted into emacs, as I am not an expert
>> on the possible security implications.
>>
>> Other than that, emacs already has the code for calculating (LA)LR parsing
>> tables in the semantic packages.  The tree-sitter grammar compiler may have
>> additional logic for providing multiple starting symbols, but the parsing
>> engine should still function with a classic parsing table.
>
> Thanks.  Such an approach would let us treat tree-sitter grammars a lot
> more like font-lock-keywords, and I think for some modes, that'd be a
> good option.  (Of course, SHTDI.)
>
> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube
> Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-)

I was wondering the same. How the hell? There had been some talks to
support a more lightweight JavaScript interpreter as an alternative but
it hasn't gone anyway. Somehow because compatibility reason. I don't how
could node be dependency for these. Grammars are mostly without
dependencies except some have dependencies to other grammars on the
source level such as the C++ require the C grammar.

> Do you happen to know whether the subset of Rust that gccrs recognizes
> is sufficient to compile the tree sitter grammar compiler?  If so, we
> could in principle combine gccrs with a bare-bones embedded JS
> interpreter like https://duckjs.org/ to produce a mechanism that would
> let us customize and rebuild tree sitter grammars as easily as we do
> elisp files, even on obscure platforms like DJGPP.

I don't know 100% but it does not look that way reading their latest
report:
- https://raw.githubusercontent.com/Rust-GCC/Reporting/refs/heads/main/2024/2024-12-09-report.org
- https://rust-gcc.github.io/2024/12/02/2024-11-monthly-report.html

Really strange that GCCrs doesn't use sourceware.org

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 29, 2024, at 3:29 PM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
> 
> Daniel Colascione <dancol@dancol.org> writes:
> 
>> Lynn Winebarger <owinebar@gmail.com> writes:
>> 
>>> On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:
>>> 
>>>> 
>>>> 
>>>> It's a shame there's no way to write TS grammars in plain elisp. I figure
>>>> vendoring both the source and the generated code would be best, as it'd
>>>> allow building Emacs anywhere but still make it convenient on systems with
>>>> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
>>>> with any scheme involving checking in generated outputs, the source and
>>>> output can get out of sync, but I think there are build time guardrails we
>>>> can build to make sure it doesn't happen.
>>>> 
>>> 
>>> I looked into this last year.  The tree-sitter library provides a parsing
>>> engine that references a fairly standard LR type parsing table in binary
>>> form.  I got stuck in adding a generic primitive functionality for reading
>>> and writing arbitrary binary data structures based on a data description
>>> DSL, since I wouldn't want to tie the interpreter core to the data
>>> structures of an external, dynamically-loadable library.  But, I wasn't
>>> sure such an extension would be accepted into emacs, as I am not an expert
>>> on the possible security implications.
>>> 
>>> Other than that, emacs already has the code for calculating (LA)LR parsing
>>> tables in the semantic packages.  The tree-sitter grammar compiler may have
>>> additional logic for providing multiple starting symbols, but the parsing
>>> engine should still function with a classic parsing table.
>> 
>> Thanks.  Such an approach would let us treat tree-sitter grammars a lot
>> more like font-lock-keywords, and I think for some modes, that'd be a
>> good option.  (Of course, SHTDI.)
>> 
>> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube
>> Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-)

> I was wondering the same. How the hell? There had been some talks to
> support a more lightweight JavaScript interpreter as an alternative but
> it hasn't gone anyway. Somehow because compatibility reason. I don't how
> could node be dependency for these. Grammars are mostly without
> dependencies except some have dependencies to other grammars on the
> source level such as the C++ require the C grammar.

I don’t think you need nodejs to build the grammar. You might need it to develop the grammar, but compiling grammar.js to parser.c only requires the tree-sitter CLI which is written in Rust.

Yuan

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 7:30:52 PM EST, Yuan Fu <casouri@gmail.com> wrote:
>
>
>> On Dec 29, 2024, at 3:29 PM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
>> 
>> Daniel Colascione <dancol@dancol.org> writes:
>> 
>>> Lynn Winebarger <owinebar@gmail.com> writes:
>>> 
>>>> On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:
>>>> 
>>>>> 
>>>>> 
>>>>> It's a shame there's no way to write TS grammars in plain elisp. I figure
>>>>> vendoring both the source and the generated code would be best, as it'd
>>>>> allow building Emacs anywhere but still make it convenient on systems with
>>>>> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
>>>>> with any scheme involving checking in generated outputs, the source and
>>>>> output can get out of sync, but I think there are build time guardrails we
>>>>> can build to make sure it doesn't happen.
>>>>> 
>>>> 
>>>> I looked into this last year.  The tree-sitter library provides a parsing
>>>> engine that references a fairly standard LR type parsing table in binary
>>>> form.  I got stuck in adding a generic primitive functionality for reading
>>>> and writing arbitrary binary data structures based on a data description
>>>> DSL, since I wouldn't want to tie the interpreter core to the data
>>>> structures of an external, dynamically-loadable library.  But, I wasn't
>>>> sure such an extension would be accepted into emacs, as I am not an expert
>>>> on the possible security implications.
>>>> 
>>>> Other than that, emacs already has the code for calculating (LA)LR parsing
>>>> tables in the semantic packages.  The tree-sitter grammar compiler may have
>>>> additional logic for providing multiple starting symbols, but the parsing
>>>> engine should still function with a classic parsing table.
>>> 
>>> Thanks.  Such an approach would let us treat tree-sitter grammars a lot
>>> more like font-lock-keywords, and I think for some modes, that'd be a
>>> good option.  (Of course, SHTDI.)
>>> 
>>> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube
>>> Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-)
>
>> I was wondering the same. How the hell? There had been some talks to
>> support a more lightweight JavaScript interpreter as an alternative but
>> it hasn't gone anyway. Somehow because compatibility reason. I don't how
>> could node be dependency for these. Grammars are mostly without
>> dependencies except some have dependencies to other grammars on the
>> source level such as the C++ require the C grammar.
>
>I don’t think you need nodejs to build the grammar. You might need it to develop the grammar, but compiling grammar.js to parser.c only requires the tree-sitter CLI which is written in Rust.
>
>Yuan

Doesn't the CLI shell out to Node?

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 29, 2024, at 4:36 PM, Daniel Colascione <dancol@dancol.org> wrote:
> 
> 
> 
> On December 29, 2024 7:30:52 PM EST, Yuan Fu <casouri@gmail.com> wrote:
>> 
>> 
>>> On Dec 29, 2024, at 3:29 PM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
>>> 
>>> Daniel Colascione <dancol@dancol.org> writes:
>>> 
>>>> Lynn Winebarger <owinebar@gmail.com> writes:
>>>> 
>>>>> On Fri, Dec 27, 2024, 9:25 AM Daniel Colascione <dancol@dancol.org> wrote:
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> It's a shame there's no way to write TS grammars in plain elisp. I figure
>>>>>> vendoring both the source and the generated code would be best, as it'd
>>>>>> allow building Emacs anywhere but still make it convenient on systems with
>>>>>> needed tools (JS runtime, Rust, etc.) to update and modify the grammar. As
>>>>>> with any scheme involving checking in generated outputs, the source and
>>>>>> output can get out of sync, but I think there are build time guardrails we
>>>>>> can build to make sure it doesn't happen.
>>>>>> 
>>>>> 
>>>>> I looked into this last year.  The tree-sitter library provides a parsing
>>>>> engine that references a fairly standard LR type parsing table in binary
>>>>> form.  I got stuck in adding a generic primitive functionality for reading
>>>>> and writing arbitrary binary data structures based on a data description
>>>>> DSL, since I wouldn't want to tie the interpreter core to the data
>>>>> structures of an external, dynamically-loadable library.  But, I wasn't
>>>>> sure such an extension would be accepted into emacs, as I am not an expert
>>>>> on the possible security implications.
>>>>> 
>>>>> Other than that, emacs already has the code for calculating (LA)LR parsing
>>>>> tables in the semantic packages.  The tree-sitter grammar compiler may have
>>>>> additional logic for providing multiple starting symbols, but the parsing
>>>>> engine should still function with a classic parsing table.
>>>> 
>>>> Thanks.  Such an approach would let us treat tree-sitter grammars a lot
>>>> more like font-lock-keywords, and I think for some modes, that'd be a
>>>> good option.  (Of course, SHTDI.)
>>>> 
>>>> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube
>>>> Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-)
>> 
>>> I was wondering the same. How the hell? There had been some talks to
>>> support a more lightweight JavaScript interpreter as an alternative but
>>> it hasn't gone anyway. Somehow because compatibility reason. I don't how
>>> could node be dependency for these. Grammars are mostly without
>>> dependencies except some have dependencies to other grammars on the
>>> source level such as the C++ require the C grammar.
>> 
>> I don’t think you need nodejs to build the grammar. You might need it to develop the grammar, but compiling grammar.js to parser.c only requires the tree-sitter CLI which is written in Rust.
>> 
>> Yuan
> 
> Doesn't the CLI shell out to Node?

No, the CLI  is written in Rust. The nodejs package is just a shim (it just downloads pre-built binary from GitHub releases).

Yuan

Re: Tree-sitter maturity

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

On Fri, 27 Dec 2024, 14:20 Philip Kaludercic, <philipk@posteo.net> wrote:

> Daniel Colascione <dancol@dancol.org> writes:
>
> > Why not just vendor all the grammars with the Emacs modes that use them?
>
> I am guessing part of the reason is that TS grammars are not fun to
> build.  IIRC they are specified in a Javascript DSL (that used to
> require node.js but AFAIU works with other implementations as well),
> that a program written in Rust translates to C code.


It's probably worth mentioning that the generated C code is included in the
Git repositories, these days along with a Makefile.  So, if you're only
building rather than developing the grammars, you can simply run `make &&
make install` or similar.

--
Peter Oliver

[-- Attachment #2: Type: text/html, Size: 1263 bytes --]

Re: Tree-sitter maturity

[Philip Kaludercic]

Peter Oliver <p.d.oliver@mavit.org.uk> writes:

> On Fri, 27 Dec 2024, 14:20 Philip Kaludercic, <philipk@posteo.net> wrote:
>
>> Daniel Colascione <dancol@dancol.org> writes:
>>
>> > Why not just vendor all the grammars with the Emacs modes that use them?
>>
>> I am guessing part of the reason is that TS grammars are not fun to
>> build.  IIRC they are specified in a Javascript DSL (that used to
>> require node.js but AFAIU works with other implementations as well),
>> that a program written in Rust translates to C code.
>
>
> It's probably worth mentioning that the generated C code is included in the
> Git repositories, these days along with a Makefile.  So, if you're only
> building rather than developing the grammars, you can simply run `make &&
> make install` or similar.

That's true, and also the reason why `treesit-install-language-grammar'
can work without the additional dependencies, but on the other hand that
doesn't make it any easier to modify the grammars, as one just accepts
that the repositories are tracking generated code. 

> --
> Peter Oliver

Re: Tree-sitter maturity

[Björn Bidar]

Peter Oliver <p.d.oliver@mavit.org.uk> writes:

> On Fri, 27 Dec 2024, 14:20 Philip Kaludercic, <philipk@posteo.net> wrote:
>
>> Daniel Colascione <dancol@dancol.org> writes:
>>
>> > Why not just vendor all the grammars with the Emacs modes that use them?
>>
>> I am guessing part of the reason is that TS grammars are not fun to
>> build.  IIRC they are specified in a Javascript DSL (that used to
>> require node.js but AFAIU works with other implementations as well),
>> that a program written in Rust translates to C code.
>
>
> It's probably worth mentioning that the generated C code is included in the
> Git repositories, these days along with a Makefile.  So, if you're only
> building rather than developing the grammars, you can simply run `make &&
> make install` or similar.

Note this isn't practical as the C code is a built time artifact which
is checked into repositories to avoid issues in some of the projects
using the grammars. However that is bad practice for several reasons:
- the grammar is C code yes but it is so big that is effectively a blob.
  Doing so duplicates the blob every time it is generated to a degree.
- Since the blob is a build time artifact it is against most
  distributions (including GNU I assume) to use these. It is also not
  reproducible since it might have been generated by a different
  tree-sitter library version, that should not be an issue but still it
  isn't right.
- Further it is hard to verify if the blob hasn't been modified after
  regenerating.  

The practice is temporary and could stop at any point in time. As a
maintainer of some of these grammar packages the checked in grammars
bother me very much.

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Fri, 27 Dec 2024 08:46:06 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: rms@gnu.org, manphiz@gmail.com
> 
> >> It might take a while for that to happen, which is why I still believe
> >> it would be better if tree-sitter major modes would populate
> >> `treesit-language-source-alist' on their own, and point to the specific
> >> checkouts that the major mode developer tested their implementation
> >> against.
> >
> >We could have done that, but there's no way we could keep the value of
> >treesit-language-source-alist up-to-date, because the grammar
> >libraries put out new versions much more frequently than Emacs
> >releases, especially if you consider libraries that have no official
> >versions at all (in which case we can only point to some revision in
> >their repository).
> >
> >The question that bothers me is how useful is it to have
> >treesit-language-source-alist that is outdated?  What do we expect the
> >users to do with such an outdated value?
> >
> 
> Why not just vendor all the grammars with the Emacs modes that use them?

We'd need to ask their developers to agree to this.  Other than that,
I don't see how is that different from pointing to a specific version
of each grammar: both will be outdated a short time after we point to
the version or release Emacs with that version.

So why do you think this is better?

Re: Tree-sitter maturity

[Daniel Colascione]

On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: rms@gnu.org, manphiz@gmail.com
>> 
>> >> It might take a while for that to happen, which is why I still believe
>> >> it would be better if tree-sitter major modes would populate
>> >> `treesit-language-source-alist' on their own, and point to the specific
>> >> checkouts that the major mode developer tested their implementation
>> >> against.
>> >
>> >We could have done that, but there's no way we could keep the value of
>> >treesit-language-source-alist up-to-date, because the grammar
>> >libraries put out new versions much more frequently than Emacs
>> >releases, especially if you consider libraries that have no official
>> >versions at all (in which case we can only point to some revision in
>> >their repository).
>> >
>> >The question that bothers me is how useful is it to have
>> >treesit-language-source-alist that is outdated?  What do we expect the
>> >users to do with such an outdated value?
>> >
>> 
>> Why not just vendor all the grammars with the Emacs modes that use them?
>
>We'd need to ask their developers to agree to this. 

Why? They're free software. For copyright assignment? Seems like an exception would make sense here.

> Other than that,
>I don't see how is that different from pointing to a specific version
>of each grammar: both will be outdated a short time after we point to
>the version or release Emacs with that version.
>
>So why do you think this is better?

Vendoring enables building a full featured Emacs without a network connection and guarantees build reproducibility in perpetuity.

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Fri, 27 Dec 2024 10:05:47 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: emacs-devel@gnu.org, philipk@posteo.net, rms@gnu.org, manphiz@gmail.com
> 
> >> Why not just vendor all the grammars with the Emacs modes that use them?
> >
> >We'd need to ask their developers to agree to this. 
> 
> Why? They're free software. For copyright assignment? Seems like an exception would make sense here.

AFAIK, that was the policy until now: we should have written agreement
by authors to include any code in Emacs.  RMS might know more, because
he asked for that.

> > Other than that,
> >I don't see how is that different from pointing to a specific version
> >of each grammar: both will be outdated a short time after we point to
> >the version or release Emacs with that version.
> >
> >So why do you think this is better?
> 
> Vendoring enables building a full featured Emacs without a network connection and guarantees build reproducibility in perpetuity.

Is a network connection really a serious problem nowadays?

Re: Tree-sitter maturity

[Daniel Colascione]

On December 27, 2024 10:31:52 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Fri, 27 Dec 2024 10:05:47 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: emacs-devel@gnu.org, philipk@posteo.net, rms@gnu.org, manphiz@gmail.com
>> 
>> >> Why not just vendor all the grammars with the Emacs modes that use them?
>> >
>> >We'd need to ask their developers to agree to this. 
>> 
>> Why? They're free software. For copyright assignment? Seems like an exception would make sense here.
>
>AFAIK, that was the policy until now: we should have written agreement
>by authors to include any code in Emacs.  RMS might know more, because
>he asked for that.
>
>> > Other than that,
>> >I don't see how is that different from pointing to a specific version
>> >of each grammar: both will be outdated a short time after we point to
>> >the version or release Emacs with that version.
>> >
>> >So why do you think this is better?
>> 
>> Vendoring enables building a full featured Emacs without a network connection and guarantees build reproducibility in perpetuity.
>
>Is a network connection really a serious problem nowadays?

It's just another thing that can go wrong. And if, as discussed elsewhere on the thread, the network fetch is of HEAD rather than some specific commit with a known-good hash, that's a security issue. There's a reason many organizations ptohibit network access during builds or at least require that all fetched dependencies be hash-locked.

Re: Tree-sitter maturity

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 27 Dec 2024 10:05:47 -0500
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: emacs-devel@gnu.org, philipk@posteo.net, rms@gnu.org, manphiz@gmail.com
>>
>> >> Why not just vendor all the grammars with the Emacs modes that use them?
>> >
>> >We'd need to ask their developers to agree to this.
>>
>> Why? They're free software. For copyright assignment? Seems like an exception would make sense here.
>
> AFAIK, that was the policy until now: we should have written agreement
> by authors to include any code in Emacs.  RMS might know more, because
> he asked for that.

We can add non-assigned files to our tree if we consider them as _not_ a
part of Emacs.  In that case, there is no need to copyright assign them
to the FSF.

For example, as explained in admin/notes/copyright:

    lwlib/
    rms (2007/02/17): "lwlib is not assigned to the FSF; we don't
    consider it part of Emacs. [...] Therefore non-FSF copyrights are ok
    in lwlib."

In my view, vendored tree-sitter grammars would be analogous to that.

Re: Tree-sitter maturity

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For example, as explained in admin/notes/copyright:

  >     lwlib/
  >     rms (2007/02/17): "lwlib is not assigned to the FSF; we don't
  >     consider it part of Emacs. [...] Therefore non-FSF copyrights are ok
  >     in lwlib."

  > In my view, vendored tree-sitter grammars would be analogous to that.

That is valid, as regards assignments.

But there are practical eeasons why we MIGHT not want to include those
grammar files in Emacs at all.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Tree-sitter maturity

[Daniel Colascione]

On December 28, 2024 11:19:04 PM EST, Richard Stallman <rms@gnu.org> wrote:
>[[[ To any NSA and FBI agents reading my email: please consider    ]]]
>[[[ whether defending the US Constitution against all enemies,     ]]]
>[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>  > For example, as explained in admin/notes/copyright:
>
>  >     lwlib/
>  >     rms (2007/02/17): "lwlib is not assigned to the FSF; we don't
>  >     consider it part of Emacs. [...] Therefore non-FSF copyrights are ok
>  >     in lwlib."
>
>  > In my view, vendored tree-sitter grammars would be analogous to that.
>
>That is valid, as regards assignments.
>
>But there are practical eeasons why we MIGHT not want to include those
>grammar files in Emacs at all.
>


Such as?

Re: Tree-sitter maturity

[tomas]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

On Sat, Dec 28, 2024 at 11:21:10PM -0500, Daniel Colascione wrote:
> 
> 
> On December 28, 2024 11:19:04 PM EST, Richard Stallman <rms@gnu.org> wrote:
> >[[[ To any NSA and FBI agents reading my email: please consider    ]]]
> >[[[ whether defending the US Constitution against all enemies,     ]]]
> >[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> >
> >  > For example, as explained in admin/notes/copyright:
> >
> >  >     lwlib/
> >  >     rms (2007/02/17): "lwlib is not assigned to the FSF; we don't
> >  >     consider it part of Emacs. [...] Therefore non-FSF copyrights are ok
> >  >     in lwlib."
> >
> >  > In my view, vendored tree-sitter grammars would be analogous to that.
> >
> >That is valid, as regards assignments.
> >
> >But there are practical eeasons why we MIGHT not want to include those
> >grammar files in Emacs at all.
> >
> 
> 
> Such as?

More difficult to get rid of?

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 1:41:28 AM EST, tomas@tuxteam.de wrote:
>On Sat, Dec 28, 2024 at 11:21:10PM -0500, Daniel Colascione wrote:
>> 
>> 
>> On December 28, 2024 11:19:04 PM EST, Richard Stallman <rms@gnu.org> wrote:
>> >[[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> >[[[ whether defending the US Constitution against all enemies,     ]]]
>> >[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>> >
>> >  > For example, as explained in admin/notes/copyright:
>> >
>> >  >     lwlib/
>> >  >     rms (2007/02/17): "lwlib is not assigned to the FSF; we don't
>> >  >     consider it part of Emacs. [...] Therefore non-FSF copyrights are ok
>> >  >     in lwlib."
>> >
>> >  > In my view, vendored tree-sitter grammars would be analogous to that.
>> >
>> >That is valid, as regards assignments.
>> >
>> >But there are practical eeasons why we MIGHT not want to include those
>> >grammar files in Emacs at all.
>> >
>> 
>> 
>> Such as?
>
>More difficult to get rid of?
>
>Cheers


Can you elaborate?

Re: Tree-sitter maturity

[tomas]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

On Sun, Dec 29, 2024 at 01:43:28AM -0500, Daniel Colascione wrote:
> 
> 
> On December 29, 2024 1:41:28 AM EST, tomas@tuxteam.de wrote:
> >On Sat, Dec 28, 2024 at 11:21:10PM -0500, Daniel Colascione wrote:

[including treesitter grammars in Emacs]

> >> >But there are practical eeasons why we MIGHT not want to include those
> >> >grammar files in Emacs at all.
> >> >
> >> 
> >> 
> >> Such as?
> >
> >More difficult to get rid of?
> >
> >Cheers
> 
> 
> Can you elaborate?

Sure. I observe that the whole treesitter culture is one far
away from the Gnu culture. So I adopt a careful "wait and see"
approach. I'll compile my Emacs without tree sitter support
for as long as I don't understand the situation. Perhaps I'll
want to avoid the whole thing forever, it depends.

This will become the more and more difficult the more TS grammars
start displacing regular Emacs grammars.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 1:54:57 AM EST, tomas@tuxteam.de wrote:
>On Sun, Dec 29, 2024 at 01:43:28AM -0500, Daniel Colascione wrote:
>> 
>> 
>> On December 29, 2024 1:41:28 AM EST, tomas@tuxteam.de wrote:
>> >On Sat, Dec 28, 2024 at 11:21:10PM -0500, Daniel Colascione wrote:
>
>[including treesitter grammars in Emacs]
>
>> >> >But there are practical eeasons why we MIGHT not want to include those
>> >> >grammar files in Emacs at all.
>> >> >
>> >> 
>> >> 
>> >> Such as?
>> >
>> >More difficult to get rid of?
>> >
>> >Cheers
>> 
>> 
>> Can you elaborate?
>
>Sure. I observe that the whole treesitter culture is one far
>away from the Gnu culture. So I adopt a careful "wait and see"
>approach. I'll compile my Emacs without tree sitter support
>for as long as I don't understand the situation. Perhaps I'll
>want to avoid the whole thing forever, it depends.
>
>This will become the more and more difficult the more TS grammars
>start displacing regular Emacs grammars.
>
>Cheers

I'm honestly confused. Tree sitter grammars are free software. How are they "far away from GNU culture"? Is "Gnu culture" just doing what we were doing in 2010, but over and over again, forever?

Re: Tree-sitter maturity

[tomas]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

On Sun, Dec 29, 2024 at 02:05:25AM -0500, Daniel Colascione wrote:

[...]

> I'm honestly confused. Tree sitter grammars are free software. How are they "far away from GNU culture"? Is "Gnu culture" just doing what we were doing in 2010, but over and over again, forever?

So is Microsoft's VSCode. I still mistrust them.

You can use "Free Software" (what they call Open Source, because they
detest freedom) for purely strategic reasons.

Note that I am not prescribing what you use or not: I'm just trying
to explain why *I* prefer not to touch some things.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Tree-sitter maturity

[Björn Bidar]

tomas@tuxteam.de writes:

> On Sun, Dec 29, 2024 at 01:43:28AM -0500, Daniel Colascione wrote:
>> 
>> 
>> On December 29, 2024 1:41:28 AM EST, tomas@tuxteam.de wrote:
>> >On Sat, Dec 28, 2024 at 11:21:10PM -0500, Daniel Colascione wrote:
>
> [including treesitter grammars in Emacs]
>
>> >> >But there are practical eeasons why we MIGHT not want to include those
>> >> >grammar files in Emacs at all.
>> >> >
>> >> 
>> >> 
>> >> Such as?
>> >
>> >More difficult to get rid of?
>> >
>> >Cheers
>> 
>> 
>> Can you elaborate?
>
> Sure. I observe that the whole treesitter culture is one far
> away from the Gnu culture. So I adopt a careful "wait and see"
> approach. I'll compile my Emacs without tree sitter support
> for as long as I don't understand the situation. Perhaps I'll
> want to avoid the whole thing forever, it depends.

From my point of view the issue is more on the fire and forget approach
of many of these projects. Some projects (not just tree-sitter-grammars)
don't keep good practices by for example storing blobs in git, modifying
sources using git filters after checkout or using unstable in tree
dependencies.
The latter doesn't affect our situation right now since none of the
in-tree dependencies are used.


> This will become the more and more difficult the more TS grammars
> start displacing regular Emacs grammars.

For me the issue is more that it is easier to find support for niche
languages by trying to target a broader audiences which do not depend
on one specific editor to support.

Some modes might pop up for languages which were not supported by Emacs
before using tree-sitter grammars only without support for not using
tree-sitter.

Re: Tree-sitter maturity

[Björn Bidar]

Daniel Colascione <dancol@dancol.org> writes:

> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>>> From: Daniel Colascione <dancol@dancol.org>
>>> CC: rms@gnu.org, manphiz@gmail.com
>>> 
>>> >> It might take a while for that to happen, which is why I still believe
>>> >> it would be better if tree-sitter major modes would populate
>>> >> `treesit-language-source-alist' on their own, and point to the specific
>>> >> checkouts that the major mode developer tested their implementation
>>> >> against.
>>> >
>>> >We could have done that, but there's no way we could keep the value of
>>> >treesit-language-source-alist up-to-date, because the grammar
>>> >libraries put out new versions much more frequently than Emacs
>>> >releases, especially if you consider libraries that have no official
>>> >versions at all (in which case we can only point to some revision in
>>> >their repository).
>>> >
>>> >The question that bothers me is how useful is it to have
>>> >treesit-language-source-alist that is outdated?  What do we expect the
>>> >users to do with such an outdated value?
>>> >
>>> 
>>> Why not just vendor all the grammars with the Emacs modes that use them?
>>
>>We'd need to ask their developers to agree to this. 
>
> Why? They're free software. For copyright assignment? Seems like an exception would make sense here.
>
>> Other than that,
>>I don't see how is that different from pointing to a specific version
>>of each grammar: both will be outdated a short time after we point to
>>the version or release Emacs with that version.
>>
>>So why do you think this is better?
>
> Vendoring enables building a full featured Emacs without a network connection and guarantees build reproducibility in perpetuity.

Did you think of the long term consequences?

The embedded dependencies would have to be maintained first by Emacs and
later by packagers.

All the infrastructure around syncing of grammars is time spend that
could spend on more long term efforts such as stabilizing the
tree-sitter based modes to not break as easy on grammar changes or to
improve tree-sitter it self.

Re: Tree-sitter maturity

[Daniel Colascione]

On December 29, 2024 10:05:26 AM EST, "Björn Bidar" <bjorn.bidar@thaodan.de> wrote:
>Daniel Colascione <dancol@dancol.org> writes:
>
>> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>>>> From: Daniel Colascione <dancol@dancol.org>
>>>> CC: rms@gnu.org, manphiz@gmail.com
>>>> 
>>>> >> It might take a while for that to happen, which is why I still believe
>>>> >> it would be better if tree-sitter major modes would populate
>>>> >> `treesit-language-source-alist' on their own, and point to the specific
>>>> >> checkouts that the major mode developer tested their implementation
>>>> >> against.
>>>> >
>>>> >We could have done that, but there's no way we could keep the value of
>>>> >treesit-language-source-alist up-to-date, because the grammar
>>>> >libraries put out new versions much more frequently than Emacs
>>>> >releases, especially if you consider libraries that have no official
>>>> >versions at all (in which case we can only point to some revision in
>>>> >their repository).
>>>> >
>>>> >The question that bothers me is how useful is it to have
>>>> >treesit-language-source-alist that is outdated?  What do we expect the
>>>> >users to do with such an outdated value?
>>>> >
>>>> 
>>>> Why not just vendor all the grammars with the Emacs modes that use them?
>>>
>>>We'd need to ask their developers to agree to this. 
>>
>> Why? They're free software. For copyright assignment? Seems like an exception would make sense here.
>>
>>> Other than that,
>>>I don't see how is that different from pointing to a specific version
>>>of each grammar: both will be outdated a short time after we point to
>>>the version or release Emacs with that version.
>>>
>>>So why do you think this is better?
>>
>> Vendoring enables building a full featured Emacs without a network connection and guarantees build reproducibility in perpetuity.
>
>Did you think of the long term consequences?
>
>The embedded dependencies would have to be maintained first by Emacs and
>later by packagers.
>
>All the infrastructure around syncing of grammars is time spend that
>could spend on more long term efforts such as stabilizing the
>tree-sitter based modes to not break as easy on grammar changes or to
>improve tree-sitter it self.

I've vendored plenty of things. Works fine in practice. Big programs like Firefox vendor the world too, and they work fine. It's really not that much work. It eliminates entire classes of problem. It's going to take more time to deal with the problems of taking a dependency and the headaches of not having a stable interface than it would to set up a few git subtrees or submodules and invoke their build system from that of Emacs.

It's not even the precise mechanics: pulling down a grammar by hash is tantamount to just checking in the grammar, but with more moving parts. You still pair one to one the grammar and the Lisp code meant to use it so you don't end up chasing down weird compatibility issues. IMHO, since they're tightly coupled anyway, we might as well distribute them together.

As for changing TS grammars not to break: why do you think that would be feasible? So far TS grammar authors haven't felt particularly obligated to maintain compatibility.

Re: Tree-sitter maturity

[Björn Bidar]

Daniel Colascione <dancol@dancol.org> writes:

> On December 29, 2024 10:05:26 AM EST, "Björn Bidar"
> <bjorn.bidar@thaodan.de> wrote:
>>Daniel Colascione <dancol@dancol.org> writes:
>>
>>> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>>>>> From: Daniel Colascione <dancol@dancol.org>
>>>>> CC: rms@gnu.org, manphiz@gmail.com
>>>>> 
>>>>> >> It might take a while for that to happen, which is why I still
>>>>> >> believe
>>>>> >> it would be better if tree-sitter major modes would populate
>>>>> >> `treesit-language-source-alist' on their own, and point to the
>>>>> >> specific
>>>>> >> checkouts that the major mode developer tested their implementation
>>>>> >> against.
>>>>> >
>>>>> >We could have done that, but there's no way we could keep the value of
>>>>> >treesit-language-source-alist up-to-date, because the grammar
>>>>> >libraries put out new versions much more frequently than Emacs
>>>>> >releases, especially if you consider libraries that have no official
>>>>> >versions at all (in which case we can only point to some revision in
>>>>> >their repository).
>>>>> >
>>>>> >The question that bothers me is how useful is it to have
>>>>> >treesit-language-source-alist that is outdated?  What do we expect the
>>>>> >users to do with such an outdated value?
>>>>> >
>>>>> 
>>>>> Why not just vendor all the grammars with the Emacs modes that
>>>>> use them?
>>>>
>>>>We'd need to ask their developers to agree to this. 
>>>
>>> Why? They're free software. For copyright assignment? Seems like an
>>> exception would make sense here.
>>>
>>>> Other than that,
>>>>I don't see how is that different from pointing to a specific version
>>>>of each grammar: both will be outdated a short time after we point to
>>>>the version or release Emacs with that version.
>>>>
>>>>So why do you think this is better?
>>>
>>> Vendoring enables building a full featured Emacs without a network
>>> connection and guarantees build reproducibility in perpetuity.
>>
>>Did you think of the long term consequences?
>>
>>The embedded dependencies would have to be maintained first by Emacs and
>>later by packagers.
>>
>>All the infrastructure around syncing of grammars is time spend that
>>could spend on more long term efforts such as stabilizing the
>>tree-sitter based modes to not break as easy on grammar changes or to
>>improve tree-sitter it self.
>
> I've vendored plenty of things. Works fine in practice. Big programs
> like Firefox vendor the world too, and they work fine. It's really not
> that much work. It eliminates entire classes of problem. It's going to
> take more time to deal with the problems of taking a dependency and
> the headaches of not having a stable interface than it would to set up
> a few git subtrees or submodules and invoke their build system from
> that of Emacs.

Big programs like Firefox vendor the world only for packagers to have
to revert those. Vendoring only works long enough until the dependencies
you have vendored are not out of date. It is something which only works
in projects who control most of their dependency chain and/or have a
fire and forget approach of software development.

> It's not even the precise mechanics: pulling down a grammar by hash is
> tantamount to just checking in the grammar, but with more moving
> parts. You still pair one to one the grammar and the Lisp code meant
> to use it so you don't end up chasing down weird compatibility
> issues. IMHO, since they're tightly coupled anyway, we might as well
> distribute them together.
>
> As for changing TS grammars not to break: why do you think that would
> be feasible? So far TS grammar authors haven't felt particularly
> obligated to maintain compatibility.

I don't know exactly to be honst but I don't think we are alone with
this issue. If we are we should check out it is handled in other
editors.

Re: Tree-sitter maturity

[Björn Bidar]

Daniel Colascione <dancol@dancol.org> writes:

> On December 29, 2024 11:02:47 AM EST, "Björn Bidar" <bjorn.bidar@thaodan.de> wrote:
>>Daniel Colascione <dancol@dancol.org> writes:
>>
>>> On December 29, 2024 10:05:26 AM EST, "Björn Bidar"
>>> <bjorn.bidar@thaodan.de> wrote:
>>>>Daniel Colascione <dancol@dancol.org> writes:
>>>>
>>>>> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>>>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>>>>>>> From: Daniel Colascione <dancol@dancol.org>
>>>>>>> CC: rms@gnu.org, manphiz@gmail.com
>>>>>>> 
>>>>>>> >> It might take a while for that to happen, which is why I still
>>>>>>> >> believe
>>>>>>> >> it would be better if tree-sitter major modes would populate
>>>>>>> >> `treesit-language-source-alist' on their own, and point to the
>>>>>>> >> specific
>>>>>>> >> checkouts that the major mode developer tested their implementation
>>>>>>> >> against.
>>>>>>> >
>>>>>>> >We could have done that, but there's no way we could keep the value of
>>>>>>> >treesit-language-source-alist up-to-date, because the grammar
>>>>>>> >libraries put out new versions much more frequently than Emacs
>>>>>>> >releases, especially if you consider libraries that have no official
>>>>>>> >versions at all (in which case we can only point to some revision in
>>>>>>> >their repository).
>>>>>>> >
>>>>>>> >The question that bothers me is how useful is it to have
>>>>>>> >treesit-language-source-alist that is outdated?  What do we expect the
>>>>>>> >users to do with such an outdated value?
>>>>>>> >
>>>>>>> 
>>>>>>> Why not just vendor all the grammars with the Emacs modes that
>>>>>>> use them?
>>>>>>
>>>>>>We'd need to ask their developers to agree to this. 
>>>>>
>>>>> Why? They're free software. For copyright assignment? Seems like an
>>>>> exception would make sense here.
>>>>>
>>>>>> Other than that,
>>>>>>I don't see how is that different from pointing to a specific version
>>>>>>of each grammar: both will be outdated a short time after we point to
>>>>>>the version or release Emacs with that version.
>>>>>>
>>>>>>So why do you think this is better?
>>>>>
>>>>> Vendoring enables building a full featured Emacs without a network
>>>>> connection and guarantees build reproducibility in perpetuity.
>>>>
>>>>Did you think of the long term consequences?
>>>>
>>>>The embedded dependencies would have to be maintained first by Emacs and
>>>>later by packagers.
>>>>
>>>>All the infrastructure around syncing of grammars is time spend that
>>>>could spend on more long term efforts such as stabilizing the
>>>>tree-sitter based modes to not break as easy on grammar changes or to
>>>>improve tree-sitter it self.
>>>
>>> I've vendored plenty of things. Works fine in practice. Big programs
>>> like Firefox vendor the world too, and they work fine. It's really not
>>> that much work. It eliminates entire classes of problem. It's going to
>>> take more time to deal with the problems of taking a dependency and
>>> the headaches of not having a stable interface than it would to set up
>>> a few git subtrees or submodules and invoke their build system from
>>> that of Emacs.
>>
>>Big programs like Firefox vendor the world only for packagers to have
>>to revert those. 
>
> These packagers are wrong, FWIW. Unbundling is needless and often
> introduces bugs.

It introduces bugs in software with generally unstable APIs/ABIs or software
using unfinished/unreleased versions of software. Partially the problem
we are facing here right now.

The rat tail of issues this can entrail can be long.

FWIW You called over half of the Unix community wrong where bundled
dependencies are frowned or forbidden upon.

> In the mobile world, popular OSes seldom provide
> libraries. Apps are expected to bundle their dependencies. The sky
> doesn't fall. In fact, the mobile app ecosystem is healthier and more
> secure than the desktop one precisely because it isn't burdened by
> ideas that no longer make sense in a modern technical context,
> e.g. that apps should casually share libraries.

I work on mobile operating systems, what you describe is double edged
sword. The applications size's increase and the party to blame for
security issues moves from the os to the application developer. The
mobile OS would had plenty of issues from this practice notably for
example the log4you debacle.

Most mobile operating systems provide their own set of available
libraries, apps are not expected to bundle dependencies unless they are
not available for that OS.

Part of the issue is that library dependencies moving faster than many
operating systems can or with stable APIs. The end result of such lets
bundled everything approach is that you have use the exact chain of
dependencies to build a software which is awesome if you like the fire
and forget approach of software development.


To me what you write reads like mobile operating systems = JavaScript/AI developers.

>> Vendoring only works long enough until the dependencies
>>you have vendored are not out of date. 
>
> It doesn't matter whether the dependency is out of date so long as
> it's in sync with code that interacts with it. It's even worse when
> the dependency doesn't make any compatibility guarantees. IMHO, the
> only reasonable way to consume a dependency with an unstable interface
> is to bundle or hash-lock or outright vendor it.

If you have software which has short life cycles this can work but I
don't think this works for Emacs.
Further bundled dependencies require to patch the software bundling the
dependency to fix bugs and security issue. Bugfixes are not really an
issue with grammars but with libtree-sitter which Emacs depend on.
Putting bundled grammars and libtree-sitter in this equation makes it
harder to maintain the Emacs package since I have to watch to not break
the embedded grammars when updating tree-sitter or it's dependencies.

>> It is something which only works
>>in projects who control most of their dependency chain and/or have a
>>fire and forget approach of software development.
>>
>>> It's not even the precise mechanics: pulling down a grammar by hash is
>>> tantamount to just checking in the grammar, but with more moving
>>> parts. You still pair one to one the grammar and the Lisp code meant
>>> to use it so you don't end up chasing down weird compatibility
>>> issues. IMHO, since they're tightly coupled anyway, we might as well
>>> distribute them together.
>>>
>>> As for changing TS grammars not to break: why do you think that would
>>> be feasible? So far TS grammar authors haven't felt particularly
>>> obligated to maintain compatibility.
>>
>>I don't know exactly to be honst but I don't think we are alone with
>>this issue. If we are we should check out it is handled in other
>>editors.
>
> You're right that this is a problem everyone should be hitting.

Maybe there's a way around it. The only way is to reach out to other
projects using the library or upstream.

Re: Tree-sitter maturity

[Yuan Fu]

> On Dec 29, 2024, at 11:06 AM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
> 
> Daniel Colascione <dancol@dancol.org> writes:
> 
>> On December 29, 2024 11:02:47 AM EST, "Björn Bidar" <bjorn.bidar@thaodan.de> wrote:
>>> Daniel Colascione <dancol@dancol.org> writes:
>>> 
>>>> On December 29, 2024 10:05:26 AM EST, "Björn Bidar"
>>>> <bjorn.bidar@thaodan.de> wrote:
>>>>> Daniel Colascione <dancol@dancol.org> writes:
>>>>> 
>>>>>> On December 27, 2024 9:59:14 AM EST, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>>>>> Date: Fri, 27 Dec 2024 08:46:06 -0500
>>>>>>>> From: Daniel Colascione <dancol@dancol.org>
>>>>>>>> CC: rms@gnu.org, manphiz@gmail.com
>>>>>>>> 
>>>>>>>>>> It might take a while for that to happen, which is why I still
>>>>>>>>>> believe
>>>>>>>>>> it would be better if tree-sitter major modes would populate
>>>>>>>>>> `treesit-language-source-alist' on their own, and point to the
>>>>>>>>>> specific
>>>>>>>>>> checkouts that the major mode developer tested their implementation
>>>>>>>>>> against.
>>>>>>>>> 
>>>>>>>>> We could have done that, but there's no way we could keep the value of
>>>>>>>>> treesit-language-source-alist up-to-date, because the grammar
>>>>>>>>> libraries put out new versions much more frequently than Emacs
>>>>>>>>> releases, especially if you consider libraries that have no official
>>>>>>>>> versions at all (in which case we can only point to some revision in
>>>>>>>>> their repository).
>>>>>>>>> 
>>>>>>>>> The question that bothers me is how useful is it to have
>>>>>>>>> treesit-language-source-alist that is outdated?  What do we expect the
>>>>>>>>> users to do with such an outdated value?
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> Why not just vendor all the grammars with the Emacs modes that
>>>>>>>> use them?
>>>>>>> 
>>>>>>> We'd need to ask their developers to agree to this. 
>>>>>> 
>>>>>> Why? They're free software. For copyright assignment? Seems like an
>>>>>> exception would make sense here.
>>>>>> 
>>>>>>> Other than that,
>>>>>>> I don't see how is that different from pointing to a specific version
>>>>>>> of each grammar: both will be outdated a short time after we point to
>>>>>>> the version or release Emacs with that version.
>>>>>>> 
>>>>>>> So why do you think this is better?
>>>>>> 
>>>>>> Vendoring enables building a full featured Emacs without a network
>>>>>> connection and guarantees build reproducibility in perpetuity.
>>>>> 
>>>>> Did you think of the long term consequences?
>>>>> 
>>>>> The embedded dependencies would have to be maintained first by Emacs and
>>>>> later by packagers.
>>>>> 
>>>>> All the infrastructure around syncing of grammars is time spend that
>>>>> could spend on more long term efforts such as stabilizing the
>>>>> tree-sitter based modes to not break as easy on grammar changes or to
>>>>> improve tree-sitter it self.
>>>> 
>>>> I've vendored plenty of things. Works fine in practice. Big programs
>>>> like Firefox vendor the world too, and they work fine. It's really not
>>>> that much work. It eliminates entire classes of problem. It's going to
>>>> take more time to deal with the problems of taking a dependency and
>>>> the headaches of not having a stable interface than it would to set up
>>>> a few git subtrees or submodules and invoke their build system from
>>>> that of Emacs.
>>> 
>>> Big programs like Firefox vendor the world only for packagers to have
>>> to revert those. 
>> 
>> These packagers are wrong, FWIW. Unbundling is needless and often
>> introduces bugs.
> 
> It introduces bugs in software with generally unstable APIs/ABIs or software
> using unfinished/unreleased versions of software. Partially the problem
> we are facing here right now.
> 
> The rat tail of issues this can entrail can be long.
> 
> FWIW You called over half of the Unix community wrong where bundled
> dependencies are frowned or forbidden upon.
> 
>> In the mobile world, popular OSes seldom provide
>> libraries. Apps are expected to bundle their dependencies. The sky
>> doesn't fall. In fact, the mobile app ecosystem is healthier and more
>> secure than the desktop one precisely because it isn't burdened by
>> ideas that no longer make sense in a modern technical context,
>> e.g. that apps should casually share libraries.
> 
> I work on mobile operating systems, what you describe is double edged
> sword. The applications size's increase and the party to blame for
> security issues moves from the os to the application developer. The
> mobile OS would had plenty of issues from this practice notably for
> example the log4you debacle.
> 
> Most mobile operating systems provide their own set of available
> libraries, apps are not expected to bundle dependencies unless they are
> not available for that OS.
> 
> Part of the issue is that library dependencies moving faster than many
> operating systems can or with stable APIs. The end result of such lets
> bundled everything approach is that you have use the exact chain of
> dependencies to build a software which is awesome if you like the fire
> and forget approach of software development.
> 
> 
> To me what you write reads like mobile operating systems = JavaScript/AI developers.
> 
>>> Vendoring only works long enough until the dependencies
>>> you have vendored are not out of date. 
>> 
>> It doesn't matter whether the dependency is out of date so long as
>> it's in sync with code that interacts with it. It's even worse when
>> the dependency doesn't make any compatibility guarantees. IMHO, the
>> only reasonable way to consume a dependency with an unstable interface
>> is to bundle or hash-lock or outright vendor it.
> 
> If you have software which has short life cycles this can work but I
> don't think this works for Emacs.
> Further bundled dependencies require to patch the software bundling the
> dependency to fix bugs and security issue. Bugfixes are not really an
> issue with grammars but with libtree-sitter which Emacs depend on.
> Putting bundled grammars and libtree-sitter in this equation makes it
> harder to maintain the Emacs package since I have to watch to not break
> the embedded grammars when updating tree-sitter or it's dependencies.
> 
>>> It is something which only works
>>> in projects who control most of their dependency chain and/or have a
>>> fire and forget approach of software development.
>>> 
>>>> It's not even the precise mechanics: pulling down a grammar by hash is
>>>> tantamount to just checking in the grammar, but with more moving
>>>> parts. You still pair one to one the grammar and the Lisp code meant
>>>> to use it so you don't end up chasing down weird compatibility
>>>> issues. IMHO, since they're tightly coupled anyway, we might as well
>>>> distribute them together.
>>>> 
>>>> As for changing TS grammars not to break: why do you think that would
>>>> be feasible? So far TS grammar authors haven't felt particularly
>>>> obligated to maintain compatibility.
>>> 
>>> I don't know exactly to be honst but I don't think we are alone with
>>> this issue. If we are we should check out it is handled in other
>>> editors.
>> 
>> You're right that this is a problem everyone should be hitting.
> 
> Maybe there's a way around it. The only way is to reach out to other
> projects using the library or upstream.

Nvim vendors grammars, it also has a “database” repo that pins grammars (https://github.com/nvim-treesitter/nvim-treesitter), that repo also provides a command to install grammars.

IIUC Pulsar (community-supported Atom successor) vendors grammars too, because that’s what Atom did originally.

Helix I think only provide a command to install grammars (plus their database pinning grammar versions).

For context, Emacs also has a command to install grammars, but we don’t provide a database nor version pinning.

Nvim and Pulsar have plans to move towards using wasm grammars. Tree-sitter recently gained the ability to compile grammars into wasm object files and load wasm grammars. Pulsar is built on electron so naturally has access to wasm, nvim is adding a wasm runtime as an optional dependency.

Yuan

Re: Tree-sitter maturity

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>> 
>> Richard Stallman <rms@gnu.org> writes:
>> 
>> > If we add something like this to Emacs, there is an issue we need to
>> > take care about: to make carefully sure that it does not install
>> > any nonfree grammars.  I don't know how those grammars are released,
>> > ir by whom, or how much they care about free software.  We can't
>> > take for granted that they do.
>> >
>> > Perhaps we could check automatically that the grammar found is properly
>> > licenses, and disregard any grammars that are not free.
>> >
>> > By contrast, if grammars are going to be packaged and released for
>> > distros, and chosen for installation by users, then it is the user's
>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>> > (and the GNU/Linux distro might insist on that).
>> 
>> It might take a while for that to happen, which is why I still believe
>> it would be better if tree-sitter major modes would populate
>> `treesit-language-source-alist' on their own, and point to the specific
>> checkouts that the major mode developer tested their implementation
>> against.
>
> We could have done that, but there's no way we could keep the value of
> treesit-language-source-alist up-to-date, because the grammar
> libraries put out new versions much more frequently than Emacs
> releases, especially if you consider libraries that have no official
> versions at all (in which case we can only point to some revision in
> their repository).

Is there a reason we need to keep them *that* up to date?  What happens
when someone is on an older version of Emacs, and with Tree Sitter
changes in the meantime the current development tip is not compatible
with the library that Emacs binds against?  It seems more reliable to me
to pin what commit the grammar was tested against, if only as a hint.

And the release issue can be circumvented if we either list the
specific revisions to use  in `treesit-language-source-alist' or just
download the tarball directly.  Each -ts-mode.el file can just hint at
the commit to use with a 

  (add-to-list
   'treesit-language-source-alist
   '(python "https://github.com/tree-sitter/py-tree-sitter/tarball/d65416ba825a99054f7b64649c6b93c6fa7f5e04"))

expression.

> The question that bothers me is how useful is it to have
> treesit-language-source-alist that is outdated?  What do we expect the
> users to do with such an outdated value?

One idea is to first download the pinned version, and if that can't be
built to download the tip.  We could also prompt the user to check with
them.

If this is a serious issue (which I cannot estimate), we can provide an
official package similar to the on MELPA with updated references.

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: rms@gnu.org,  manphiz@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 27 Dec 2024 14:11:01 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> It might take a while for that to happen, which is why I still believe
> >> it would be better if tree-sitter major modes would populate
> >> `treesit-language-source-alist' on their own, and point to the specific
> >> checkouts that the major mode developer tested their implementation
> >> against.
> >
> > We could have done that, but there's no way we could keep the value of
> > treesit-language-source-alist up-to-date, because the grammar
> > libraries put out new versions much more frequently than Emacs
> > releases, especially if you consider libraries that have no official
> > versions at all (in which case we can only point to some revision in
> > their repository).
> 
> Is there a reason we need to keep them *that* up to date?

There is: later versions generally improve on older ones and fix some
bugs.

> What happens when someone is on an older version of Emacs, and with
> Tree Sitter changes in the meantime the current development tip is
> not compatible with the library that Emacs binds against?  It seems
> more reliable to me to pin what commit the grammar was tested
> against, if only as a hint.

It's more reliable, but it could have the effect of keeping people
from upgrading when they can.  What is better?  I don't know.

> > The question that bothers me is how useful is it to have
> > treesit-language-source-alist that is outdated?  What do we expect the
> > users to do with such an outdated value?
> 
> One idea is to first download the pinned version, and if that can't be
> built to download the tip.  We could also prompt the user to check with
> them.

If we do that, it is basically what we have today: the responsibility
for deciding which version to install is on the user.

> If this is a serious issue (which I cannot estimate), we can provide an
> official package similar to the on MELPA with updated references.

Yes, if we have volunteers to keep these up-to-date (which involves
testing the new versions when they come out).

Re: Tree-sitter maturity

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> We could have done that, but there's no way we could keep the value of
> treesit-language-source-alist up-to-date, because the grammar
> libraries put out new versions much more frequently than Emacs
> releases, especially if you consider libraries that have no official
> versions at all (in which case we can only point to some revision in
> their repository).

May this list be updated from ELPA? Via a special package or maybe via compat.el.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Philip Kaludercic <philipk@posteo.net>, rms@gnu.org, manphiz@gmail.com,
>  emacs-devel@gnu.org
> Date: Fri, 27 Dec 2024 18:29:47 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > We could have done that, but there's no way we could keep the value of
> > treesit-language-source-alist up-to-date, because the grammar
> > libraries put out new versions much more frequently than Emacs
> > releases, especially if you consider libraries that have no official
> > versions at all (in which case we can only point to some revision in
> > their repository).
> 
> May this list be updated from ELPA? Via a special package or maybe via compat.el.

Everything is possible, if someone volunteers to do the job.  (There's
still the question I asked whether such a package will be useful and
used by enough users, but that is secondary.)

In general, people are suggesting all kinds of semi-solutions for a
problem that has no solution, and isn't ours to solve to begin with.
I don't mind providing such semi-solutions, if someone volunteers, I'm
just pointing out their (almost obvious) downsides, so people would
not make a mistake of thinking we can solve this like we are used to
solve such problems when they are under our complete control.

Re: Tree-sitter maturity

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> > We could have done that, but there's no way we could keep the value of
>> > treesit-language-source-alist up-to-date...
>> 
>> May this list be updated from ELPA? Via a special package or maybe via compat.el.
>
> Everything is possible, if someone volunteers to do the job.  (There's
> still the question I asked whether such a package will be useful and
> used by enough users, but that is secondary.)

To be clear, I mostly wanted to show that keeping the value up to date
is not impossible (contrary to your assertion).
Whether it is a viable practical option is another question. But the
idea should not be rejected outright.

> In general, people are suggesting all kinds of semi-solutions for a
> problem that has no solution, and isn't ours to solve to begin with.
> I don't mind providing such semi-solutions, if someone volunteers, I'm
> just pointing out their (almost obvious) downsides, so people would
> not make a mistake of thinking we can solve this like we are used to
> solve such problems when they are under our complete control.

Fair.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: philipk@posteo.net, rms@gnu.org, manphiz@gmail.com, emacs-devel@gnu.org
> Date: Sat, 28 Dec 2024 08:11:08 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > We could have done that, but there's no way we could keep the value of
> >> > treesit-language-source-alist up-to-date...
> >> 
> >> May this list be updated from ELPA? Via a special package or maybe via compat.el.
> >
> > Everything is possible, if someone volunteers to do the job.  (There's
> > still the question I asked whether such a package will be useful and
> > used by enough users, but that is secondary.)
> 
> To be clear, I mostly wanted to show that keeping the value up to date
> is not impossible (contrary to your assertion).
> Whether it is a viable practical option is another question. But the
> idea should not be rejected outright.

I didn't reject the idea outright; expressing doubts and citing
downsides is not rejection.  However, what bothers me is indeed the
practical, not theoretical, viability of this.

Re: Tree-sitter maturity

[Björn Bidar]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Not sure whether this has been mentioned: there is a "treesit-auto"[1]
>   > addon on melpa that can detect missing treesitter grammars and handle
>   > their installation automatically.  It would be great if something
>   > similar can be integrated into core, and if possible, grammar version
>   > handling and compatibility with distribution supplied grammar would be
>   > good to have.
>
> If we add something like this to Emacs, there is an issue we need to
> take care about: to make carefully sure that it does not install
> any nonfree grammars.  I don't know how those grammars are released,
> ir by whom, or how much they care about free software.  We can't
> take for granted that they do.

Grammars are built from source (exluding the parser.c blob which I
mentioned a few messages early maybe). No blob which has not been built
from source is loaded into Emacs.

> Perhaps we could check automatically that the grammar found is properly
> licenses, and disregard any grammars that are not free.

Who decides what grammar is free or not? It is not Emacs to judge to restrict
user freedom. What you described is akin to Free-Software DRM. 

Re: Tree-sitter maturity

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Ideally, users should be able install the grammars separately, perhaps
  > > using their `make install' or `apt', and Emacs would only use them
  > > where they normally get installed.  That is ideal because it is not
  > > very integrated -- it preserves modularity.  In particular, it assures
  > > that the details of how they get installed are not a direct concern of
  > > Emacs maintenance.

  > Emacs needs to be built with the tree-sitter library to support the
  > modes based on it, and the grammar library needs to be installed.

That makes sense, but are we talking about two different issues?  I
though we were talking about various grammars, each for some specific
language to be parsed.  When you say "the grammar library", does it
mean those same grammars?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Tree-sitter maturity

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: bjorn.bidar@thaodan.de, emacs-devel@gnu.org
> Date: Wed, 25 Dec 2024 23:32:28 -0500
> 
>   > > Ideally, users should be able install the grammars separately, perhaps
>   > > using their `make install' or `apt', and Emacs would only use them
>   > > where they normally get installed.  That is ideal because it is not
>   > > very integrated -- it preserves modularity.  In particular, it assures
>   > > that the details of how they get installed are not a direct concern of
>   > > Emacs maintenance.
> 
>   > Emacs needs to be built with the tree-sitter library to support the
>   > modes based on it, and the grammar library needs to be installed.
> 
> That makes sense, but are we talking about two different issues?  I
> though we were talking about various grammars, each for some specific
> language to be parsed.  When you say "the grammar library", does it
> mean those same grammars?

Yes.  A tree-sitter grammar is a small shared library.

Re: Tree-sitter maturity

[Björn Bidar]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Ideally, users should be able install the grammars separately, perhaps
>   > > using their `make install' or `apt', and Emacs would only use them
>   > > where they normally get installed.  That is ideal because it is not
>   > > very integrated -- it preserves modularity.  In particular, it assures
>   > > that the details of how they get installed are not a direct concern of
>   > > Emacs maintenance.
>
>   > Emacs needs to be built with the tree-sitter library to support the
>   > modes based on it, and the grammar library needs to be installed.
>
> That makes sense, but are we talking about two different issues?  I
> though we were talking about various grammars, each for some specific
> language to be parsed.  When you say "the grammar library", does it
> mean those same grammars?

The grammar DSL is converted into a generated parser with tree-sitter,
the generated output is a .c file. The file is that compiled into a
library when is then loaded using the regular methods to load libraries
with the addition of possible extra path if the user has set them.

Re: Tree-sitter maturity

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]

On Wed, 18 Dec 2024, Eli Zaretskii wrote:

> Many language grammars don't make official releases and thus don't
> have versions.

I don’t believe this is true of any of the parsers used by modes that are currently part of Emacs, is it?

If a parser hasn’t made an official release yet, that’s probably a good signal that it’s not suitable for us at the moment.

-- 
Peter Oliver

Re: Tree-sitter maturity

[Eli Zaretskii]

> Date: Thu, 19 Dec 2024 12:23:12 +0000 (GMT)
> From: Peter Oliver <p.d.oliver@mavit.org.uk>
> cc: Yuan Fu <casouri@gmail.com>, emacs-devel@gnu.org
> 
> On Wed, 18 Dec 2024, Eli Zaretskii wrote:
> 
> > Many language grammars don't make official releases and thus don't
> > have versions.
> 
> I don’t believe this is true of any of the parsers used by modes that are currently part of Emacs, is it?

I don't know, I didn't check which ones are in Emacs and which aren't.
I have more than 70 grammar libraries on my system, and most of them
don't have versions.  If most of those in Emacs do, that's just sheer
luck which can change any moment, if we add more modes.

> If a parser hasn’t made an official release yet, that’s probably a good signal that it’s not suitable for us at the moment.

That would exclude too many grammars, I'm afraid.  And I don't see a
reason to exclude them, since IME they are all perfectly usable and
useful.

But mentioning the commit SHA is fine by me, as long as we tell users
not to assume that any newer version will not work.

Re: Tree-sitter maturity

[Vincenzo Pupillo]

In data giovedì 19 dicembre 2024 13:23:12 Ora standard dell’Europa centrale, 
Peter Oliver ha scritto:
> On Wed, 18 Dec 2024, Eli Zaretskii wrote:
> > Many language grammars don't make official releases and thus don't
> > have versions.
> 
> I don’t believe this is true of any of the parsers used by modes that are
> currently part of Emacs, is it?
No, phpdoc for e.g. doesn't have an official release.

> 
> If a parser hasn’t made an official release yet, that’s probably a good
> signal that it’s not suitable for us at the moment.


Vincenzo

Re: Tree-sitter maturity

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Yuan Fu <casouri@gmail.com>
>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>>  Stefan Kangas <stefankangas@gmail.com>,
>>  Emacs Devel <emacs-devel@gnu.org>,
>>  Eli Zaretskii <eliz@gnu.org>
>> 
>> >> It’s also worth noting that Tree-sitter itself is somewhat
>> > immature; the developers say that until it reaches version 1.0, we
>> > should be wary of potentially unannounced incompatible changes
>> > (although they are trying harder to avoid this, over time).
>> > 
>> > 
>> > [1] https://build.opensuse.org/package/show/editors/tree-sitter
>> 
>> I wonder if we can formalize a way for tree-sitter major modes to
>> state the compatible version of language grammar it uses. Maybe a
>> package.el cookies, or a variable that set, or even just comments in
>> the beginning of the file.
>> 
>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>> 
>> I especially want built-in major modes to give a version, so that
>> packagers can package Emacs with the right version of tree-sitter
>> grammar. I know Eli has problems with pinning a grammar version for
>> builtin modes before, but I wonder what’s he’s stance now?
>
> What's changed?
>
> Many language grammars don't make official releases and thus don't
> have versions.  Moreover, AFAIK there's no API to determine the
> version of the grammar library we load.  So how can we manage such
> version-pinning in a way that (a) is up-to-date, and (b) doesn't
> preclude people from using a grammar library due to false negatives?

There isn't any version pinning from what I know. There is no stable
definition of the grammar versions. Each grammar is used to generate a
parser based on the tree-sitter version.

I don't think pinning version does make sense especially in this
instance.

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Stefan.

On Wed, Nov 20, 2024 at 20:28:58 -0600, Stefan Kangas wrote:
> Hi Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > The immediate reason is that, as maintainer of CC Mode, CC Mode's
> > symbols, its names, were taken by Emacs and used for other purposes
> > without informing me, much less consulting me.  That makes my position as
> > CC Mode maintainer here untenable.

> That is highly regrettable.  You are a valued member of our team, and
> it's sad to see you go.

Thanks for that!

> > These symbols have been appropriated by Emacs to mean "the current
> > preferred mode for C", etc., rather than C Mode, C++ Mode etc.  In
> > certain circumstances, in particular, in Local Variables: sections and
> > auto-mode-alist, there is now no longer any way unambiguously to specify
> > C Mode or C++ Mode.  Up till recently ("\\.myc\\'" . c-mode) in
> > auto-mode-alist meant C Mode, and would have had the effect of
> > auto-loading CC Mode, if needed, and running C Mode.

> From my point of view, we are still in early days when it comes to the
> new tree-sitter modes.  For starters, we do not recommend them by
> default, and some language modes are also not yet ready for prime-time.
> I'm not even sure that a majority of distros ship the feature in a
> useful form yet, but I didn't really check.

> AFAIU, the purpose of `major-mode-remap-alist` is to provide a mechanism
> to respect what users want.  Where there is disagrement, it concerns the
> technical details of how to best achieve that, and to which extent we
> should set things up automatically based on indicators such as the user
> actions "running a mode", "loading a file", or "running a command".

> But the feature has teething problems.  My understanding was that we
> agreed in Bug#74339 that the situation in Emacs 30 is already better
> than in Emacs 29, and that we will continue working on this in Emacs 31.

> For example, it has been suggested that we should replace the automatic
> setting of `major-mode-remap-defaults` with an entirely new command like
> `foo-ts-mode-prefer`, that would be used as the canonical indication
> that a user wants to use the tree-sitter mode everywhere.  There surely
> exist other options that we could evaluate also.

> For this reason, I hope that there is still room to reconsider your
> decision to resign.

> > Stefan's habit of making big changes in Emacs without seeking consensus
> > is at the heart of why I am resigning.  These changes have caused Emacs a
> > lot of damage over the years and have caused other contributors,
> > including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
> > character.  On the one hand, he's a very capable hacker, and is always
> > ready to help others with technical questions.  On the other hand, as
> > mentioned, he is contemptuous of the Emacs conventions, and unlike
> > Richard and Eli, does not have the gift of knowing what the Right Thing
> > is.

> This is where I have to disagree quite strongly.  I find the charges
> directed at Stefan Monnier both unfair and one-sided.  I fail to see
> which of his actions or words that could possibly warrant such a
> negative interpretation, or that would justify assuming any ill intent.

For starters: The change in the meaning of `c-mode' and `c++-mode' he
introduced in bug#69191, discussed at length in my last post.  Stefan is
not stupid.  He knew full well what he was doing in bypassing open
discussion about major-mode-remap-defaults.

Number 2: In late January 2024, Stefan decided to replace the customary
list form of interpreted functions with opaque atoms, because the list
form "annoyed" him.  In the ensuing discussion, Richard described the
proposal as "perverse", and both Eli and I were asking questions as to
the purpose of the change.  Only evasive non-answers came back.  There
was certainly no consensus around the proposal.  Nevertheless, Stefan
quietly committed his patch on 2024-03-11 in commit
f2bccae22bd47a2e7e0937b78ea06131711b935a.  Emacs is slightly less
powerful as a result, in that macros can no longer operate on the code
of a function in a reasonable fashion.

Number 3: Stefan introduced pcase.el without any open discussion, and
proliferated it rapidly around the Emacs core, leading to confusion
around the use of ` and ,, certainly on my part.  Now it can be argued
that pcase has been a success, but it could have been so much better if
it had been developed cooperatively.  For years there was no adequate
doc string for `pcase', and even now the doc strings for things like
pcase-let* are woefully inadequate.  Stefan is not good at documenting;
nobody can be good at everything.

Number 4: Some years ago, Stefan removed the documentation of defadvice
from the elisp manual without any discussion.  Despite widespread
protest, he refused to put it back again.  Quite coincidentally, he had
just written and pushed nadvice.el.

Number 5: Previously, there had been an embargo on the use of the CL
library, except at compile time.  This kept the size of the Emacs Lisp
language manageable, and the language easy to understand, and made
maintainers' and beginners' lives easier.  At some stage this embargo
was lifted, and the use of CL rapidly proliferated through the Emacs
core.  Now, it could be argued that the facilities and expressiveness of
the CL lib outweigh these disadvantages.  But it was not so argued.  It
just happened.  Maybe somebody else but Stefan made this change, but it
seems unlikely.  Incidentally, the CL library is badly documented; most
of its functions, macros, and variables lack doc strings, and comments
are sparse indeed.  For example, in cl-generic.el, there is no
description of the structures and algorithms used to implement generic
functions.  "Maintainable" isn't an adjective which springs to mind for
this library.

> I have to agree with Eli.  Although it would, in hindsight, certainly
> have been better to discuss these particular changes in more detail in
> advance, I don't see that he has done anything very unusual or different
> from what most other core contributors do on a routine basis.

This "be nice to everybody no matter what they do" and "always assume
the best of everybody" creates the perfect atmosphere for a monster to
flourish in.  Stefan is such a monster; not all the time, not even most
of the time, but in doing the things detailed above, and other things, I
don't understand why you are defending him.

I've had continual trouble over the last ~20 years with what Stefan has
done, and how he's done it.  Nobody else even comes close.  As I said,
this is the root cause of why I'm leaving the Emacs team.  Most of the
time, he is extremely helpful and efficient at maintaining, and I'm
grateful for all the help he has given me over the years.  As I said, a
Jekyll-and-Hyde character.

> I also do not appreciate where it veers into ad-hominem, such as talking
> about Stefan M's character, etc.  That is strictly off-topic here, as
> you well know, and does not reach the usual high level of standard that
> one would expect from one of your posts.

I have not come anywhere near ad hominem.  It is true that many forums
degenerate into slanging matches which repel decent posters.
emacs-devel is the opposite extreme, sort of touchy-feely where nobody's
allowed to offend anybody else at all, no matter what they do, why and
how they do it.  This is just as unhealthy as the the continual abuse
forums; it leads to the build up of repressed resentment.

Sometimes the truth must be told bluntly, and that is what I have tried
to do here.  

> Can we please all remember that we share the same goal here; that we all
> want to help advance Emacs and free software?

> > I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> > maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> > Mode issues which arise in Emacs.  Please post these to
> > bug-cc-mode@gnu.org.

> > It just remains to say that my respect for Eli and the other maintainers
> > remains undiminished, and that I wish all of them and the Emacs project
> > all success in the future.

> Thanks for continuing to maintain CC-mode, and likewise.

Thanks!

> I hope that you will seriously consider the idea to reverse your
> decision to quit Emacs development.  It would be much better if we could
> find a way where we can all continue working together.  I'd suggest
> giving the idea at least a couple of days to fully consider, though I'll
> of course respect your decision either way.

I can't honestly see myself changing my mind in the space of days.
Maybe in months, or a year or two.  But I would ask you and the other
maintainers to take seriously the criticisms I've made yesterday and
today.

> Meanwhile, if there is anything I can do to help improve things, please
> feel free to reach out.  Thanks again for all your work on Emacs.

And thanks for the project.  All in all, it's been a great project to
work on.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Eli Zaretskii]

> Date: Thu, 21 Nov 2024 13:01:52 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> I've had continual trouble over the last ~20 years with what Stefan has
> done, and how he's done it.  Nobody else even comes close.

I can only say that I completely disagree with your unfavorable (to
say the least) description of Stefan's conduct here, and regret and am
very sorry that you somehow came to these conclusions, which IMO are
very wrong.

Re: My resignation from Emacs development

[Alfred M. Szmidt]

   > I've had continual trouble over the last ~20 years with what Stefan has
   > done, and how he's done it.  Nobody else even comes close.

   I can only say that I completely disagree with your unfavorable (to
   say the least) description of Stefan's conduct here, and regret and am
   very sorry that you somehow came to these conclusions, which IMO are
   very wrong.

This has been brewing for a long time and you're putting your head
into the sand Eli, Alan is utterly on point.  Your comment is not
helping to mitigate the damage that has been caused in the least, and
making it worse.

Re: My resignation from Emacs development

[Po Lu]

"Alfred M. Szmidt" <ams@gnu.org> writes:

>    > I've had continual trouble over the last ~20 years with what Stefan has
>    > done, and how he's done it.  Nobody else even comes close.
>
>    I can only say that I completely disagree with your unfavorable (to
>    say the least) description of Stefan's conduct here, and regret and am
>    very sorry that you somehow came to these conclusions, which IMO are
>    very wrong.
>
> This has been brewing for a long time and you're putting your head
> into the sand Eli, Alan is utterly on point.  Your comment is not
> helping to mitigate the damage that has been caused in the least, and
> making it worse.

Yes, there has lately appeared a ``relentless modernization drive'', if
you will.  Most of these changes are not worth the controversy they
create.

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  acm@muc.de,  emacs-devel@gnu.org
> Date: Fri, 22 Nov 2024 08:01:37 +0800
> 
> "Alfred M. Szmidt" <ams@gnu.org> writes:
> 
> >    > I've had continual trouble over the last ~20 years with what Stefan has
> >    > done, and how he's done it.  Nobody else even comes close.
> >
> >    I can only say that I completely disagree with your unfavorable (to
> >    say the least) description of Stefan's conduct here, and regret and am
> >    very sorry that you somehow came to these conclusions, which IMO are
> >    very wrong.
> >
> > This has been brewing for a long time and you're putting your head
> > into the sand Eli, Alan is utterly on point.  Your comment is not
> > helping to mitigate the damage that has been caused in the least, and
> > making it worse.
> 
> Yes, there has lately appeared a ``relentless modernization drive'', if
> you will.  Most of these changes are not worth the controversy they
> create.

Given that many community members seem to think our current approach
is actually too conservative and prevents faster progress, I guess the
reality is somewhere in the middle, probably quite farther from that
"relentless modernization" than you seem to imply.

Re: My resignation from Emacs development

[Robert Pluim]

>>>>> On Fri, 22 Nov 2024 09:03:33 +0200, Eli Zaretskii <eliz@gnu.org> said:
    >> Yes, there has lately appeared a ``relentless modernization drive'', if
    >> you will.  Most of these changes are not worth the controversy they
    >> create.

    Eli> Given that many community members seem to think our current approach
    Eli> is actually too conservative and prevents faster progress, I guess the
    Eli> reality is somewhere in the middle, probably quite farther from that
    Eli> "relentless modernization" than you seem to imply.

I suspect that Po Lu (and Alan) were talking more about the increase
in the use of cl and pcase (which I still havenʼt got my head around,
and now we already have cond* as well).

Robert
-- 

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Po Lu <luangruo@yahoo.com>,  ams@gnu.org,  acm@muc.de,  emacs-devel@gnu.org
> Date: Fri, 22 Nov 2024 09:14:55 +0100
> 
> >>>>> On Fri, 22 Nov 2024 09:03:33 +0200, Eli Zaretskii <eliz@gnu.org> said:
>     >> Yes, there has lately appeared a ``relentless modernization drive'', if
>     >> you will.  Most of these changes are not worth the controversy they
>     >> create.
> 
>     Eli> Given that many community members seem to think our current approach
>     Eli> is actually too conservative and prevents faster progress, I guess the
>     Eli> reality is somewhere in the middle, probably quite farther from that
>     Eli> "relentless modernization" than you seem to imply.
> 
> I suspect that Po Lu (and Alan) were talking more about the increase
> in the use of cl and pcase (which I still havenʼt got my head around,
> and now we already have cond* as well).

The increase in their use is real, but we don't allow it without
careful consideration.  In particular, they should not be used where
simpler constructs can reasonably do the job.

Other than that, this is a community project.  People must understand
that the maintainers are just stewards; we don't "own" the project,
and thus have no moral right to stubbornly oppose de-facto coding
practices that are overwhelmingly favored by the community (including
at least some of the maintainers, judging by their code).  Any demands
that we forbid use of cl-lib or pcase in Emacs are thus unfair at
best.

Re: My resignation from Emacs development

[Po Lu]

Robert Pluim <rpluim@gmail.com> writes:

> I suspect that Po Lu (and Alan) were talking more about the increase
> in the use of cl and pcase (which I still havenʼt got my head around,
> and now we already have cond* as well).
>
> Robert

Not this time.  My anecdote for this round of controversy is the
obsoletion of if-let at the whim of a select group of people in the bug
tracker, exactly when most users have just integrated it into their
muscle memory.

What is this doc string meant to imply, for example?

    This macro will be marked obsolete in Emacs 31.1; prefer `if-let*'
    in new code.

Why is there now a manner of "pre-obsoleting" functions?

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  ams@gnu.org,  acm@muc.de,
>   emacs-devel@gnu.org
> Date: Sat, 23 Nov 2024 07:59:38 +0800
> 
> Robert Pluim <rpluim@gmail.com> writes:
> 
> > I suspect that Po Lu (and Alan) were talking more about the increase
> > in the use of cl and pcase (which I still havenʼt got my head around,
> > and now we already have cond* as well).
> >
> > Robert
> 
> Not this time.  My anecdote for this round of controversy is the
> obsoletion of if-let at the whim of a select group of people in the bug
> tracker, exactly when most users have just integrated it into their
> muscle memory.
> 
> What is this doc string meant to imply, for example?
> 
>     This macro will be marked obsolete in Emacs 31.1; prefer `if-let*'
>     in new code.
> 
> Why is there now a manner of "pre-obsoleting" functions?

Emacs 31.1 is pretty far away, so there's still time to discuss this
and change the decision, if there's an agreement.  Jonas Bernoulli
posted a comprehensive critique of that decision, and we should expect
the people who involved in that discussion to consider his arguments
very seriously.

It is unrealistic to expect that no controversial decisions will ever
be made in a community where more than a single individual installs
changes.  What is important is to keep cooperative spirit even when
we disagree with the eventual decisions.  Because presumably our
common goal is more important than disagreements here and there.

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Eli.

On Thu, Nov 21, 2024 at 15:48:06 +0200, Eli Zaretskii wrote:
> > Date: Thu, 21 Nov 2024 13:01:52 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I've had continual trouble over the last ~20 years with what Stefan has
> > done, and how he's done it.  Nobody else even comes close.

> I can only say that I completely disagree with your unfavorable (to
> say the least) description of Stefan's conduct here, and regret and am
> very sorry that you somehow came to these conclusions, which IMO are
> very wrong.

The five anecdotes I outlined in detail in my post to Stefan K happened
as I described.  How can one avoid the conclusion I came to?

In the second of these (about Stefan M's change of interpreted functions
to opaque atoms) Richard posted to the thread five times.  In one of
these posts he described the proposed change as "perverse".  Stefan
ignored all five of Richard's posts, and bulldozered his change through
anyway.  Richard was unhappy about the change, and you were uncertain
about it, to say the least.  Stefan ignored both of you (as well as
being discourteous to me) and just ploughed ahead.

What does all this say about Stefan Monnier?

It was Stefan's commit after that thread, and the fact that nothing was
done about it, that caused me finally to lose enthusiasm for the Emacs
project.  Since then, I've kept going, basically by inertia and habit.

If you want to look at this thread again, it starts with this post:

From: Stefan Monnier <monnier@iro.umontreal.ca>
To: emacs-devel@gnu.org
Subject: Distinguishing `consp` and `functionp`
Date: Thu, 25 Jan 2024 18:15:48 -0500

..

One way or another, Stefan will have become aware of this thread.  He's
at liberty to answer and contest all the points I've made about him.  I
doubt he will do so.

Anyhow, I'm leaving.  I think I've now made it abundantly clear _why_
I'm leaving.

I still wish Emacs a successful future.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Adam Porter]

Dear Alan,

I have not corresponded with you before, but as a user and contributor
myself, I appreciate your contributions to Emacs.

Now, I usually steer clear of threads like these; I see little benefit
to taking sides or passing judgment on those who are implicated.  Were
you simply saying farewell, I'd be glad to wish you the same, and leave
it at that.

But I can't, in good conscience, stand by and say nothing after your
comments like this:

> This "be nice to everybody no matter what they do" and "always 
> assume the best of everybody" creates the perfect atmosphere for a 
> monster to flourish in.  Stefan is such a monster; not all the time,
> not even most of the time, but in doing the things detailed above,
> and other things, I don't understand why you are defending him.
> 
> I've had continual trouble over the last ~20 years with what Stefan 
> has done, and how he's done it.  Nobody else even comes close.  As I
> said, this is the root cause of why I'm leaving the Emacs team. Most
> of the time, he is extremely helpful and efficient at maintaining,
> and I'm grateful for all the help he has given me over the years. As
> I said, a Jekyll-and-Hyde character.

These accusations are beyond unfair and unkind.  You even followed it up
immediately with:

> I have not come anywhere near ad hominem.

If calling someone a "monster" is not ad hominem, I don't know what is.
  As well, your other comments about Stefan M., including your list of
historical grievances, are essentially a form of character
assassination.  I have watched as you have publicly impugned Stefan's
motivation and character several times before; it was wrong then, and it
is even more so now, as you essentially accuse him of being a bully--
ironic, since "bullying" seems like an apt characterization of your
comments about him.

> It is true that many forums degenerate into slanging matches which 
> repel decent posters. emacs-devel is the opposite extreme, sort of 
> touchy-feely where nobody's allowed to offend anybody else at all, 
> no matter what they do, why and how they do it.  This is just as 
> unhealthy as the the continual abuse forums; it leads to the build 
> up of repressed resentment.

I don't find that characterization of this mailing list to be accurate
at all.  There is infrequent, but consistently repellent content from
certain participants; thankfully, it is usually not repaid in kind;
occasionally it draws a tame chastisement.  It's to be expected, when
you consider the variety of backgrounds present, combined with strong
personalities and enthusiastic participation.  There are far, far worse
forums to be found.

As to participation, everyone is a volunteer here; everyone is free to
contribute or not, as he sees fit.  You should do what seems best to 
you, whether that means continuing to contribute, scaling back your 
contributions, or ceasing to contribute.  If it is no longer enjoyable 
for you to contribute, for whatever reason, then you probably shouldn't, 
for your own sake.  I would say that to anyone, including myself.  At 
the same time, I would encourage you to reconsider the decision to cease 
contributing altogether, for your sake and others', as your 
contributions have been valuable to innumerable people; and as hobbies 
go, this is a pretty good one.  And there's nothing wrong with taking a 
break.

But it is not okay for you to blame Stefan for your decision to leave.
  As you know, in the past he served as the Emacs maintainer, and now he
remains a prominent contributor, and a maintainer of some parts of
Emacs, but not of the overall project.  So if you can't abide some
technical decisions that have been made by Stefan M., you ought to take
them up with Eli, Andrea, and Stefan K.  And if they disagree with you
and won't overturn those decisions, and you decide to leave, you ought 
to ascribe that responsibility simply and honestly, not by publicly
defaming Stefan M. like this.  It does not behoove you, nor the GNU
Emacs project, to act this way.

Sincerely,
Adam

Re: My resignation from Emacs development

[Madhu]

* Adam Porter <169c6564-4722-4338-a049-5f8f3ce69394@alphapapa.net> :
Wrote on Thu, 21 Nov 2024 23:35:35 -0600:

> But it is not okay for you to blame Stefan for your decision to leave.
>  As you know, in the past he served as the Emacs maintainer, and now he
> remains a prominent contributor, and a maintainer of some parts of
> Emacs, but not of the overall project.  So if you can't abide some
> technical decisions that have been made by Stefan M., you ought to take
> them up with Eli, Andrea, and Stefan K.  And if they disagree with you
> and won't overturn those decisions, and you decide to leave, you ought
> to ascribe that responsibility simply and honestly, not by publicly
> defaming Stefan M. like this.  It does not behoove you, nor the GNU
> Emacs project, to act this way.

The correct response to these complaints would be if Eli and the
maintainers would reign in Stefan and address the genuine concerns
rather than gerrymandering on with the present course.

This again purposely misses the point and does not address the isssue
raised by the resignation. That there are problems with Stefans
intentions which are not as stated which are of concern to emacs
development as a whole, these are being swept under the floor. There are
repeated concerns that the way Stefan is leading development us
destroying the value of core which RMS brought to us from the 70s. This
is being set aside through bulldozering narrative.

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Madhu <enometh@meer.net>
> Date: Fri, 22 Nov 2024 12:54:01 +0530
> 
> * Adam Porter <169c6564-4722-4338-a049-5f8f3ce69394@alphapapa.net> :
> Wrote on Thu, 21 Nov 2024 23:35:35 -0600:
> 
> > But it is not okay for you to blame Stefan for your decision to leave.
> >  As you know, in the past he served as the Emacs maintainer, and now he
> > remains a prominent contributor, and a maintainer of some parts of
> > Emacs, but not of the overall project.  So if you can't abide some
> > technical decisions that have been made by Stefan M., you ought to take
> > them up with Eli, Andrea, and Stefan K.  And if they disagree with you
> > and won't overturn those decisions, and you decide to leave, you ought
> > to ascribe that responsibility simply and honestly, not by publicly
> > defaming Stefan M. like this.  It does not behoove you, nor the GNU
> > Emacs project, to act this way.
> 
> The correct response to these complaints would be if Eli and the
> maintainers would reign in Stefan and address the genuine concerns
> rather than gerrymandering on with the present course.
> 
> This again purposely misses the point and does not address the isssue
> raised by the resignation. That there are problems with Stefans
> intentions which are not as stated which are of concern to emacs
> development as a whole, these are being swept under the floor. There are
> repeated concerns that the way Stefan is leading development us
> destroying the value of core which RMS brought to us from the 70s. This
> is being set aside through bulldozering narrative.

When we see such problems, they are _never_ swept under the carpet.
On the contrary, the reaction is usually immediate and quite harsh,
including (but not limited to) reverting the offending changes in a
non-negotiable way.

The reason this didn't happen with Stefan Monnier is that at least I
don't see any particular problem of this kind in what Stefan does (and
did during the decades of his very active involvement in the project).
The Emacs model of development is that we completely trust leading
contributors to install changes without discussing them.  This trust
works well and keeps our development moving forward very fast,
although sometimes there are good-faith mistakes, which then require
discussions a-posteriori, and sometimes (rarely) end up with changes
being reverted or radically modified.  All of the leading
contributors, including yours truly, have sometimes, rarely, made such
mistakes.  Stefan's record is not different in this regard from any
other's.  The changes he installed in March indeed should have been
discussed more, but I don't expect us rejecting them as result.
Moreover, Alan himself made such a mistake when he installed his
cc-mode.el change back in May, the change which led to bug#74339, and
eventually to this sad result (because Alan staunchly opposed to
modifying his change from back then, even though the modifications
proposed to him would not affect the effect of his change in any way).

So there's nothing here that requires any "reigning in", just the
normal practice of Emacs development, which hasn't changed in decades,
because we think it fits well the way this community is structured,
and the nature and the vast span of expertise needed to develop an
maintain Emacs.

I cannot speak for Stefan Kangas and Andrea, but I'd be very surprised
if they didn't agree with what I say above.  We definitely don't agree
that the many changes developed and installed by Stefan are
"destroying the value of core which RMS brought to us from the 70s."
Quite the opposite.

So no, we are not "gerrymandering", and the reason we don't even
consider "reigning in" Stefan is because we see absolutely no problems
with his conduct, certainly no malice.  And while it is easy to
bad-mouth our ways of leading the project "from the fences" by people
who are not really involved with the day in and day out hard work on
helping this community move forward, and never contributed anything
significant to that movement, the truth is very far from these claims.
You and Alfred, and even Po Lu, are well advised to tone down your
claims, and exercise much more humility when you post such vicious
personal attacks on Stefan and others.

Re: My resignation from Emacs development

[Madhu]

* Eli Zaretskii <86ldxbofgw.fsf@gnu.org> :
Wrote on Fri, 22 Nov 2024 10:11:27 +0200:
> So no, we are not "gerrymandering", and the reason we don't even
> consider "reigning in" Stefan is because we see absolutely no problems
> with his conduct, certainly no malice.  And while it is easy to
> bad-mouth our ways of leading the project "from the fences" by people
> who are not really involved with the day in and day out hard work on
> helping this community move forward, and never contributed anything
> significant to that movement, the truth is very far from these claims.
> You and Alfred, and even Po Lu, are well advised to tone down your
> claims, and exercise much more humility when you post such vicious
> personal attacks on Stefan and others.

I offer the criticism in good faith, not out of any malice for Stefan
but because I see the direction as being detrimental for the original
goals of the project. (Mis)Characterising criticisms as vicious personal
attacks and doubling on the claims and refuse to acknowledge the valid
criticism does not help anyone except probably the project management. I
understand your "advice" as a threat that further criticism will be
cancelled and not dealt with.

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Madhu <enometh@meer.net>
> Date: Fri, 22 Nov 2024 14:56:39 +0530
> 
> 
> * Eli Zaretskii <86ldxbofgw.fsf@gnu.org> :
> Wrote on Fri, 22 Nov 2024 10:11:27 +0200:
> > So no, we are not "gerrymandering", and the reason we don't even
> > consider "reigning in" Stefan is because we see absolutely no problems
> > with his conduct, certainly no malice.  And while it is easy to
> > bad-mouth our ways of leading the project "from the fences" by people
> > who are not really involved with the day in and day out hard work on
> > helping this community move forward, and never contributed anything
> > significant to that movement, the truth is very far from these claims.
> > You and Alfred, and even Po Lu, are well advised to tone down your
> > claims, and exercise much more humility when you post such vicious
> > personal attacks on Stefan and others.
> 
> I offer the criticism in good faith, not out of any malice for Stefan
> but because I see the direction as being detrimental for the original
> goals of the project.

I was responding to your criticism of the Emacs maintainers.  You
expect us to "reign in" Stefan based on your assessment of his
actions, and accuse us of "gerrymandering" because we don't.  If your
criticism of Stefan is in good faith, then please limit it to the
changes that Stefan installs and the developments he leads.  If we
agree with your criticism, we will certainly act on it.  But as long
as we the maintainers disagree with your assessment of Stefan's
activities, you cannot expect us to act against them, and have no
right to reprimand us for not taking those actions.

> (Mis)Characterising criticisms as vicious personal attacks

No such mis-characterization was present in what I wrote.

> and doubling on the claims and refuse to acknowledge the valid
> criticism

Your criticism was indeed valid, but there's no obligation for us to
agree with it.

> I understand your "advice" as a threat that further criticism will
> be cancelled and not dealt with.

There were no threats, none at all.  Any criticism is taken seriously
and is considered.  But again, don't expect us to agree with
everything you (or anyone else) says.

And I would expect you to tone down your claims because you are not
involved in Emacs maintenance, and so lack both the broad perspective
and the detailed information about what is going on and how decisions
are being made.  That doesn't mean you cannot ask tough questions and
offer radically different perspectives on these issues, but please
choose words carefully on the assumption that you might not know
enough to pass definitive judgments.

Re: My resignation from Emacs development

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Madhu <enometh@meer.net>
>> Date: Fri, 22 Nov 2024 12:54:01 +0530
>>
>> The correct response to these complaints would be if Eli and the
>> maintainers would reign in Stefan and address the genuine concerns
>> rather than gerrymandering on with the present course.
>>
>> This again purposely misses the point and does not address the isssue
>> raised by the resignation. That there are problems with Stefans
>> intentions which are not as stated which are of concern to emacs
>> development as a whole, these are being swept under the floor. There are
>> repeated concerns that the way Stefan is leading development us
>> destroying the value of core which RMS brought to us from the 70s. This
>> is being set aside through bulldozering narrative.
>
> When we see such problems, they are _never_ swept under the carpet.
> On the contrary, the reaction is usually immediate and quite harsh,
> including (but not limited to) reverting the offending changes in a
> non-negotiable way.
>
> The reason this didn't happen with Stefan Monnier is that at least I
> don't see any particular problem of this kind in what Stefan does (and
> did during the decades of his very active involvement in the project).
> The Emacs model of development is that we completely trust leading
> contributors to install changes without discussing them.  This trust
> works well and keeps our development moving forward very fast,
> although sometimes there are good-faith mistakes, which then require
> discussions a-posteriori, and sometimes (rarely) end up with changes
> being reverted or radically modified.  All of the leading
> contributors, including yours truly, have sometimes, rarely, made such
> mistakes.  Stefan's record is not different in this regard from any
> other's.  The changes he installed in March indeed should have been
> discussed more, but I don't expect us rejecting them as result.
> Moreover, Alan himself made such a mistake when he installed his
> cc-mode.el change back in May, the change which led to bug#74339, and
> eventually to this sad result (because Alan staunchly opposed to
> modifying his change from back then, even though the modifications
> proposed to him would not affect the effect of his change in any way).
>
> So there's nothing here that requires any "reigning in", just the
> normal practice of Emacs development, which hasn't changed in decades,
> because we think it fits well the way this community is structured,
> and the nature and the vast span of expertise needed to develop an
> maintain Emacs.
>
> I cannot speak for Stefan Kangas and Andrea, but I'd be very surprised
> if they didn't agree with what I say above.  We definitely don't agree
> that the many changes developed and installed by Stefan are
> "destroying the value of core which RMS brought to us from the 70s."
> Quite the opposite.
>
> So no, we are not "gerrymandering", and the reason we don't even
> consider "reigning in" Stefan is because we see absolutely no problems
> with his conduct, certainly no malice.  And while it is easy to
> bad-mouth our ways of leading the project "from the fences" by people
> who are not really involved with the day in and day out hard work on
> helping this community move forward, and never contributed anything
> significant to that movement, the truth is very far from these claims.
> You and Alfred, and even Po Lu, are well advised to tone down your
> claims, and exercise much more humility when you post such vicious
> personal attacks on Stefan and others.

In case there is any doubt, I very much agree with everything that Eli
writes here.

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Eli.

On Fri, Nov 22, 2024 at 10:11:27 +0200, Eli Zaretskii wrote:
> > From: Madhu <enometh@meer.net>
> > Date: Fri, 22 Nov 2024 12:54:01 +0530

> > * Adam Porter <169c6564-4722-4338-a049-5f8f3ce69394@alphapapa.net> :
> > Wrote on Thu, 21 Nov 2024 23:35:35 -0600:

> > > But it is not okay for you to blame Stefan for your decision to leave.
> > >  As you know, in the past he served as the Emacs maintainer, and now he
> > > remains a prominent contributor, and a maintainer of some parts of
> > > Emacs, but not of the overall project.  So if you can't abide some
> > > technical decisions that have been made by Stefan M., you ought to take
> > > them up with Eli, Andrea, and Stefan K.  And if they disagree with you
> > > and won't overturn those decisions, and you decide to leave, you ought
> > > to ascribe that responsibility simply and honestly, not by publicly
> > > defaming Stefan M. like this.  It does not behoove you, nor the GNU
> > > Emacs project, to act this way.

> > The correct response to these complaints would be if Eli and the
> > maintainers would reign in Stefan and address the genuine concerns
> > rather than gerrymandering on with the present course.

> > This again purposely misses the point and does not address the isssue
> > raised by the resignation. That there are problems with Stefans
> > intentions which are not as stated which are of concern to emacs
> > development as a whole, these are being swept under the floor. There are
> > repeated concerns that the way Stefan is leading development us
> > destroying the value of core which RMS brought to us from the 70s. This
> > is being set aside through bulldozering narrative.

> When we see such problems, they are _never_ swept under the carpet.
> On the contrary, the reaction is usually immediate and quite harsh,
> including (but not limited to) reverting the offending changes in a
> non-negotiable way.

> The reason this didn't happen with Stefan Monnier is that at least I
> don't see any particular problem of this kind in what Stefan does (and
> did during the decades of his very active involvement in the project).

I have taken the trouble to outline and analyse in detail several places
where Stefan's contributions have given rise to trouble, at least for me
personally, and certainly for some other contributors.  Up till now, none
of you, Stefan K, or Andrea have replied with the same level of detail,
or even acknowledged what I wrote.  I'm disappointed at this.

> The Emacs model of development is that we completely trust leading
> contributors to install changes without discussing them.  This trust
> works well and keeps our development moving forward very fast,
> although sometimes there are good-faith mistakes, which then require
> discussions a-posteriori, and sometimes (rarely) end up with changes
> being reverted or radically modified.

The model of development for all contributors bar Stefan M has involved
open discussion on emacs-devel before big changes.  No other leading
contributor has violated this convention that I'm aware of.  For example,
when you implemented the display-line-numbers facility, you did so
entirely in public.  This resulted in several bugs being found by other
people and corrected, and resulted in a smoother introduction of the
feature than otherwise would have been the case.

> All of the leading contributors, including yours truly, have sometimes,
> rarely, made such mistakes.  Stefan's record is not different in this
> regard from any other's.

Eli, how can you say this?  Stefan's record is _very_ different.  I could
not have posted five anecdotes about any other contributor the way I did
yesterday for Stefan.  The way he makes big changes to Emacs could almost
be calculated to cause "mistakes".

> The changes he installed in March indeed should have been discussed
> more, but I don't expect us rejecting them as result.

I have been trying to get you, Stefan K, and Andrea to analyse _why_
these changes were not discussed more.  Those changes are what led to my
resignation.

> Moreover, Alan himself made such a mistake when he installed his
> cc-mode.el change back in May, the change which led to bug#74339, and
> eventually to this sad result (because Alan staunchly opposed to
> modifying his change from back then, even though the modifications
> proposed to him would not affect the effect of his change in any way).

My mistake was more political than technical.  Had I been more forthright
in exposing the problem in May, I still doubt anything would have been
done about it, precisely because the commit causing it was made by "a
leading contributor".  That is an expectation I wouldn't have had two
days ago.

> So there's nothing here that requires any "reigning in", just the
> normal practice of Emacs development, which hasn't changed in decades,
> because we think it fits well the way this community is structured,
> and the nature and the vast span of expertise needed to develop and
> maintain Emacs.

I have regrettably resigned from Emacs, precisely because of this "normal
practice of Emacs development".

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Stefan Kangas]

Alan Mackenzie <acm@muc.de> writes:

> I have taken the trouble to outline and analyse in detail several places
> where Stefan's contributions have given rise to trouble, at least for me
> personally, and certainly for some other contributors.  Up till now, none
> of you, Stefan K, or Andrea have replied with the same level of detail,
> or even acknowledged what I wrote.  I'm disappointed at this.

Please allow for more than 24 hours before drawing any conclusions.
Yesterday, I barely had time to read your message, much less reply.

I will reply to your message in detail soon.

Re: My resignation from Emacs development

[Eli Zaretskii]

> Date: Fri, 22 Nov 2024 13:06:40 +0000
> Cc: Madhu <enometh@meer.net>, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > The reason this didn't happen with Stefan Monnier is that at least I
> > don't see any particular problem of this kind in what Stefan does (and
> > did during the decades of his very active involvement in the project).
> 
> I have taken the trouble to outline and analyse in detail several places
> where Stefan's contributions have given rise to trouble, at least for me
> personally, and certainly for some other contributors.  Up till now, none
> of you, Stefan K, or Andrea have replied with the same level of detail,
> or even acknowledged what I wrote.  I'm disappointed at this.

Like I said: good-faith mistakes do happen, but they happen rarely.
The cases you mentioned don't change and don't contradict that, even
if one agrees 100% with your assessment of what happened in those
cases.

> 
> > The Emacs model of development is that we completely trust leading
> > contributors to install changes without discussing them.  This trust
> > works well and keeps our development moving forward very fast,
> > although sometimes there are good-faith mistakes, which then require
> > discussions a-posteriori, and sometimes (rarely) end up with changes
> > being reverted or radically modified.
> 
> The model of development for all contributors bar Stefan M has involved
> open discussion on emacs-devel before big changes.  No other leading
> contributor has violated this convention that I'm aware of.  For example,
> when you implemented the display-line-numbers facility, you did so
> entirely in public.

Bad example.  The line-number display was a feature I implemented
because others requested it; I myself don't use it and consider it
un-Emacsy, as I told back then.  So I needed the feedback very much
because I couldn't myself make decisions about a feature I don't and
won't use.

If you want a better example, compare with development of
bidirectional editing support.  There, I posted a very small number of
messages describing my design decisions, but never asked for their
approval -- because in that area I know more than most anyone here.

For a smaller and more recent example, consider the implementation of
TTY menus.  Or even the recent support for thumbnails on MS-Windows.

> > All of the leading contributors, including yours truly, have sometimes,
> > rarely, made such mistakes.  Stefan's record is not different in this
> > regard from any other's.
> 
> Eli, how can you say this?  Stefan's record is _very_ different.

As I made it abundantly clear, I disagree.

> > Moreover, Alan himself made such a mistake when he installed his
> > cc-mode.el change back in May, the change which led to bug#74339, and
> > eventually to this sad result (because Alan staunchly opposed to
> > modifying his change from back then, even though the modifications
> > proposed to him would not affect the effect of his change in any way).
> 
> My mistake was more political than technical.  Had I been more forthright
> in exposing the problem in May, I still doubt anything would have been
> done about it, precisely because the commit causing it was made by "a
> leading contributor".  That is an expectation I wouldn't have had two
> days ago.

If you had explained what you are about to do, I would have objected
right there and then.  Instead, I was surprised to discover this the
hard way when I installed the 2nd pretest of Emacs 30 (I have no idea
how I missed that with the first pretest).

> > So there's nothing here that requires any "reigning in", just the
> > normal practice of Emacs development, which hasn't changed in decades,
> > because we think it fits well the way this community is structured,
> > and the nature and the vast span of expertise needed to develop and
> > maintain Emacs.
> 
> I have regrettably resigned from Emacs, precisely because of this "normal
> practice of Emacs development".

And I regret your decision very much.  I think and hope our common
goal of developing Emacs should allow us to cooperate even with people
with whom we occasionally disagree, even when the disagreements are
radical.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

If we explicitly establish that normal practice of Emacs development
is to to discuss every significant feature change or redesign on
emacs-devel -- as I have advcated before -- would you be willing to
rejoin Emacs development?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Richard.

On Sun, Nov 24, 2024 at 23:28:14 -0500, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

> If we explicitly establish that normal practice of Emacs development
> is to to discuss every significant feature change or redesign on
> emacs-devel -- as I have advcated before -- would you be willing to
> rejoin Emacs development?

I think that's been the normal practice in Emacs for just about ever.

My gripe is specifically with Stefan Monnier, who has failed to follow
this practice on quite a few occasions over the years, as I have
catalogued in some detail in this thread.

Also, when promoting his own changes, Stefan has frequently answered
reasonable questions with "politicians' answers" - evasive non-answers.
The latest such post was his post in this thread, where he somehow
managed to avoid addressing my criticisms of his past behaviour.  In
particular, he failed to admit any wrongdoing, and failed to give any
undertakings about the future.

Also, the current three maintainers have all explicitly said they see no
problem with Stefan's behaviour here.  If I were to rejoin the Emacs
project (assuming I'd be welcome, which might not be the case after all
I've written), nothing would have changed.  All the unpleasantness in
this thread would have been for nothing.  And I'd likely continue to get
angry about Stefan in the future.

The immediate problem - the use of CC Mode's symbols to mean other
things - has no available compromise.  Eli and I failed to reach such a
compromise after extensive exchanges, both on emacs-devel and in private
email.

So it doesn't look like I'll be rejoining the Emacs project any time
soon.  Sorry.

> -- 
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Please forgive my delay in getting back to this.

  > The immediate problem - the use of CC Mode's symbols to mean other
  > things - has no available compromise.

I'm surprised by that.  I proposed more than one idea, and there was
some discussion.  But I didn't see where it ended up.  Did that
discussion peter out without arriving anywhere?  Was any alternative
interface seriously considered?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Richard.

On Thu, Dec 12, 2024 at 23:35:28 -0500, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

> Please forgive my delay in getting back to this.

No problem!

>   > The immediate problem - the use of CC Mode's symbols to mean other
>   > things - has no available compromise.

> I'm surprised by that.  I proposed more than one idea, and there was
> some discussion.  But I didn't see where it ended up.  Did that
> discussion peter out without arriving anywhere?  Was any alternative
> interface seriously considered?

What has been implemented, "remapping", is a sort of extreme version of
advice: it supersedes a symbol's function by some other function.
What's more, it is the core Emacs functions which do this, not some
wierd user setting.  CC Mode's symbol `c-mode' now sometimes means C
Mode, sometimes c-ts-mode.

I don't think this is a good technical solution for whatever problem it
was intended to solve.

I was not involved in the discussion which decided to implement this,
assuming there was such a discussion.  I have been unable to find it in
the archives, and nobody has given me a reference to it, despite it
being relevant to this thread.

The lack of available compromise is largely due to needing/wanting to
get the upcoming release released on time, without making any
significant changes to the code which might make an extra pretest
necessary.

> -- 
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Eli Zaretskii]

> Date: Sun, 15 Dec 2024 15:27:31 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> What has been implemented, "remapping", is a sort of extreme version of
> advice: it supersedes a symbol's function by some other function.
> What's more, it is the core Emacs functions which do this, not some
> wierd user setting.  CC Mode's symbol `c-mode' now sometimes means C
> Mode, sometimes c-ts-mode.
> 
> I don't think this is a good technical solution for whatever problem it
> was intended to solve.
> 
> I was not involved in the discussion which decided to implement this,
> assuming there was such a discussion.  I have been unable to find it in
> the archives, and nobody has given me a reference to it, despite it
> being relevant to this thread.

Remapping of major modes was introduced in Sep 2022, and was discussed
in bug#58075 (which was opened for that purpose).  From my POV, it's
just a convenient user option, so discussing it as a feature-request
bug report was appropriate.

Btw, I suspect that when Richard says "I proposed more than one idea",
he refers to a much more recent discussion, not about what happened
when mode-remapping was added to Emacs.

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Eli.

On Sun, Dec 15, 2024 at 17:48:20 +0200, Eli Zaretskii wrote:
> > Date: Sun, 15 Dec 2024 15:27:31 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > What has been implemented, "remapping", is a sort of extreme version of
> > advice: it supersedes a symbol's function by some other function.
> > What's more, it is the core Emacs functions which do this, not some
> > wierd user setting.  CC Mode's symbol `c-mode' now sometimes means C
> > Mode, sometimes c-ts-mode.

> > I don't think this is a good technical solution for whatever problem it
> > was intended to solve.

> > I was not involved in the discussion which decided to implement this,
> > assuming there was such a discussion.  I have been unable to find it in
> > the archives, and nobody has given me a reference to it, despite it
> > being relevant to this thread.

> Remapping of major modes was introduced in Sep 2022, and was discussed
> in bug#58075 (which was opened for that purpose).

Thanks, I've read it now.

> From my POV, it's just a convenient user option, so discussing it as a
> feature-request bug report was appropriate.

To me, it screems out "there be dragons, here", even from the very first
post.

> Btw, I suspect that when Richard says "I proposed more than one idea",
> he refers to a much more recent discussion, not about what happened
> when mode-remapping was added to Emacs.

OK.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What has been implemented, "remapping", is a sort of extreme version of
  > advice: it supersedes a symbol's function by some other function.

I agree.  There are various cleaner ways to do this particular job.

  > What's more, it is the core Emacs functions which do this, not some
  > wierd user setting.  CC Mode's symbol `c-mode' now sometimes means C
  > Mode, sometimes c-ts-mode.

This situation is complex because what was previously a single
operation has to be replaced by a choice of two partly similar
operations.  That is always somewhat messy, but we can make it less
messy than this.

One of the two new operations is, "Select C mode, no TS."
the other new operation is, "Select my usual mode for C."
They used to be entirely equiva;ent, but now they are not.

We could define a name for each of those operations.

Maybe there could be `c-no-ts-mode' and `c-ts-mode', as well as
`c-mode' which would follow the user's previously stated choice
between the other two.

  > The lack of available compromise is largely due to needing/wanting to
  > get the upcoming release released on time, without making any
  > significant changes to the code which might make an extra pretest
  > necessary.

That is not necessarily a horrible error if it is _temporary_.
What would be horrible is to let this hurried expedient decide
the commands of Emacs forever.



-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Wed, 18 Dec 2024 23:22:01 -0500
> 
>   > What has been implemented, "remapping", is a sort of extreme version of
>   > advice: it supersedes a symbol's function by some other function.
> 
> I agree.

I don't agree.  This is not the first (nor the last) time we have
similar "remapping" in Emacs.  We have "<remap> <SOMETHING>" in
keymaps (see "Remapping Commands" in the ELisp manual).  We have
face-remapping-alist, which remaps a named face into some different
face.  We have shortcuts, which "remap" symbols.  And I'm sure I can
find more examples of similar "remappings".

>   > What's more, it is the core Emacs functions which do this, not some
>   > wierd user setting.  CC Mode's symbol `c-mode' now sometimes means C
>   > Mode, sometimes c-ts-mode.
> 
> This situation is complex because what was previously a single
> operation has to be replaced by a choice of two partly similar
> operations.  That is always somewhat messy, but we can make it less
> messy than this.
> 
> One of the two new operations is, "Select C mode, no TS."
> the other new operation is, "Select my usual mode for C."
> They used to be entirely equiva;ent, but now they are not.
> 
> We could define a name for each of those operations.
> 
> Maybe there could be `c-no-ts-mode' and `c-ts-mode', as well as
> `c-mode' which would follow the user's previously stated choice
> between the other two.
> 
>   > The lack of available compromise is largely due to needing/wanting to
>   > get the upcoming release released on time, without making any
>   > significant changes to the code which might make an extra pretest
>   > necessary.
> 
> That is not necessarily a horrible error if it is _temporary_.
> What would be horrible is to let this hurried expedient decide
> the commands of Emacs forever.

We are discussing the various ways of having a cleaner set of commands
and features in future Emacs releases.  But none of them will remove
major-mode-remap-alist from Emacs, though we might decide not to use
it for some of these features.

Re: My resignation from Emacs development

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Madhu <enometh@meer.net>
>> Date: Fri, 22 Nov 2024 12:54:01 +0530
>> 
>> * Adam Porter <169c6564-4722-4338-a049-5f8f3ce69394@alphapapa.net> :
>> Wrote on Thu, 21 Nov 2024 23:35:35 -0600:
>> 
>> > But it is not okay for you to blame Stefan for your decision to leave.
>> >  As you know, in the past he served as the Emacs maintainer, and now he
>> > remains a prominent contributor, and a maintainer of some parts of
>> > Emacs, but not of the overall project.  So if you can't abide some
>> > technical decisions that have been made by Stefan M., you ought to take
>> > them up with Eli, Andrea, and Stefan K.  And if they disagree with you
>> > and won't overturn those decisions, and you decide to leave, you ought
>> > to ascribe that responsibility simply and honestly, not by publicly
>> > defaming Stefan M. like this.  It does not behoove you, nor the GNU
>> > Emacs project, to act this way.
>> 
>> The correct response to these complaints would be if Eli and the
>> maintainers would reign in Stefan and address the genuine concerns
>> rather than gerrymandering on with the present course.
>> 
>> This again purposely misses the point and does not address the isssue
>> raised by the resignation. That there are problems with Stefans
>> intentions which are not as stated which are of concern to emacs
>> development as a whole, these are being swept under the floor. There are
>> repeated concerns that the way Stefan is leading development us
>> destroying the value of core which RMS brought to us from the 70s. This
>> is being set aside through bulldozering narrative.
>
> When we see such problems, they are _never_ swept under the carpet.
> On the contrary, the reaction is usually immediate and quite harsh,
> including (but not limited to) reverting the offending changes in a
> non-negotiable way.
>
> The reason this didn't happen with Stefan Monnier is that at least I
> don't see any particular problem of this kind in what Stefan does (and
> did during the decades of his very active involvement in the project).
> The Emacs model of development is that we completely trust leading
> contributors to install changes without discussing them.  This trust
> works well and keeps our development moving forward very fast,
> although sometimes there are good-faith mistakes, which then require
> discussions a-posteriori, and sometimes (rarely) end up with changes
> being reverted or radically modified.  All of the leading
> contributors, including yours truly, have sometimes, rarely, made such
> mistakes.  Stefan's record is not different in this regard from any
> other's.  The changes he installed in March indeed should have been
> discussed more, but I don't expect us rejecting them as result.
> Moreover, Alan himself made such a mistake when he installed his
> cc-mode.el change back in May, the change which led to bug#74339, and
> eventually to this sad result (because Alan staunchly opposed to
> modifying his change from back then, even though the modifications
> proposed to him would not affect the effect of his change in any way).
>
> So there's nothing here that requires any "reigning in", just the
> normal practice of Emacs development, which hasn't changed in decades,
> because we think it fits well the way this community is structured,
> and the nature and the vast span of expertise needed to develop an
> maintain Emacs.
>
> I cannot speak for Stefan Kangas and Andrea, but I'd be very surprised
> if they didn't agree with what I say above.  We definitely don't agree
> that the many changes developed and installed by Stefan are
> "destroying the value of core which RMS brought to us from the 70s."
> Quite the opposite.

Thanks Eli for being so clear on this, this is indeed absolutely agreed.

  Andrea

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Adam.

On Thu, Nov 21, 2024 at 23:35:35 -0600, Adam Porter wrote:

> Dear Alan,

> I have not corresponded with you before, but as a user and contributor
> myself, I appreciate your contributions to Emacs.

> Now, I usually steer clear of threads like these; I see little benefit
> to taking sides or passing judgment on those who are implicated.  Were
> you simply saying farewell, I'd be glad to wish you the same, and leave
> it at that.

> But I can't, in good conscience, stand by and say nothing after your
> comments like this:

> > This "be nice to everybody no matter what they do" and "always 
> > assume the best of everybody" creates the perfect atmosphere for a 
> > monster to flourish in.  Stefan is such a monster; not all the time,
> > not even most of the time, but in doing the things detailed above,
> > and other things, I don't understand why you are defending him.

> > I've had continual trouble over the last ~20 years with what Stefan 
> > has done, and how he's done it.  Nobody else even comes close.  As I
> > said, this is the root cause of why I'm leaving the Emacs team. Most
> > of the time, he is extremely helpful and efficient at maintaining,
> > and I'm grateful for all the help he has given me over the years. As
> > I said, a Jekyll-and-Hyde character.

> These accusations are beyond unfair and unkind.  You even followed it up
> immediately with:

> > I have not come anywhere near ad hominem.

> If calling someone a "monster" is not ad hominem, I don't know what is.

There's a lot of misunderstanding on the net about what ad hominem means.
It means critisizing something BECAUSE of who wrote/said/did it.  I have
not done this at all, here.  I am critisizing what Stefan has done,
fairly and squarely, and the (lack of) reaction from the various
maintainers.

If anything, it is Eli, Stefan K, and Andrea who are guilty of a sort of
"ad hominem defence".  None of them appeared seriously to have considered
what I have said about Stefan M and the running of the Emacs project.
They have defended his action because they were HIS actions.

>   As well, your other comments about Stefan M., including your list of
> historical grievances, are essentially a form of character
> assassination.  I have watched as you have publicly impugned Stefan's
> motivation and character several times before; it was wrong then, and it
> is even more so now, as you essentially accuse him of being a bully--
> ironic, since "bullying" seems like an apt characterization of your
> comments about him.

If I am attacking Stefan, it is because of what he has done.  If you were
being serious, you would analyse my main gripes about Stefan - that he
makes big changes to Emacs without prior discussion on emacs-devel, and
that he is discourteous on mailing lists - and point out where I am
mistaken.  I posted five anecdotes yesterday in my post to Stefan K.  You
could have challenged these in detail, possibly, the three current
maintainers certainly could have done.  None of you have done so.

> > It is true that many forums degenerate into slanging matches which 
> > repel decent posters. emacs-devel is the opposite extreme, sort of 
> > touchy-feely where nobody's allowed to offend anybody else at all, 
> > no matter what they do, why and how they do it.  This is just as 
> > unhealthy as the the continual abuse forums; it leads to the build 
> > up of repressed resentment.

> I don't find that characterization of this mailing list to be accurate
> at all.  There is infrequent, but consistently repellent content from
> certain participants; thankfully, it is usually not repaid in kind;
> occasionally it draws a tame chastisement.  It's to be expected, when
> you consider the variety of backgrounds present, combined with strong
> personalities and enthusiastic participation.  There are far, far worse
> forums to be found.

Again, you don't address my point, here.  That was that the extreme
delicacy in the Emacs mailing lists is exaggerated and conter-productive.

> As to participation, everyone is a volunteer here; everyone is free to
> contribute or not, as he sees fit.  You should do what seems best to 
> you, whether that means continuing to contribute, scaling back your 
> contributions, or ceasing to contribute.  If it is no longer enjoyable 
> for you to contribute, for whatever reason, then you probably shouldn't, 
> for your own sake.  I would say that to anyone, including myself.  At 
> the same time, I would encourage you to reconsider the decision to cease 
> contributing altogether, for your sake and others', as your 
> contributions have been valuable to innumerable people; and as hobbies 
> go, this is a pretty good one.  And there's nothing wrong with taking a 
> break.

> But it is not okay for you to blame Stefan for your decision to leave.

It is Stefan's actions, both in the past and no doubt on-going, that have
had negative effects on me, and his failure to adhere to the standards
that other contributors do.

Stefan will have learned, one way or another, of this thread.  He is
quite capable of responding, himself.  I doubt he will.

> As you know, in the past he served as the Emacs maintainer, and now he
> remains a prominent contributor, and a maintainer of some parts of
> Emacs, but not of the overall project.  So if you can't abide some
> technical decisions that have been made by Stefan M., you ought to take
> them up with Eli, Andrea, and Stefan K.  And if they disagree with you
> and won't overturn those decisions, and you decide to leave, you ought
> to ascribe that responsibility simply and honestly, not by publicly
> defaming Stefan M. like this.  It does not behoove you, nor the GNU
> Emacs project, to act this way.

There is no question of defamation.  What I have written is true and
verifiable, or fair comment at the very least.  You have faied to address
the substatntive points I've made.  I wonder why you posted.

> Sincerely,
> Adam

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Adam Porter]

Alan,

On 11/22/24 04:57, Alan Mackenzie wrote:

>> If calling someone a "monster" is not ad hominem, I don't know what is.
> 
> There's a lot of misunderstanding on the net about what ad hominem means.
> It means critisizing something BECAUSE of who wrote/said/did it.  I have
> not done this at all, here.  I am critisizing what Stefan has done,
> fairly and squarely, and the (lack of) reaction from the various
> maintainers.

A common definition of "ad hominem" is:

   appealing to personal considerations (rather than to fact
     or reason)

I think that calling someone a "monster" fits that definition. 
Regardless, calling Stefan, or anyone else here, a "monster" is not civil.

> If anything, it is Eli, Stefan K, and Andrea who are guilty of a sort of
> "ad hominem defence".  None of them appeared seriously to have considered
> what I have said about Stefan M and the running of the Emacs project.
> They have defended his action because they were HIS actions.

That is your characterization of their reasoning.

> If I am attacking Stefan, it is because of what he has done.  If you were
> being serious, you would analyse my main gripes about Stefan - that he
> makes big changes to Emacs without prior discussion on emacs-devel, and
> that he is discourteous on mailing lists - and point out where I am
> mistaken.  I posted five anecdotes yesterday in my post to Stefan K.  You
> could have challenged these in detail, possibly, the three current
> maintainers certainly could have done.  None of you have done so.

I am being quite serious, but it is not my responsibility to analyze and 
judge your technical disagreements with anyone, especially regarding 
issues I have not been involved with.  I am intentionally not commenting 
on them, because they are beside the point, which is your public 
behavior toward Stefan.

>> I don't find that characterization of this mailing list to be accurate
>> at all.  There is infrequent, but consistently repellent content from
>> certain participants; thankfully, it is usually not repaid in kind;
>> occasionally it draws a tame chastisement.  It's to be expected, when
>> you consider the variety of backgrounds present, combined with strong
>> personalities and enthusiastic participation.  There are far, far worse
>> forums to be found.
> 
> Again, you don't address my point, here.  That was that the extreme
> delicacy in the Emacs mailing lists is exaggerated and conter-productive.

I addressed it: I don't find your characterization to be accurate.  As I 
described, this list is remarkably tolerant of uncivil words, and any 
corrections are gentle.

>> But it is not okay for you to blame Stefan for your decision to leave.
> 
> It is Stefan's actions, both in the past and no doubt on-going, that have
> had negative effects on me, and his failure to adhere to the standards
> that other contributors do.

Do you think that any of your actions have had negative effects on him? 
On anyone else?  Or is he, in your eyes, the only guilty party?

Whose responsibility is it to judge whether he adheres to the project's 
standards?

> Stefan will have learned, one way or another, of this thread.  He is
> quite capable of responding, himself.  I doubt he will.

Generally, the subject of comments like yours would be wise to remain 
silent.  He needn't defend himself here--we know him.

> There is no question of defamation.  What I have written is true and
> verifiable, or fair comment at the very least.  You have faied to address
> the substatntive points I've made.  I wonder why you posted.
What you have written is your characterization of his actions and 
motivation.  As has been made clear, your opinion is not universally 
shared.

You have made several very unfair comments about him.  These are what 
I'm addressing, and I do so because you have made them publicly, and 
repeatedly.  I offer no opinion on the technical matters, because that 
is not my responsibility here; and because this is not a technical 
discussion, despite your framing it as one.

--Adam

Re: My resignation from Emacs development

[Daniel Radetsky]

On Thu, Nov 21, 2024 at 11:35:35PM -0600, Adam Porter wrote:
> But it is not okay for you to blame Stefan for your decision to leave.

I disagree. If he chooses to leave, and the reason for this
choice is Stefan's behavior or decisions, then blaming
Stefan seems straightforwardly informative.

I'm not as much of a veteran of this list as some of you,
and my few interactions with Stefan have been positive. So I
can't really speak to who's in the right and don't think I
should. But it's broadly better to have information about
what's going on and what decisions are being made and how
everyone feels about those decisions than to not have this
information.

Basically, if Stefan made a decisions, and this made Alan so
unhappy that he wants to leave, this is something everyone
should know. Sometimes this is the price of a decision. We
need to know the price to make informed choices.

I'm not accusing you of this specifically, but it seems like
in situations like this there's a desire to make the
situation black and white. Either Stefan made a bad decision
which ought to be reversed, and the fact that it is not
being reversed would justify Alan leaving, or Alan is being
unreasonable and thus his decision to leave is a foregone
conclusion being unfairly blamed on Stefan. Thus if we don't
want to reverse Stefan's decision, we must believe that Alan
is being unreasonable.

But it's also possible that e.g. Stefan made a good decision
in the big picture, but this was locally problematic for
Alan. And even though we prefer Stefan's good decision, we
prefer a worse decision with the benefit of Alan's continued
contribution than the alternative. Or maybe not, but this is
why we want to surface the costs of decisions. It's better
than pretending that hard decisions don't need to be made,
and that the true costs of those decisions are just somebody
being unreasonable and thus not worth counting on the "cost"
side of the ledger.

If Alan isn't happy with Stefan's decision then even if we
think it was overall a good decision, this doesn't mean we
have to be unhappy with Alan. We can just ask ourselves if
the whole thing is worth it. Or rather, the rest of you can
ask it; I don't have an opinion on the specifics.

--dmr

My resignation from Emacs development

[Christopher Dimech]

> Sent: Wednesday, November 27, 2024 at 7:01 AM
> From: "Daniel Radetsky" <dradetsky@gmail.com>
> To: "Adam Porter" <adam@alphapapa.net>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> On Thu, Nov 21, 2024 at 11:35:35PM -0600, Adam Porter wrote:
> > But it is not okay for you to blame Stefan for your decision to leave.
>
> I disagree. If he chooses to leave, and the reason for this
> choice is Stefan's behavior or decisions, then blaming
> Stefan seems straightforwardly informative.

In the matter of blaming maintainers for decisions - whether directly or
indirectly - the question of whether maintainers should be allowed to
break their own rules is critical.  A compelling case exists that they
should.  Strict adherence to lengthy review periods or
consensus-building processes is often impractical, especially in
situations where only maintainers possess the necessary expertise to
advance the program.

Maintainers breaking their own rules represents a pragmatic approach,
prioritizing progress and functionality over rigid adherence to dogmatic
processes. This flexibility ensures that the project continues to evolve
and adapt to its challenges.

That said, while maintainers must retain the ability to make such
decisions - even if they sometimes result in dissent or the departure of
contributors - there is a clear responsibility to avoid fostering a
culture of arbitrary rule-breaking.  Transparency, accountability, and
judicious use of this authority are essential to maintain the integrity
of the program, especially in a collaborative environment heavily
reliant on contributor involvement.

> I'm not as much of a veteran of this list as some of you,
> and my few interactions with Stefan have been positive. So I
> can't really speak to who's in the right and don't think I
> should. But it's broadly better to have information about
> what's going on and what decisions are being made and how
> everyone feels about those decisions than to not have this
> information.
>
> Basically, if Stefan made a decisions, and this made Alan so
> unhappy that he wants to leave, this is something everyone
> should know. Sometimes this is the price of a decision. We
> need to know the price to make informed choices.
>
> I'm not accusing you of this specifically, but it seems like
> in situations like this there's a desire to make the
> situation black and white. Either Stefan made a bad decision
> which ought to be reversed, and the fact that it is not
> being reversed would justify Alan leaving, or Alan is being
> unreasonable and thus his decision to leave is a foregone
> conclusion being unfairly blamed on Stefan. Thus if we don't
> want to reverse Stefan's decision, we must believe that Alan
> is being unreasonable.
>
> But it's also possible that e.g. Stefan made a good decision
> in the big picture, but this was locally problematic for
> Alan. And even though we prefer Stefan's good decision, we
> prefer a worse decision with the benefit of Alan's continued
> contribution than the alternative. Or maybe not, but this is
> why we want to surface the costs of decisions. It's better
> than pretending that hard decisions don't need to be made,
> and that the true costs of those decisions are just somebody
> being unreasonable and thus not worth counting on the "cost"
> side of the ledger.
>
> If Alan isn't happy with Stefan's decision then even if we
> think it was overall a good decision, this doesn't mean we
> have to be unhappy with Alan. We can just ask ourselves if
> the whole thing is worth it. Or rather, the rest of you can
> ask it; I don't have an opinion on the specifics.
>
> --dmr
>
>

Re: My resignation from Emacs development

[Adam Porter]

Christopher,

On 11/26/24 13:51, Christopher Dimech wrote:
> In the matter of blaming maintainers for decisions - whether directly or
> indirectly - the question of whether maintainers should be allowed to
> break their own rules is critical.  A compelling case exists that they
> should.  Strict adherence to lengthy review periods or
> consensus-building processes is often impractical, especially in
> situations where only maintainers possess the necessary expertise to
> advance the program.
> 
> Maintainers breaking their own rules represents a pragmatic approach,
> prioritizing progress and functionality over rigid adherence to dogmatic
> processes. This flexibility ensures that the project continues to evolve
> and adapt to its challenges.

You seem to imply that some kind of rule-breaking has happened.  I don't 
think this is so--unless the rule were "No one may make any change 
unless everyone agrees to it."  The technical matters in question have 
been thoroughly discussed.  A change was made.  The maintainers support 
it (in absence of a better solution, which they have not found).  One 
contributor refuses to tolerate it--regrettable, but solely his decision 
to make.  There's little else--factually--to say.

> That said, while maintainers must retain the ability to make such 
> decisions - even if they sometimes result in dissent or the
> departure of contributors - there is a clear responsibility to avoid
> fostering a culture of arbitrary rule-breaking.  Transparency,
> accountability, and judicious use of this authority are essential to
> maintain the integrity of the program, especially in a collaborative
> environment heavily reliant on contributor involvement.
You seem to imply some kind of secrecy is involved.  Everything I see 
indicates the opposite: lengthy, public discussions, long-considered but 
finally needed decisions, and further lengthy, public discussions (with 
unfairly implied chastisement of the maintainers for implied secrecy). 
One could hardly find a more transparently run project.

You even mention integrity, as if to suggest that the maintainers' is in 
question.  Please be careful that your words don't imply criticism where 
none is deserved.

--Adam

Re: My resignation from Emacs development

[Daniel Radetsky]

On Tue, Nov 26, 2024 at 08:18:17PM -0600, Adam Porter wrote:
> You even mention integrity, as if to suggest that the maintainers' is in
> question.  Please be careful that your words don't imply criticism where
> none is deserved.

He referred to the integrity of "the program" not any
individual. I for one did not read this as implying that the
integrity of any person was in question.

I think you ought to consider what effect your own words are
having before being so ready to warn others about their
words. Your zeal to stamp out anything that might even
slightly imply criticism puts me in mind of "The wicked flee
where no man pursueth."

Don't get me wrong, my opinion is still that you just take
civility and reputation seriously, but a less-perceptive (or
less charitable) observer could mistake it for consciousness
of guilt.

Re: My resignation from Emacs development

[Christopher Dimech]

> Sent: Wednesday, November 27, 2024 at 2:18 PM
> From: "Adam Porter" <adam@alphapapa.net>
> To: "Christopher Dimech" <dimech@gmx.com>, "Daniel Radetsky" <dradetsky@gmail.com>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> Christopher,
>
> On 11/26/24 13:51, Christopher Dimech wrote:
> > In the matter of blaming maintainers for decisions - whether directly or
> > indirectly - the question of whether maintainers should be allowed to
> > break their own rules is critical.  A compelling case exists that they
> > should.  Strict adherence to lengthy review periods or
> > consensus-building processes is often impractical, especially in
> > situations where only maintainers possess the necessary expertise to
> > advance the program.
> >
> > Maintainers breaking their own rules represents a pragmatic approach,
> > prioritizing progress and functionality over rigid adherence to dogmatic
> > processes. This flexibility ensures that the project continues to evolve
> > and adapt to its challenges.
>
> You seem to imply that some kind of rule-breaking has happened.  I don't
> think this is so--unless the rule were "No one may make any change
> unless everyone agrees to it."  The technical matters in question have
> been thoroughly discussed.  A change was made.  The maintainers support
> it (in absence of a better solution, which they have not found).  One
> contributor refuses to tolerate it--regrettable, but solely his decision
> to make.  There's little else--factually--to say.

Even when rule-breaking occurs, this should not inherently be a problem.
Maintainers are justified to make decisions.

> > That said, while maintainers must retain the ability to make such
> > decisions - even if they sometimes result in dissent or the
> > departure of contributors - there is a clear responsibility to avoid
> > fostering a culture of arbitrary rule-breaking.  Transparency,
> > accountability, and judicious use of this authority are essential to
> > maintain the integrity of the program, especially in a collaborative
> > environment heavily reliant on contributor involvement.

> You seem to imply some kind of secrecy is involved.  Everything I see
> indicates the opposite: lengthy, public discussions, long-considered but
> finally needed decisions, and further lengthy, public discussions (with
> unfairly implied chastisement of the maintainers for implied secrecy).
> One could hardly find a more transparently run project.
>
> You even mention integrity, as if to suggest that the maintainers' is in
> question.  Please be careful that your words don't imply criticism where
> none is deserved.

No criticism.  Integrity referred to emacs as a computer program, not of
maintainers.  Contributors should acknowledge that the practical demands
of maintaining necessitate decisions being made without exhaustive
discourse like in this case.  Finally, it is users that have to adapt
accordingly with the tools provided.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You seem to imply that some kind of rule-breaking has happened.  I don't 
  > think this is so--unless the rule were "No one may make any change 
  > unless everyone agrees to it."  The technical matters in question have 
  > been thoroughly discussed.

I think the crucial question was _where_ this discussion took place.
It was in the bug reporting list, not emacs-devel.

I have complained before that the practice of discussing a feature
change on the bug list caused feature change ideas not to be
recognized as such.  People should help notify all of us
that we are discussing a feature change.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 29 Nov 2024 22:52:55 -0500
> 
>   > You seem to imply that some kind of rule-breaking has happened.  I don't 
>   > think this is so--unless the rule were "No one may make any change 
>   > unless everyone agrees to it."  The technical matters in question have 
>   > been thoroughly discussed.
> 
> I think the crucial question was _where_ this discussion took place.
> It was in the bug reporting list, not emacs-devel.
> 
> I have complained before that the practice of discussing a feature
> change on the bug list caused feature change ideas not to be
> recognized as such.  People should help notify all of us
> that we are discussing a feature change.

When such discussions happen on the bug list, we generally try to add
to the discussion people who might be relevant or whose opinions we
want to hear.  In other cases, someone insists to move the discussion
to emacs-devel.

Of course, mistakes can and do happen.  But the understanding that
some issues need broader discussions does exist, and there's no
intentional avoidance of that for some covert reasons.

Please also keep in mind that a person interested in some subject
could be off-line for some reason, not reading any of the Emacs lists,
when the discussion happens.  So even making sure all such discussions
happen on emacs-devel is not a guarantee that everyone who should be
involved will be.  (I do agree that emacs-devel is a better place for
discussing important changes, I'm just saying it is not a panacea.)

Discuss new features/enhancements or large changes for users in emacs-devel [was: My resignation from Emacs development]

[Drew Adams]

> > I think the crucial question was _where_ this discussion took place.
> > It was in the bug reporting list, not emacs-devel.
> >
> > I have complained before that the practice of discussing a feature
> > change on the bug list caused feature change ideas not to be
> > recognized as such.  People should help notify all of us
> > that we are discussing a feature change.
> 
> When such discussions happen on the bug list, we generally try to add
> to the discussion people who might be relevant or whose opinions we
> want to hear.  In other cases, someone insists to move the discussion
> to emacs-devel.
> 
> Of course, mistakes can and do happen.  But the understanding that
> some issues need broader discussions does exist, and there's no
> intentional avoidance of that for some covert reasons.
> 
> Please also keep in mind that a person interested in some subject
> could be off-line for some reason, not reading any of the Emacs lists,
> when the discussion happens.  So even making sure all such discussions
> happen on emacs-devel is not a guarantee that everyone who should be
> involved will be.  (I do agree that emacs-devel is a better place for
> discussing important changes, I'm just saying it is not a panacea.)

Good to hear.  IMO it does happen too often that
something that could benefit from discussion on -
or at least exposure to - emacs-devel stays on
the bug list.

I understand that it's not always obvious when or
if some part of a bug-list thread should be moved
to emacs-devel.  But I think everyone could make
a more concerted effort to do so.

Just one opinion.

Re: Discuss new features/enhancements or large changes for users in emacs-devel [was: My resignation from Emacs development]

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "adam@alphapapa.net" <adam@alphapapa.net>,
>         "emacs-devel@gnu.org"
> 	<emacs-devel@gnu.org>
> Date: Sat, 30 Nov 2024 16:22:23 +0000
> 
> > > I think the crucial question was _where_ this discussion took place.
> > > It was in the bug reporting list, not emacs-devel.
> > >
> > > I have complained before that the practice of discussing a feature
> > > change on the bug list caused feature change ideas not to be
> > > recognized as such.  People should help notify all of us
> > > that we are discussing a feature change.
> > 
> > When such discussions happen on the bug list, we generally try to add
> > to the discussion people who might be relevant or whose opinions we
> > want to hear.  In other cases, someone insists to move the discussion
> > to emacs-devel.
> > 
> > Of course, mistakes can and do happen.  But the understanding that
> > some issues need broader discussions does exist, and there's no
> > intentional avoidance of that for some covert reasons.
> > 
> > Please also keep in mind that a person interested in some subject
> > could be off-line for some reason, not reading any of the Emacs lists,
> > when the discussion happens.  So even making sure all such discussions
> > happen on emacs-devel is not a guarantee that everyone who should be
> > involved will be.  (I do agree that emacs-devel is a better place for
> > discussing important changes, I'm just saying it is not a panacea.)
> 
> Good to hear.  IMO it does happen too often that
> something that could benefit from discussion on -
> or at least exposure to - emacs-devel stays on
> the bug list.

How often is "too often"?  Do you have some quantitative data about
the percentage of such discussions that happen on the bug tracker?

RE: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was: My resignation from Emacs development]

[Drew Adams]

> > > > I think the crucial question was _where_ this discussion took
> place.
> > > > It was in the bug reporting list, not emacs-devel.
> > > >
> > > > I have complained before that the practice of discussing a feature
> > > > change on the bug list caused feature change ideas not to be
> > > > recognized as such.  People should help notify all of us
> > > > that we are discussing a feature change.
> > >
> > > When such discussions happen on the bug list, we generally try to add
> > > to the discussion people who might be relevant or whose opinions we
> > > want to hear.  In other cases, someone insists to move the discussion
> > > to emacs-devel.
> > >
> > > Of course, mistakes can and do happen.  But the understanding that
> > > some issues need broader discussions does exist, and there's no
> > > intentional avoidance of that for some covert reasons.
> > >
> > > Please also keep in mind that a person interested in some subject
> > > could be off-line for some reason, not reading any of the Emacs
> > > lists, when the discussion happens.  So even making sure all such
> > > discussions happen on emacs-devel is not a guarantee that everyone
> > > who should be involved will be.  (I do agree that emacs-devel is
> > > a better place for discussing important changes, I'm just saying
> > > it is not a panacea.)
> >
> > Good to hear.  IMO it does happen too often that
> > something that could benefit from discussion on -
> > or at least exposure to - emacs-devel stays on
> > the bug list.
> 
> How often is "too often"?  Do you have some quantitative data about
> the percentage of such discussions that happen on the bug tracker?

Too often in my opinion.  Too often for what I think
is most helpful and healthy for Emacs.

As I said:

> > Just one opinion.

Re: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was: My resignation from Emacs development]

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "rms@gnu.org" <rms@gnu.org>, "adam@alphapapa.net" <adam@alphapapa.net>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Sat, 30 Nov 2024 21:06:30 +0000
> 
> > > Good to hear.  IMO it does happen too often that
> > > something that could benefit from discussion on -
> > > or at least exposure to - emacs-devel stays on
> > > the bug list.
> > 
> > How often is "too often"?  Do you have some quantitative data about
> > the percentage of such discussions that happen on the bug tracker?
> 
> Too often in my opinion.  Too often for what I think
> is most helpful and healthy for Emacs.

That might be highly subjective, because its in human nature to
exaggerate the negative situations over positive ones.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I suggested

  > > I have complained before that the practice of discussing a feature
  > > change on the bug list caused feature change ideas not to be
  > > recognized as such.  People should help notify all of us
  > > that we are discussing a feature change.

You replied, pointing out some details of how things tend to work
currently:

  > When such discussions happen on the bug list, we generally try to add
  > to the discussion people who might be relevant or whose opinions we
  > want to hear.

Adding specific pertinent people is a helpful think to do, I agree.

                   In other cases, someone insists to move the discussion
  > to emacs-devel.

I'm simply suggesting that we make a point of thinking of this
in certain sitations.

  > Of course, mistakes can and do happen.  But the understanding that
  > some issues need broader discussions does exist, and there's no
  > intentional avoidance of that for some covert reasons.

None of us is perfect, and I don't think anyone had a ban intention
in handling that bug report thread.

By thinking consciously of the importance of bringing a feature
discussion to emacs-devel, we'll learn to remember hat more often.

  > Please also keep in mind that a person interested in some subject
  > could be off-line for some reason, not reading any of the Emacs lists,
  > when the discussion happens.  So even making sure all such discussions
  > happen on emacs-devel is not a guarantee that everyone who should be
  > involved will be.

I know.  It won't be guaranteed to happen.  But with all of us aware
that moving such discussions to emacs-devel, we will start noticing
oppprtunities to do that.

                       (I do agree that emacs-devel is a better place for
  > discussing important changes, I'm just saying it is not a panacea.)

I agree.  There are bo panaceas in this field.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: adam@alphapapa.net, emacs-devel@gnu.org
> Date: Tue, 03 Dec 2024 02:26:13 -0500
> 
> I suggested
> 
>   > > I have complained before that the practice of discussing a feature
>   > > change on the bug list caused feature change ideas not to be
>   > > recognized as such.  People should help notify all of us
>   > > that we are discussing a feature change.
> 
> You replied, pointing out some details of how things tend to work
> currently:
> 
>   > When such discussions happen on the bug list, we generally try to add
>   > to the discussion people who might be relevant or whose opinions we
>   > want to hear.
> 
> Adding specific pertinent people is a helpful think to do, I agree.
> 
>                    In other cases, someone insists to move the discussion
>   > to emacs-devel.
> 
> I'm simply suggesting that we make a point of thinking of this
> in certain sitations.

I definitely agree, as a general rule.

>   > Of course, mistakes can and do happen.  But the understanding that
>   > some issues need broader discussions does exist, and there's no
>   > intentional avoidance of that for some covert reasons.
> 
> None of us is perfect, and I don't think anyone had a ban intention
> in handling that bug report thread.
> 
> By thinking consciously of the importance of bringing a feature
> discussion to emacs-devel, we'll learn to remember hat more often.
> 
>   > Please also keep in mind that a person interested in some subject
>   > could be off-line for some reason, not reading any of the Emacs lists,
>   > when the discussion happens.  So even making sure all such discussions
>   > happen on emacs-devel is not a guarantee that everyone who should be
>   > involved will be.
> 
> I know.  It won't be guaranteed to happen.  But with all of us aware
> that moving such discussions to emacs-devel, we will start noticing
> oppprtunities to do that.
> 
>                        (I do agree that emacs-devel is a better place for
>   > discussing important changes, I'm just saying it is not a panacea.)
> 
> I agree.  There are bo panaceas in this field.

Full agreement.

Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Drew Adams]

> I think the crucial question was _where_ this discussion took
> place.  It was in the bug reporting list, not emacs-devel.
> 
> I have complained before that the practice of discussing a feature
> change on the bug list caused feature change ideas not to be
> recognized as such.  People should help notify all of us
> that we are discussing a feature change.

FWIW -

This is exactly the point I would have made,
if I had posted to this thread. ;-)

It doesn't _really_ matter (IMHO) who's
doing it - even if a discusser is a decider
(maintainer).  What matters is to get the
discussion viewed (and maybe participated
in) by a _wider audience_.
___

In bug thread 73853 I wrote this about it:

  I don't understand why language-design discussions,
  even major ones sometimes, are carried out in
  debbugs and not always in emacs-devel@gnu.org.

  I can understand that some real BUG discussion can
  diverge or expand to a design discussion, but even
  then I'd think that that design discussion should
  be gently moved to emacs-devel.

  Some such discussions could even benefit from a
  mention in other places, such as help-gnu-emacs,
  so more users could tune in to the discussion in
  emacs-devel if they're interested.

Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Sat, 30 Nov 2024 16:21:44 +0000
> 
> In bug thread 73853 I wrote this about it:
> 
>   I don't understand why language-design discussions,
>   even major ones sometimes, are carried out in
>   debbugs and not always in emacs-devel@gnu.org.

Bug#73853 was not about language-design decisions, it was about
deprecating a macro.  And you were part of that discussion since its
beginning.

RE: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Drew Adams]

> > In bug thread 73853 I wrote this about it:
> >
> >   I don't understand why language-design discussions,
> >   even major ones sometimes, are carried out in
> >   debbugs and not always in emacs-devel@gnu.org.
> 
> Bug#73853 was not about language-design decisions, it was about
> deprecating a macro.

The bug might not have been about language-design,
but some of the bug thread was - which is the point
here.

My post that I quoted (above) was a +1 support for
a post from Jonas Bernoulli which was largely about
the same point being discussed here, now.  E.g.:

  This could have been prevented if more people
  (including non-debbugs and non-emacs-devel regulars)
  were given a chance to think about the problem
  and time to articulate their concerns and proposals,
  before facts were created.  Or even if the people
  who did take part in past conversations had spend
  more time actually talking things through.

> And you were part of that discussion since its
> beginning.

Once again, it's not about me.  That I was (a small)
part of that bug discussion is irrelevant to RMS's
point, to which I replied here creating this new
thread.

Re: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Eli Zaretskii]

> From: Drew Adams <drew.adams@oracle.com>
> CC: "rms@gnu.org" <rms@gnu.org>, "adam@alphapapa.net" <adam@alphapapa.net>,
>         "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> Date: Sat, 30 Nov 2024 21:09:21 +0000
> 
> > > In bug thread 73853 I wrote this about it:
> > >
> > >   I don't understand why language-design discussions,
> > >   even major ones sometimes, are carried out in
> > >   debbugs and not always in emacs-devel@gnu.org.
> > 
> > Bug#73853 was not about language-design decisions, it was about
> > deprecating a macro.
> 
> The bug might not have been about language-design,
> but some of the bug thread was - which is the point
> here.

That people participating in a discussion bring up arguments based on
language design doesn't necessarily (and generally shouldn't) make the
discussion to be about language design.  Language design is a tangent
in a discussion about deprecating a macro.

> My post that I quoted (above) was a +1 support for
> a post from Jonas Bernoulli which was largely about
> the same point being discussed here, now.  E.g.:
> 
>   This could have been prevented if more people
>   (including non-debbugs and non-emacs-devel regulars)
>   were given a chance to think about the problem
>   and time to articulate their concerns and proposals,
>   before facts were created.  Or even if the people
>   who did take part in past conversations had spend
>   more time actually talking things through.

The quality and efficiency of a discussion don't necessarily improve
by asking more people to participate.  The main consideration for
having discussions on emacs-devel rather than elsewhere is because we
think we have the moral obligation to make it known to more people,
even if this makes the discussion much less effective, as it many
times does.  Thus, claiming that opening a discussion will necessarily
make the decisions better is basically missing the point.

Also, you were part of the discussion from its very beginning, but
never objected to the changes, until Jonas chimed in, much later.

> > And you were part of that discussion since its
> > beginning.
> 
> Once again, it's not about me.

You are one example of people who keep criticizing our decision-making
process, so looking into your participation in that case is very
relevant.

RE: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Drew Adams]

> > > > In bug thread 73853 I wrote this about it:
> > > >
> > > >   I don't understand why language-design discussions,
> > > >   even major ones sometimes, are carried out in
> > > >   debbugs and not always in emacs-devel@gnu.org.
> > >
> > > Bug#73853 was not about language-design decisions, it was about
> > > deprecating a macro.
> >
> > The bug might not have been about language-design,
> > but some of the bug thread was - which is the point
> > here.
> 
> That people participating in a discussion bring up arguments based on
> language design doesn't necessarily (and generally shouldn't) make the
> discussion to be about language design.  Language design is a tangent
> in a discussion about deprecating a macro.

Yes.

The bug was posted to deprecate a macro.  But the
discussion veered quickly into language design and
its process: whether and how mistakes were made in
this case, why, & how to avoid this in the future. 

And multiple macros were discussed when trying to
realize what went wrong and figure out the best
way to fix things.  It was no longer only about
whether to deprecate a particular macro.

The discussion became even more general: should
ad hoc design discussions be moved from the bug
list to emacs-devel more often - and would that
have helped avoid some of the problems discussed
wrt design of the *-let macros.

> > > And you were part of that discussion since its
> > > beginning.
> >
> > Once again, it's not about me.
> 
> You are one example of people who keep criticizing our decision-making
> process, so looking into your participation in that case is very
> relevant.

Can't wait till Santa Eli posts his complete lists
of the Naughty & Nice. ;-)

I _support_ the decision-making process in general.
And I most often _agree_ with the decisions made,
including those made by you.  Sometimes I say so
explicitly; more often I don't chime in just to
say that I agree.

In particular, I'll say again that I often agree
with a decision _not_ to make some change that's
been proposed.  Wrt continuity, deprecation,
backward compatibility, etc., I'm conservative,
and I appreciate your approach in such cases.

There are zillions of users who don't frequent the
Emacs "social media" spaces, and Emacs development
needs to - and generally does - keep them in mind,
even if some enthusiastic change-promoters speak
loudly and often.

And I often _don't_ chime in when I disagree.
You may think I "keep criticizing", but I submit
that that's long been ingrained in your mind and
your opinion of me.  That, despite the many times
I've explicitly posted saying I'm grateful that
you, in particular, maintain Emacs, and that I
think you do a very good job in general.

You also, however, can be very defensive and take
things personally - that's a weakness that works
against you, not for you.  It seems there's no
setting your mind at ease.  You've made your
Enemies List, and you can't help adding notches
to it each time you feel attacked.  No one's
perfect.

Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Bug#73853 was not about language-design decisions, it was about
  > deprecating a macro.

I can see two possible meanings for what "Bug#73853" refers
to in that sentenbce:

1. The original bug report which started that thread in bug-gnu-emacs.

2. The discussion thread which started with that bug report.

Could it be that (1) was about deprecating that macro, 
and (2) started off there by wandered into a discussion of
choosing major modes for various programming languages?

I don't know for a fact how that discussion went, but that history
would fit with what has been said about it since.  So I'll suppose
tentatively that was what happened.

While the discussion was about deprecating a macro, discussing it
in a bu report ticket made sense.

Once it changed to be about proposals for controlling major modes,
someone should have moved it to emacs-devel.

Idesally, whoever was ispired by that bug discussion to raise an idea
for controlling major modes should have sent that message to
emacs-devel and started a new threat.  But everyone forgets sometimes.
After one person hadn't remembered to do this, it wasn't too late.
Someone else should have moved and renamed the thread.





-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Tue, 03 Dec 2024 02:25:43 -0500
> 
>   > Bug#73853 was not about language-design decisions, it was about
>   > deprecating a macro.
> 
> I can see two possible meanings for what "Bug#73853" refers
> to in that sentenbce:
> 
> 1. The original bug report which started that thread in bug-gnu-emacs.
> 
> 2. The discussion thread which started with that bug report.

It is both, actually.

> Could it be that (1) was about deprecating that macro, 
> and (2) started off there by wandered into a discussion of
> choosing major modes for various programming languages?

No.  This is a misunderstanding: bug#73853 has nothing to do with
choosing major modes.  That bug was about and-let* and when-let*, and
was brought up here as an example of a discussion that should have
been taken to emacs-devel, because it (allegedly) was about the
language design of Emacs Lisp.

> While the discussion was about deprecating a macro, discussing it
> in a bu report ticket made sense.
> 
> Once it changed to be about proposals for controlling major modes,
> someone should have moved it to emacs-devel.

It never did.  Instead, someone in that discussion mentioned in
passing some aspects of language design related to the discussion.
That was a tangent that fortunately didn't deflect the discussion.

I think you confuse this with another bug, bug#69191.  That one was
about remapping major modes from the get-go.

Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It never did.  Instead, someone in that discussion mentioned in
  > passing some aspects of language design related to the discussion.
  > That was a tangent that fortunately didn't deflect the discussion.

  > I think you confuse this with another bug, bug#69191.  That one was
  > about remapping major modes from the get-go.

I guess you could say that, thought it might give the wrong idea.  I
don't normally read bug ticket discussions -- only when I know of a
special reason, and that didn't happen for those.  I started reading
about the major mode issue when I saw complaints in emacs-devel.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It doesn't _really_ matter (IMHO) who's
  > doing it - even if a discusser is a decider
  > (maintainer).  What matters is to get the
  > discussion viewed (and maybe participated
  > in) by a _wider audience_.

I agree completely.

When the topic of a thread wanders from A to B, there's no reason to
blame anyone.  It is natural that sometimes discussions wander, and
sometimes this wandering can convert a discussion of fixing a bug into
a decision about features.

When that happens, we should recognize that the topic has changed,
and handle the new topic in the way that it calls for.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Discuss new features/enhancements or large changes for users in emacs-devel

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: adam@alphapapa.net, emacs-devel@gnu.org
> Date: Sun, 01 Dec 2024 23:09:19 -0500
> 
>   > It doesn't _really_ matter (IMHO) who's
>   > doing it - even if a discusser is a decider
>   > (maintainer).  What matters is to get the
>   > discussion viewed (and maybe participated
>   > in) by a _wider audience_.
> 
> I agree completely.
> 
> When the topic of a thread wanders from A to B, there's no reason to
> blame anyone.  It is natural that sometimes discussions wander, and
> sometimes this wandering can convert a discussion of fixing a bug into
> a decision about features.
> 
> When that happens, we should recognize that the topic has changed,
> and handle the new topic in the way that it calls for.

I don't think anyone will disagree, at least not in general.

That said, I would like to point out a few aspects that AFAIU are at
the real core of the issues which prompted this:

  . a discussion could wander into a tangent, in which case TRT is to
    ask people to make the tangent a separate discussion, instead of
    moving it to emacs-devel
  . we encourage people to submit "feature-request" bug reports (and
    Emacs recently acquired the "M-x submit-emacs-patch" command for
    that reason), in which case the bug list _is_ the proper place to
    discuss that.  When the feature is significant and/or affects
    Emacs or our users in prominent ways, prudence would mandate that
    we move such general discussions to emacs-devel, but that's a
    judgment call, not an automatic knee-jerk reaction

RE: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel

[Drew Adams]

> >   > It doesn't _really_ matter (IMHO) who's
> >   > doing it - even if a discusser is a decider
> >   > (maintainer).  What matters is to get the
> >   > discussion viewed (and maybe participated
> >   > in) by a _wider audience_.
> >
> > I agree completely.
> >
> > When the topic of a thread wanders from A to B, there's no reason to
> > blame anyone.  It is natural that sometimes discussions wander, and
> > sometimes this wandering can convert a discussion of fixing a bug into
> > a decision about features.
> >
> > When that happens, we should recognize that the topic has changed,
> > and handle the new topic in the way that it calls for.
> 
> I don't think anyone will disagree, at least not in general.
> 
> That said, I would like to point out a few aspects that AFAIU are at
> the real core of the issues which prompted this:
> 
>   . a discussion could wander into a tangent, in which case TRT is to
>     ask people to make the tangent a separate discussion, instead of
>     moving it to emacs-devel
>   . we encourage people to submit "feature-request" bug reports (and
>     Emacs recently acquired the "M-x submit-emacs-patch" command for
>     that reason), in which case the bug list _is_ the proper place to
>     discuss that.  When the feature is significant and/or affects
>     Emacs or our users in prominent ways, prudence would mandate that
>     we move such general discussions to emacs-devel, but that's a
>     judgment call, not an automatic knee-jerk reaction

FWIW, I agree.

But if the tangent (first bullet) is not another
bug and isn't an enhancement, and the list is the
bug list, then a different list is likely better
(emacs-tangent, gnu-emacs-help, emacs-devel).

Re: Discuss new features/enhancements or large changes for users in emacs-devel

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That said, I would like to point out a few aspects that AFAIU are at
  > the real core of the issues which prompted this:

  >   . a discussion could wander into a tangent, in which case TRT is to
  >     ask people to make the tangent a separate discussion, instead of
  >     moving it to emacs-devel

If the new topic is a tangent to the bug discussion, where that should go
depends on what it is about.

It might be of little importance to Emacs, in which case maybe it
ought to move to emacs-tangents.  Or it be a suggestion for some
bigger change in Emacs.   That should go to emacs-devel, I think.

  >   . we encourage people to submit "feature-request" bug reports (and
  >     Emacs recently acquired the "M-x submit-emacs-patch" command for
  >     that reason), in which case the bug list _is_ the proper place to
  >     discuss that.  When the feature is significant and/or affects
  >     Emacs or our users in prominent ways, prudence would mandate that
  >     we move such general discussions to emacs-devel, but that's a
  >     judgment call, not an automatic knee-jerk reaction

Jdgment calls are like this.  There is no precise rule that gives a
perect result every time.  That is too much to ask for.

What we can hope for is to recognize when a local change idea
has wandered into an idea for a change that would affect a lot
more situations.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Discuss new features/enhancements or large changes for users in emacs-devel

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: drew.adams@oracle.com, adam@alphapapa.net, emacs-devel@gnu.org
> Date: Thu, 05 Dec 2024 00:08:07 -0500
> 
>   >   . a discussion could wander into a tangent, in which case TRT is to
>   >     ask people to make the tangent a separate discussion, instead of
>   >     moving it to emacs-devel
> 
> If the new topic is a tangent to the bug discussion, where that should go
> depends on what it is about.
> 
> It might be of little importance to Emacs, in which case maybe it
> ought to move to emacs-tangents.  Or it be a suggestion for some
> bigger change in Emacs.   That should go to emacs-devel, I think.

When we tell people to make the tangent a separate discussion, it goes
without saying that the separate discussion should have its best place
determined as part of starting it.

>   >   . we encourage people to submit "feature-request" bug reports (and
>   >     Emacs recently acquired the "M-x submit-emacs-patch" command for
>   >     that reason), in which case the bug list _is_ the proper place to
>   >     discuss that.  When the feature is significant and/or affects
>   >     Emacs or our users in prominent ways, prudence would mandate that
>   >     we move such general discussions to emacs-devel, but that's a
>   >     judgment call, not an automatic knee-jerk reaction
> 
> Jdgment calls are like this.  There is no precise rule that gives a
> perect result every time.  That is too much to ask for.
> 
> What we can hope for is to recognize when a local change idea
> has wandered into an idea for a change that would affect a lot
> more situations.

Agreed.

RE: [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development]

[Drew Adams]

>   > It doesn't _really_ matter (IMHO) who's
>   > doing it - even if a discusser is a decider
>   > (maintainer).  What matters is to get the
>   > discussion viewed (and maybe participated
>   > in) by a _wider audience_.
> 
> I agree completely.
> 
> When the topic of a thread wanders from A to B, there's no reason to
> blame anyone.  It is natural that sometimes discussions wander, and
> sometimes this wandering can convert a discussion of fixing a bug into
> a decision about features.
> 
> When that happens, we should recognize that the topic has changed,
> and handle the new topic in the way that it calls for.

Yes, I meant what you just said, and I probably
should have made that explicit.

We all do it occasionally: keep the same thread,
in the same group going, even when the discussion
digresses.  There's no reason for anyone to feel
particularly responsible (singled out) for this.
Anyone can start a new thread or suggest one.

Re: My resignation from Emacs development

[Adam Porter]

On 11/26/24 13:01, Daniel Radetsky wrote:
> On Thu, Nov 21, 2024 at 11:35:35PM -0600, Adam Porter wrote:
>> But it is not okay for you to blame Stefan for your decision to leave.
> 
> I disagree. If he chooses to leave, and the reason for this
> choice is Stefan's behavior or decisions, then blaming
> Stefan seems straightforwardly informative.

1.  As has been clearly stated, Stefan is not in charge of this 
project--the maintainers are.  Any change made by anyone, including 
Stefan, only persists with their approval.

So to blame Stefan is to imply a responsibility he does not bear, which 
is unfair and wrong.

2.  As has been clearly stated, the technical matters in question have 
been discussed extensively by the maintainers, and the status quo 
appears to be somewhat of a compromise, an approximation of the best 
that could be done given the circumstances and the timeframe.  It 
doesn't seem that anyone is truly content with the current 
implementation, yet no agreement on how to improve it has been reached.

So to blame Stefan for an implementation that even he is not fully 
satisfied with, yet no one has been able improve upon with consensus, is 
unfair and wrong.

3.  As has been admitted by Alan himself, he made a relevant change 
without discussing it first, and one that apparently forced the hand of 
the maintainers to deal with.  That would seem to imply his own having 
committed the same kind of misdeed which he accuses Stefan of committing.

So to blame Stefan for an action which Alan himself has taken, in the 
same context, even, is unfair and wrong.

> I'm not as much of a veteran of this list as some of you,
> and my few interactions with Stefan have been positive. So I
> can't really speak to who's in the right and don't think I
> should. But it's broadly better to have information about
> what's going on and what decisions are being made and how
> everyone feels about those decisions than to not have this
> information.

You seem to imply that this information has only now been revealed.  As 
has been clearly stated, these discussions about the technical matters 
are not recent, and they were primarily conducted in public.  You can 
review them and come to your own conclusions.  The only new information 
here is Alan's recent decision to stop contributing, which is not a 
technical matter.

> Basically, if Stefan made a decisions, and this made Alan so
> unhappy that he wants to leave, this is something everyone
> should know. Sometimes this is the price of a decision. We
> need to know the price to make informed choices.

Alan's feelings about and reaction to these technical issues are Alan's 
concern.  If he chooses to make them public, that's his decision, but it 
doesn't necessarily make them our concern.  Disagreements about how to 
manage a project like this are common, and they needn't always be made 
public--especially, they should not be in the form of public character 
assassination and ritual defamation.

> I'm not accusing you of this specifically, but it seems like
> in situations like this there's a desire to make the
> situation black and white. Either Stefan made a bad decision
> which ought to be reversed, and the fact that it is not
> being reversed would justify Alan leaving, or Alan is being
> unreasonable and thus his decision to leave is a foregone
> conclusion being unfairly blamed on Stefan. Thus if we don't
> want to reverse Stefan's decision, we must believe that Alan
> is being unreasonable.

You imply circumstances which are not so.  See earlier paragraphs.

> But it's also possible that e.g. Stefan made a good decision
> in the big picture, but this was locally problematic for
> Alan. And even though we prefer Stefan's good decision, we
> prefer a worse decision with the benefit of Alan's continued
> contribution than the alternative. Or maybe not, but this is
> why we want to surface the costs of decisions. It's better
> than pretending that hard decisions don't need to be made,
> and that the true costs of those decisions are just somebody
> being unreasonable and thus not worth counting on the "cost"
> side of the ledger.

I don't think that any project ought to govern itself by acceding to "my 
way or the highway" demands--what could be more unhealthy.

> If Alan isn't happy with Stefan's decision then even if we
> think it was overall a good decision, this doesn't mean we
> have to be unhappy with Alan. We can just ask ourselves if
> the whole thing is worth it. Or rather, the rest of you can
> ask it; I don't have an opinion on the specifics.
Respectfully, it seems like you have spoken without fully informing 
yourself of the context.  This whole situation is very simple (not being 
a technical problem, though originating in one): 1. Stefan made a 
change, 2. Alan doesn't like it, 3. no consensus has been reached on how 
to improve it, 4. the maintainers don't think that reverting the change 
would be an improvement, and 5. Alan has decided to cease contributing.

All of that is fine, though Alan's decision is regrettable.  What isn't 
fine is to misplace blame on Stefan, for a decision that the maintainers 
themselves support, and one that no one is fully satisfied with.

All parties would be well advised to set aside ego, territorialism, and 
grudges, and seek the best of the project, which is our common interest, 
the reason we are here.  Technical problems can be solved, and social 
ones can be forgiven--if the parties are willing.

But each one must decide for himself.  And once a participant has made 
his decision, for the good of the project and its participants, he ought 
to stop publicly litigating it.

And any outside participants who feel a duty to offer their input ought 
to do so with the utmost care, and only after fully informing themselves 
of the context and all parties' views, lest they only throw fuel on the 
fire.

--Adam

Re: My resignation from Emacs development

[Daniel Radetsky]

On Tue, Nov 26, 2024 at 08:06:53PM -0600, Adam Porter wrote:
> 1.  As has been clearly stated, Stefan is not in charge of this project--the
> maintainers are.  Any change made by anyone, including Stefan, only persists
> with their approval.
> 
> So to blame Stefan is to imply a responsibility he does not bear, which is
> unfair and wrong.

I'm not sure that's necessarily true; if Stefan is making a
positive decision and the other maintainers are merely
acquiescing, then blaming Stefan on the grounds that he's
the driving force behind the decision isn't totally
unreasonable.

But it's aside the point: Whether or not its right to blame
Stefan, it seems that Alan _does_ blame Stefan, and this is
useful information for anyone who seeks to reconcile the
parties or improve future decisionmaking. When you tell Alan
he's unfair and wrong to blame Stefan, you encourage him and
others in the future to just shut up and not air their
greivances. This might reduce on-list drama, but I don't
think it will convince the Alans of the world to change his
perspective or not leave the project. At best it will cause
them to leave more quietly, which I don't think is what
anyone wants. Or more likely, as in this case, it won't even
do that.

> 3.  As has been admitted by Alan himself, he made a relevant change without
> discussing it first, and one that apparently forced the hand of the
> maintainers to deal with.  That would seem to imply his own having committed
> the same kind of misdeed which he accuses Stefan of committing.

I don't think you get what's going on here. This isn't a
debate you can win. I mean, you can win it, but you don't
get anything for winning. So this kind of Tu Quoquery isn't
of any use even if it was apposite. And as it happens, I
don't think it is; it's not just a question of whether a
similar act was committed, but whether it was committed in
circumstances in which it produced a similar amount of harm.
Whether or not Alan's previous action was also a violation
of the hypothetical rule, it demonstrably did not make
anyone angry enough to resign the project.

Maybe such an accusation might shame the original speaker
into dropping his objection on the grounds that none are
without sin or something, but it doesn't do anything to
redress his injury. And as it doesn't seem to be shaming
Alan into silence, I'd let this point go.

> You seem to imply that this information has only now been revealed.

No, only that I only now became aware of it.

> Alan's feelings about and reaction to these technical issues are Alan's
> concern.

They're also our concern if we want him to continue with the
project. Your line of reasoning seems to be bending in the
direction of "if he can't control his feelings about these
issues, we don't need him on the project." If that is your
position, you should be explicit. But I think it's a silly
position.

> Disagreements about how to
> manage a project like this are common, and they needn't always be made
> public--especially, they should not be in the form of public character
> assassination and ritual defamation.

People should ideally not get angry enough to say mean
things about other people in the course of a project, but
sometimes they do.

I don't know about you, but I didn't take anything Alan said
about Stefan particularly seriously. As in, as far as the
concrete accusations, I barely took them in. The
overwhelming issue for me was: some member of the project is
extremely unhappy and wants to leave. Can this be salvaged?
So I don't see any need to reprimand Alan for "character
assassination" insofar as he didn't even come close to
successfully assassinating Stefan's character for me. Maybe
I'm weird for being able to reserve judgment on Stefan for
this, I don't know.

> I don't think that any project ought to govern itself by acceding to "my way
> or the highway" demands--what could be more unhealthy.

I don't know; what is your way?

The point is that it's not a question of bowing or not
bowing to arguably-unreasonable demands. That's just not the
right way to think about this. Instead: everyone involved
has the right to negotiate for their position, and it's the
job of project leadership to decide if they're getting a
good deal or not. If Alan decides that this particular point
is a my-way-or-the-highway situation for him, that's fine,
he's entitled to feel that way. We can then think it over
and then say very politely "Ultimately we decided to go with
'highway'. Thank you very much for all your help in the
past." What we've done here is not to "stand up to his
unreasonable demands" or "refuse to let him walk over us" or
any such silly framing of the issue. Instead we decided the
best way to get what we want, which was to reject his deal.
Make sense?

> All of that is fine, though Alan's decision is regrettable.  What isn't fine
> is to misplace blame on Stefan, for a decision that the maintainers
> themselves support, and one that no one is fully satisfied with.

Again, I don't see why this is all that important...

> social [problems] can be forgiven--if the parties are
> willing.

Except insofar as, in my opinion, every time you say "You
should not misplace blame on Stefan" it gets slightly harder
to hear the part where you say "If you decide to come back,
all is forgiven." Right now, I wouldn't bother saying the
first thing _at all_.

> But each one must decide for himself.  And once a participant has made his
> decision, for the good of the project and its participants, he ought to stop
> publicly litigating it.

To be fair, he's publicly litigating it because, in addition
to trying to talk him down, people are also challenging him
on it. I could just as well say that once he's made his
decision, for the good of the project people should stop
inviting him to litigate it. But I know that's unreasonable.
Inevitably, people are going to talk about this because
that's what people do. Those big brains aren't just for
show.

> And any outside participants who feel a duty to offer their input ought to
> do so with the utmost care, and only after fully informing themselves of the
> context and all parties' views, lest they only throw fuel on the fire.

Respectfully, I feel like your comments pose a greater risk
of fueling the fire than mine do. It doesn't seem to me like
you're keeping your eye on the not-fueling-the-fire bit with
sufficient assiduity. You have (understandable) opinions
about proper conduct and respect for reputations, and you're
putting them forward at a time when you might be better
served by holding them back for the time being. And I do
mean _you_ might be better served, in the sense that you
personally might thereby acquire a slightly better emacs.

--dmr

Re: My resignation from Emacs development

[Stefan Kangas]

Hi Alan,

Alan Mackenzie <acm@muc.de> writes:

> For starters: The change in the meaning of `c-mode' and `c++-mode' he
> introduced in bug#69191, discussed at length in my last post.  Stefan is
> not stupid.  He knew full well what he was doing in bypassing open
> discussion about major-mode-remap-defaults.

The patch was posted to Bug#69191 on 2024-02-16, received its first
reply on 2024-02-21 ("SGTM"), and was committed on 2024-03-03, more than
two weeks later.  That all seems bog-standard to me.

Meanwhile, the patch that you installed 1fdf0f68ccf0 was _not_ posted
anywhere before it was installed, but it turned out to be controversial.
Should you be condemned for that?  Eli didn't seem to think so when he
filed Bug#74339, and I agree.  Some questions were asked, though.

I also don't understand why anyone would have reason to suspect that you
(or anyone else, really) would find the change in Bug#69191 so
objectionable.  I would never have guessed that myself, and the
controversy therefore caught many of us by surprise.  I still find some
of the objections bewildering but would be happy to continue working to
resolve any outstanding concerns.

> Number 2: In late January 2024, Stefan decided to replace the customary
> list form of interpreted functions with opaque atoms, because the list
> form "annoyed" him.  In the ensuing discussion, Richard described the
> proposal as "perverse", and both Eli and I were asking questions as to
> the purpose of the change.  Only evasive non-answers came back.  There
> was certainly no consensus around the proposal.  Nevertheless, Stefan
> quietly committed his patch on 2024-03-11 in commit
> f2bccae22bd47a2e7e0937b78ea06131711b935a.  Emacs is slightly less
> powerful as a result, in that macros can no longer operate on the code
> of a function in a reasonable fashion.

I disagree, actually.  I recall a lengthy discussion with several
participants that clarified why the change was indeed beneficial.

I spent quite some time thinking about the issue myself, and also came
to the conclusion that it was a good change, though I didn't have much
technical content to add at that point which hadn't already been
explained.

Perhaps no one pointed out that using opaque function objects as we now
do is already standard practice in both Common Lisp and Scheme; both are
clearly examples of very powerful and capable Lisp dialects.  I don't
see that this change has made Emacs any less powerful.

> Number 3: Stefan introduced pcase.el without any open discussion, and
> proliferated it rapidly around the Emacs core, leading to confusion
> around the use of ` and ,, certainly on my part.  Now it can be argued
> that pcase has been a success, but it could have been so much better if
> it had been developed cooperatively.  For years there was no adequate
> doc string for `pcase', and even now the doc strings for things like
> pcase-let* are woefully inadequate.  Stefan is not good at documenting;
> nobody can be good at everything.

I submit that the reason why pcase has proliferated both in core and
externally is that it is fundamentally useful.  This proliferation is
not due to a master plan concocted by Stefan Monnier (who, for the
record, was an Emacs maintainer when pcase.el was installed).  I'm sure
that he did take the chance to use it to improve some existing code
though, work that we still benefit from and can be grateful for.

In practice, however, the decision to use pcase is typically made by the
person writing some piece of code, and we cannot police the style of
individual contributions on that level.  That would make no one happy,
and it would largely be detrimental to the project.  So if people like
pcase (or when-let, etc.), they will use it.

What should we blame Stefan Monnier for?  For implementing a novel
feature that has proven both highly useful and popular?

> Number 4: Some years ago, Stefan removed the documentation of defadvice
> from the elisp manual without any discussion.  Despite widespread
> protest, he refused to put it back again.  Quite coincidentally, he had
> just written and pushed nadvice.el.

We routinely remove references to obsolete or deprecated functions and
macros in the manual.  There is nothing strange about that in and of
itself.  In this case, I also find nadvice.el a significant improvement
over advice.el; therefore, I think removing it was likely the right
decision.  (That said, I wasn't around at the time, so I didn't read the
discussions.)

Moreover, when nadvice.el was installed, and these changes were made in
the manual, Stefan Monnier was an Emacs maintainer.  Making such
decisions, even amid controversy, is quite literally a part of the job
description.  Our current maintainers, I'm sure, have also taken
decisions that some people have disagreed with.  C'est la vie.

We can certainly discuss changing our current development model and
decision-making process.  But for now, it is what we have.

> Number 5: Previously, there had been an embargo on the use of the CL
> library, except at compile time.  This kept the size of the Emacs Lisp
> language manageable, and the language easy to understand, and made
> maintainers' and beginners' lives easier.

You have raised this point in the past, and my impression is that few
people agree with this sweeping generalization.  I do not agree that our
current use of cl-lib constitutes a problem.

It is true that we could improve cl-lib.  It would be even better if we
added some of the more useful functions and macros to Emacs Lisp itself,
so that we would have less need for a separate library.

> At some stage this embargo was lifted, and the use of CL rapidly
> proliferated through the Emacs core.  Now, it could be argued that the
> facilities and expressiveness of the CL lib outweigh these
> disadvantages.  But it was not so argued.  It just happened.  Maybe
> somebody else but Stefan made this change, but it seems unlikely.

Similar to pcase, the reason cl-lib has proliferated is that many people
find it useful.  Perhaps you may need to accept that you are in the
minority on this issue.

> I have not come anywhere near ad hominem.  It is true that many forums
> degenerate into slanging matches which repel decent posters.
> emacs-devel is the opposite extreme, sort of touchy-feely where nobody's
> allowed to offend anybody else at all, no matter what they do, why and
> how they do it.  This is just as unhealthy as the the continual abuse
> forums; it leads to the build up of repressed resentment.
>
> Sometimes the truth must be told bluntly, and that is what I have tried
> to do here.

Phrases such as "Jekyll-and-Hyde character", "contemptuous of the Emacs
conventions", "does not have the gift of knowing what the Right Thing
is", along with terms like "monster" and asking us "what does that say
about [him]", etc., are clearly ad hominem in my view.

These statements focus on personal characteristics rather than on
actions or statements made by an individual.

>> I have to agree with Eli.  Although it would, in hindsight, certainly
>> have been better to discuss these particular changes in more detail in
>> advance, I don't see that he has done anything very unusual or different
>> from what most other core contributors do on a routine basis.
>
> This "be nice to everybody no matter what they do" and "always assume
> the best of everybody" creates the perfect atmosphere for a monster to
> flourish in.  Stefan is such a monster; not all the time, not even most
> of the time, but in doing the things detailed above, and other things, I
> don't understand why you are defending him.

OK, let's be frank then.

First, I understand that there are things that you are unhappy about,
and that is valid and to be taken seriously.

However, I find your attempt to portray Stefan Monnier's character
negatively completely beyond the pale.  Using words like a "monster" to
refer to another core contributor is just appalling, and completely
uncalled for.

Thus, I am standing up for basic decency in the face of what is more and
more starting to look like an attempt at public character assassination,
using harsh language (e.g., "monster") and enumerating alleged
"mistakes" from as far back as 15 years.  I urge you to retract these
comments, and apologize.

I have already expressed my opinions on the technical issues that you
listed above.  However, from a strictly procedural standpoint, I do not
see that Stefan Monnier has exhibited a pattern of behaviour that stands
out as culpable or even unusual within our development model.

Note that I am not claiming he has never made mistakes, or that
everything he has done is perfect.  I am quite sure that he has done
many mistakes, both technical and procedural.  But so has every single
person on this list.  That does not justify singling him out, nor does
it justify the attacks directed at him here.

Please understand that the changes that Stefan Monnier has authored and
that you find objectionable, in the minds of many other hackers are
improvements.  In my view, for example, he has done tremendous work to
improve Emacs, and not least to make Emacs Lisp more powerful.  The
examples you give are some of the things that I would point to if I
wanted to demonstrate that.

---

Ultimately, all of us are volunteers who care deeply about our work.
This means that feelings can sometimes run high, and that is
understandable, because we are all human beings with different ideas,
interests, and motives.

Our team of core contributors are, despite our usually polite demeanor,
a relatively rowdy group of strong-willed, and often opinionated
individuals.  This description applies to all of us; we wouldn't be
running Emacs, let alone hack on it, if it weren't true.  We are a small
church, and it is in our best interest to collaborate.

Precisely for this reason, especially when we disagree, we must make
every effort to treat each other courteously and with respect.  This
does not mean sweeping issues under the rug, or maintaining some
superficial peace merely for appearances.  But it does mean that we have
a strong preference for discussing the specific issues at hand, and that
we ask everyone to meet a minimum standard, which we refer to as "kind
communication", when participating in Emacs development.  This is not
likely to change.

I sincerely hope that you reconsider your resignation.  It is sad to see
a capable member of our core team leave.  Regardless of what happens, I
look forward to continue collaborating with you in your capacity as CC
mode maintainer.

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Stefan.

Thanks for the detailed reply.

On Fri, Nov 22, 2024 at 10:36:23 -0500, Stefan Kangas wrote:
> Hi Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > For starters: The change in the meaning of `c-mode' and `c++-mode' he
> > introduced in bug#69191, discussed at length in my last post.  Stefan is
> > not stupid.  He knew full well what he was doing in bypassing open
> > discussion about major-mode-remap-defaults.

> The patch was posted to Bug#69191 on 2024-02-16, received its first
> reply on 2024-02-21 ("SGTM"), and was committed on 2024-03-03, more than
> two weeks later.  That all seems bog-standard to me.

For a big change that changes the way people work with Emacs?

> Meanwhile, the patch that you installed 1fdf0f68ccf0 was _not_ posted
> anywhere before it was installed, but it turned out to be controversial.
> Should you be condemned for that?

Yes, actually, I think I should.  It was a bad idea, that idea being to
attract attention to the patch for bug#69191, and it failed miserably.
It took well over five months before Eli noticed.  I really ought to
have raised the issue in emacs-devel.

> Eli didn't seem to think so when he filed Bug#74339, and I agree.
> Some questions were asked, though.

> I also don't understand why anyone would have reason to suspect that you
> (or anyone else, really) would find the change in Bug#69191 so
> objectionable.  I would never have guessed that myself, and the
> controversy therefore caught many of us by surprise.  I still find some
> of the objections bewildering but would be happy to continue working to
> resolve any outstanding concerns.

It enabled and encouraged software libraries to make uncontrolled
changes to other libraries' interfaces, in particular to hi-jack them.
This is just a Bad Thing.  In particular, it changed the symbols
`c-mode' and `c++-mode' from meaning, unambiguously, C Mode and C++
Mode, to meaning something else.  How can that not be controversial?

> > Number 2: In late January 2024, Stefan decided to replace the customary
> > list form of interpreted functions with opaque atoms, because the list
> > form "annoyed" him.  In the ensuing discussion, Richard described the
> > proposal as "perverse", and both Eli and I were asking questions as to
> > the purpose of the change.  Only evasive non-answers came back.  There
> > was certainly no consensus around the proposal.  Nevertheless, Stefan
> > quietly committed his patch on 2024-03-11 in commit
> > f2bccae22bd47a2e7e0937b78ea06131711b935a.  Emacs is slightly less
> > powerful as a result, in that macros can no longer operate on the code
> > of a function in a reasonable fashion.

> I disagree, actually.  I recall a lengthy discussion with several
> participants that clarified why the change was indeed beneficial.

I looked at the thread closely last night.  There was no clarification
at all.  The matter was left hanging in the air with unanswered
questions.

> I spent quite some time thinking about the issue myself, and also came
> to the conclusion that it was a good change, though I didn't have much
> technical content to add at that point which hadn't already been
> explained.

> Perhaps no one pointed out that using opaque function objects as we now
> do is already standard practice in both Common Lisp and Scheme; both are
> clearly examples of very powerful and capable Lisp dialects.  I don't
> see that this change has made Emacs any less powerful.

Somebody did point out its presence in other Lisps, yes.  Nobody gave a
compelling reason why it would be good for Emacs Lisp.  The change broke
an old macro in CC Mode, which luckily wasn't being used for much.  I
was able to fix that.

> > Number 3: Stefan introduced pcase.el without any open discussion, and
> > proliferated it rapidly around the Emacs core, leading to confusion
> > around the use of ` and ,, certainly on my part.  Now it can be argued
> > that pcase has been a success, but it could have been so much better if
> > it had been developed cooperatively.  For years there was no adequate
> > doc string for `pcase', and even now the doc strings for things like
> > pcase-let* are woefully inadequate.  Stefan is not good at documenting;
> > nobody can be good at everything.

> I submit that the reason why pcase has proliferated both in core and
> externally is that it is fundamentally useful.  This proliferation is
> not due to a master plan concocted by Stefan Monnier (who, for the
> record, was an Emacs maintainer when pcase.el was installed).  I'm sure
> that he did take the chance to use it to improve some existing code
> though, work that we still benefit from and can be grateful for.

I don't doubt its usefulness.  However, it could have been better if it
had been openly discussed, possibly engaging help in documenting it
better.

I don't recall any other contributer, maintainer or not, making such
large changes without consultation.

> In practice, however, the decision to use pcase is typically made by the
> person writing some piece of code, and we cannot police the style of
> individual contributions on that level.  That would make no one happy,
> and it would largely be detrimental to the project.  So if people like
> pcase (or when-let, etc.), they will use it.

I don't recall any discussion over the introduction of when-let, etc.
Maybe there was.  There has recently been a long thread in emacs-devel
showing how confusing contributers have found it.

> What should we blame Stefan Monnier for?  For implementing a novel
> feature that has proven both highly useful and popular?

For doing so without open discussion on emacs-devel.

> > Number 4: Some years ago, Stefan removed the documentation of defadvice
> > from the elisp manual without any discussion.  Despite widespread
> > protest, he refused to put it back again.  Quite coincidentally, he had
> > just written and pushed nadvice.el.

> We routinely remove references to obsolete or deprecated functions and
> macros in the manual.  There is nothing strange about that in and of
> itself.

The feature was in widespread use throughout Emacs and external
libraries.  The documentation was still needed.  defadvice had never
been marked as deprecated up till that point.

> In this case, I also find nadvice.el a significant improvement over
> advice.el; therefore, I think removing it was likely the right
> decision.  (That said, I wasn't around at the time, so I didn't read
> the discussions.)

> Moreover, when nadvice.el was installed, and these changes were made in
> the manual, Stefan Monnier was an Emacs maintainer.  Making such
> decisions, even amid controversy, is quite literally a part of the job
> description.  Our current maintainers, I'm sure, have also taken
> decisions that some people have disagreed with.  C'est la vie.

No other maintainer, that I'm aware of, has made big changes to Emacs
without public discussion first.  Eli doesn't do it, you don't, and I'm
sure Andrea doesn't either.

> We can certainly discuss changing our current development model and
> decision-making process.  But for now, it is what we have.

That is what I have been urging you (plural) to do for a few days, now.

> > Number 5: Previously, there had been an embargo on the use of the CL
> > library, except at compile time.  This kept the size of the Emacs Lisp
> > language manageable, and the language easy to understand, and made
> > maintainers' and beginners' lives easier.

> You have raised this point in the past, and my impression is that few
> people agree with this sweeping generalization.  I do not agree that our
> current use of cl-lib constitutes a problem.

When I was actively trying to implement bug#67455 (getting function
names into backtraces where there is currently only an anonymous hex
address) the widespread use of cl-lib probably slowed my progress by a
factor of two.  I was continually having to look up (usually
non-existent) doc strings, and spent much time crawling through cl-lib
source code working out what functions did.

The last time we talked about the use of cl-lib, the discussion was
terminated by somebody (I can't remember who) undertaking to document
cl-lib.  I don't think anything ever came from that intention.

> It is true that we could improve cl-lib.  It would be even better if we
> added some of the more useful functions and macros to Emacs Lisp itself,
> so that we would have less need for a separate library.

> > At some stage this embargo was lifted, and the use of CL rapidly
> > proliferated through the Emacs core.  Now, it could be argued that the
> > facilities and expressiveness of the CL lib outweigh these
> > disadvantages.  But it was not so argued.  It just happened.  Maybe
> > somebody else but Stefan made this change, but it seems unlikely.

> Similar to pcase, the reason cl-lib has proliferated is that many people
> find it useful.  Perhaps you may need to accept that you are in the
> minority on this issue.

Again, we don't and cannot know.  There was no discussion where people
could have raised objections.  My own impression is that cl-lib does not
help contributers express things better; only differently.  But the
people who write cl-lib constructs are often not those who have to
maintain them, and those maintainers are denied the choice.

> > I have not come anywhere near ad hominem.  It is true that many forums
> > degenerate into slanging matches which repel decent posters.
> > emacs-devel is the opposite extreme, sort of touchy-feely where nobody's
> > allowed to offend anybody else at all, no matter what they do, why and
> > how they do it.  This is just as unhealthy as the the continual abuse
> > forums; it leads to the build up of repressed resentment.

> > Sometimes the truth must be told bluntly, and that is what I have tried
> > to do here.

> Phrases such as "Jekyll-and-Hyde character", "contemptuous of the Emacs
> conventions", "does not have the gift of knowing what the Right Thing
> is", along with terms like "monster" and asking us "what does that say
> about [him]", etc., are clearly ad hominem in my view.

No.  Ad hominem has a very specific meaning.  It means attacking the
author of some writing or act in place of criticising the writing or act
itself.  It does not mean simple denigration, or even insult.
Everything I've written here is about what Stefan has DONE, together
with speculation as to why.

> These statements focus on personal characteristics rather than on
> actions or statements made by an individual.

They do, yes.  You haven't commented explicitly on whether or not they
are true.

To the five anecdotes I've described, you have given a different
justification for each one.  Taken individually, each of them could
easily hold.  Taken together, they point with high probability to there
being a problem in Stefan's approach.

Maybe you could explain why I have had no such problems with any other
contributors or maintainers.  Not with you, not with Andrea, not with
Eli, despite having had quite a few disagreements with him over the
years.

> >> I have to agree with Eli.  Although it would, in hindsight, certainly
> >> have been better to discuss these particular changes in more detail in
> >> advance, I don't see that he has done anything very unusual or different
> >> from what most other core contributors do on a routine basis.

> > This "be nice to everybody no matter what they do" and "always assume
> > the best of everybody" creates the perfect atmosphere for a monster to
> > flourish in.  Stefan is such a monster; not all the time, not even most
> > of the time, but in doing the things detailed above, and other things, I
> > don't understand why you are defending him.

> OK, let's be frank then.

> First, I understand that there are things that you are unhappy about,
> and that is valid and to be taken seriously.

> However, I find your attempt to portray Stefan Monnier's character
> negatively completely beyond the pale.  Using words like a "monster" to
> refer to another core contributor is just appalling, and completely
> uncalled for.

I think you've changed my meaning somewhat by taking these phrases out
of context.

I was quite explicit about my view of Stefan; he is mostly a positive
force on Emacs, but has negative attributes, too.  I think you (plural)
are trying to pretend these negative bits don't exist, or don't matter.

> Thus, I am standing up for basic decency in the face of what is more and
> more starting to look like an attempt at public character assassination,
> using harsh language (e.g., "monster") and enumerating alleged
> "mistakes" from as far back as 15 years.  I urge you to retract these
> comments, and apologize.

I think the important factor is whether or not my assertions are true.
What would I have to gain by such a character assassination?

If they are true, then I have nothing to apologise for.  If they are
mistaken, I would indeed apologise.  Quite a few posters on this thread,
and several by private email, have said they recognise that what I have
written has a basis in fact.

> I have already expressed my opinions on the technical issues that you
> listed above.  However, from a strictly procedural standpoint, I do not
> see that Stefan Monnier has exhibited a pattern of behaviour that stands
> out as culpable or even unusual within our development model.

Do you have any other explanation as to why I should be so dissatisfied
that I am leaving the project?

> Note that I am not claiming he has never made mistakes, or that
> everything he has done is perfect.  I am quite sure that he has done
> many mistakes, both technical and procedural.  But so has every single
> person on this list.  That does not justify singling him out, nor does
> it justify the attacks directed at him here.

No.  Stefan M is different from all other maintainers, as I have pointed
out at great length.

> Please understand that the changes that Stefan Monnier has authored and
> that you find objectionable, in the minds of many other hackers are
> improvements.  In my view, for example, he has done tremendous work to
> improve Emacs, and not least to make Emacs Lisp more powerful.  The
> examples you give are some of the things that I would point to if I
> wanted to demonstrate that.

I agree with the above paragraph, mostly.  Stefan has done a lot
positive as well.

> ---

> Ultimately, all of us are volunteers who care deeply about our work.
> This means that feelings can sometimes run high, and that is
> understandable, because we are all human beings with different ideas,
> interests, and motives.

> Our team of core contributors are, despite our usually polite demeanor,
> a relatively rowdy group of strong-willed, and often opinionated
> individuals.  This description applies to all of us; we wouldn't be
> running Emacs, let alone hack on it, if it weren't true.  We are a small
> church, and it is in our best interest to collaborate.

> Precisely for this reason, especially when we disagree, we must make
> every effort to treat each other courteously and with respect.  This
> does not mean sweeping issues under the rug, or maintaining some
> superficial peace merely for appearances.  But it does mean that we have
> a strong preference for discussing the specific issues at hand, and that
> we ask everyone to meet a minimum standard, which we refer to as "kind
> communication", when participating in Emacs development.  This is not
> likely to change.

> I sincerely hope that you reconsider your resignation.  It is sad to see
> a capable member of our core team leave.  Regardless of what happens, I
> look forward to continue collaborating with you in your capacity as CC
> mode maintainer.

Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Stefan Monnier via Emacs development discussions.]

Hi Alan,

[ Someone just made me aware of this ... interesting thread.  ]

> For starters: The change in the meaning of `c-mode' and `c++-mode' he
> introduced in bug#69191, discussed at length in my last post.  Stefan is
> not stupid.  He knew full well what he was doing in bypassing open
> discussion about major-mode-remap-defaults.

Actually, I must admit that in this specific case you caught me by
surprise.  The change of meaning was fundamentally introduced AFAICT by
`major-mode-remap-alist` (for which I also plead guilty) back in
Emacs-29.  I saw it as mostly a cleanup, and I expected you'd be rather
favorable to that change.  I still can't understand what it is that
bothers you so much there and that didn't bother you in Emacs-29's
`major-mode-remap-alist`.

[ I'm no fan of the way loading files like `c-ts-mode.el` (and now
  `cc-mode.el` as well) changes the behavior of Emacs.  The patch for
  bug#69191 did not fundamentally change that (because Emacs maintainers
  had already made it clear they did not intend to accept such
  a change), and the purpose was solely to make those changes cleaner.

  E.g. previously the fact that `c-ts-mode` becomes preferred after
  loading `c-ts-mode.el` was limited to those files whose mode is chosen
  via `auto-mode-alist` rather than other methods like file-local
  cookies.
  Also, undoing that change was somewhat complicated.  ]

> Number 3: Stefan introduced pcase.el without any open discussion, and

Thanks.  Sometimes I feel like my years as Emacs contributor have been
spent doing nothing else than janitorial work, so it's good to be
reminded that I also did make some more significant contributions.

> Number 4: Some years ago, Stefan removed the documentation of defadvice
> from the elisp manual without any discussion.  Despite widespread
> protest, he refused to put it back again.  Quite coincidentally, he had
> just written and pushed nadvice.el.

"coincidentally" is wrong here: the removal was made very much
explicitly in relation to the addition of `nadvice.el`, with the
explicit purpose to discourage the use of `defadvice` in new code in
favor of the new nadvice functions.  And an important reason why I was
comfortable removing that doc is because virtually the same text was
(and still is) available in the `Commentary:` section of `advice.el`.

> Number 5: Previously, there had been an embargo on the use of the CL
> library, except at compile time.  This kept the size of the Emacs Lisp
> language manageable, and the language easy to understand, and made
> maintainers' and beginners' lives easier.

That's an opinion, not a fact.  It made some things easier, and
others harder.  Who benefited most and who suffered most is hard
to tell.  I don't think anyone here really knows.

BTW, things like pushing for the eventual removal of `defadvice` is done
for the purpose of keeping the language smaller to make maintainers' and
beginners' lives easier.  The transient state is worse, admittedly.
If Emacs dies soon, it was clearly the wrong call.  I made the decision
based on the assumption that it will live for many more years.

> At some stage this embargo was lifted, and the use of CL rapidly
> proliferated through the Emacs core.  Now, it could be argued that the
> facilities and expressiveness of the CL lib outweigh these
> disadvantages.  But it was not so argued.  It just happened.
> Maybe somebody else but Stefan made this change, but it
> seems unlikely.

I plead guilty to turning CL into CL-lib so that it can be used at
run-time.  No doubt about that.  This was the best compromise I could
find to satisfy the constant complaints about not being able to use the
CL library in core Emacs code.

> Incidentally, the CL library is badly documented; most of its
> functions, macros, and variables lack doc strings, and comments are
> sparse indeed.  For example, in cl-generic.el, there is no description
> of the structures and algorithms used to implement generic functions.
> "Maintainable" isn't an adjective which springs to mind for
> this library.

[ Despite its name, `cl-generic.el` is not part of CL-lib.  ]

BTW, you forgot lexical-binding in your list of my accomplishments.

You can return to regularly scheduled Lisp hacking now.


        Stefan


PS: Oh, I think I saw a mention of `if-let` in this thread, so I want
    to state clearly that I do *not* plead guilty for this one.  I'm not
    a big fan of these `if/when/while/and-let` and can't remember taking
    part in their development at all, except recently trying
    (unsuccessfully) to get rid of `and-let*`, and advocating for the
    reduction in the number of those macros.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I have to agree with Eli.  Although it would, in hindsight, certainly
  > have been better to discuss these particular changes in more detail in
  > advance,

I too have complained that major changes were not being discussed in the
way that they call for.

I think this incident shows that we need to make a rule to ensure
they get discused properly.  Some ideas occur to me, which could form
a part of this.

Here's one possible approach that occurs to me.
It's not the only approach that could be good.

* A new feature must be brought up on emacs-devel with a clear description.

* If anyone on the list says, "This feature calls for careful discussion,"
then maintainers must respond saying, "We will now have 14 days for discussion
of this feature."

* 14 later, a maintainer can post, "We have had 14 days of discussion for
the feature XyZ.  Here's how the plan stands now.  Does anyone call for further
discussion?"

* If anyone on the list says, "This feature calls for further
discussion," then maintainers must respond saying, "We will now have 7
days for further discussion of this feature."

* 7 days later, the maintainers can install the feature.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Date: Sat, 23 Nov 2024 01:10:42 -0500
> 
>   > I have to agree with Eli.  Although it would, in hindsight, certainly
>   > have been better to discuss these particular changes in more detail in
>   > advance,
> 
> I too have complained that major changes were not being discussed in the
> way that they call for.

My impression is that complaints about "changes installed without
being discussed" assume that discussing them would have stopped them
from being installed.  When there's a chance that this is the case,
according to the best judgment of the maintainers, we insist on
discussions (at least I do) as prerequisite for accepting the changes.
There are enough changes which were rejected this way.

However, as I've written elsewhere in this thread, Emacs maintainers
are just stewards.  They don't "own" the project, and cannot dictate
their preferences when some change is widely favored by the community,
not without good reasons, which are considered "good enough" by the
community.  So when we feel that a change will be widely supported,
regardless of discussions, we see no reason to delay its installation.
(Of course, code review and various minor issues like proper
documentation etc. are still in effect, and many times require
re-submission of patches before they are installed.)

> I think this incident shows that we need to make a rule to ensure
> they get discused properly.  Some ideas occur to me, which could form
> a part of this.
> 
> Here's one possible approach that occurs to me.
> It's not the only approach that could be good.
> 
> * A new feature must be brought up on emacs-devel with a clear description.
> 
> * If anyone on the list says, "This feature calls for careful discussion,"
> then maintainers must respond saying, "We will now have 14 days for discussion
> of this feature."
> 
> * 14 later, a maintainer can post, "We have had 14 days of discussion for
> the feature XyZ.  Here's how the plan stands now.  Does anyone call for further
> discussion?"
> 
> * If anyone on the list says, "This feature calls for further
> discussion," then maintainers must respond saying, "We will now have 7
> days for further discussion of this feature."
> 
> * 7 days later, the maintainers can install the feature.

We already have those rules, albeit not formally.  But rules are not
always followed to the letter in a community such as ours, and we have
no real means of enforcing them.  (Extreme examples of disregard for
those rules cause radical measures like public harsh reprimands of the
offenders, and reverting of the offending changes, but that is a
doomsday weapon that must be employed extremely sparingly.  And we
have nothing else except telling people a-posteriori they should have
known better.)

I object to codifying such rules, for two reasons:

  . Rules that force people to behave decently and respectfully to
    others send a message that people are not trusted to do it
    otherwise, which, to me, smells of totalitarian societies and
    by-default incrimination of people we know nothing about.  We have
    Gnu Kind Communication Guidelines where other projects have the
    notorious Codes of Conduct, for this very reason.
  . If such rules are broken, we have no real power to do anything.
    IOW, the "or else" clause of any such rule is basically empty.
    The only measure we have is to take away their write access, which
    is again a doomsday weapon, and will certainly avert contributors
    if applied when, say, some change was installed without waiting
    for 14 days.

Beyond the above, I personally have no will to enforce such rules, and
if the decision is to introduce them, someone else will have to do
that, because I will not.  It is against my vision of how such
communities should be led, and no one can ask me to do this job
against my principles and my best judgment.

Re: My resignation from Emacs development

[Christopher Dimech]

> Sent: Saturday, November 23, 2024 at 7:48 PM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: rms@gnu.org
> Cc: stefankangas@gmail.com, acm@muc.de, emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> > From: Richard Stallman <rms@gnu.org>
> > Cc: acm@muc.de, emacs-devel@gnu.org
> > Date: Sat, 23 Nov 2024 01:10:42 -0500
> > 
> >   > I have to agree with Eli.  Although it would, in hindsight, certainly
> >   > have been better to discuss these particular changes in more detail in
> >   > advance,
> > 
> > I too have complained that major changes were not being discussed in the
> > way that they call for.
> 
> My impression is that complaints about "changes installed without
> being discussed" assume that discussing them would have stopped them
> from being installed.  When there's a chance that this is the case,
> according to the best judgment of the maintainers, we insist on
> discussions (at least I do) as prerequisite for accepting the changes.
> There are enough changes which were rejected this way.
> 
> However, as I've written elsewhere in this thread, Emacs maintainers
> are just stewards.  They don't "own" the project, and cannot dictate
> their preferences when some change is widely favored by the community,
> not without good reasons, which are considered "good enough" by the
> community.  So when we feel that a change will be widely supported,
> regardless of discussions, we see no reason to delay its installation.
> (Of course, code review and various minor issues like proper
> documentation etc. are still in effect, and many times require
> re-submission of patches before they are installed.)

I challenge the notion that Emacs maintainers act solely as stewards.
The term implies neutrality, but maintainers inherently influence the
project's trajectory.

Maintainers have a fiduciary duty to both the project's longevity and
its contributors' collective vision.  Thus the robustness of the project
and the ease of improvements are important.

> > I think this incident shows that we need to make a rule to ensure
> > they get discused properly.  Some ideas occur to me, which could form
> > a part of this.
> > 
> > Here's one possible approach that occurs to me.
> > It's not the only approach that could be good.
> > 
> > * A new feature must be brought up on emacs-devel with a clear description.
> > 
> > * If anyone on the list says, "This feature calls for careful discussion,"
> > then maintainers must respond saying, "We will now have 14 days for discussion
> > of this feature."
> > 
> > * 14 later, a maintainer can post, "We have had 14 days of discussion for
> > the feature XyZ.  Here's how the plan stands now.  Does anyone call for further
> > discussion?"
> > 
> > * If anyone on the list says, "This feature calls for further
> > discussion," then maintainers must respond saying, "We will now have 7
> > days for further discussion of this feature."
> > 
> > * 7 days later, the maintainers can install the feature.
> 
> We already have those rules, albeit not formally.  But rules are not
> always followed to the letter in a community such as ours, and we have
> no real means of enforcing them.  (Extreme examples of disregard for
> those rules cause radical measures like public harsh reprimands of the
> offenders, and reverting of the offending changes, but that is a
> doomsday weapon that must be employed extremely sparingly.  And we
> have nothing else except telling people a-posteriori they should have
> known better.)
> 
> I object to codifying such rules, for two reasons:
> 
>   . Rules that force people to behave decently and respectfully to
>     others send a message that people are not trusted to do it
>     otherwise, which, to me, smells of totalitarian societies and
>     by-default incrimination of people we know nothing about.  We have
>     Gnu Kind Communication Guidelines where other projects have the
>     notorious Codes of Conduct, for this very reason.
>   . If such rules are broken, we have no real power to do anything.
>     IOW, the "or else" clause of any such rule is basically empty.
>     The only measure we have is to take away their write access, which
>     is again a doomsday weapon, and will certainly avert contributors
>     if applied when, say, some change was installed without waiting
>     for 14 days.

They also impose significant logistical burdens on maintainers,
particularly if enforcement becomes a regular necessity or must happen
at inopportune times.  Enforcing rules consistently and addressing
violations, especially during critical project milestones or personal
time constraints, is unsustainable.

Ultimately, the sustainability of enforcement must align with the
maintainers' capacity,

> Beyond the above, I personally have no will to enforce such rules, and
> if the decision is to introduce them, someone else will have to do
> that, because I will not.  It is against my vision of how such
> communities should be led, and no one can ask me to do this job
> against my principles and my best judgment.

Let's put aside competing visions and principles and prioritize
practical judgment. The abundance of conflicting ideals has only added
complexity. It’s time for the maintainers to make a clear decision on
this matter, now that the discussion has been fully laid out.

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: rms@gnu.org, stefankangas@gmail.com, acm@muc.de, emacs-devel@gnu.org
> Date: Sat, 23 Nov 2024 12:06:10 +0100
> 
> > However, as I've written elsewhere in this thread, Emacs maintainers
> > are just stewards.  They don't "own" the project, and cannot dictate
> > their preferences when some change is widely favored by the community,
> > not without good reasons, which are considered "good enough" by the
> > community.  So when we feel that a change will be widely supported,
> > regardless of discussions, we see no reason to delay its installation.
> > (Of course, code review and various minor issues like proper
> > documentation etc. are still in effect, and many times require
> > re-submission of patches before they are installed.)
> 
> I challenge the notion that Emacs maintainers act solely as stewards.
> The term implies neutrality, but maintainers inherently influence the
> project's trajectory.

The term does not imply neutrality, it implies taking care of the
project and keeping it in good shape, and that by its very definition
does influence the development.

Anyway, focusing on this aspect is taking out the main part of what I
wrote above.

> Maintainers have a fiduciary duty to both the project's longevity and
> its contributors' collective vision.  Thus the robustness of the project
> and the ease of improvements are important.

And you are saying that this doesn't happen here?

Re: My resignation from Emacs development

[Christopher Dimech]

> Sent: Saturday, November 23, 2024 at 11:54 PM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: rms@gnu.org, stefankangas@gmail.com, emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: rms@gnu.org, stefankangas@gmail.com, acm@muc.de, emacs-devel@gnu.org
> > Date: Sat, 23 Nov 2024 12:06:10 +0100
> >
> > > However, as I've written elsewhere in this thread, Emacs maintainers
> > > are just stewards.  They don't "own" the project, and cannot dictate
> > > their preferences when some change is widely favored by the community,
> > > not without good reasons, which are considered "good enough" by the
> > > community.  So when we feel that a change will be widely supported,
> > > regardless of discussions, we see no reason to delay its installation.
> > > (Of course, code review and various minor issues like proper
> > > documentation etc. are still in effect, and many times require
> > > re-submission of patches before they are installed.)
> >
> > I challenge the notion that Emacs maintainers act solely as stewards.
> > The term implies neutrality, but maintainers inherently influence the
> > project's trajectory.
>
> The term does not imply neutrality, it implies taking care of the
> project and keeping it in good shape, and that by its very definition
> does influence the development.
>
> Anyway, focusing on this aspect is taking out the main part of what I
> wrote above.
>
> > Maintainers have a fiduciary duty to both the project's longevity and
> > its contributors' collective vision.  Thus the robustness of the project
> > and the ease of improvements are important.
>
> And you are saying that this doesn't happen here?

I strongly support the need to avoid implementing rules that impose
excessive burdens on maintainers --- especially those that elevate
contributors' collective vision above the practical demands of
maintaining and improving the code. Ensuring the functionality and
integrity of the code, while keeping it amenable to enhancements, must
remain the top priority, as these are the cornerstones of the project's
long-term success. Your dedication upon those priorities is invaluable.

Re: My resignation from Emacs development

[Adam Porter]

> I object to codifying such rules, for two reasons:
> 
>   . Rules that force people to behave decently and respectfully to
>     others send a message that people are not trusted to do it
>     otherwise, which, to me, smells of totalitarian societies and
>     by-default incrimination of people we know nothing about.  We have
>     Gnu Kind Communication Guidelines where other projects have the
>     notorious Codes of Conduct, for this very reason.
>   . If such rules are broken, we have no real power to do anything.
>     IOW, the "or else" clause of any such rule is basically empty.
>     The only measure we have is to take away their write access, which
>     is again a doomsday weapon, and will certainly avert contributors
>     if applied when, say, some change was installed without waiting
>     for 14 days.
> 
> Beyond the above, I personally have no will to enforce such rules, and
> if the decision is to introduce them, someone else will have to do
> that, because I will not.  It is against my vision of how such
> communities should be led, and no one can ask me to do this job
> against my principles and my best judgment.

FWIW, I completely agree with Eli.  Emacs is not like the others, both 
technically, as software, and socially, as a project and community.  We 
assume good faith by default; we are forgiving of mistakes and "bad 
days"; we don't look for reasons to accuse others of various violations, 
to impose penalties or threaten to.  This place is a welcome respite 
from what seems to be the norm.

And so we don't make lists of such potential violations to hang over our 
heads as a lingering threat.  That's something my elementary school 
teachers did, when we were learning how to behave and work together in 
groups.  I don't want to regress to such a state, and I think it isn't 
needed here; beyond that, it would have negative effects.

We already know how we should behave here.  Yet, we are people, so 
sometimes we fail.  And the world keeps turning, and emacs.git keeps 
receiving commits, and people keep coming and going as life goes on.  I 
expect that, someday, when I am no longer present on this world, 
emacs.vcs-of-the-future will still be receiving commits.  And I hope 
that the participants then are as generally friendly, tolerant, and 
forgiving as they are now--not looking to cite each other for rule 
violations.

--Adam

Re: My resignation from Emacs development

[Sean Whitton]

Hello,

On Sat 23 Nov 2024 at 05:59pm -06, Adam Porter wrote:

> FWIW, I completely agree with Eli.  Emacs is not like the others, both
> technically, as software, and socially, as a project and community.  We assume
> good faith by default; we are forgiving of mistakes and "bad days"; we don't
> look for reasons to accuse others of various violations, to impose penalties
> or threaten to.  This place is a welcome respite from what seems to be the
> norm.

Yes, it is indeed such a relief from other Free Software working
groups/communities I interact with.

We exchange thoughtful messages, we don't have to open a web browser
most of the time, and we make nice commits.  It's great.

-- 
Sean Whitton

Re: My resignation from Emacs development

[tomas]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

On Sun, Dec 01, 2024 at 11:50:21AM +0800, Sean Whitton wrote:
> Hello,
> 
> On Sat 23 Nov 2024 at 05:59pm -06, Adam Porter wrote:
> 
> > FWIW, I completely agree with Eli.  Emacs is not like the others, both
> > technically, as software, and socially, as a project and community.  We assume
> > good faith by default; we are forgiving of mistakes and "bad days"; we don't
> > look for reasons to accuse others of various violations, to impose penalties
> > or threaten to.  This place is a welcome respite from what seems to be the
> > norm.
> 
> Yes, it is indeed such a relief from other Free Software working
> groups/communities I interact with.
> 
> We exchange thoughtful messages, we don't have to open a web browser
> most of the time, and we make nice commits.  It's great.

Couldn't agree more.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: My resignation from Emacs development

[Suhail Singh]

Richard Stallman <rms@gnu.org> writes:

> I too have complained that major changes were not being discussed in the
> way that they call for.
>
> I think this incident shows that we need to make a rule to ensure
> they get discused properly.

When rules are brought up, it's important to clarify a few things:

1. Who would the rules apply to?  Presumably this would be anyone with
   commit access.
2. Who would be enforcing the rules?  Would this be the maintainers or
   someone else?
3. What would be the recourse should the rules be broken?  Would they
   result in behaviour that is meaningfully different than what happens
   today?
4. What about errors in enforcement?  How are those to be handled?
5. Specifically, in this case: how does one determine which change ought
   to be first discussed on emacs-devel vs on debbugs vs not at all?

Perhaps 2-5 above are already specified elsewhere for the project.  If
so, please ignore.

From my perspective (that of a relative outsider), what the community
may perhaps benefit from might be some guidelines regarding conflict
resolution and mediation.  Even as I state it, I feel compelled to hedge
because I am not entirely convinced that it would've resulted in a
different outcome.  In particular, it's entirely possible that
reasonable steps to resolve the conflict had already been taken by Eli
and/or others prior to Alan's post on emacs-devel.

-- 
Suhail

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 1. Who would the rules apply to?  Presumably this would be anyone with
  >    commit access.

Everyone who writes or proposes feature changes for Emacs would be
asked to start discussions about them.

  > 2. Who would be enforcing the rules?  Would this be the maintainers or
  >    someone else?

All contributors would have that responsibility, and the maintainers
should be leaders in carring it out.

  > 3. What would be the recourse should the rules be broken?  Would they
  >    result in behaviour that is meaningfully different than what happens
  >    today?

We are all trying to work together here, so I don't think we need
"recourse" other than to remind people gently that "We should discuss
this on emacs-devel."  Any of us can start that discussion.  So if you
see that a change is being considered that calls for discussion on
emacs-devel, just forward one of the messages about it to emacs-devel.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Suhail Singh]

Richard Stallman <rms@gnu.org> writes:

> We are all trying to work together here, so I don't think we need
> "recourse" other than to remind people gently that "We should discuss
> this on emacs-devel."  Any of us can start that discussion.  So if you
> see that a change is being considered that calls for discussion on
> emacs-devel, just forward one of the messages about it to emacs-devel.

I see.  The change you're proposing, if I understand correctly, is to
make the above (that anyone can start a discussion on emacs-devel, and
that we should all aspire to do so for "feature changes") common
knowledge (in addition to defining particulars regd. duration of
discussion etc).

I was under the impression that this was already common knowledge, but
perhaps not.  Regardless, while it's not clear whether such a guideline
would have helped in the triggering event that started the present
discussion, I do think there's some value in noting explicitly that
everyone has the shared responsibilities to discuss features (for some
definition of feature) on emacs-devel before pushing them.  And that if
one such omission comes to light it is the observer's responsibility to
treat the omission as having been accidental (as opposed to from
ill-intent) and to gently start the discussion on emacs-devel.

-- 
Suhail

Re: My resignation from Emacs development

[Gerd Möllmann]

Alan Mackenzie <acm@muc.de> writes:

> I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> Mode issues which arise in Emacs.  Please post these to
> bug-cc-mode@gnu.org.

Thanks Alan, for everything.

Will you make a package out of cc-mode, or should one use the Hg repo?

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Gerd.

On Thu, Nov 21, 2024 at 06:59:23 +0100, Gerd Möllmann wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> > maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> > Mode issues which arise in Emacs.  Please post these to
> > bug-cc-mode@gnu.org.

> Thanks Alan, for everything.

Appreciated.

> Will you make a package out of cc-mode, or should one use the Hg repo?

I haven't really thought that far ahead, yet.  Maybe I'll just commit
changes in the CC Mode repo, and post patches to the OPs.  But the
patches frequently need work on them because of differences in
whitespace, spelling, use of older CC Mode macros, and things like this.

We'll see!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Eli Zaretskii]

> Date: Fri, 22 Nov 2024 11:36:55 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > Will you make a package out of cc-mode, or should one use the Hg repo?
> 
> I haven't really thought that far ahead, yet.  Maybe I'll just commit
> changes in the CC Mode repo, and post patches to the OPs.  But the
> patches frequently need work on them because of differences in
> whitespace, spelling, use of older CC Mode macros, and things like this.
> 
> We'll see!

Is the Hg repository whose URL is here:

  https://cc-mode.sourceforge.net/hgaccess.php

the up-to-date one?  I'm asking because the last commit there was in
July 2023, and because there's a branch called Branch_5_35 whereas the
CC Mode files in Emacs say they are from version 5.33.1.  So I wonder
where are the up-to-date sources of CC Mode, and where do you intend
to develop it, so we could sync with that from time to time.

Thanks.

Re: My resignation from Emacs development

[Alan Mackenzie]

Hello, Eli.

On Fri, Nov 22, 2024 at 13:52:51 +0200, Eli Zaretskii wrote:
> > Date: Fri, 22 Nov 2024 11:36:55 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > Will you make a package out of cc-mode, or should one use the Hg repo?

> > I haven't really thought that far ahead, yet.  Maybe I'll just commit
> > changes in the CC Mode repo, and post patches to the OPs.  But the
> > patches frequently need work on them because of differences in
> > whitespace, spelling, use of older CC Mode macros, and things like this.

> > We'll see!

> Is the Hg repository whose URL is here:

>   https://cc-mode.sourceforge.net/hgaccess.php

> the up-to-date one?

No, it is not.  I gave up on SourceForge when they changed their terms
and conditions to allow them to sell members' details to advertising
agencies.  I got most of the site over to savannah at
https://www.nongnu.org/cc-mode, but got interrupted before I could get
the mailing list transferred.  I also failed to make any announcement on
the list (bug-cc-mode@gnu.org).  I need to do both of these things,
still.

The repository being maintained is at
http://hg.savannah.nongnu.org/cc-mode, or with a savannah user name,
ssh://USERNAME@hg.savannah.nongnu.org/cc-mode.

> I'm asking because the last commit there was in July 2023, and because
> there's a branch called Branch_5_35 whereas the CC Mode files in Emacs
> say they are from version 5.33.1.  So I wonder where are the
> up-to-date sources of CC Mode, and where do you intend to develop it,
> so we could sync with that from time to time.

See above.

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: My resignation from Emacs development

[Eli Zaretskii]

> Date: Sat, 23 Nov 2024 10:36:15 +0000
> Cc: gerd.moellmann@gmail.com, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > Is the Hg repository whose URL is here:
> 
> >   https://cc-mode.sourceforge.net/hgaccess.php
> 
> > the up-to-date one?
> 
> No, it is not.  I gave up on SourceForge when they changed their terms
> and conditions to allow them to sell members' details to advertising
> agencies.  I got most of the site over to savannah at
> https://www.nongnu.org/cc-mode, but got interrupted before I could get
> the mailing list transferred.  I also failed to make any announcement on
> the list (bug-cc-mode@gnu.org).  I need to do both of these things,
> still.
> 
> The repository being maintained is at
> http://hg.savannah.nongnu.org/cc-mode

The correct URL seems to be

   https://hg.savannah.nongnu.org/hgweb/cc-mode/

> or with a savannah user name,
> ssh://USERNAME@hg.savannah.nongnu.org/cc-mode.

Thanks.

Re: My resignation from Emacs development

[Andrea Corallo]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Emacs.
>
> I'm resigning my position as Emacs contributor.
>
> The immediate reason is that, as maintainer of CC Mode, CC Mode's
> symbols, its names, were taken by Emacs and used for other purposes
> without informing me, much less consulting me.  That makes my position as
> CC Mode maintainer here untenable.
>
> Eli Zaretskii and I have had extensive discussions, both in public and in
> private email, over the last week or so, but we have been unable to reach
> any satisfactory compromise solution.
>
> Names are important.  They have power.  To take somebody's/somthing's
> name and misuse it is an exercise of aggression.  Try using "Emacs" or
> even "free software" to mean something different, and see just how
> quickly you would hear back from Richard Stallman.  This misuse of CC
> Mode's "trademarks", the symbols `c-mode', `c++-mode', and perhaps
> `c-or-c++-mode', is just such an act of aggression.
>
> These symbols have been appropriated by Emacs to mean "the current
> preferred mode for C", etc., rather than C Mode, C++ Mode etc.  In
> certain circumstances, in particular, in Local Variables: sections and
> auto-mode-alist, there is now no longer any way unambiguously to specify
> C Mode or C++ Mode.  Up till recently ("\\.myc\\'" . c-mode) in
> auto-mode-alist meant C Mode, and would have had the effect of
> auto-loading CC Mode, if needed, and running C Mode.
>
> The change took place in the commit for bug#69191 "New var
> `major-mode-remap-defaults`, for packages".  It sounds so innocent, but
> is an extremely bad solution for whatever problem (unspecified in the
> commit message) it was intended to solve.  A major mode using it changes
> the interfaces of other libraries in an uncontrolled way.  This is not
> good software engineering.
>
> This bug was raised and committed by Stefan Monnier.  Despite the fact
> that the bug fix directly impinged upon CC Mode, and there was even a
> change to cc-mode.el in the patch, he failed even to inform me.  The only
> two modes substantially affected by this change were ruby-mode and CC
> Mode, and it is clear that Dmitry Gutov, maintainer of ruby-mode, was
> aware of the change.  Had I known of this proposal, I would certainly
> have objected to it.  Stefan is intelligent enough to have realised this,
> and maybe his avoidance of open discussion was motivated by this.
>
> Bug#69191 was a big change.  In Emacs, we have a convention whereby big
> changes are discussed openly on emacs-devel and a consensus reached
> before the change is made.  Stefan Monnier has regularly violated this
> convention, possibly believing that his ideas for Emacs are so good as to
> be beyond question.  Any attempt to question his ideas is likely to be
> met by evasive non-answers, if any response at all is forthcoming.  I
> could give several paragraphs worth of justification for these
> assertions, but I think everybody here knows I am right.
>
> In Emacs there is also a convention of treating eachother with respect on
> the mailing lists.  Sadly this convention is superficial, and seems only
> to mean things like not using swear words.  The truly contemptuous
> communication style, this evasive non-answering, seems to be regarded as
> acceptable.  I suggest that this change.
>
> Stefan's habit of making big changes in Emacs without seeking consensus
> is at the heart of why I am resigning.  These changes have caused Emacs a
> lot of damage over the years and have caused other contributors,
> including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
> character.  On the one hand, he's a very capable hacker, and is always
> ready to help others with technical questions.  On the other hand, as
> mentioned, he is contemptuous of the Emacs conventions, and unlike
> Richard and Eli, does not have the gift of knowing what the Right Thing
> is.
>
> I strongly recommend that Stefan somehow be reigned in and required to
> observe Emacs's conventions about open discussion and courteous
> communication.  As I mentioned, his violations of these are at the core
> of why I feel unable to continue contributing to Emacs.
>
> I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> Mode issues which arise in Emacs.  Please post these to
> bug-cc-mode@gnu.org.
>
> It just remains to say that my respect for Eli and the other maintainers
> remains undiminished, and that I wish all of them and the Emacs project
> all success in the future.
>
> --
> Alan Mackenzie (Nuremberg, Germany).

Hi Alan,

I don't have much to add as I share all Eli's and Stefan's opinions
here.

I as well don't agree with your critic of S.M. actions, but more
importantly, please assume always good faith from other Emacs developers
(as suggested here [1]).  Our goal here of just making a better Emacs,
this is shared by every one, and would be a pity to loose your
contribution.

Please reconsider your position, sometimes stress and other factors can
tweak our conclusion momentary, but let's not forget our goals.

Thanks

  Andrea

[1] <https://www.gnu.org/philosophy/kind-communication.html>

Re: My resignation from Emacs development

[Alfred M. Szmidt]

Making a better Emacs is making sure that e.g. Alan feels welcome and
happy to contribute, clearly that has failed completley.  That you,
Eli, and Stefan K hide behind "oh, we don't think there is anything
wrong, just take a break" and think it is just fine is beyond
bewildering.

Please take a step back (instead of quoting the GKC), and consider
what is going on currently is not working to the satisfaction of many
people, to the point of very much appreciated hackers like Alan.

Re: My resignation from Emacs development

[Christopher Dimech]

> Sent: Friday, November 22, 2024 at 7:01 AM
> From: "Alfred M. Szmidt" <ams@gnu.org>
> To: "Andrea Corallo" <acorallo@gnu.org>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Subject: Re: My resignation from Emacs development
>
> Making a better Emacs is making sure that e.g. Alan feels welcome and
> happy to contribute, clearly that has failed completley.  That you,
> Eli, and Stefan K hide behind "oh, we don't think there is anything
> wrong, just take a break" and think it is just fine is beyond
> bewildering.
>
> Please take a step back (instead of quoting the GKC), and consider
> what is going on currently is not working to the satisfaction of many
> people, to the point of very much appreciated hackers like Alan.

It is right to do so.  Still, working with emacs code needs to be natural,
with generic names always meaning the built-in default mode.  The rest
with precise names to know what one is using.  This should be a minor thing,
but is discussed as a major problem.  There is a similar problem with Latex-like
modes --- there is too much confusion on what is being actually used.  Regardless
of the name of the mode, emacs then opts for some other thing instead.  This
problem has been there for a few years now.

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: "Alfred M. Szmidt" <ams@gnu.org>
> Cc: acm@muc.de, emacs-devel@gnu.org
> Date: Thu, 21 Nov 2024 14:01:13 -0500
> 
> Making a better Emacs is making sure that e.g. Alan feels welcome and
> happy to contribute, clearly that has failed completley.

That is true, on both counts.

Btw, making a better Emacs is also about making sure that e.g. Stefan
Monnier (as well as anyone else) feels welcome and happy to
contribute.

> That you, Eli, and Stefan K hide behind "oh, we don't think there is
> anything wrong, just take a break" and think it is just fine is
> beyond bewildering.

We don't hide.  We are several people who independently came to very
similar conclusions in this matter, while Alan disagreed.  It's not
like we conspired to make Alan unhappy.  And no one said that nothing
is wrong, quite the contrary.

> Please take a step back (instead of quoting the GKC), and consider
> what is going on currently is not working to the satisfaction of many
> people, to the point of very much appreciated hackers like Alan.

I don't think you have a complete view of what happened.  You didn't
participate in the relevant discussions, neither back in May nor now.
Moreover, part of the discussion was in private email between Alan and
myself, so only we two know what was said there.  I assure you that
every effort was made to accommodate Alan's complaints, at least on
the technical level (since we are talking about bugs and how to fix
them).

That we failed is a fact, but it's not like we didn't try not to fail,
and tried hard and for a long time.  It is thus at least unfair to
accuse us in dismissing this or in being insincere.  However, I don't
claim to be exceptionally good at this, so if someone is better and
can convince Alan to stay, by all means please go ahead, and my hat's
off to you if you succeed.

In any case, if you are right, and "what is going on currently is not
working to the satisfaction of many people", then I'd appreciate if
those people voiced their concerns, so that we could at least try to
improve the situation.

Re: My resignation from Emacs development

[Jim Porter]

On 11/20/2024 7:13 AM, Alan Mackenzie wrote:
> I'm resigning my position as Emacs contributor.
> 
> The immediate reason is that, as maintainer of CC Mode, CC Mode's
> symbols, its names, were taken by Emacs and used for other purposes
> without informing me, much less consulting me.  That makes my position as
> CC Mode maintainer here untenable.

We haven't interacted much directly on the lists, but I'll be sorry to 
see you go. cc-mode is one of the most essential modes in Emacs for me, 
and without it I probably wouldn't use Emacs at all (and thus would 
never have become the current Eshell maintainer).

I won't get into the technical aspects of your message since I wasn't 
following the original discussions closely. However, I did want to say 
that we're all doing this for free (I think), and I've always felt that 
if it stops being fun, it's time for a break at minimum.

More generally, I do think there are important things for everyone to 
take from this. I'm confident that everyone here wants what's best for 
Emacs, but that's not only a technical question. Emacs development is a 
lot of work, and it's important to feel that your contributions (of any 
kind; not just code) are appreciated.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I agree with what you said.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Po Lu]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Emacs.
>
> I'm resigning my position as Emacs contributor.
>
> The immediate reason is that, as maintainer of CC Mode, CC Mode's
> symbols, its names, were taken by Emacs and used for other purposes
> without informing me, much less consulting me.  That makes my position as
> CC Mode maintainer here untenable.
>
> Eli Zaretskii and I have had extensive discussions, both in public and in
> private email, over the last week or so, but we have been unable to reach
> any satisfactory compromise solution.
>
> Names are important.  They have power.  To take somebody's/somthing's
> name and misuse it is an exercise of aggression.  Try using "Emacs" or
> even "free software" to mean something different, and see just how
> quickly you would hear back from Richard Stallman.  This misuse of CC
> Mode's "trademarks", the symbols `c-mode', `c++-mode', and perhaps
> `c-or-c++-mode', is just such an act of aggression.
>
> These symbols have been appropriated by Emacs to mean "the current
> preferred mode for C", etc., rather than C Mode, C++ Mode etc.  In
> certain circumstances, in particular, in Local Variables: sections and
> auto-mode-alist, there is now no longer any way unambiguously to specify
> C Mode or C++ Mode.  Up till recently ("\\.myc\\'" . c-mode) in
> auto-mode-alist meant C Mode, and would have had the effect of
> auto-loading CC Mode, if needed, and running C Mode.
>
> The change took place in the commit for bug#69191 "New var
> `major-mode-remap-defaults`, for packages".  It sounds so innocent, but
> is an extremely bad solution for whatever problem (unspecified in the
> commit message) it was intended to solve.  A major mode using it changes
> the interfaces of other libraries in an uncontrolled way.  This is not
> good software engineering.
>
> This bug was raised and committed by Stefan Monnier.  Despite the fact
> that the bug fix directly impinged upon CC Mode, and there was even a
> change to cc-mode.el in the patch, he failed even to inform me.  The only
> two modes substantially affected by this change were ruby-mode and CC
> Mode, and it is clear that Dmitry Gutov, maintainer of ruby-mode, was
> aware of the change.  Had I known of this proposal, I would certainly
> have objected to it.  Stefan is intelligent enough to have realised this,
> and maybe his avoidance of open discussion was motivated by this.
>
> Bug#69191 was a big change.  In Emacs, we have a convention whereby big
> changes are discussed openly on emacs-devel and a consensus reached
> before the change is made.  Stefan Monnier has regularly violated this
> convention, possibly believing that his ideas for Emacs are so good as to
> be beyond question.  Any attempt to question his ideas is likely to be
> met by evasive non-answers, if any response at all is forthcoming.  I
> could give several paragraphs worth of justification for these
> assertions, but I think everybody here knows I am right.
>
> In Emacs there is also a convention of treating eachother with respect on
> the mailing lists.  Sadly this convention is superficial, and seems only
> to mean things like not using swear words.  The truly contemptuous
> communication style, this evasive non-answering, seems to be regarded as
> acceptable.  I suggest that this change.
>
> Stefan's habit of making big changes in Emacs without seeking consensus
> is at the heart of why I am resigning.  These changes have caused Emacs a
> lot of damage over the years and have caused other contributors,
> including me, extra work and difficulty.  Stefan is a Jekyll-and-Hyde
> character.  On the one hand, he's a very capable hacker, and is always
> ready to help others with technical questions.  On the other hand, as
> mentioned, he is contemptuous of the Emacs conventions, and unlike
> Richard and Eli, does not have the gift of knowing what the Right Thing
> is.
>
> I strongly recommend that Stefan somehow be reigned in and required to
> observe Emacs's conventions about open discussion and courteous
> communication.  As I mentioned, his violations of these are at the core
> of why I feel unable to continue contributing to Emacs.
>
> I will shortly be unsubscribing from emacs-devel.  I intend to carry on
> maintaining stand alone CC Mode, and I'm prepared to deal with any CC
> Mode issues which arise in Emacs.  Please post these to
> bug-cc-mode@gnu.org.
>
> It just remains to say that my respect for Eli and the other maintainers
> remains undiminished, and that I wish all of them and the Emacs project
> all success in the future.

I trust that I speak for everyone in saying that we regret that it has
come to a departure on such bad terms, and would all of us prefer it if
you were to reconsider your decision.  I'm inclined to agree with your
observations concerning communication on this list.

On committing significant and/or controversial changes (was: My resignation from Emacs development)

[Ihor Radchenko]

Alan Mackenzie <acm@muc.de> writes:

> I'm resigning my position as Emacs contributor.
>
> The immediate reason is that, as maintainer of CC Mode, CC Mode's
> symbols, its names, were taken by Emacs and used for other purposes
> without informing me, much less consulting me.  That makes my position as
> CC Mode maintainer here untenable.
>
> Eli Zaretskii and I have had extensive discussions, both in public and in
> private email, over the last week or so, but we have been unable to reach
> any satisfactory compromise solution.
> ...

It looks like most of the discussion in the original thread shifted
towards personalities and specific example cases. I would like to create
a new thread that will exclude that part and instead focus on possible
constructive changes that might convince you Alan to re-consider the
resignation.

AFAIU, there are two main issues you are annoyed with:

1. Large changes are _sometimes_ committed without notice or discussion by
   Emacs maintainers.

2. When discussing controversial changes, Emacs maintainers _sometimes_
   make a judgment call and commit something without making it clear in
   the discussion thread that the decision has been made.

Would you re-consider if we somehow solve these issues?


Tentative proposal:

1. Make a rule that non-trivial changes and new features _must_ be
   announced on emacs-devel at least a month (or week?) in advance
   before committing them, and are only committed if there is no
   significant discussion or after the discussion is settled

   If no announcement is made, they are reverted (temporarily), the
   announcement is made, so that discussion has a chance to happen.

2. Make a rule that judgment calls are clearly indicated. If some change
   sparks controversy/discussion and maintainer has to choose among
   multiple solutions, such decision should be done in a separate,
   clearly marked email, with a link to commit.

WDYT?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: On committing significant and/or controversial changes (was: My resignation from Emacs development)

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 3017 bytes --]

Considering how cheap git branches are, I would add that contributors could
create a branch with their potentially controversial changes committed in
the branch for people to better appreciate vs. users speculatively applying
patches in their own private branches. I agree with your other suggestions.
I remain a relative Emacs contribution outsider, however, despite decades
of Emacs experience as a user, and as a developer in general. I greatly
appreciate the level of discussion among Emacs developers vs. what I
experience (despite encouragement) in the commercial and educational
universes.

-Stephane

On Fri, Nov 22, 2024 at 12:25 PM Ihor Radchenko <yantar92@posteo.net> wrote:

> Alan Mackenzie <acm@muc.de> writes:
>
> > I'm resigning my position as Emacs contributor.
> >
> > The immediate reason is that, as maintainer of CC Mode, CC Mode's
> > symbols, its names, were taken by Emacs and used for other purposes
> > without informing me, much less consulting me.  That makes my position as
> > CC Mode maintainer here untenable.
> >
> > Eli Zaretskii and I have had extensive discussions, both in public and in
> > private email, over the last week or so, but we have been unable to reach
> > any satisfactory compromise solution.
> > ...
>
> It looks like most of the discussion in the original thread shifted
> towards personalities and specific example cases. I would like to create
> a new thread that will exclude that part and instead focus on possible
> constructive changes that might convince you Alan to re-consider the
> resignation.
>
> AFAIU, there are two main issues you are annoyed with:
>
> 1. Large changes are _sometimes_ committed without notice or discussion by
>    Emacs maintainers.
>
> 2. When discussing controversial changes, Emacs maintainers _sometimes_
>    make a judgment call and commit something without making it clear in
>    the discussion thread that the decision has been made.
>
> Would you re-consider if we somehow solve these issues?
>
>
> Tentative proposal:
>
> 1. Make a rule that non-trivial changes and new features _must_ be
>    announced on emacs-devel at least a month (or week?) in advance
>    before committing them, and are only committed if there is no
>    significant discussion or after the discussion is settled
>
>    If no announcement is made, they are reverted (temporarily), the
>    announcement is made, so that discussion has a chance to happen.
>
> 2. Make a rule that judgment calls are clearly indicated. If some change
>    sparks controversy/discussion and maintainer has to choose among
>    multiple solutions, such decision should be done in a separate,
>    clearly marked email, with a link to commit.
>
> WDYT?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>
>

[-- Attachment #2: Type: text/html, Size: 3960 bytes --]

Re: On committing significant and/or controversial changes (was: My resignation from Emacs development)

[Eli Zaretskii]

> From: Ship Mints <shipmints@gmail.com>
> Date: Fri, 22 Nov 2024 12:47:22 -0500
> Cc: Alan Mackenzie <acm@muc.de>, emacs-devel@gnu.org
> 
> Considering how cheap git branches are, I would add that contributors could create a branch with their
> potentially controversial changes committed in the branch for people to better appreciate vs. users
> speculatively applying patches in their own private branches.

Branches are "cheap" to create, but are "expensive" because too many
active branches tend to increase the probability of mistakes when
people push changes to the wrong branch or merge from/to the wrong
branch.

Branches also make it a bit harder to track changes people install.

So I don't think I like this proposal for changing our procedures.

Re: On committing significant and/or controversial changes

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ship Mints <shipmints@gmail.com>
>> Date: Fri, 22 Nov 2024 12:47:22 -0500
>> Cc: Alan Mackenzie <acm@muc.de>, emacs-devel@gnu.org
>> 
>> Considering how cheap git branches are, I would add that contributors could create a branch with their
>> potentially controversial changes committed in the branch for people to better appreciate vs. users
>> speculatively applying patches in their own private branches.
>
> Branches are "cheap" to create, but are "expensive" because too many
> active branches tend to increase the probability of mistakes when
> people push changes to the wrong branch or merge from/to the wrong
> branch.

Rules on such branches should help for example for Glibc there are rules
for namespace specifically to avoid this kind of issue.[1]

These test branches exist to experiment, create commits such as fixup
commits that won't reflect the actual commits that would be merged after
the changes in said branch would be in a working state.


> Branches also make it a bit harder to track changes people install.

It depends on how they are organized. E.g. if topic branches are merged
into master which only contain extra commits related to that topic they
are easy to track. However if a topic branch contains merge commits
where they should have been rebased first before merging instead of
merging mater back then these can cause unnecessary noise in the commit
history.

> So I don't think I like this proposal for changing our procedures.

Change is hard but without change issues will potentially keep existing.
Making use of features that different systems offer can help to relieve
some of the friction and make communication of changes easier.

It would be best to take the things learned by people outside of the
personal bubble into account. Often they can show use things that we
have become blind to or would have never thought of in the first place.


[1] https://sourceware.org/glibc/wiki/GlibcGit#Branch_Name_Space_Conventions

Re: On committing significant and/or controversial changes

[Adam Porter]

Björn,

> Change is hard but without change issues will potentially keep existing.
> Making use of features that different systems offer can help to relieve
> some of the friction and make communication of changes easier.
> 
> It would be best to take the things learned by people outside of the
> personal bubble into account. Often they can show use things that we
> have become blind to or would have never thought of in the first place.

Your comments could be interpreted as condescending, so as to suggest 
that Eli and the Emacs maintainers were stubbornly ignorant of 
development practices outside of the GNU Emacs project.

The history of this project, recent and not, and of these maintainers' 
practices, shows clearly that that is not the case.  The Emacs software, 
as well as its development methodologies, have evolved steadily, 
sometimes at a pace that is criticized for being too fast.  As well, 
none of these maintainers are paid for their efforts, so they bring to 
this task the lessons they have learned in their professional work, 
which they have done for many years, and which is often on software very 
different from Emacs, and developed very differently from how Emacs is.

So to imply that the maintainers are not well versed in alternatives, 
and that their decisions for this project are made carelessly, borders 
on rudeness.  Eli patiently explained why these practices are used, and 
until you wrangle a project as large and varied as this one, you ought 
to present your suggestions more respectfully.

--Adam

Re: On committing significant and/or controversial changes

[Björn Bidar]

Adam Porter <adam@alphapapa.net> writes:

> Björn,
>
>> Change is hard but without change issues will potentially keep existing.
>> Making use of features that different systems offer can help to relieve
>> some of the friction and make communication of changes easier.
>> It would be best to take the things learned by people outside of the
>> personal bubble into account. Often they can show use things that we
>> have become blind to or would have never thought of in the first place.
>
> Your comments could be interpreted as condescending, so as to suggest
> that Eli and the Emacs maintainers were stubbornly ignorant of
> development practices outside of the GNU Emacs project.

Hm? I didn't write such a thing. I'm just saying sometimes someone
outside of ones own social bubble can bring viewpoints or influences
that we usually are not exposed to. Being stubborn is a
totally different thing.

> [...]
>
> So to imply that the maintainers are not well versed in alternatives,
> and that their decisions for this project are made carelessly, borders
> on rudeness.  Eli patiently explained why these practices are used,
> and until you wrangle a project as large and varied as this one, you
> ought to present your suggestions more respectfully.

You imply that I imply. I just suggest to use more features from
software such as git and look at FOSS projects outside of GNU for
improvements.
Implying something which I didn't write is also rude.

Re: On committing significant and/or controversial changes

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: Ship Mints <shipmints@gmail.com>,  yantar92@posteo.net,
>   emacs-devel@gnu.org
> Date: Sun, 24 Nov 2024 04:35:49 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Ship Mints <shipmints@gmail.com>
> >> Date: Fri, 22 Nov 2024 12:47:22 -0500
> >> Cc: Alan Mackenzie <acm@muc.de>, emacs-devel@gnu.org
> >> 
> >> Considering how cheap git branches are, I would add that contributors could create a branch with their
> >> potentially controversial changes committed in the branch for people to better appreciate vs. users
> >> speculatively applying patches in their own private branches.
> >
> > Branches are "cheap" to create, but are "expensive" because too many
> > active branches tend to increase the probability of mistakes when
> > people push changes to the wrong branch or merge from/to the wrong
> > branch.
> 
> Rules on such branches should help for example for Glibc there are rules
> for namespace specifically to avoid this kind of issue.[1]

We already have two namespaces: 'feature' and 'scratch'.  I don't
think we need more, but significantly, even these two get people
confused as to where to create branches in each case.  You can see
enough questions about that on the list to realize that this is a
non-trivial burden.

> These test branches exist to experiment, create commits such as fixup
> commits that won't reflect the actual commits that would be merged after
> the changes in said branch would be in a working state.

That's not the issue here; for code that is not ready for mainline we
already use branches.  In the context of this discussion, branches
were suggested as ways to install controversial changes.  I don't see
any significant advantages to that, only disadvantages.  See below.

> > Branches also make it a bit harder to track changes people install.
> 
> It depends on how they are organized.

Not for "tracking" I had in mind.  I usually review all the changes
that get installed into our Git repository, so that I could fix or
flag or start discussion for any of them that didn't do a clean job or
left some left-overs, or might be plain wrong.  More branches means
harder work for this purpose.

> E.g. if topic branches are merged
> into master which only contain extra commits related to that topic they
> are easy to track. However if a topic branch contains merge commits
> where they should have been rebased first before merging instead of
> merging mater back then these can cause unnecessary noise in the commit
> history.

The latter is indeed what happens in most cases with long-lived
feature branches.

> > So I don't think I like this proposal for changing our procedures.
> 
> Change is hard but without change issues will potentially keep existing.

We know that, right?  Which is why changes in this area must have
advantages that outweigh the disadvantages.  In this case, I don't see
any significant advantages, but I do see disadvantages.  One
significant disadvantage is that changes on a branch don't get tried
by too many people (how many of the people here are routinely
building, let alone using, the two feature branches we have now?), so
feedback is much harder to get and much slower to come in.  For
changes that might be controversial, this is from my POV a crucial
disadvantage, because decision-making in those cases must have
feedback.

> Making use of features that different systems offer can help to relieve
> some of the friction and make communication of changes easier.

I don't believe the friction can be relieved by these measures, no.
Branches in our Git repository are still considered "blessed" to a
degree, and don't really cause people with high emotions to cool down.
We had enough examples of that, so this is not theory, it's fact.
Branches for this purpose is a form of procrastination, and won't help
making the decision or relive the friction.

> It would be best to take the things learned by people outside of the
> personal bubble into account. Often they can show use things that we
> have become blind to or would have never thought of in the first place.

Indeed.  Which is why we do that all the time; the fact that there are
3 co-maintainers, each one with his specific experience and views, and
we consult among us when controversial issues arise before taking
actions, should tell you that at least the things learned by us,
directly and indirectly, are brought to bear in these situations.

Re: On committing significant and/or controversial changes

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Fri, 22 Nov 2024 17:26:17 +0000
> 
> 1. Make a rule that non-trivial changes and new features _must_ be
>    announced on emacs-devel at least a month (or week?) in advance
>    before committing them, and are only committed if there is no
>    significant discussion or after the discussion is settled
> 
>    If no announcement is made, they are reverted (temporarily), the
>    announcement is made, so that discussion has a chance to happen.

There's no need for a rule, since this is common sense.  All the
contributors are expected to behave like that, and if they don't, they
should expect post-factum objections, discussions, and the danger of
reverting the offending changes.

> 2. Make a rule that judgment calls are clearly indicated. If some change
>    sparks controversy/discussion and maintainer has to choose among
>    multiple solutions, such decision should be done in a separate,
>    clearly marked email, with a link to commit.

This is also common sense, and already followed.  Of course "clearly
indicated" is in the eyes of the beholder, so what is clearly
indicated" for me might not be so for others.

Bottom line: we already expect everyone to behave like that, and I see
no need to codify what common sense tells us.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Names are important.  They have power.  To take somebody's/somthing's
  > name and misuse it is an exercise of aggression.  Try using "Emacs" or
  > even "free software" to mean something different, and see just how
  > quickly you would hear back from Richard Stallman.

Is this really the case?  I am trying to figure out what concrete events
those descriptions could refer to, and I can't be sure.  Depending on
the details of the practcie in quetsion, I might say that it causes
a big problem.  Or I might not.

But if everyone involved is motivated by good will, I would not call
it "aggression".  That word asserts injustice and bad will.

I'm sure the people now disagreeing here all want to make Emacs as
good as can be, and that all are pursuing goals that are
indirividually desirable, even if they conflict in practice.

So let's see if we can find a compromise solution that gives the best
of both goals.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: My resignation from Emacs development

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 23 Nov 2024 01:10:30 -0500
> 
>   > Names are important.  They have power.  To take somebody's/somthing's
>   > name and misuse it is an exercise of aggression.  Try using "Emacs" or
>   > even "free software" to mean something different, and see just how
>   > quickly you would hear back from Richard Stallman.
> 
> Is this really the case?  I am trying to figure out what concrete events
> those descriptions could refer to, and I can't be sure.

If I may summarize those events and the technical aspects of this,
they are as follows:

 . Emacs 29 introduced a new user option called
   major-mode-remap-alist.  It allows to "remap" one major mode into
   another, and is obeyed by normal-mode, so if FOO-mode is remapped
   to BAR-mode, then any time FOO-mode is found to be appropriate for
   a buffer, whether by virtue of auto-mode-alist, or file-local
   variables, or anything else, normal-mode would actually activate
   BAR-mode instead.
 . When we added to Emacs major modes based on the tree-sitter library
   for programming languages for which we already had "traditional"
   major modes, this mode remapping was suggested as the means for
   users to express their preference of using the tree-sitter modes
   (which by default are disabled, as auto-mode-alist still names the
   traditional modes for the respective file types, to preserve past
   behavior and avoid surprising users).  I disagreed with using this
   feature, because the value of major-mode-remap-alist is not simple
   enough for users to customize.  Instead, we arranged for the
   tree-sitter modes to modify auto-mode-alist when the mode is
   loaded, and told users that by loading the mode manually, they will
   have their preference be known to Emacs.
 . Changing global data structure, such as auto-mode-alist, when a
   Lisp package is merely loaded is not the best idea, and has some
   negative consequences.  For example, byte-compiling a package in a
   running Emacs session loads that package, and thus might modify
   auto-mode-alist when the user didn't really mean that.  And getting
   auto-mode-alist back to its previous value is not easy, unless you
   are a Lisp programmer.  Moreover, modifying auto-mode-alist doesn't
   affect visiting files that have a mode cookie or set their
   major-mode in the file-local variables.  So this solution had known
   issues, but was still used for its simplicity in the typical use
   case, where the user starts a session and wants to switch to using
   a certain tree-sitter mode from that moment onwards.
 . In March 2024 Stefan Monnier installed a change for Emacs 30 that
   introduced a new variable, major-mode-remap-defaults.  This
   variable is consulted by normal-mode if major-mode-remap-alist
   (which is a user option, and thus can be changed only by users)
   doesn't have any associations for a mode, and if
   major-mode-remap-defaults does have such an association, the mode
   is remapped as described above.  Stefan then used this new variable
   to replace the code which modified auto-mode-alist when a
   tree-sitter mode was loaded.  Specifically, when c-ts-mode is
   loaded, it adds associations to major-mode-remap-defaults that
   remap c-mode and c++-mode to their tree-sitter counterparts; and
   when ruby-ts-mode is loaded, it adds associations that remap
   ruby-mode to its tree-sitter counterpart.  The main motivation for
   these changes was that they make the process of preferring a
   tree-sitter mode by the user cleaner, by not messing with
   auto-mode-alist, and by making sure normal-mode will _always_ use
   the preferred mode.  This change was discussed (as part of
   bug#69191), but it should have been discussed more, and in
   particular its effects on normal-mode and its callers should have
   been explicitly described and discussed.
 . In May 2024 Alan Mackenzie installed in cc-mode.el a change that
   countermanded the use of major-mode-remap-defaults for remapping
   c-mode and c++-mode to the tree-sitter counterparts.  The change he
   installed defeated the code in c-ts-mode which added associations
   to major-mode-remap-defaults.  The result was that loading
   c-ts-mode no longer had the effect of using c-ts-mode for C files,
   contrary to the documentation which said it would.  The exact way
   this change worked was never described anywhere but the commit log
   message, although the change itself was announced in a message
   posted by Alan to emacs-devel
   (https://lists.gnu.org/archive/html/emacs-devel/2024-05/msg01294.html).
   Needless to say, the adverse effects of that change on people who
   prefer using c-ts-mode went unknown until Emacs 30 went into
   pretest.  When asked (in bug#74339) to fix that code which allowed
   the documented procedure for users to express their preference for
   c-ts-mode to work, Alan said he will not agree to such a change,
   and proposed other ways of solving this, which would go back to
   using auto-mode-alist (something which we already decided we want
   to abandon in favor of better ways) and would delay the release of
   Emacs 30.1 due to non-trivial changes in data structures and code
   central to Emacs as a whole, not just to these two modes.

I will leave it to Alan to describe why he thinks the mode-remapping
feature is wrong when it affects c-mode, and/or why it is wrong to do
that via major-mode-remap-defaults.  From my POV, mode-remapping is
just a convenience feature which allows users to substitute one mode
for another without a lot of modifications in auto-mode-alist and
similar data strictures used by normal-mode, editing all the files
which have the mode cookie or specify the mode in file-local
variables, etc.  I don't see any problem with the user saying that the
symbol c-mode should be interpreted as meaning some other mode should
be invoked when normal-mode is called with that symbol as its first
argument.

Re: My resignation from Emacs development

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Names are important.  They have power.  To take somebody's/somthing's
  > name and misuse it is an exercise of aggression.  Try using "Emacs" or
  > even "free software" to mean something different, and see just how
  > quickly you would hear back from Richard Stallman.

Is this really the case?  I am trying to figure out what concrete events
those descriptions could refer to, and I can't be sure.  Depending on
the details of the practcie in quetsion, I might say that it causes
a big problem.  Or I might not.

But if everyone involved is motivated by good will, I would not call
it "aggression".  That word asserts injustice and intent to be nasty.
We may have hurt feelings but I don't think anyone intended to hurt.

I'm sure the people now disagreeing here all want to make Emacs as
good as can be, and that all are pursuing goals that are
indirividually desirable, even if they conflict in practice.

I can't tell exactly what the dispute is about, but I get
the general idea that, due to new features, there are now
two natural meanings for -*-c-*- in a file mode line:

* Use one specific mode, C mode.

* Use the user's stated premerence among a few possible modes for C.

Is that it?  If that is not entirely correct, please correct me.

Also, has such a change affected any other language modes likewise?

If it is something like that, I think we can find a compromise
solution that does a pretty good job of giving users most of all the
goals, if we try to be flecible while keeping the user goals in mind.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)