bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Jacob Faibussowitsch]

[-- Attachment #1: Type: text/plain, Size: 149 bytes --]

C++ ts-mode is missing “alignof” as a builtin keyword like sizeof.

Best Regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)


[-- Attachment #2: Type: text/html, Size: 734 bytes --]

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Yuan Fu]

> On Dec 31, 2024, at 8:01 AM, Jacob Faibussowitsch <jacob.fai@gmail.com> wrote:
> 
> C++ ts-mode is missing “alignof” as a builtin keyword like sizeof.
> 
> Best Regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> 

Thanks for reporting this. It seems tree-sitter-c/cpp supports '__alignof__', '__alignof', '_alignof', 'alignof', ‘_Alignof’, I can add all of them as keywords. The feature is added in tree-sitter-c in Aug 9, 2023 [1], and appears in v0.20.5.

Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?

[1] https://github.com/tree-sitter/tree-sitter-c/commit/8e919ab298b6e4d05ee29b10482de4fcfb075074

Yuan

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Stefan Kangas]

Yuan Fu <casouri@gmail.com> writes:

> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30
> and have this change queued for Emacs 30.2? Or should I wait a bit?

I'm not Eli, but to have this in Emacs 30.2, you need to wait until
Emacs 30.1 is released first.

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Eli Zaretskii]

> Cc: 75226@debbugs.gnu.org
> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 1 Jan 2025 19:28:35 -0800
> 
> 
> 
> > On Dec 31, 2024, at 8:01 AM, Jacob Faibussowitsch <jacob.fai@gmail.com> wrote:
> > 
> > C++ ts-mode is missing “alignof” as a builtin keyword like sizeof.
> > 
> > Best Regards,
> > 
> > Jacob Faibussowitsch
> > (Jacob Fai - booss - oh - vitch)
> > 
> 
> Thanks for reporting this. It seems tree-sitter-c/cpp supports '__alignof__', '__alignof', '_alignof', 'alignof', ‘_Alignof’, I can add all of them as keywords. The feature is added in tree-sitter-c in Aug 9, 2023 [1], and appears in v0.20.5.
> 
> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?

If this just adds a bunch of strings to c-ts-mode--keywords, it's okay
to add them on emacs-30.

Thanks.

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Jacob Faibussowitsch]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]

Should also do `alignas` and friends while you’re at it. I noticed it is missing from `c-ts-mode--keywords` as well.

Best Regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)

> On Jan 2, 2025, at 1:41 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> Cc: 75226@debbugs.gnu.org
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 1 Jan 2025 19:28:35 -0800
>> 
>> 
>> 
>>> On Dec 31, 2024, at 8:01 AM, Jacob Faibussowitsch <jacob.fai@gmail.com> wrote:
>>> 
>>> C++ ts-mode is missing “alignof” as a builtin keyword like sizeof.
>>> 
>>> Best Regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> 
>> 
>> Thanks for reporting this. It seems tree-sitter-c/cpp supports '__alignof__', '__alignof', '_alignof', 'alignof', ‘_Alignof’, I can add all of them as keywords. The feature is added in tree-sitter-c in Aug 9, 2023 [1], and appears in v0.20.5.
>> 
>> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?
> 
> If this just adds a bunch of strings to c-ts-mode--keywords, it's okay
> to add them on emacs-30.
> 
> Thanks.


[-- Attachment #2: Type: text/html, Size: 4742 bytes --]

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Eli Zaretskii]

> From: Jacob Faibussowitsch <jacob.fai@gmail.com>
> Date: Thu, 2 Jan 2025 09:03:26 -0500
> Cc: Yuan Fu <casouri@gmail.com>,
>  75226@debbugs.gnu.org
> 
> Should also do `alignas` and friends while you’re at it. I noticed it is missing from `c-ts-mode--keywords` as
> well.

The keywords are all explicit in the grammar's JS source, so how about
check them all, while we are at it?

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Yuan Fu]

> On Jan 1, 2025, at 10:41 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> Cc: 75226@debbugs.gnu.org
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 1 Jan 2025 19:28:35 -0800
>> 
>> 
>> 
>>> On Dec 31, 2024, at 8:01 AM, Jacob Faibussowitsch <jacob.fai@gmail.com> wrote:
>>> 
>>> C++ ts-mode is missing “alignof” as a builtin keyword like sizeof.
>>> 
>>> Best Regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> 
>> 
>> Thanks for reporting this. It seems tree-sitter-c/cpp supports '__alignof__', '__alignof', '_alignof', 'alignof', ‘_Alignof’, I can add all of them as keywords. The feature is added in tree-sitter-c in Aug 9, 2023 [1], and appears in v0.20.5.
>> 
>> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?
> 
> If this just adds a bunch of strings to c-ts-mode--keywords, it's okay
> to add them on emacs-30.

Not quite, right now emacs-30’s c-ts-mode works with grammar as far back as v0.19.0 [1]. If we just add the keywords, c-ts-mode wouldn’t work with grammar v0.20.4 and earlier. So I’m going to add the new keywords with a check, which disables the new keywords if grammar doesn’t recognize them. Also there’re many missing keywords, none of which I have seen before :-), I need time to go over them and see how are they supposed by be fontified (keyword face or something else).

So let’s wait for 30.2.

[1] It might work with even earlier versions but I didn’t test, v0.19.0 (4 years ago) is the earliest version that still has compatible LANGUAGE_VERSION (13) with the latest tree-sitter lib.

Yuan

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 3 Jan 2025 00:50:46 -0800
> Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
>  75226@debbugs.gnu.org
> 
> > On Jan 1, 2025, at 10:41 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?
> > 
> > If this just adds a bunch of strings to c-ts-mode--keywords, it's okay
> > to add them on emacs-30.
> 
> Not quite, right now emacs-30’s c-ts-mode works with grammar as far back as v0.19.0 [1]. If we just add the keywords, c-ts-mode wouldn’t work with grammar v0.20.4 and earlier. So I’m going to add the new keywords with a check, which disables the new keywords if grammar doesn’t recognize them. Also there’re many missing keywords, none of which I have seen before :-), I need time to go over them and see how are they supposed by be fontified (keyword face or something else).
> 
> So let’s wait for 30.2.

If so, please install on master, and let's consider backporting it to
the emacs-30 branch after 30.1 is released.

Thanks.

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Yuan Fu]

> On Jan 3, 2025, at 3:38 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Fri, 3 Jan 2025 00:50:46 -0800
>> Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
>> 75226@debbugs.gnu.org
>> 
>>> On Jan 1, 2025, at 10:41 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?
>>> 
>>> If this just adds a bunch of strings to c-ts-mode--keywords, it's okay
>>> to add them on emacs-30.
>> 
>> Not quite, right now emacs-30’s c-ts-mode works with grammar as far back as v0.19.0 [1]. If we just add the keywords, c-ts-mode wouldn’t work with grammar v0.20.4 and earlier. So I’m going to add the new keywords with a check, which disables the new keywords if grammar doesn’t recognize them. Also there’re many missing keywords, none of which I have seen before :-), I need time to go over them and see how are they supposed by be fontified (keyword face or something else).
>> 
>> So let’s wait for 30.2.
> 
> If so, please install on master, and let's consider backporting it to
> the emacs-30 branch after 30.1 is released.
> 
> Thanks.

Sounds good.

Yuan

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Yuan Fu]

> On Jan 3, 2025, at 11:34 PM, Yuan Fu <casouri@gmail.com> wrote:
> 
> 
> 
>> On Jan 3, 2025, at 3:38 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>>> From: Yuan Fu <casouri@gmail.com>
>>> Date: Fri, 3 Jan 2025 00:50:46 -0800
>>> Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
>>> 75226@debbugs.gnu.org
>>> 
>>>> On Jan 1, 2025, at 10:41 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>>>> 
>>>>> Eli, what’s the status of Emacs 30 right now? Can I push to emacs-30 and have this change queued for Emacs 30.2? Or should I wait a bit?
>>>> 
>>>> If this just adds a bunch of strings to c-ts-mode--keywords, it's okay
>>>> to add them on emacs-30.
>>> 
>>> Not quite, right now emacs-30’s c-ts-mode works with grammar as far back as v0.19.0 [1]. If we just add the keywords, c-ts-mode wouldn’t work with grammar v0.20.4 and earlier. So I’m going to add the new keywords with a check, which disables the new keywords if grammar doesn’t recognize them. Also there’re many missing keywords, none of which I have seen before :-), I need time to go over them and see how are they supposed by be fontified (keyword face or something else).
>>> 
>>> So let’s wait for 30.2.
>> 
>> If so, please install on master, and let's consider backporting it to
>> the emacs-30 branch after 30.1 is released.
>> 
>> Thanks.
> 
> Sounds good.
> 
> Yuan

I found that both c++-mode and c++-ts-mode fontifies C++ alternative operators like and_eq, xor, etc as keywords. Maybe we should fontify them in operator face (added in Emacs 29) instead?

Yuan

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Tue, 7 Jan 2025 19:09:40 -0800
> Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
>  75226@debbugs.gnu.org
> 
> I found that both c++-mode and c++-ts-mode fontifies C++ alternative operators like and_eq, xor, etc as keywords. Maybe we should fontify them in operator face (added in Emacs 29) instead?

What do other editors do?

Alan, WDYT about this?

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Jacob Faibussowitsch]

[-- Attachment #1: Type: text/plain, Size: 1066 bytes --]

A good rule of thumb for this could be: if you can define an “operator” overload for something, then it is an operator and should be fontified as such.

Some examples:

1. `&&` or `and`, can define `operator &&()` or `operator and()` -> operator face.
2. `and_eq` or `&=`, can operator &=()` or `operator and_eq()` -> operator face.
3. `for`, cannot define `operator for()` -> keyword face.
4. `while`, cannot define `operator while()` -> keyword face. 

Just my pair of pennies...

Best Regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)

> On Jan 8, 2025, at 7:49 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Tue, 7 Jan 2025 19:09:40 -0800
>> Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
>> 75226@debbugs.gnu.org
>> 
>> I found that both c++-mode and c++-ts-mode fontifies C++ alternative operators like and_eq, xor, etc as keywords. Maybe we should fontify them in operator face (added in Emacs 29) instead?
> 
> What do other editors do?
> 
> Alan, WDYT about this?


[-- Attachment #2: Type: text/html, Size: 1983 bytes --]

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Alan Mackenzie]

Hello, Eli and Jacob.

On Wed, Jan 08, 2025 at 14:49:50 +0200, Eli Zaretskii wrote:
> > From: Yuan Fu <casouri@gmail.com>
> > Date: Tue, 7 Jan 2025 19:09:40 -0800
> > Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
> >  75226@debbugs.gnu.org

> > I found that both c++-mode and c++-ts-mode fontifies C++ alternative
> > operators like and_eq, xor, etc as keywords. Maybe we should fontify
> > them in operator face (added in Emacs 29) instead?

> What do other editors do?

> Alan, WDYT about this?

For me personally, another face for "alphabetic operators" would
come dangerously close to angry fruit salad.

and_eq, xor, etc. _are_ keywords, so I think it makes sense to fontify
them as such.  The keywords in C++ could be divided into several
categories, including alphabetic operators, and to give each category
its own face would, I think, overload the typical user, who wouldn't
instantly grasp what all the faces meant.  This would be
counterproductive.  I don't think it makes too much sense to single out
just one category of keywords for special treatment.

Currently, C++ Mode has just two faces for these things:
font-lock-keyword-face for alphabetic ones, and the default-face for
those consisting of symbol characters like =, +, ;, .....  This is
adequate, probably, for someone whose native language uses a Latin
script.  Maybe users of other languages see things differently.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Dmitry Gutov]

On 08/01/2025 16:49, Alan Mackenzie wrote:
>>> I found that both c++-mode and c++-ts-mode fontifies C++ alternative
>>> operators like and_eq, xor, etc as keywords. Maybe we should fontify
>>> them in operator face (added in Emacs 29) instead?
>> What do other editors do?
>> Alan, WDYT about this?
> For me personally, another face for "alphabetic operators" would
> come dangerously close to angry fruit salad.

One of the benefits of the current setup is that font-lock-operator-face 
by default just inherits the attributes from the 'default' face.

But custom themes (and individual users) can add distinction by 
customizing this face.

bug#75226: [31.0.50] C++ ts-mode missing "alignof" as keyword

[Yuan Fu]

> On Jan 8, 2025, at 6:49 AM, Alan Mackenzie <acm@muc.de> wrote:
> 
> Hello, Eli and Jacob.
> 
> On Wed, Jan 08, 2025 at 14:49:50 +0200, Eli Zaretskii wrote:
>>> From: Yuan Fu <casouri@gmail.com>
>>> Date: Tue, 7 Jan 2025 19:09:40 -0800
>>> Cc: Jacob Faibussowitsch <jacob.fai@gmail.com>,
>>> 75226@debbugs.gnu.org
> 
>>> I found that both c++-mode and c++-ts-mode fontifies C++ alternative
>>> operators like and_eq, xor, etc as keywords. Maybe we should fontify
>>> them in operator face (added in Emacs 29) instead?
> 
>> What do other editors do?
> 
>> Alan, WDYT about this?
> 
> For me personally, another face for "alphabetic operators" would
> come dangerously close to angry fruit salad.

As Dmitry mentioned, this face inherits from default, so unless the user wants to, it doesn’t have color. And some user do like angry fruit salad, from what I learned.

> and_eq, xor, etc. _are_ keywords, so I think it makes sense to fontify
> them as such.  The keywords in C++ could be divided into several
> categories, including alphabetic operators, and to give each category
> its own face would, I think, overload the typical user, who wouldn't
> instantly grasp what all the faces meant.  This would be
> counterproductive.  I don't think it makes too much sense to single out
> just one category of keywords for special treatment.
> 
> Currently, C++ Mode has just two faces for these things:
> font-lock-keyword-face for alphabetic ones, and the default-face for
> those consisting of symbol characters like =, +, ;, .....  This is
> adequate, probably, for someone whose native language uses a Latin
> script.  Maybe users of other languages see things differently.

Thinking about it, I agree that we should use keyword face for and_eq and friends, and don’t fontify +/-/etc by default. In the end of the day, it’s about feelings; and for programmers, it feels right to highlight and_eq in keyword face and not highlight symbol operators; even thought technically and_eq and &= are both operators.

I’ll clean up my patch and push it later.

Yuan