bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

c-ts-mode cannot fontify the following code:

ifsubmitbufl (tp)
{
  register	*kb1 = tp;
  extern	 tbase, i_len;
  register	 l = len;

again:
  
  highprio (); /* tbase is used by protoureg */
  iomap (&UBR, tbase += UBR_SIZ);
  UBR->addr_l = kb1;
  UBR->addr_m = kb1 >> 010;
  UBR->mode_l = 07777 &~ UBR_INST;
  UBR->mode_m = 0;
  UBR->long_l = l;
  UBR->long_m = l >> 010;

  iounmap ();
  tally_iostat (tbase);
  iospoke (UBR, P_LOCK);
  resetprio ();

  SWITCH (S_WAIT|S_HIGH|S_IMMED, 0700, again); /* this returns to
						  BR->senv[0], except
						  if CL_HI */
}

here, ifsubmitbufl, tbase, l and again are fontified as types.  They are
fontified correctly by CC Mode, as three identifiers and one label.

Trying to indent after "again:" results in text being indented to column
0.

In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu) of 2022-11-21 built
 on trinity
Repository revision: c1a95056214d9bf18bc2c34e938205f1da916bba
Repository branch: feature/tree-sitter
Windowing system distributor 'The X.Org Foundation', version 11.0.12101099
System Description: Fedora Linux 37 (Workstation Edition)

Configured using:
 'configure --with-tree-sitter --with-x-toolkit=no --without-cairo
 --with-dumping=unexec'

Configured features:
ACL DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON LCMS2
LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY OLDXMENU PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TREE_SITTER UNEXEC WEBP X11 XDBE XFT
XIM XINPUT2 XPM ZLIB

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Yuan Fu]

> On Nov 20, 2022, at 10:09 PM, Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org> wrote:
> 
> 
> c-ts-mode cannot fontify the following code:
> 
> ifsubmitbufl (tp)
> {
>  register	*kb1 = tp;
>  extern	 tbase, i_len;
>  register	 l = len;
> 
> again:
> 
>  highprio (); /* tbase is used by protoureg */
>  iomap (&UBR, tbase += UBR_SIZ);
>  UBR->addr_l = kb1;
>  UBR->addr_m = kb1 >> 010;
>  UBR->mode_l = 07777 &~ UBR_INST;
>  UBR->mode_m = 0;
>  UBR->long_l = l;
>  UBR->long_m = l >> 010;
> 
>  iounmap ();
>  tally_iostat (tbase);
>  iospoke (UBR, P_LOCK);
>  resetprio ();
> 
>  SWITCH (S_WAIT|S_HIGH|S_IMMED, 0700, again); /* this returns to
> 						  BR->senv[0], except
> 						  if CL_HI */
> }
> 
> here, ifsubmitbufl, tbase, l and again are fontified as types.  They are
> fontified correctly by CC Mode, as three identifiers and one label.

Thanks, I fixe the label’s face. As for ifsubmitbufl, tbase, and l, I don’t really understand the syntax here, and I think neither does tree-sitter . In general tree-sitter struggles with nonstandard (in its book anyway) syntax. And there aren’t much we can do (except for maybe improve tree-sitter-c’s parser).

> 
> Trying to indent after "again:" results in text being indented to column
> 0.

Seems to work fine here. I don’t see anything I changed recently that could affect this, either.

Yuan

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Yuan Fu <casouri@gmail.com> writes:

>> On Nov 20, 2022, at 10:09 PM, Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org> wrote:
>> 
>> 
>> c-ts-mode cannot fontify the following code:
>> 
>> ifsubmitbufl (tp)
>> {
>>  register	*kb1 = tp;
>>  extern	 tbase, i_len;
>>  register	 l = len;
>> 
>> again:
>> 
>>  highprio (); /* tbase is used by protoureg */
>>  iomap (&UBR, tbase += UBR_SIZ);
>>  UBR->addr_l = kb1;
>>  UBR->addr_m = kb1 >> 010;
>>  UBR->mode_l = 07777 &~ UBR_INST;
>>  UBR->mode_m = 0;
>>  UBR->long_l = l;
>>  UBR->long_m = l >> 010;
>> 
>>  iounmap ();
>>  tally_iostat (tbase);
>>  iospoke (UBR, P_LOCK);
>>  resetprio ();
>> 
>>  SWITCH (S_WAIT|S_HIGH|S_IMMED, 0700, again); /* this returns to
>> 						  BR->senv[0], except
>> 						  if CL_HI */
>> }
>> 
>> here, ifsubmitbufl, tbase, l and again are fontified as types.  They are
>> fontified correctly by CC Mode, as three identifiers and one label.
>
> Thanks, I fixe the label’s face.

Thanks.

> As for ifsubmitbufl, tbase, and l, I don’t really understand the
> syntax here, and I think neither does tree-sitter . In general
> tree-sitter struggles with nonstandard (in its book anyway) syntax.

Well, that code is 100% valid C.  "*kb1, tbase, i_len, l" are implicitly
int (which on that machine is 16 bits wide), and so is the value of
"ifsubmitbufl" and "tp".

> And there aren’t much we can do (except for maybe improve
> tree-sitter-c’s parser).

Would someone try that?

> Seems to work fine here. I don’t see anything I changed recently that
> could affect this, either.

Sorry, I think I meant "before".  Place point after "l = len;", then
press RET.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Eli Zaretskii]

> Cc: 59435@debbugs.gnu.org
> Date: Mon, 21 Nov 2022 19:57:08 +0800
> From:  Po Lu via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> > As for ifsubmitbufl, tbase, and l, I don’t really understand the
> > syntax here, and I think neither does tree-sitter . In general
> > tree-sitter struggles with nonstandard (in its book anyway) syntax.
> 
> Well, that code is 100% valid C.

It's K&R C, which is used very rarely these days, and it sounds like the
next version of GCC will reject it, since C2x makes it invalid.

So I see no catastrophe that c-ts-mode doesn't support this.  You can always
opt not to use c-ts-mode if you need this style supported.  Of course, if
tree-sitter's parser is updated to add such support, we'll all be happier.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's K&R C, which is used very rarely these days, and it sounds like the
  > next version of GCC will reject it, since C2x makes it invalid.

The GNU policy is to continue supporting old forms of syntax for many
years after the latest version of a language specification has
deprecated it.  Old programs are sometimes very important.  We should
refuse to aid the crusades to compel people to "upgrade".

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Richard Stallman <rms@gnu.org> writes:

> The GNU policy is to continue supporting old forms of syntax for many
> years after the latest version of a language specification has
> deprecated it.  Old programs are sometimes very important.  We should
> refuse to aid the crusades to compel people to "upgrade".

Well, it seems that one such crusade is already taking place.  The
crusade will even affect software like Autoconf, especially the
countless configure scripts that have already been generated and are
being included with software:

  https://gcc.gnu.org/pipermail/gcc/2022-November/239883.html
  https://lwn.net/Articles/913505/

Looking at the comments in the latter article, you can see that the
crusade has already been demonstrated to break many pieces of software:
SDL, zsh, just to name a few.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I will look at those articles.
Is there any option to tell GCC to accept older syntax?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Richard Stallman <rms@gnu.org> writes:

> Is there any option to tell GCC to accept older syntax?

Yes.  But Autoconf does not know about those options, so generated
configure scripts that use the old syntax in tests will not work right.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes.  But Autoconf does not know about those options, so generated
  > configure scripts that use the old syntax in tests will not work right.

In principle it should not be hard to fix Autoconf to do this, but it does
not have a real maintainer to do it.

Would anyone like to do easy things on Autoconf now?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Richard Stallman <rms@gnu.org> writes:

> In principle it should not be hard to fix Autoconf to do this, but it does
> not have a real maintainer to do it.
>
> Would anyone like to do easy things on Autoconf now?

Does anyone have a list of those easy things?

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Yuan Fu]

> On Nov 24, 2022, at 3:38 PM, Richard Stallman <rms@gnu.org> wrote:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>> It's K&R C, which is used very rarely these days, and it sounds like the
>> next version of GCC will reject it, since C2x makes it invalid.
> 
> The GNU policy is to continue supporting old forms of syntax for many
> years after the latest version of a language specification has
> deprecated it.  Old programs are sometimes very important.  We should
> refuse to aid the crusades to compel people to "upgrade”.

If I can support them,I definitely would. It’s just that they’ll have a lower priority when I decide where to put my limited resources on. And we are definitely not actively removing any support.

Yuan

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: luangruo@yahoo.com, 59435@debbugs.gnu.org, casouri@gmail.com
> Date: Thu, 24 Nov 2022 18:38:30 -0500
> 
>   > It's K&R C, which is used very rarely these days, and it sounds like the
>   > next version of GCC will reject it, since C2x makes it invalid.
> 
> The GNU policy is to continue supporting old forms of syntax for many
> years after the latest version of a language specification has
> deprecated it.  Old programs are sometimes very important.  We should
> refuse to aid the crusades to compel people to "upgrade".

My point was that such code is extremely rare nowadays (or at least is
extremely rarely _edited_), and thus failure to fontify and/or indent it
correctly isn't a catastrophe IMO.  Unless you disagree with that opinion,
what you say is a tangent for this discussion (although I agree with what
you say regardless).

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > My point was that such code is extremely rare nowadays (or at least is
  > extremely rarely _edited_), and thus failure to fontify and/or indent it
  > correctly isn't a catastrophe IMO.

I agree with you on that point.

But it's a good thing it led to informing about that "crusade".
I'll talk with people about it off this list.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Florian Weimer]

* Eli Zaretskii:

>> Cc: 59435@debbugs.gnu.org
>> Date: Mon, 21 Nov 2022 19:57:08 +0800
>> From:  Po Lu via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> > As for ifsubmitbufl, tbase, and l, I don’t really understand the
>> > syntax here, and I think neither does tree-sitter . In general
>> > tree-sitter struggles with nonstandard (in its book anyway) syntax.
>> 
>> Well, that code is 100% valid C.
>
> It's K&R C, which is used very rarely these days, and it sounds like the
> next version of GCC will reject it, since C2x makes it invalid.

Some clarification:

Implicit ints were removed from the language in 1999.  GCC 14 (to be
released in 2024) will likely no longer accept them by default, along
with implicit function declarations (also removed in 1999).  But you can
still get them back using -std=gnu89, and there are no plans to remove
that.

Old-style function definitions will finally be removed in C2X (which
will probably be called C23), but I've been told that it will be several
years (but probably not anything close to 25) until GCC switches to
-std=gnu23 (or whatever the year will be the year of the standard in the
end).  Function declarations which are not a prototype—void foo();—will
change meaning and denote a function with an empty parameter list, same
as today: foo(void);.

Old-style function declarations are still in C18, and so are function
declarations without a prototype.

Thanks,
Florian

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Florian Weimer <fweimer@redhat.com> writes:

> Some clarification:
>
> Implicit ints were removed from the language in 1999.  GCC 14 (to be
> released in 2024) will likely no longer accept them by default, along
> with implicit function declarations (also removed in 1999).  But you can
> still get them back using -std=gnu89, and there are no plans to remove
> that.

What about -std=gnu99? Is there a way to get them back there?

> Old-style function definitions will finally be removed in C2X (which
> will probably be called C23), but I've been told that it will be several
> years (but probably not anything close to 25) until GCC switches to
> -std=gnu23 (or whatever the year will be the year of the standard in the
> end).  Function declarations which are not a prototype—void foo();—will
> change meaning and denote a function with an empty parameter list, same
> as today: foo(void);.

No more:

  int (*pFillSpans) ();

  (*pFillSpans) (pDrawable, pGc, nInit, pptInit, pwidthInit, fSorted)

?

Will there be an option to get that back in gnu23?

> Old-style function declarations are still in C18, and so are function
> declarations without a prototype.

Thanks.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Florian Weimer]

* Po Lu:

> Florian Weimer <fweimer@redhat.com> writes:
>
>> Some clarification:
>>
>> Implicit ints were removed from the language in 1999.  GCC 14 (to be
>> released in 2024) will likely no longer accept them by default, along
>> with implicit function declarations (also removed in 1999).  But you can
>> still get them back using -std=gnu89, and there are no plans to remove
>> that.
>
> What about -std=gnu99? Is there a way to get them back there?

It's odd to ask for a C99 mode explicitly and depend on a feature that
was removed from C99.  If you write C89 code, you should probably use
-std=gnu89.

On the other hand, the largest benefit will come from change the
default.  From that perspective, it won't be necessary to change the
-std=gnu99 behavior.

>> Old-style function definitions will finally be removed in C2X (which
>> will probably be called C23), but I've been told that it will be several
>> years (but probably not anything close to 25) until GCC switches to
>> -std=gnu23 (or whatever the year will be the year of the standard in the
>> end).  Function declarations which are not a prototype—void foo();—will
>> change meaning and denote a function with an empty parameter list, same
>> as today: foo(void);.
>
> No more:
>
>   int (*pFillSpans) ();
>
>   (*pFillSpans) (pDrawable, pGc, nInit, pptInit, pwidthInit, fSorted)
>
> ?

Right, it's an argument list mismatch.

> Will there be an option to get that back in gnu23?

I'm not the C frontend maintainer.  It seems unlikely that this is going
to be supported because it's one of the major C23 changes.

Before -std=gnu23 becomes the default, GCC will probably start warning
about calling ()-declared functions with extra arguments, to help with
porting to a future -std=gnu32-by-default change.  -Wstrict-prototypes
is not a good proxy for this because it will warn about ()-style
declarations even in contexts where C23 will align with what the
programmer intended (no arguments accepted).

Thanks,
Florian

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Florian Weimer <fweimer@redhat.com> writes:

> It's odd to ask for a C99 mode explicitly and depend on a feature that
> was removed from C99.  If you write C89 code, you should probably use
> -std=gnu89.

Well, it may not have been in C99, but it was in "GNU99".  The code I
have in mind (not the one that started this bug report) is written that
way, and with -Wno-implicit-int...

> On the other hand, the largest benefit will come from change the
> default.  From that perspective, it won't be necessary to change the
> -std=gnu99 behavior.

Thanks for clarifying.

> Right, it's an argument list mismatch.

Thanks.

> I'm not the C frontend maintainer.  It seems unlikely that this is going
> to be supported because it's one of the major C23 changes.
>
> Before -std=gnu23 becomes the default, GCC will probably start warning
> about calling ()-declared functions with extra arguments, to help with
> porting to a future -std=gnu32-by-default change.  -Wstrict-prototypes
> is not a good proxy for this because it will warn about ()-style
> declarations even in contexts where C23 will align with what the
> programmer intended (no arguments accepted).

Thanks for explaining.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Stefan Kangas]

Yuan Fu <casouri@gmail.com> writes:

> And there aren’t much we can do (except for maybe improve
> tree-sitter-c’s parser).

We should report these cases as bugs though, right?

IOW, I'm thinking that, going forward, perhaps in these cases it makes
sense for us to close bugs once there is a corresponding bug reported
against the relevant treesitter grammar.

bug#59435: 29.0.50; tree-sitter fails to fontify or indent simple function

[Yuan Fu]

Po Lu <luangruo@yahoo.com> writes:

> Yuan Fu <casouri@gmail.com> writes:
>
>>> On Nov 20, 2022, at 10:09 PM, Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org> wrote:
>>> 
>>> 
>>> c-ts-mode cannot fontify the following code:
>>> 
>>> ifsubmitbufl (tp)
>>> {
>>>  register	*kb1 = tp;
>>>  extern	 tbase, i_len;
>>>  register	 l = len;
>>> 
>>> again:
>>> 
>>>  highprio (); /* tbase is used by protoureg */
>>>  iomap (&UBR, tbase += UBR_SIZ);
>>>  UBR->addr_l = kb1;
>>>  UBR->addr_m = kb1 >> 010;
>>>  UBR->mode_l = 07777 &~ UBR_INST;
>>>  UBR->mode_m = 0;
>>>  UBR->long_l = l;
>>>  UBR->long_m = l >> 010;
>>> 
>>>  iounmap ();
>>>  tally_iostat (tbase);
>>>  iospoke (UBR, P_LOCK);
>>>  resetprio ();
>>> 
>>>  SWITCH (S_WAIT|S_HIGH|S_IMMED, 0700, again); /* this returns to
>>> 						  BR->senv[0], except
>>> 						  if CL_HI */
>>> }
>>> 
>>> here, ifsubmitbufl, tbase, l and again are fontified as types.  They are
>>> fontified correctly by CC Mode, as three identifiers and one label.
>>
>> Thanks, I fixe the label’s face.
>
> Thanks.
>
>> As for ifsubmitbufl, tbase, and l, I don’t really understand the
>> syntax here, and I think neither does tree-sitter . In general
>> tree-sitter struggles with nonstandard (in its book anyway) syntax.
>
> Well, that code is 100% valid C.  "*kb1, tbase, i_len, l" are implicitly
> int (which on that machine is 16 bits wide), and so is the value of
> "ifsubmitbufl" and "tp".
>
>> And there aren’t much we can do (except for maybe improve
>> tree-sitter-c’s parser).
>
> Would someone try that?
>
>> Seems to work fine here. I don’t see anything I changed recently that
>> could affect this, either.
>
> Sorry, I think I meant "before".  Place point after "l = len;", then
> press RET.

That seems to work too. It is indented as much as the previous line.

Yuan