c-ts-mode

c-ts-mode

[Ergus]

Hi all:

I have been trying to use c++-ts-mode and I found this apparently wrong
indentation:

int a(in x
       )

The ) is placed differently compared with the previous c++-mode and
inconsistent with the current linux coding standard.

The treesit-check-indent shows actually a diff there (same with for
loops)

I see this entry `((node-is ")") parent 1)` in c-ts-mode--indent-styles
so there must e a reason I am not aware of... however:

Is it possible to specify small differences in the existent styles like
we used to do with c-set-offset? either in the init file or the
.dir-locals.el?

a way to put something like: `((node-is ")") parent 0`??

If I need to define some completely different indentation style (google
for example) whats the intended method to do it without reinventing the
wheel?

Thanks in advance,
Ergus

Re: c-ts-mode

[Yuan Fu]

> On Aug 30, 2023, at 4:52 PM, Ergus <spacibba@aol.com> wrote:
> 
> Hi all:
> 
> I have been trying to use c++-ts-mode and I found this apparently wrong
> indentation:
> 
> int a(in x
>      )
> 
> The ) is placed differently compared with the previous c++-mode and
> inconsistent with the current linux coding standard.
> 
> The treesit-check-indent shows actually a diff there (same with for
> loops)
> 
> I see this entry `((node-is ")") parent 1)` in c-ts-mode--indent-styles
> so there must e a reason I am not aware of... however:
> 
> Is it possible to specify small differences in the existent styles like
> we used to do with c-set-offset? either in the init file or the
> .dir-locals.el?
> 
> a way to put something like: `((node-is ")") parent 0`??
> 
> If I need to define some completely different indentation style (google
> for example) whats the intended method to do it without reinventing the
> wheel?

For now at least, I think you can try modifying treesit-simple-indent-rules. You can prepend rules to the list which would take precedence over the rest.

C-ts-mode could definitely be improved in terms of customization, and any feature in general. Any help is welcome!

Yuan

Re: c-ts-mode

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

On Fri, Sep 1, 2023, 05:15 Yuan Fu <casouri@gmail.com> wrote:

> .
>
> C-ts-mode could definitely be improved in terms of customization, and any
> feature in general. Any help is welcome!
>

How difficult would it be, in your opinion, to have c-ts-mode or
c++-ts-mode import cc-mode styles as treesitter rules? I'd be interested in
that feature even if the importation is not 100% accurate.

João


In

>

[-- Attachment #2: Type: text/html, Size: 1049 bytes --]

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Sep 2023 10:25:28 +0100
> Cc: Ergus <spacibba@aol.com>, emacs-devel <emacs-devel@gnu.org>
> 
> On Fri, Sep 1, 2023, 05:15 Yuan Fu <casouri@gmail.com> wrote:
> 
>  C-ts-mode could definitely be improved in terms of customization, and any feature in general.
>  Any help is welcome!
> 
> How difficult would it be, in your opinion, to have c-ts-mode or c++-ts-mode import cc-mode styles as
> treesitter rules? I'd be interested in that feature even if the importation is not 100% accurate.

Some of the styles are already supported.  The rest should also be
possible (at least most of them), but someone will have to write the
code.  It is not a simple matter of "importing", since CC Mode
implements these features based on infrastructure that either doesn't
exist or makes no sense in treesit-supported modes.

Re: c-ts-mode

[João Távora]

On Thu, Sep 7, 2023 at 10:37 AM Eli Zaretskii <eliz@gnu.org> wrote:

> Some of the styles are already supported.  The rest should also be
> possible (at least most of them), but someone will have to write the
> code.  It is not a simple matter of "importing", since CC Mode
> implements these features based on infrastructure that either doesn't
> exist or makes no sense in treesit-supported modes.

Thanks for clarifying.  My current cc-mode user style is very
simple (I think) it inherits from the 'gnu' style and adds
minimal things.

(c-add-style "user"
             '("gnu"
               (c-special-indent-hook)
               (c-offsets-alist
                (substatement-open . 0)
                (namespace-open . -2)
                (innamespace . -2)
                (extern-lang-open . -2)
                (inextern-lang . -2))))

Basically I'd just like to keep code inside top-level
C++ namespaces and extern blocks unindented.
As to substatement-open I don't even know what
the purpose is anymore.

I see that c++-ts-mode already has a gnu user style, so
I would just need to add my preferences.  Is there a manual
entry somewhere for learning how to do this?

Also, if I may go on a tangent:  Is the forward-sexp-function
for c++-ts-mode currently undergoing work on master?  It
doesn't seem to work and all C-M-foo navigation is broken as
a result.

Debugger entered--Lisp error: (treesit-invalid-predicate nil)
  treesit-node-match-p(#<treesit-node ";" in 41-42> sexp t)
  #f(compiled-function (node) #<bytecode
0x197a9d284470c5e2>)(#<treesit-node ";" in 41-42>)
  treesit-node-top-level(#<treesit-node ";" in 41-42>
#f(compiled-function (node) #<bytecode 0x197a9d284470c5e2>) t)
  treesit--things-around(47 sexp)
  treesit--navigate-thing(47 1 end sexp restricted)
  treesit-end-of-thing(sexp 1 restricted)
  treesit-forward-sexp(1)

Setting the variable to nil brings me to more familar
and satisfactory terrain where, and I suppose the non-treesit
syntax table is being used for that.  Is there any big
advantage in switching to treesitter's forward-sexp-function?

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Sep 2023 16:58:29 +0100
> Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> 
> On Thu, Sep 7, 2023 at 10:37 AM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Some of the styles are already supported.  The rest should also be
> > possible (at least most of them), but someone will have to write the
> > code.  It is not a simple matter of "importing", since CC Mode
> > implements these features based on infrastructure that either doesn't
> > exist or makes no sense in treesit-supported modes.
> 
> Thanks for clarifying.  My current cc-mode user style is very
> simple (I think) it inherits from the 'gnu' style and adds
> minimal things.
> 
> (c-add-style "user"
>              '("gnu"
>                (c-special-indent-hook)
>                (c-offsets-alist
>                 (substatement-open . 0)
>                 (namespace-open . -2)
>                 (innamespace . -2)
>                 (extern-lang-open . -2)
>                 (inextern-lang . -2))))
> 
> Basically I'd just like to keep code inside top-level
> C++ namespaces and extern blocks unindented.
> As to substatement-open I don't even know what
> the purpose is anymore.
> 
> I see that c++-ts-mode already has a gnu user style, so
> I would just need to add my preferences.  Is there a manual
> entry somewhere for learning how to do this?

I think c-ts-mode--indent-styles needs to be refactored toallow fine
control on the indentation parameters that are currently hard-coded
for each supported style.  They will need to have different names,
because tree-sitter doesn't use the CC Mode terminology for the
syntactical constructs.  Also, there seem to be many more parameters
than in CC Mode.

> Also, if I may go on a tangent:  Is the forward-sexp-function
> for c++-ts-mode currently undergoing work on master?  It
> doesn't seem to work and all C-M-foo navigation is broken as
> a result.
> 
> Debugger entered--Lisp error: (treesit-invalid-predicate nil)
>   treesit-node-match-p(#<treesit-node ";" in 41-42> sexp t)
>   #f(compiled-function (node) #<bytecode
> 0x197a9d284470c5e2>)(#<treesit-node ";" in 41-42>)
>   treesit-node-top-level(#<treesit-node ";" in 41-42>
> #f(compiled-function (node) #<bytecode 0x197a9d284470c5e2>) t)
>   treesit--things-around(47 sexp)
>   treesit--navigate-thing(47 1 end sexp restricted)
>   treesit-end-of-thing(sexp 1 restricted)
>   treesit-forward-sexp(1)

First, please submit a bug report about this with all the details.

And second, I don't seem to be able to reproduce this, assuming that
by C-M-foo you meant C-M-f and C-M-b.  Maybe this needs some
particular source file to reproduce?  (I tried with one C file and one
C++ file.)

> Setting the variable to nil brings me to more familar
> and satisfactory terrain where, and I suppose the non-treesit
> syntax table is being used for that.  Is there any big
> advantage in switching to treesitter's forward-sexp-function?

Yes, see treesit-defun-tactic.

Re: c-ts-mode

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Date: Thu, 7 Sep 2023 16:58:29 +0100
>> Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
>> 
> I think c-ts-mode--indent-styles needs to be refactored toallow fine
> control on the indentation parameters that are currently hard-coded
> for each supported style.  They will need to have different names,
> because tree-sitter doesn't use the CC Mode terminology for the
> syntactical constructs.  Also, there seem to be many more parameters
> than in CC Mode.

I've been doing some experiments with c-ts-mode-indent-style, which can
be set to a function that returns a list that adds new items in front of
the list returned by with:

   (alist-get 'gnu (c-ts-mode--indent-styles 'cpp))

It's not super clean (notice the '--'), but not very dirty either.

The hard part is writing the rule in question, but I'll get there I
think.

> First, please submit a bug report about this with all the details.

Done.  bug#65810

> And second, I don't seem to be able to reproduce this, assuming that
> by C-M-foo you meant C-M-f and C-M-b.  Maybe this needs some
> particular source file to reproduce?  (I tried with one C file and one
> C++ file.)

You assumed correctly, And no, it works with an empty file even.  The
recipe is so simple I might as well repost it here:

  ~/Source/Emacs/emacs/src/emacs -nw -Q ~/tmp/simple.cpp -f c++-ts-mode

Then just try any C-M-f or C-M-b or C-M-u etc, and see the error
happening.

I installed my grammar with M-x treesit-install-language-grammar today.
But it also happens with a grammar installed in the same way many months
ago.

Doesn't seem to happen in c-ts-mode.

>> Setting the variable to nil brings me to more familar
>> and satisfactory terrain where, and I suppose the non-treesit
>> syntax table is being used for that.  Is there any big
>> advantage in switching to treesitter's forward-sexp-function?
>
> Yes, see treesit-defun-tactic.

Will check it out, thanks.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: casouri@gmail.com,  spacibba@aol.com,  emacs-devel@gnu.org
> Date: Thu, 07 Sep 2023 18:53:58 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: João Távora <joaotavora@gmail.com>
> >> Date: Thu, 7 Sep 2023 16:58:29 +0100
> >> Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> >> 
> > I think c-ts-mode--indent-styles needs to be refactored toallow fine
> > control on the indentation parameters that are currently hard-coded
> > for each supported style.  They will need to have different names,
> > because tree-sitter doesn't use the CC Mode terminology for the
> > syntactical constructs.  Also, there seem to be many more parameters
> > than in CC Mode.
> 
> I've been doing some experiments with c-ts-mode-indent-style, which can
> be set to a function that returns a list that adds new items in front of
> the list returned by with:
> 
>    (alist-get 'gnu (c-ts-mode--indent-styles 'cpp))
> 
> It's not super clean (notice the '--'), but not very dirty either.

My preference would be to provide the same interface as CC Mode: an
alist with the parameters and their values, or something similar
(e.g., a keyword/value plist).  Asking users to write Lisp functions
to customize indentation style is less friendly, especially if the
user comes from CC Mode.

Re: c-ts-mode

[João Távora]

On Thu, Sep 7, 2023 at 7:13 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > I've been doing some experiments with c-ts-mode-indent-style, which can
> > be set to a function that returns a list that adds new items in front of
> > the list returned by with:
> >
> >    (alist-get 'gnu (c-ts-mode--indent-styles 'cpp))
> >
> > It's not super clean (notice the '--'), but not very dirty either.
>
> My preference would be to provide the same interface as CC Mode: an
> alist with the parameters and their values, or something similar
> (e.g., a keyword/value plist).  Asking users to write Lisp functions
> to customize indentation style is less friendly, especially if the
> user comes from CC Mode.

Agree.  The function I'm writing is so simple that it needn't
really be a function.  If there was a public way to "grab"
the rules named of a indentation style (like 'gnu' in my case)
it would be almost trivial to get rid of the function.

The real challenge is writing the rules themselves.  I'm missing
a kind of "debug rule" that doesn't do anything but prints out
contextual information from the node, parent-node, grandparents.
I made one but it's not very good.  Is there something like that?
Wouldn't even need to be an indentation rule, more like a "describe
AST at point"...

João

PS: I see in the meantime you reproduced the c++-ts-mode-bug.  Great!

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Sep 2023 19:23:33 +0100
> Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> 
> The real challenge is writing the rules themselves.  I'm missing
> a kind of "debug rule" that doesn't do anything but prints out
> contextual information from the node, parent-node, grandparents.
> I made one but it's not very good.  Is there something like that?
> Wouldn't even need to be an indentation rule, more like a "describe
> AST at point"...

Did you try "M-x treesit-explore-mode RET"?

Re: c-ts-mode

[João Távora]

On Thu, Sep 7, 2023 at 7:32 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Thu, 7 Sep 2023 19:23:33 +0100
> > Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> >
> > The real challenge is writing the rules themselves.  I'm missing
> > a kind of "debug rule" that doesn't do anything but prints out
> > contextual information from the node, parent-node, grandparents.
> > I made one but it's not very good.  Is there something like that?
> > Wouldn't even need to be an indentation rule, more like a "describe
> > AST at point"...
>
> Did you try "M-x treesit-explore-mode RET"?

That's a great find, and so is M-x treesit-inspect-mode.

My rules are now done.

(setq c-ts-mode-indent-style
      (lambda ()
        (append '(((n-p-gp nil nil "namespace_definition") grand-parent 0)
                  ((n-p-gp nil nil "linkage_specification") grand-parent 0))
                (alist-get 'gnu (c-ts-mode--indent-styles 'cpp)))))

The lambda, cl-list*, the alist-get and the '--' are ugly but
beyond that, it's better than cc-mode's system, to be honest.

Anyway, to get the ugly out, here's an idea.

IMHO making c-ts-mode--indent-styles a public CL-style
generic function would be a good possibility.

It's a simple patch to allow that.

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 5f685e016d2..337c8c4eed1 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -165,9 +165,12 @@ c-ts-mode--get-indent-style
   (let ((style
          (if (functionp c-ts-mode-indent-style)
              (funcall c-ts-mode-indent-style)
-           (alist-get c-ts-mode-indent-style
(c-ts-mode--indent-styles mode)))))
+           (c-ts-mode-indent-styles c-ts-mode-indent-style mode))))
     `((,mode ,@style))))

+(cl-defgeneric c-ts-mode-indent-styles (style mode)
+  (alist-get style (c-ts-mode--indent-styles mode)))
+
 (defun c-ts-mode--prompt-for-style ()
   "Prompt for an indent style and return the symbol for it."
   (let ((mode (if (derived-mode-p 'c-ts-mode) 'c 'c++)))


With this patch, my rules become:

(cl-defmethod c-ts-mode-indent-styles ((_style (eql gnu)) (_m (eql cpp)))
  (append '(((n-p-gp nil nil "namespace_definition") grand-parent 0)
            ((n-p-gp nil nil "linkage_specification") grand-parent 0))
          (cl-call-next-method)))

Or maybe just

(cl-defmethod c-ts-mode-indent-styles (_style (_m (eql cpp)))
  (append '(((n-p-gp nil nil "namespace_definition") grand-parent 0)
            ((n-p-gp nil nil "linkage_specification") grand-parent 0))
          (cl-call-next-method)))

if I would like them to apply to other non-gnu styles, too.

A defcustom-style thing for customize lovers can also be added,
later for people that don't like defgeneric.  Seems like a pretty
large DSL to code up in customize, though.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 7 Sep 2023 23:01:58 +0100
> Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> 
> On Thu, Sep 7, 2023 at 7:32 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: João Távora <joaotavora@gmail.com>
> > > Date: Thu, 7 Sep 2023 19:23:33 +0100
> > > Cc: casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> > >
> > > The real challenge is writing the rules themselves.  I'm missing
> > > a kind of "debug rule" that doesn't do anything but prints out
> > > contextual information from the node, parent-node, grandparents.
> > > I made one but it's not very good.  Is there something like that?
> > > Wouldn't even need to be an indentation rule, more like a "describe
> > > AST at point"...
> >
> > Did you try "M-x treesit-explore-mode RET"?
> 
> That's a great find, and so is M-x treesit-inspect-mode.
> 
> My rules are now done.
> 
> (setq c-ts-mode-indent-style
>       (lambda ()
>         (append '(((n-p-gp nil nil "namespace_definition") grand-parent 0)
>                   ((n-p-gp nil nil "linkage_specification") grand-parent 0))
>                 (alist-get 'gnu (c-ts-mode--indent-styles 'cpp)))))
> 
> The lambda, cl-list*, the alist-get and the '--' are ugly but
> beyond that, it's better than cc-mode's system, to be honest.
> 
> Anyway, to get the ugly out, here's an idea.
> 
> IMHO making c-ts-mode--indent-styles a public CL-style
> generic function would be a good possibility.

Sorry, I don't understand: since we already allow
c-ts-mode-indent-style to be a function, why do we need any other
function-based feature?  If the only reason is that the function form
of c-ts-mode-indent-style looks ugly to you, then I think this is in
the eyes of the beholder; it doesn't look ugly to me, FWIW.

> A defcustom-style thing for customize lovers can also be added,
> later for people that don't like defgeneric.  Seems like a pretty
> large DSL to code up in customize, though.

What I had in mind was a simple alist, like CC Mode uses, with an
infrastructure function to install it.  Patches are welcome.

Yuan and Theo, would you like to work on adding such feature to
c-ts-common?

Re: c-ts-mode

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> IMHO making c-ts-mode--indent-styles a public CL-style
>> generic function would be a good possibility.
>
> Sorry, I don't understand: since we already allow
> c-ts-mode-indent-style to be a function, why do we need any other
> function-based feature?

Not all functions are the same, and my idea is fundamentally different
than the current mechanism.  The current variable offered which can be
indeed a function has no concept of the "base set of rules" that you can
extend, modify, or splice your lists into them and neither does it
formalized way of when to do these things.

So in the current interface, the user has to specifically know where and
how to get the bae set, (which includes )exposing implementation
details).

In my proposed interface, she doesn't.  She just codes the delta and the
conditions for overridding if she so wishe.

> If the only reason is that the function form of c-ts-mode-indent-style
> looks ugly to you, then I think this is in the eyes of the beholder;
> it doesn't look ugly to me, FWIW.

You may be confusing my proposal with one of a non-generic "normal"
function that you modify with add-function.  That's more-or-less the
same as a generic funtion (with :around, :before, :after, etc...)

So if you don't like CL stuff that has a more emacsy feel (but is
generally the same).  And of course a hook has an even more emacsy feel.

The advantage of my approach is the specialization in the arguments.

>> A defcustom-style thing for customize lovers can also be added,
>> later for people that don't like defgeneric.  Seems like a pretty
>> large DSL to code up in customize, though.
>
> What I had in mind was a simple alist, like CC Mode uses, with an
> infrastructure function to install it.  Patches are welcome.

It would certainly work for me, at least for my very simple case, and I
would be happy for this.  Not sure it is half as powerful, for example
how would you make that simple alist express cases where you want to add
the rules _after_ the base set?

Two alists?  I don't know if that is useful, but I seem to have read
somewhere in ts code that ordering is important.

Anyway, looking forward to see those patches, for the moment I've
supplied one.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: Theodor Thornhill <theo@thornhill.no>,  casouri@gmail.com,
>   spacibba@aol.com,  emacs-devel@gnu.org
> Date: Fri, 08 Sep 2023 08:25:50 +0100
> 
> > What I had in mind was a simple alist, like CC Mode uses, with an
> > infrastructure function to install it.  Patches are welcome.
> 
> It would certainly work for me, at least for my very simple case, and I
> would be happy for this.  Not sure it is half as powerful, for example
> how would you make that simple alist express cases where you want to add
> the rules _after_ the base set?

How does CC Mode support this (if it does)?  I think we should allow
users of CC Mode easy migration to c-ts-mode.  Anything beyond that
is, of course, welcome, but it is less important from my POV.

I'll let Theo and Yuan chime in on the other points you make.

Re: c-ts-mode

[João Távora]

On Fri, Sep 8, 2023 at 12:26 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Cc: Theodor Thornhill <theo@thornhill.no>,  casouri@gmail.com,
> >   spacibba@aol.com,  emacs-devel@gnu.org
> > Date: Fri, 08 Sep 2023 08:25:50 +0100
> >
> > > What I had in mind was a simple alist, like CC Mode uses, with an
> > > infrastructure function to install it.  Patches are welcome.
> >
> > It would certainly work for me, at least for my very simple case, and I
> > would be happy for this.  Not sure it is half as powerful, for example
> > how would you make that simple alist express cases where you want to add
> > the rules _after_ the base set?
>
> How does CC Mode support this (if it does)?

No idea, I don't know CC Mode's styling interface well beyond
the trivial example I showed.

It does seems like CC Mode have different mechanics?
Maybe order doesn't matter there?

> Anything beyond that is, of course, welcome, but it is
> less important from my POV.

Sure makes sense.  As I said I'm just conjecturing someone
will need that flexibility because rule ordering is a fundamental
part of the mechanics in TS and I don't think we abstract it
away.  But personally, for those ridiculous two rules, I didn't
need it at all, adding at the front was just fine

Anyway, it's worth pointing out that unless you're extremely adept
at crafting indentation styles for Emacs, you're always in a losing
battle when working on multiple projects, since many projects
nowadays (not just C/C++) use an external format definition.
A .clang-format  (even we have one since 2017) or other similar
files.  These are understood by other editors and tools.

Maybe in the past CC Mode's styles were very useful, but I don't
think they attract the same interest today because there are these
external tools.  For the same reason, I don't predict ts
indentation styles to become widely used.

Of course, this is a big annoyance, because there always multiple
competing sources of indentation rules.  Emacs TAB use one thing,
everyone else uses something else.

Personally I've not been fighting this, i.e. I gave up.  I use
Emacs indentation rules to more or less align the file with the rules
and then make sure to run the tool (maybe via eglot-format)
before committing.  It'd be great if I could rely on indent-region
or C-M-q to produce the canonical indentation like I do in
Lisp modes, but I can't.

A more promising solution to this whole problem would be
to somehow bring these external formatter's rules to meet Emacs's
idea of "just indent these lines".

I tried to do that with Eglot (and Theo helped out) via
indent-region-function but we alas it didn't work very well,
because -- at least for clangd -- the LSP formatter (which
delegates to clang-format under the hood) has a tendency to add
and remove newlines even if you ask it to stay within the
same line.  So basic things like indenting an empty line is
essentially impossible.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Sep 2023 13:38:58 +0100
> Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, 
> 	emacs-devel@gnu.org
> 
> Anyway, it's worth pointing out that unless you're extremely adept
> at crafting indentation styles for Emacs, you're always in a losing
> battle when working on multiple projects, since many projects
> nowadays (not just C/C++) use an external format definition.
> A .clang-format  (even we have one since 2017) or other similar
> files.  These are understood by other editors and tools.

I guess this means we should add a TODO item for supporting such
external specifications?

Re: c-ts-mode

[Eli Zaretskii]

> Date: Fri, 08 Sep 2023 16:11:46 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> 
> > From: João Távora <joaotavora@gmail.com>
> > Date: Fri, 8 Sep 2023 13:38:58 +0100
> > Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, 
> > 	emacs-devel@gnu.org
> > 
> > Anyway, it's worth pointing out that unless you're extremely adept
> > at crafting indentation styles for Emacs, you're always in a losing
> > battle when working on multiple projects, since many projects
> > nowadays (not just C/C++) use an external format definition.
> > A .clang-format  (even we have one since 2017) or other similar
> > files.  These are understood by other editors and tools.
> 
> I guess this means we should add a TODO item for supporting such
> external specifications?

Done; patches welcome.

Re: c-ts-mode

[João Távora]

On Fri, Sep 8, 2023 at 2:32 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > Date: Fri, 08 Sep 2023 16:11:46 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, emacs-devel@gnu.org
> >
> > > From: João Távora <joaotavora@gmail.com>
> > > Date: Fri, 8 Sep 2023 13:38:58 +0100
> > > Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com,
> > >     emacs-devel@gnu.org
> > >
> > > Anyway, it's worth pointing out that unless you're extremely adept
> > > at crafting indentation styles for Emacs, you're always in a losing
> > > battle when working on multiple projects, since many projects
> > > nowadays (not just C/C++) use an external format definition.
> > > A .clang-format  (even we have one since 2017) or other similar
> > > files.  These are understood by other editors and tools.
> >
> > I guess this means we should add a TODO item for supporting such
> > external specifications?
>
> Done; patches welcome.

I couldn't find your TODO item (in etc/TODO at least).  But I'm curious
as to how you will phrase it: "supporting" can have many meanings.

If it means "have some kind of interface for using" then Emacs already
has one, which is `eglot-format`.  Although indirect (because LSP), it
is a pretty effective in abstracting away different specs of different
such tools (clang-format, prettier, eslint, etc).

But if "supporting" means "plug into indent-line-function and
indent-region-function", then it's going to be relatively hard, because
as I explained, these are formatters, not indenters, so it's a bit
of a round-peg, square-hole problem.  So if you want to keep the
existing interface of those two functions (which would be ideal, since
a lot of tooling already depends on them), there would have to be some
way to communicate with these tools so that they only talk about
indentation.  Not saying it's impossible, but it's hard, at least
when LSP is used for abstracting away differences.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Sep 2023 16:15:09 +0100
> Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, 
> 	emacs-devel@gnu.org
> 
> > > I guess this means we should add a TODO item for supporting such
> > > external specifications?
> >
> > Done; patches welcome.
> 
> I couldn't find your TODO item (in etc/TODO at least).

Chrystal ball says you looked on master.  I made my change on the
emacs-29 branch instead.

> But I'm curious as to how you will phrase it: "supporting" can have
> many meanings.
> 
> If it means "have some kind of interface for using" then Emacs already
> has one, which is `eglot-format`.  Although indirect (because LSP), it
> is a pretty effective in abstracting away different specs of different
> such tools (clang-format, prettier, eslint, etc).

I think we should try to support this without LSP servers.

> But if "supporting" means "plug into indent-line-function and
> indent-region-function", then it's going to be relatively hard, because
> as I explained, these are formatters, not indenters, so it's a bit
> of a round-peg, square-hole problem.

Which, as we all know, is a problem that was solved at least once.

> So if you want to keep the existing interface of those two functions
> (which would be ideal, since a lot of tooling already depends on
> them), there would have to be some way to communicate with these
> tools so that they only talk about indentation.  Not saying it's
> impossible, but it's hard, at least when LSP is used for abstracting
> away differences.

Let's talk after you read what I wrote in TODO.

Re: c-ts-mode

[João Távora]

On Fri, Sep 8, 2023 at 4:34 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Fri, 8 Sep 2023 16:15:09 +0100
> > Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com,
> >       emacs-devel@gnu.org
> >
> > > > I guess this means we should add a TODO item for supporting such
> > > > external specifications?
> > >
> > > Done; patches welcome.
> >
> > I couldn't find your TODO item (in etc/TODO at least).
>
> Chrystal ball says you looked on master.  I made my change on the
> emacs-29 branch instead.

Ah yes.

> I think we should try to support this without LSP servers.

OK, but then you'll have to deal with the abstraction problem which
is being solved by LSP servers at the moment, quite effectively.
It's fine to want this but the motivation you'll find
for working on this item is affected  That's because that "minimum
command" you mention already exists in practice

Are you aware of eslint, prettier, gofmt, rustfmt, and whatever
other formatter programs are out there?    Or do you just want
to support clang-format for the Emacs project and other C/C++
projects?

Also I'm not sure that formatters for some languages aren't
exclusively LSP.

Anyway, do you also want to support this without external tools
at all  (i.e. without relying on the clang-format program at all?)
Your TODO item seems to say so.

If so, then going that route means basically writing a translator
of clang-format -> Emacs treesitter rules.  Maybe it's easier than
I think, and the main advantage is it will plug into
indent-line-function and indent-region-function rather easily.
And, who knows, some code can be reused for writing translators of
other formatters.

Anyway I wouldn't discard the "LSP shortcut" idea.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Sep 2023 16:56:20 +0100
> Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, 
> 	emacs-devel@gnu.org
> 
> OK, but then you'll have to deal with the abstraction problem which
> is being solved by LSP servers at the moment, quite effectively.
> It's fine to want this but the motivation you'll find
> for working on this item is affected  That's because that "minimum
> command" you mention already exists in practice
> 
> Are you aware of eslint, prettier, gofmt, rustfmt, and whatever
> other formatter programs are out there?    Or do you just want
> to support clang-format for the Emacs project and other C/C++
> projects?

I'd start with the latter.  But that's something for the volunteer who
takes up that job to figure out.

> Anyway, do you also want to support this without external tools
> at all  (i.e. without relying on the clang-format program at all?)
> Your TODO item seems to say so.

Yes, if possible.

> If so, then going that route means basically writing a translator
> of clang-format -> Emacs treesitter rules.  Maybe it's easier than
> I think, and the main advantage is it will plug into
> indent-line-function and indent-region-function rather easily.
> And, who knows, some code can be reused for writing translators of
> other formatters.

Yes.

> Anyway I wouldn't discard the "LSP shortcut" idea.

I didn't discard it.

Re: c-ts-mode

[João Távora]

On Fri, Sep 8, 2023 at 7:23 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > If so, then going that route means basically writing a translator
> > of clang-format -> Emacs treesitter rules.  Maybe it's easier than
> > I think, and the main advantage is it will plug into
> > indent-line-function and indent-region-function rather easily.
> > And, who knows, some code can be reused for writing translators of
> > other formatters.
> Yes.
>
> > Anyway I wouldn't discard the "LSP shortcut" idea.
>
> I didn't discard it.

OK.  But the TODO item doesn't mention it.  I think prospective
volunteers should eventually be steered to this discussion to
learn about these trade-offs.  Some entries in etc/TODO have
links to lists.gnu.org.  I suggest to add one in the item to our
conversation.

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Sep 2023 19:30:15 +0100
> Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, 
> 	emacs-devel@gnu.org
> 
> On Fri, Sep 8, 2023 at 7:23 PM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > > If so, then going that route means basically writing a translator
> > > of clang-format -> Emacs treesitter rules.  Maybe it's easier than
> > > I think, and the main advantage is it will plug into
> > > indent-line-function and indent-region-function rather easily.
> > > And, who knows, some code can be reused for writing translators of
> > > other formatters.
> > Yes.
> >
> > > Anyway I wouldn't discard the "LSP shortcut" idea.
> >
> > I didn't discard it.
> 
> OK.  But the TODO item doesn't mention it.

It also doesn't say it's off the table, though.  In fact, it says
nothing at all about the implementation.

If you want to add some implementation ideas to that item, I won't
object; please show the patch.

> I think prospective volunteers should eventually be steered to this
> discussion to learn about these trade-offs.  Some entries in
> etc/TODO have links to lists.gnu.org.  I suggest to add one in the
> item to our conversation.

Fine by me, thanks.

Re: c-ts-mode

[João Távora]

On Fri, Sep 8, 2023 at 7:54 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > OK.  But the TODO item doesn't mention it.
> It also doesn't say it's off the table, though.  In fact, it says
> nothing at all about the implementation.

Hmm, it talks about "read indentation rules from a file".  A more
agnostic stance would be "use indentation rules from a file".  Of course
as we discussed reading/translating has very real advantages, too.

See the patch after my sig, and feel free to reword it.

João

diff --git a/etc/TODO b/etc/TODO
index ef91c17298b..f1b3373e9e5 100644
--- a/etc/TODO
+++ b/etc/TODO
@@ -452,6 +452,10 @@ reformat the region of source code according to the rules.
 The next step is to use these rules during editing of files residing
 in a directory that has such an indentation-rules spec in it.

+For some discussion and implementation ideas (including possibly using
+LSP), see the thread starting at
+https://lists.gnu.org/archive/html/emacs-devel/2023-09/msg00609.html
+
 ** FFI (foreign function interface)
 See eg https://lists.gnu.org/r/emacs-devel/2013-10/msg00246.html

Re: c-ts-mode

[Petteri Hintsanen]

João Távora <joaotavora@gmail.com> writes:

> Maybe in the past CC Mode's styles were very useful, but I don't
> think they attract the same interest today because there are these
> external tools.  For the same reason, I don't predict ts
> indentation styles to become widely used.

I have to disagree with this.  Built-in styles are useful, and for me
they have been, in all major modes I've used (not just CC mode), close
enough for almost all use cases over the years.  I use external tools
like clang-format only to satisfy some CI systems that (IMO stupidly)
enforce certain formatting.

In general I'd find it a bit disconcerting if the tendency is to move
towards external stuff for basics like syntax highlighting and code
formatting.  I personally don't like the idea of installing a couple of
libraries plus their emacs glue libs, source code formatters along with
their dependencies, and huge language servers [*], to get first class
"editing experience" -- Emacs is supposedly a text editor, after all.
Especially given that I've got to install and use Emacs on resource
limited machines and ones where I do not have admin rights, so I cannot
install those externals just like that.

It is great that Emacs is able to leverage the power of third party libs
and tools, but I think this should not come at the expense of built in
functionality.  Of course the rant above was a rhetoric exaggeration,
but still it is perhaps something to keep in mind when designing for the
future.

Thanks,
Petteri

[*] e.g. clangd eats regularly gigabytes of memory, has tons of
dependencies, and needs tunings for the build and sysroot and whatnot

Re: c-ts-mode

[João Távora]

On Fri, Sep 8, 2023 at 8:58 PM Petteri Hintsanen <petterih@iki.fi> wrote:

> It is great that Emacs is able to leverage the power of third party libs
> and tools, but I think this should not come at the expense of built in
> functionality.  Of course the rant above was a rhetoric exaggeration,
> but still it is perhaps something to keep in mind when designing for the
> future.

It was an interesting rant IMO :-)  I very much agree with some points,
such as this last one about keeping a clean Emacs functional (which
is also why I like having things in core).  I haven't made LSP-mandated
syntax highlighting a priority for Eglot precisely because Emacs does it
quite well already.

The point I definitely don't agree with is about the "editing experience".
Editing and comprehension of complex C++ code for example is hugely
augmented by LSP (not necessarily clangd).  These tools are here to
stay hopefully.  I can't go back to the days of cscope and grep
(as for CEDET, it never worked at all for me beyond toy tricks).

Other than that, I can see your point about clangd as a gateway
to external tools that Emacs already does very reasonably well, such
as formatting code.  But the reality is not everyone in the world
uses Emacs, and you have to work with them and their code.  Even
different Emacs users will have different C/C++ styles.

> [*] e.g. clangd eats regularly gigabytes of memory, has tons of
> dependencies, and needs tunings for the build and sysroot and whatnot

I built clangd from source the other day, didn't notice that
many dependencies (just LLVM which is pretty big I guess).  By the
way building from source is a sometimes a way to bypass "admin
rights" bothers.  I didn't install clang-format or clang-tidy
separately, they are already included.

As for huge size, you get what you pay for, IMO.  I think you can
tweak it down to just a code analyzer (but analyzing lots of code
code accurately takes a lot of resources yes -- look I didn't
invent templates ;-) ).

João

Re: c-ts-mode

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 8 Sep 2023 20:42:08 +0100
> Cc: theo@thornhill.no, casouri@gmail.com, spacibba@aol.com, 
> 	emacs-devel@gnu.org
> 
> See the patch after my sig, and feel free to reword it.

It's fine, please go ahead and install.

Re: c-ts-mode

[Eli Zaretskii]

> From: Petteri Hintsanen <petterih@iki.fi>
> Cc: Eli Zaretskii <eliz@gnu.org>,  theo@thornhill.no,  casouri@gmail.com,
>   spacibba@aol.com,  emacs-devel@gnu.org
> Date: Fri, 08 Sep 2023 22:58:22 +0300
> 
> João Távora <joaotavora@gmail.com> writes:
> 
> > Maybe in the past CC Mode's styles were very useful, but I don't
> > think they attract the same interest today because there are these
> > external tools.  For the same reason, I don't predict ts
> > indentation styles to become widely used.
> 
> I have to disagree with this.  Built-in styles are useful, and for me
> they have been, in all major modes I've used (not just CC mode), close
> enough for almost all use cases over the years.  I use external tools
> like clang-format only to satisfy some CI systems that (IMO stupidly)
> enforce certain formatting.
> 
> In general I'd find it a bit disconcerting if the tendency is to move
> towards external stuff for basics like syntax highlighting and code
> formatting.

There's no such "move", not in Emacs anyway.  We will continue
supporting our built-in rules for the observable future.  This
discussion is about _allowing_ users who do want it to have Emacs obey
indentation rules specified on external files of de-facto standard
structure and format.  When this is implemented, it should be an
optional feature that users activate only if they want/need it.

> I personally don't like the idea of installing a couple of
> libraries plus their emacs glue libs, source code formatters along with
> their dependencies, and huge language servers [*], to get first class
> "editing experience" -- Emacs is supposedly a text editor, after all.
> Especially given that I've got to install and use Emacs on resource
> limited machines and ones where I do not have admin rights, so I cannot
> install those externals just like that.

You are assuming something about the implementation that still doesn't
exist.  If you mean LSP, then I agree that it should ideally not be
the prerequisite to having this feature, and I think at least for most
of that functionality we could simply convert the external rules into
our internal variables and data structures used by indentation
functions.

> It is great that Emacs is able to leverage the power of third party libs
> and tools, but I think this should not come at the expense of built in
> functionality.  Of course the rant above was a rhetoric exaggeration,
> but still it is perhaps something to keep in mind when designing for the
> future.

We always keep that in mind.

Re: c-ts-mode

[Yuan Fu]

> On Sep 8, 2023, at 4:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: João Távora <joaotavora@gmail.com>
>> Cc: Theodor Thornhill <theo@thornhill.no>,  casouri@gmail.com,
>>  spacibba@aol.com,  emacs-devel@gnu.org
>> Date: Fri, 08 Sep 2023 08:25:50 +0100
>> 
>>> What I had in mind was a simple alist, like CC Mode uses, with an
>>> infrastructure function to install it.  Patches are welcome.
>> 
>> It would certainly work for me, at least for my very simple case, and I
>> would be happy for this.  Not sure it is half as powerful, for example
>> how would you make that simple alist express cases where you want to add
>> the rules _after_ the base set?
> 
> How does CC Mode support this (if it does)?  I think we should allow
> users of CC Mode easy migration to c-ts-mode.  Anything beyond that
> is, of course, welcome, but it is less important from my POV.
> 
> I'll let Theo and Yuan chime in on the other points you make.

I don’t think users would be happy if they need to know CLOS to customize indentation rules. IMO users would be better off modifying treesit-simple-indent-rules in the major mode hook. It’s an alist of (LANGUAGE . RULES). 

So something like

(setf (alist-get 'lang treesit-simple-indent-rules)
      (append '(custom-rules...)
              (alist-get 'lang treesit-simple-indent-rules)))

Would do the trick.

Admittedly that’s a tiny bit clunky, we can add a helper function like (treesit-simple-indent-add-rules RULES &optional LANG).

Yuan

Re: c-ts-mode

[João Távora]

On Tue, Sep 12, 2023 at 1:34 AM Yuan Fu <casouri@gmail.com> wrote:

> > I'll let Theo and Yuan chime in on the other points you make.
>
> I don’t think users would be happy if they need to know CLOS to customize indentation rules.

I'll leave that to you, but FWIW I've frequently had unsolicited
positive feedback on CLOS-based customization techniques, including two
recent ones:

  bug#65418
  https://github.com/joaotavora/breadcrumb/issues/6#issuecomment-1710745508

This seems to indicate that, all other things being equal, if users
are asked to go to Elisp, they don't mind (and actually enjoy) using
CLOS (or at least defgeneric) APIs.

> IMO users would be better off modifying treesit-simple-indent-rules in the major mode hook. It’s an alist of (LANGUAGE . RULES).
>
> So something like
>
> (setf (alist-get 'lang treesit-simple-indent-rules)
>       (append '(custom-rules...)
>               (alist-get 'lang treesit-simple-indent-rules)))
>
> Would do the trick.

Would it? That's not idempotent, is it?

What is LANGUAGE exactly?  A major mode symbol, or one of those
closely related treesit symbols for the shared object library?

And where does the C/C++ base style ('gnu, 'k&r, 'stroutroup, etc)
enter here?  The typical use case is to pick one of these styles
for C/C++ and tweak it (in an idempotent fashion, of course).

João

Re: c-ts-mode

[Po Lu]

João Távora <joaotavora@gmail.com> writes:

> I'll leave that to you, but FWIW I've frequently had unsolicited
> positive feedback on CLOS-based customization techniques, including two
> recent ones:
>
>   bug#65418
>   https://github.com/joaotavora/breadcrumb/issues/6#issuecomment-1710745508
>
> This seems to indicate that, all other things being equal, if users
> are asked to go to Elisp, they don't mind (and actually enjoy) using
> CLOS (or at least defgeneric) APIs.

But Custom itself fails to support EIEIO and defgenerics, so we should
only introduce such defgenerics as user options when it is clear that
the only users with reason to set them are already adept in Emacs Lisp.

Re: c-ts-mode

[João Távora]

On Tue, Sep 12, 2023 at 9:01 AM Po Lu <luangruo@yahoo.com> wrote:
>

> But Custom itself fails to support EIEIO and defgenerics, so we should
> only introduce such defgenerics as user options when it is clear that
> the only users with reason to set them are already adept in Emacs Lisp.

That's why I specifically wrote: "if users are asked to go to Elisp...".

And, AFAIU, they _are_ being asked to go to Elisp in this particular
case.

And that's because treesitter-simple-indent-rules isn't a defcustom.

Could it ever be a defcustom?  Maybe.  But I'd say that's hard
because the ts indent rules DSL is complex and hard to express
in that manner.

And even if it were possible to do that (it probably is), it's
another question entirely if using that UI for that particular
purpose is more comfortable to the mythical "average user" than
using Elisp directly.

Furthermore, it's arguable that teaching multiple different complex
DSLs for customizing different parts of Emacs is a worse investment
than the practice of teaching basic reusable Lisp concepts that
are particularly useful for customization like user init files,
setq, eval-after-load, hooks, etc.  And why not `cl-defmethod`?
After all, people using Emacs are presumably using it as a file
editor to some capacity.

The super-popular frameworks such as Doom, Spacemacs (possibly
the natural habitat of the aforementioned mythical creature)
don't have any problems with teach their users Elisp naturally.

If we were to take a cue from these systems (and get over the some
of the CL allergies) we could -- on a case by case basis -- make
better decisions as to how to provide good customization APIs.

In summary, even though I personally don't like Custom for my
own customization I'm _not_ proposing to "get rid of it" or stop
providing defcustom specs, but to weigh the pros and cons when
considering it for a given customization point.  And of course
not use this as an argument against CL stuff, because Custom
can't express a lot of non-CL Lisp stuff either.

João

Re: c-ts-mode

[Petteri Hintsanen]

Eli Zaretskii <eliz@gnu.org> writes:

>> It is great that Emacs is able to leverage the power of third party libs
>> and tools, but I think this should not come at the expense of built in
>> functionality.  Of course the rant above was a rhetoric exaggeration,
>> but still it is perhaps something to keep in mind when designing for the
>> future.
>
> We always keep that in mind.

I know you do.  My ranting would probably have been best left unwritten.

Case in point: while updating to Emacs 29, I converted all init code to
use the new use-package macro.

There was stuff from 10+ years back I had not actually used in ages.
Nonetheless, almost everything worked flawlessly in Emacs 29.

It feels impossible to witness anything like this with any other piece
of software.

Thanks,
Petteri