Re: Re: Tree-sitter doc question

Re: Re: Tree-sitter doc question

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

>From: Eli Zaretskii <eliz@gnu.org>
>
>> Date: Thu, 8 Dec 2022 15:12:47 +0100
>> From: Ergus <spacibba@aol.com>
>>
>> Just a quick question. I see that the tree-sitter changes are already on
>> master, but I don't find any documentation. Please could you point me
>> where is it or how to use it?
>
>"No documentation" is not accurate: there's a lot in the ELisp manual.
>
>User-level documentation was not yet written, we will do that a bit
>later.

To add to Ergus' quick question. How long would it take to write a couple
of lines for the not-yet-informed with *all* steps to get a tree-sitter mode
up and running? I don't know, python-ts-mode, c-ts-mode...
Maybe that would encourage more users/testers and thus (hopefully) make
tree-sitter more tested, usable, popular, (you name it)

Just my .02c
/PA


-- 
Fragen sind nicht da um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 1718 bytes --]

Re: Tree-sitter doc question

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Fri, 9 Dec 2022 09:33:11 +0100
> 
> >User-level documentation was not yet written, we will do that a bit
> >later.
> 
> To add to Ergus' quick question. How long would it take to write a couple
> of lines for the not-yet-informed with *all* steps to get a tree-sitter mode
> up and running? I don't know, python-ts-mode, c-ts-mode... 
> Maybe that would encourage more users/testers and thus (hopefully) make
> tree-sitter more tested, usable, popular, (you name it)

?? These modes have doc strings, so it isn't like they aren't
documented at all.

And setting them up boils down to just turning them ON, something the
users who follow the development branches should know already.  Why
did you assume you'd need something beyond "M-x python-ts-mode RET"?

Re: Tree-sitter doc question

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

Hi Eli,

let me rephrase my question:

To start testing tree-sitter, do I "just" need to include
--enable-tree-sitter in my configure step or do I need something
(more|else) on a
clean-slate (Linux|macOS) system? Does the emacs build process provide all
the necessary steps or is some external configuration/installation
needed?

This is the kind of "extra documentation" some of us are yearning for ;-)

Thanks a ton in advance, /PA


On Fri, 9 Dec 2022 at 13:10, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> > Date: Fri, 9 Dec 2022 09:33:11 +0100
> >
> > >User-level documentation was not yet written, we will do that a bit
> > >later.
> >
> > To add to Ergus' quick question. How long would it take to write a couple
> > of lines for the not-yet-informed with *all* steps to get a tree-sitter
> mode
> > up and running? I don't know, python-ts-mode, c-ts-mode...
> > Maybe that would encourage more users/testers and thus (hopefully) make
> > tree-sitter more tested, usable, popular, (you name it)
>
> ?? These modes have doc strings, so it isn't like they aren't
> documented at all.
>
> And setting them up boils down to just turning them ON, something the
> users who follow the development branches should know already.  Why
> did you assume you'd need something beyond "M-x python-ts-mode RET"?
>


-- 
Fragen sind nicht da um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 2426 bytes --]

Re: Tree-sitter doc question

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Sat, 10 Dec 2022 08:29:08 +0100
> Cc: emacs-devel@gnu.org
> 
> To start testing tree-sitter, do I "just" need to include --enable-tree-sitter in my configure step or do I need
> something (more|else) on a 
> clean-slate (Linux|macOS) system? Does the emacs build process provide all the necessary steps or is
> some external configuration/installation
> needed?

You need to make sure the tree-sitter library is installed, and so are
the grammar libraries for the programming languages you want to use.
Then just rebuild Emacs; --enable-tree-sitter is ON by default, so if
you have the library installed, the build process with DTRT.

> This is the kind of "extra documentation" some of us are yearning for ;-)

The above is not different from any other optional library, such as
GnuTLS or librsvg.  The exact commands to install them depend on the
distro you are using, and we don't describe them in our documentation.

Re: Tree-sitter doc question

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 2224 bytes --]

Eli writes:

> The above is not different from any other optional library, such as
> GnuTLS or librsvg.  The exact commands to install them depend on the
> distro you are using, and we don't describe them in our documentation.

Thanks for the clarification. I was misled by finding some tree-sitter
related files
in the admin/ directory tree.  As far as I see, tree-sitter is not very
widely provided (as
opposed to GnuTLS or librsvg...)

I'm on Ubuntu 20.04 LTS and have little time to face a migration to 22.04
and I haven't
found anyone providing a PPA to install tree-sitter on my system. Same
might also
be true for many Debian users...

Well... it will have to wait until the summer (if and when I have spare
cycles to upgrade the system).
Anyhow, there are plenty of things to look inside Emacs on my practically
non-existing free cycles now.
:-)

Thanks again,
WR,/PA

On Sat, 10 Dec 2022 at 09:46, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> > Date: Sat, 10 Dec 2022 08:29:08 +0100
> > Cc: emacs-devel@gnu.org
> >
> > To start testing tree-sitter, do I "just" need to include
> --enable-tree-sitter in my configure step or do I need
> > something (more|else) on a
> > clean-slate (Linux|macOS) system? Does the emacs build process provide
> all the necessary steps or is
> > some external configuration/installation
> > needed?
>
> You need to make sure the tree-sitter library is installed, and so are
> the grammar libraries for the programming languages you want to use.
> Then just rebuild Emacs; --enable-tree-sitter is ON by default, so if
> you have the library installed, the build process with DTRT.
>
> > This is the kind of "extra documentation" some of us are yearning for ;-)
>
> The above is not different from any other optional library, such as
> GnuTLS or librsvg.  The exact commands to install them depend on the
> distro you are using, and we don't describe them in our documentation.
>


-- 
Fragen sind nicht da um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 3198 bytes --]

Re: Tree-sitter doc question

[João Paulo Labegalini de Carvalho]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

> I'm on Ubuntu 20.04 LTS and have little time to face a migration to 22.04
> and I haven't
> found anyone providing a PPA to install tree-sitter on my system. Same
> might also
> be true for many Debian users...
>

You can build tree-sitter from source by cloning the repo
https://github.com/tree-sitter/tree-sitter and running make && make install
PREFIX=<installation directory>.

Alternatively, you can grab one of the pre-build releases here:
https://github.com/tree-sitter/tree-sitter/releases

After that make sure to export PKG_CONFIG_PATH=<tree-sitter install
dir>/lib/pkgconfig.

Once that is done, emacs should build with tree-sitter smoothly.

-- 
João Paulo L. de Carvalho
Ph.D Computer Science |  IC-UNICAMP | Campinas , SP - Brazil
Postdoctoral Research Fellow | University of Alberta | Edmonton, AB - Canada
joao.carvalho@ic.unicamp.br
joao.carvalho@ualberta.ca

[-- Attachment #2: Type: text/html, Size: 1669 bytes --]

Re: Tree-sitter doc question

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1657 bytes --]

Obrigado, João Paulo

That's done :-) I now have an Emacs with tree-sitter and I have gone to the
admin/... directory to compile the language support .so.
Maybe the final step in that documentation would be to say a word on where
to place those files when testing and how to
"politely encourage" Emacs to load and use them ;-)

/PA

On Sat, 10 Dec 2022 at 16:37, João Paulo Labegalini de Carvalho <
jaopaulolc@gmail.com> wrote:

>
> I'm on Ubuntu 20.04 LTS and have little time to face a migration to 22.04
>> and I haven't
>> found anyone providing a PPA to install tree-sitter on my system. Same
>> might also
>> be true for many Debian users...
>>
>
> You can build tree-sitter from source by cloning the repo
> https://github.com/tree-sitter/tree-sitter and running make && make
> install PREFIX=<installation directory>.
>
> Alternatively, you can grab one of the pre-build releases here:
> https://github.com/tree-sitter/tree-sitter/releases
>
> After that make sure to export PKG_CONFIG_PATH=<tree-sitter install
> dir>/lib/pkgconfig.
>
> Once that is done, emacs should build with tree-sitter smoothly.
>
> --
> João Paulo L. de Carvalho
> Ph.D Computer Science |  IC-UNICAMP | Campinas , SP - Brazil
> Postdoctoral Research Fellow | University of Alberta | Edmonton, AB -
> Canada
> joao.carvalho@ic.unicamp.br
> joao.carvalho@ualberta.ca
>


-- 
Fragen sind nicht da um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 3149 bytes --]

Re: Tree-sitter doc question

[Yuan Fu]

> On Dec 11, 2022, at 12:31 AM, Pedro Andres Aranda Gutierrez <paaguti@gmail.com> wrote:
> 
> Obrigado, João Paulo
> 
> That's done :-) I now have an Emacs with tree-sitter and I have gone to the admin/... directory to compile the language support .so.
> Maybe the final step in that documentation would be to say a word on where to place those files when testing and how to 
> "politely encourage" Emacs to load and use them ;-)

admin/notes/tree-sitter/starter-guide has this:

and language definitions will be in the /dist directory. You can
either copy them to standard dynamic library locations of your system,
eg, /usr/local/lib, or leave them in /dist and later tell Emacs where
to find language definitions by setting ‘treesit-extra-load-path’.

So I think we are covered :-)

Yuan

Re: Tree-sitter doc question

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1315 bytes --]

THANKSSSS !!!
It is a bit hidden, though ;-) A bit of a highlight may not hurt for
too-quick readers like me...

/PA

On Sun, 11 Dec 2022 at 23:04, Yuan Fu <casouri@gmail.com> wrote:

>
>
> > On Dec 11, 2022, at 12:31 AM, Pedro Andres Aranda Gutierrez <
> paaguti@gmail.com> wrote:
> >
> > Obrigado, João Paulo
> >
> > That's done :-) I now have an Emacs with tree-sitter and I have gone to
> the admin/... directory to compile the language support .so.
> > Maybe the final step in that documentation would be to say a word on
> where to place those files when testing and how to
> > "politely encourage" Emacs to load and use them ;-)
>
> admin/notes/tree-sitter/starter-guide has this:
>
> and language definitions will be in the /dist directory. You can
> either copy them to standard dynamic library locations of your system,
> eg, /usr/local/lib, or leave them in /dist and later tell Emacs where
> to find language definitions by setting ‘treesit-extra-load-path’.
>
> So I think we are covered :-)
>
> Yuan



-- 
Fragen sind nicht da um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 1940 bytes --]

Re: Tree-sitter doc question

[Eric S Fraga]

On Saturday, 10 Dec 2022 at 12:14, Pedro Andres Aranda Gutierrez wrote:
> I'm on Ubuntu 20.04 LTS and have little time to face a migration to
> 22.04

You're not alone in this regard.  I am on Debian stable and the
libtree-sitter package is not available.  It is available on Debian
testing so I guess I'll just wait until it percolates downwards (or
should that be upwards?).

No big deal.  Just have to learn patience... ;-)

-- 
Eric S Fraga via gnus (Emacs 30.0.50 2022-12-02) on Debian 11.5

Re: Tree-sitter doc question

[Stefan Monnier]

> You're not alone in this regard.  I am on Debian stable and the
> libtree-sitter package is not available.

Really?

    https://packages.debian.org/unstable/libtree-sitter0
and
    https://packages.debian.org/unstable/libtree-sitter-dev

seem to disagree.


        Stefan

Re: Tree-sitter doc question

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Mon, 12 Dec 2022 12:59:01 -0500
> 
> > You're not alone in this regard.  I am on Debian stable and the
> > libtree-sitter package is not available.
> 
> Really?
> 
>     https://packages.debian.org/unstable/libtree-sitter0
> and
>     https://packages.debian.org/unstable/libtree-sitter-dev
> 
> seem to disagree.

Isn't that "UNstable"?

Re: Tree-sitter doc question

[Stefan Monnier]

>> Really?
>> 
>>     https://packages.debian.org/unstable/libtree-sitter0
>> and
>>     https://packages.debian.org/unstable/libtree-sitter-dev
>> 
>> seem to disagree.
>
> Isn't that "UNstable"?

Ah, yes, I used the wrong links, sorry, but my point still stands:

    https://packages.debian.org/bookworm/libtree-sitter0
and
    https://packages.debian.org/bookworm/libtree-sitter-dev


-- Stefan

Re: Tree-sitter doc question

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Really?
>>> 
>>>     https://packages.debian.org/unstable/libtree-sitter0
>>> and
>>>     https://packages.debian.org/unstable/libtree-sitter-dev
>>> 
>>> seem to disagree.
>>
>> Isn't that "UNstable"?
>
> Ah, yes, I used the wrong links, sorry, but my point still stands:
>
>     https://packages.debian.org/bookworm/libtree-sitter0
> and
>     https://packages.debian.org/bookworm/libtree-sitter-dev

Bookworm is Testing, not the current stable release which is named
Bullseye.

There are no libtree-sitter packages for Bullseye:

https://packages.debian.org/search?keywords=libtree-sitter&searchon=names&suite=stable&section=all

BTW, as I'm on Debian Testing getting the package is no problem. What I
don't know is how to install the parsers for the languages I'm
interested on.

Re: Tree-sitter doc question

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 320 bytes --]

On Mon, Dec 12, 2022 at 7:33 PM Óscar Fuentes <ofv@wanadoo.es> wrote:


> BTW, as I'm on Debian Testing getting the package is no problem. What I
> don't know is how to install the parsers for the languages I'm
> interested on.
>

admin/notes/tree-sitter/build-module/batch.sh

Or am I misunderstanding you?

[-- Attachment #2: Type: text/html, Size: 794 bytes --]

Re: Tree-sitter doc question

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Mon, Dec 12, 2022 at 7:33 PM Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>
>> BTW, as I'm on Debian Testing getting the package is no problem. What I
>> don't know is how to install the parsers for the languages I'm
>> interested on.
>>
>
> admin/notes/tree-sitter/build-module/batch.sh
>
> Or am I misunderstanding you?

Interesting, thank you.

Why is that script and the accompanying README under `admin'? Why are no
references to them on INSTALL?

Re: Tree-sitter doc question

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 12 Dec 2022 20:20:16 +0100
> 
> > admin/notes/tree-sitter/build-module/batch.sh
> >
> > Or am I misunderstanding you?
> 
> Interesting, thank you.
> 
> Why is that script and the accompanying README under `admin'? Why are no
> references to them on INSTALL?

Because it isn't our business to tell users how to build external
libraries.  They should ask their distros to do that, or build from
sources by themselves (I do the latter).

Re: Tree-sitter doc question

[Tim Cross]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Juanma Barranquero <lekktu@gmail.com> writes:
>
>> On Mon, Dec 12, 2022 at 7:33 PM Óscar Fuentes <ofv@wanadoo.es> wrote:
>>
>>
>>> BTW, as I'm on Debian Testing getting the package is no problem. What I
>>> don't know is how to install the parsers for the languages I'm
>>> interested on.
>>>
>>
>> admin/notes/tree-sitter/build-module/batch.sh
>>
>> Or am I misunderstanding you?
>
> Interesting, thank you.
>
> Why is that script and the accompanying README under `admin'? Why are no
> references to them on INSTALL?

I don't know the answer to those questions. However, I just tried that
script and it doesn't work on my Fedora 37 system. I also followed the
instructions for cloning the repo and building it yourself, which failed
with the same errors. The errors I get are basically of the form

Building c
/usr/bin/ld: parser.o: relocation R_X86_64_32S against `.rodata' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
cp: cannot stat 'libtree-sitter-c.so': No such file or directory

Looking at the scripts, it isn't clear where I need to add the -fPIC (it
already seems to be there?).

This occurs for every lang. Building emacs with ./configure
--with-tree-sitter seems to be fine (Emac 29 branch HEAD from this morning). 

Re: Tree-sitter doc question

[João Paulo Labegalini de Carvalho]

[-- Attachment #1.1: Type: text/plain, Size: 764 bytes --]

> Building c
> /usr/bin/ld: parser.o: relocation R_X86_64_32S against `.rodata' can not
> be used when making a shared object; recompile with -fPIC
> /usr/bin/ld: failed to set dynamic section sizes: bad value
> collect2: error: ld returned 1 exit status
> cp: cannot stat 'libtree-sitter-c.so': No such file or directory
>
> Looking at the scripts, it isn't clear where I need to add the -fPIC (it
> already seems to be there?).
>

You need to add to the like that builds the parser.c file, like in the
attached patch.

-- 
João Paulo L. de Carvalho
Ph.D Computer Science |  IC-UNICAMP | Campinas , SP - Brazil
Postdoctoral Research Fellow | University of Alberta | Edmonton, AB - Canada
joao.carvalho@ic.unicamp.br
joao.carvalho@ualberta.ca

[-- Attachment #1.2: Type: text/html, Size: 1311 bytes --]

[-- Attachment #2: 0001-Fix-scripts-that-builds-language-parsers.patch --]
[-- Type: text/x-patch, Size: 757 bytes --]

From 6085dd86945fb49aab3711911d3f7c19c4366152 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jo=C3=A3o=20P=2E=20L=2E=20de=20Carvalho?=
 <jaopaulolc@gmail.com>
Date: Mon, 12 Dec 2022 16:31:39 -0700
Subject: [PATCH] Fix scripts that builds language parsers.

---
 admin/notes/tree-sitter/build-module/build.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/admin/notes/tree-sitter/build-module/build.sh b/admin/notes/tree-sitter/build-module/build.sh
index d562f1a784..3eba8882da 100755
--- a/admin/notes/tree-sitter/build-module/build.sh
+++ b/admin/notes/tree-sitter/build-module/build.sh
@@ -39,7 +39,7 @@ grammardir=
 
 ### Build
 
-cc -c -I. parser.c
+cc -fPIC -c -I. parser.c
 # Compile scanner.c.
 if test -f scanner.c
 then
-- 
2.31.1

Re: Tree-sitter doc question

[Tim Cross]

João Paulo Labegalini de Carvalho <jaopaulolc@gmail.com> writes:

>  Building c
>  /usr/bin/ld: parser.o: relocation R_X86_64_32S against `.rodata' can not
>  be used when making a shared object; recompile with -fPIC
>  /usr/bin/ld: failed to set dynamic section sizes: bad value
>  collect2: error: ld returned 1 exit status
>  cp: cannot stat 'libtree-sitter-c.so': No such file or directory
>
>  Looking at the scripts, it isn't clear where I need to add the -fPIC (it
>  already seems to be there?).
>
> You need to add to the like that builds the parser.c file, like in the attached patch.

Thanks, that did the trick.

Re: Tree-sitter doc question

[Fraga, Eric]

On Monday, 12 Dec 2022 at 13:20, Stefan Monnier wrote:
>>> Really?
> Ah, yes, I used the wrong links, sorry, but my point still stands:

bookworm is "testing", not stable; stable is bullseye.

Yes, I could move to testing but I won't as stability is more important
(for me).  As I said in my original response, I'm happy to wait.

-- 
Eric S Fraga via gnus (Emacs 30.0.50 2022-12-02) on Debian 11.5