Andrew Tropin writes: > [[PGP Signed Part:Undecided]] > On 2023-02-10 17:02, Pierre Langlois wrote: > >> Pierre Langlois writes: >> >>> [[PGP Signed Part:Undecided]] >>> Hi Andrew, thanks for pushing this along! It's great to see things >>> getting merged. >>> >>> Andrew Tropin writes: >>> >>>> [[PGP Signed Part:Undecided]] >>>> On 2023-02-09 18:04, Andrew Tropin wrote: >>>> >>>>> On 2023-02-09 13:39, zimoun wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin wrote: >>>>>> >>>>>>> I applied tree-sitter and tree-sitter-cli patches, >>>>>> >>>>>> Just to be sure to understand, you have only applied 02/32 and 05/32, >>>>>> right? >>>>>> >>>>>> >>>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7. >>>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com >>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com >>>>>> >>>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli. >>>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com >>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com >>>>>> >>>>>> Leaving out all the others, right? >>>>> >>>>> Merged first 5 patches from 01 to 05, also added one more commit, which >>>>> addresses some things from reviews and one commit, which adds html >>>>> grammar. >>>>> >>>>> The html grammar is added for the testing purposes. It relies on >>>>> generated parser.c and scanner.c and we will need to repackage it using >>>>> grammar.js instead. I'm not sure if a separate build system is needed >>>>> for this, I guess we can just rewrite tree-sitter-grammar function, >>>>> which generates packages as in example with tree-sitter-grammar-html: >>>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158 >>>>> >>>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to >>>>> read it carefully, evaluate and either introduce new build system or >>>>> just move all needed parts to tree-sitter-grammar function. WDYT? >>>>> After we done with it we can package all other grammars. >>>> >>>> Ok, I realized that the proper build process for tree-sitter grammars is >>>> a little harder than I expected, tree-sitter-build system make sense. I >>>> reviewed it, made a small change: >>> >>> Ah great, I was going to comment to try and push for us to keep the >>> build system. I originally went with a template package and inheritance, >>> but Maxime suggested moving to a build-system which ended up making the >>> package definitions a *lot* nicer IMO (see previous discussion here >>> https://issues.guix.gnu.org/49946#144). It also allows us to deal with >>> grammars that depend on each other more nicely I think. >>> >>>> >>>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system) >>>> ;; Commentary: >>>> ;; >>>> ;; Build procedures for tree-sitter grammar packages. This is the >>>> -;; builder-side code, which builds on top fo the node build-system. >>>> +;; builder-side code, which builds on top of the node build-system. >>>> ;; >>>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native >>>> ;; shared object. The `tree-sitter generate' command invokes `node' in order >>>> @@ -114,7 +114,7 @@ (define (compile-language dir) >>>> "-fno-exceptions" >>>> "-O2" >>>> "-g" >>>> - "-o" ,(string-append lib "/" lang ".so") >>>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so") >>>> ;; An additional `scanner.{c,cc}' file is sometimes >>>> ;; provided. >>>> ,@(cond >>>> >>>> >>>> rewrote html grammar to use this build system and made it work with >>>> built-in treesit package. Also, tried examples of c and cpp grammars >>>> from patches in this thread. >>>> >>>> If you ok with it, I'll push the build system to master and update the >>>> html grammar accordingly. >> >> Oh, I forgot to say, this change to the build system LGTM! I'm really >> happy to see it merged soon :-). The path change will probably break the >> emacs-28-based tree-sitter support, but that's OK, it's better for the >> build-system to be made to target emacs 29's builtin support. I'm sure I >> can work around for emacs 28. > > Actually, I think we can build grammars with both names, just providing > two .so files instead of one. If you won't find a better workaround we > can go this way. That's no problem, I was able to work-around it quite easily. The emacs-tree-sitter-langs package expects all the grammars supported to be in a single directory, so we have to bundle them. So we can quite easily create symlinks from the bundle with the expected names, something like: --8<---------------cut here---------------start------------->8--- (define-public emacs-tree-sitter-langs-grammar-bundle (package (name "emacs-tree-sitter-langs-grammar-bundle") (source #f) (version (package-version tree-sitter)) (build-system trivial-build-system) (inputs ;; FIXME: Support for some languages is still left to package. (list tree-sitter-bash tree-sitter-c tree-sitter-c-sharp tree-sitter-cpp tree-sitter-css tree-sitter-elixir tree-sitter-elm tree-sitter-go tree-sitter-haskell tree-sitter-html tree-sitter-java tree-sitter-javascript tree-sitter-json tree-sitter-julia tree-sitter-lua tree-sitter-ocaml tree-sitter-php tree-sitter-python tree-sitter-r tree-sitter-rust tree-sitter-ruby tree-sitter-typescript)) (arguments (list #:builder (with-imported-modules '((guix build union) (guix build utils)) #~(begin (use-modules (ice-9 match) (srfi srfi-1) (guix build union) (guix build utils)) (mkdir-p #$output) (for-each (lambda (lib) (symlink lib (string-append #$output "/" (substring (basename lib) (string-length "libtree-sitter-"))))) (append-map (match-lambda ((name directory) (find-files directory "libtree-sitter-.*\\.so$"))) '#$(package-inputs this-package))))))) (synopsis #f) (description #f) (home-page #f) (license #f))) --8<---------------cut here---------------end--------------->8--- Giving us: --8<---------------cut here---------------start------------->8--- /gnu/store/fr9h59wgn55ilfifvm5df7xzxfwh38pc-emacs-tree-sitter-langs-grammar-bundle-0.20.7 ├── bash.so -> /gnu/store/9qzvcdlpryjl44klfq0i2liqf6lsp5jq-tree-sitter-bash-0.19.0/lib/tree-sitter/libtree-sitter-bash.so ├── cpp.so -> /gnu/store/my02kq2dr6h48nmrl2dnfnm5rggx837w-tree-sitter-cpp-0.20.0-1.5ead1e2/lib/tree-sitter/libtree-sitter-cpp.so ├── c_sharp.so -> /gnu/store/mp8jvbhx5xlgj5bxa52zhmk5x8i7md5h-tree-sitter-c-sharp-0.19.1/lib/tree-sitter/libtree-sitter-c_sharp.so ├── c.so -> /gnu/store/y9ln76yx68q495vx8bnqdy87x7k8ihy5-tree-sitter-c-0.20.2/lib/tree-sitter/libtree-sitter-c.so ├── css.so -> /gnu/store/fwji59sdwvqpqyb94m55qw1ak92cmys9-tree-sitter-css-0.19.0/lib/tree-sitter/libtree-sitter-css.so ├── elixir.so -> /gnu/store/f6ismnxf7hlh1sq87zzarw56avvrzsl9-tree-sitter-elixir-0.19.0-1.de20391/lib/tree-sitter/libtree-sitter-elixir.so ├── elm.so -> /gnu/store/56zjl2ljvba3z1j6zfmpfa8mq19a3g3a-tree-sitter-elm-5.6.3/lib/tree-sitter/libtree-sitter-elm.so ├── go.so -> /gnu/store/qfy5sw6za96wkszbi21adxbxzggvjz3g-tree-sitter-go-0.19.1-1.05900fa/lib/tree-sitter/libtree-sitter-go.so ├── haskell.so -> /gnu/store/cq3chmqcb8g1nf5mzb6yhzdd6x9gvg1v-tree-sitter-haskell-0.14.0-1.e30bdfd/lib/tree-sitter/libtree-sitter-haskell.so ├── html.so -> /gnu/store/5pqfyjrg3yyvxaxidk690ffls3yb7wbi-tree-sitter-html-0.19.0/lib/tree-sitter/libtree-sitter-html.so ├── javascript.so -> /gnu/store/28s09v3dfb1c9bdkq2791z9abxnih66p-tree-sitter-javascript-0.20.0/lib/tree-sitter/libtree-sitter-javascript.so ├── java.so -> /gnu/store/i7jlqf5hbv0rhlkp4qlyc2d5ndw69dx4-tree-sitter-java-0.20.1/lib/tree-sitter/libtree-sitter-java.so ├── json.so -> /gnu/store/5dv4r74p6gd9w2ncs6pjrhz5hbw47262-tree-sitter-json-0.19.0/lib/tree-sitter/libtree-sitter-json.so ├── julia.so -> /gnu/store/5k37g1sdsllgh64p0w6ggabsni6jqlkr-tree-sitter-julia-0.19.0/lib/tree-sitter/libtree-sitter-julia.so ├── lua.so -> /gnu/store/nqzn4a6kgb2rx4y44pxdm4sqf3pzcpz1-tree-sitter-lua-0.0.14/lib/tree-sitter/libtree-sitter-lua.so ├── ocaml_interface.so -> /gnu/store/3h7krcj3xxclirb8afxh65ipabw1821l-tree-sitter-ocaml-0.19.0-1.0348562/lib/tree-sitter/libtree-sitter-ocaml_interface.so ├── ocaml.so -> /gnu/store/3h7krcj3xxclirb8afxh65ipabw1821l-tree-sitter-ocaml-0.19.0-1.0348562/lib/tree-sitter/libtree-sitter-ocaml.so ├── php.so -> /gnu/store/ymxf5m8jhihbrag7v2pghgydj3byp7wh-tree-sitter-php-0.19.0-1.435fa00/lib/tree-sitter/libtree-sitter-php.so ├── python.so -> /gnu/store/ggmzicwfxb7gz1rr9lfkx8cak62bfw7v-tree-sitter-python-0.19.1-1.ed0fe62/lib/tree-sitter/libtree-sitter-python.so ├── r.so -> /gnu/store/y9dxbnbb5dyf0rq3kpar7ip4w3lq6sb9-tree-sitter-r-0.0.1-1.80efda5/lib/tree-sitter/libtree-sitter-r.so ├── ruby.so -> /gnu/store/ky8n30dw16ck6byaqnhbf9ib7xp7j0yw-tree-sitter-ruby-0.20.0/lib/tree-sitter/libtree-sitter-ruby.so ├── rust.so -> /gnu/store/25zdpwrgq1xibhv7xpg64i4g71xah6g1-tree-sitter-rust-0.20.1/lib/tree-sitter/libtree-sitter-rust.so ├── tsx.so -> /gnu/store/wqpcphz855yjrginwqrymd3xzzxb8k8l-tree-sitter-typescript-0.20.1/lib/tree-sitter/libtree-sitter-tsx.so └── typescript.so -> /gnu/store/wqpcphz855yjrginwqrymd3xzzxb8k8l-tree-sitter-typescript-0.20.1/lib/tree-sitter/libtree-sitter-typescript.so --8<---------------cut here---------------end--------------->8---