From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter introduction documentation Date: Fri, 30 Dec 2022 17:31:53 +0200 Message-ID: <83y1qo6h7a.fsf@gnu.org> References: <83edszjslp.fsf@gnu.org> <87tu1vxs3a.fsf@ledu-giraud.fr> <831qozjob7.fsf@gnu.org> <87cz8jxoat.fsf@ledu-giraud.fr> <83wn6ri7pn.fsf@gnu.org> <5e0a3185-de82-b339-0fa2-956779e63d6f@cornell.edu> <868rj6vfep.fsf@gmail.com> <4895891b-e5ea-9c37-f51b-df2e479ee758@yandex.ru> <83y1qt11xq.fsf@gnu.org> <9eb013da-d0fc-8e17-c6e3-1e8f913aebfa@yandex.ru> <83pmc50xxc.fsf@gnu.org> <71cfe4e8-3bb8-b0a6-9be5-8c0a6d92cfab@yandex.ru> <83h6xg29z3.fsf@gnu.org> <87wn6cyey5.fsf@posteo.net> <787B1EB4-1925-4679-8747-449DCD685432@gmail.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27705"; mail-complaints-to="usenet@ciao.gmane.io" Cc: philipk@posteo.net, monnier@iro.umontreal.ca, dgutov@yandex.ru, theophilusx@gmail.com, emacs-devel@gnu.org To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 30 16:32:52 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pBHNM-0006x8-06 for ged-emacs-devel@m.gmane-mx.org; Fri, 30 Dec 2022 16:32:52 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pBHMV-000502-I3; Fri, 30 Dec 2022 10:31:59 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pBHMT-0004zl-Ub for emacs-devel@gnu.org; Fri, 30 Dec 2022 10:31:58 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pBHMS-0007wF-JJ; Fri, 30 Dec 2022 10:31:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=bbH62aOgJaXWyx/KZZVwH/xc2C1KTmlL6vDxD8/8c/s=; b=bcQC2G7qXkpa 5zX3w01jMOuNNehWAPVCDQ2ke5PByCdL9MnML7JdXuz2hdQ7hvhm+29riCqChtnUkNxa5+45vAxty kq3YN4WMqzWtmlnQ2YxLjn6ifFITcco5h61CYp1mIxsWHWBDFnWMZgrXeV64VlyHxzgx2E/5EEWao nSDH8YqL3jDY8kv3xPNEYpqcFD7voryeUbCF83rNAIc4KoSsfNKgc2NmDiR8o+RWrdlghaMLl0ssl grztuVEzf5IwqDD5DG1OIuBgQthHoWNMCp9aZe02KZ58GdXZHDKjpq4iOTdhn/9vd73bsWnRnVXBE mZ1Kbypmq+qf5sZ+8FqcDw==; Original-Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pBHMR-0002Ie-C7; Fri, 30 Dec 2022 10:31:56 -0500 In-Reply-To: <787B1EB4-1925-4679-8747-449DCD685432@gmail.com> (message from Yuan Fu on Fri, 30 Dec 2022 03:06:37 -0800) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:302115 Archived-At: > From: Yuan Fu > Date: Fri, 30 Dec 2022 03:06:37 -0800 > Cc: Stefan Monnier , > Eli Zaretskii , > Dmitry Gutov , > theophilusx@gmail.com, > emacs-devel@gnu.org > > > I have asked the question before, but freedom or not, the above is a > > nuisance to run for every language. If the process is as automatic as > > the above example demonstrates, shouldn't Emacs have a command to take a > > grammar and compile+install it? I guess this could be more complicated > > if the grammar is generated using a custom tool-chain for each language > > (or is it always Javascript?), but nothing impossible. > > Though the magic of programming, such command now exists: treesit-install-language-grammar. It needs recipes to work, though. The recipe would involve https://github.com, which I guess is probably too heretical to include in Emacs source, so I left the recipes empty. I tested the install command with these recipes: > > (setq treesit-language-source-alist > '((python "https://github.com/tree-sitter/tree-sitter-python.git") > (typescript "https://github.com/tree-sitter/tree-sitter-typescript.git" > "typescript/src" "typescript"))) Thanks. I did some minor fixes to the doc strings, but this command still "needs work"(TM). See my comments below: This command requires Git, a C compiler and (sometimes) a C++ compiler, and the linker to be installed and on PATH. It also requires that the recipe for LANG exists in `treesit-language-source-alist'. I don't think treesit-language-source-alist is a good idea, especially if we don't intend populating it, at least not as a user-facing feature. Instead, the command should ask the user for the relevant values, and offer recording the values on some file that would be read next time the user wants to install an updated library. OUT-DIR is the directory to put the compiled library file, it defaults to ~/.emacs.d/tree-sitter. I don't understand what "defaults" means here, since OUT-DIR is not an optional argument of treesit--install-language-grammar-1. (let* ((lang (symbol-name lang)) (default-directory "/tmp") A literal "/tmp" is not portable and un-Emacsy; please use temporary-file-directory instead. (soext (pcase system-type ('darwin "dylib") ((or 'ms-dos 'cywin 'windows-nt) "dll") MS-DOS doesn't use DLL files. Please use dynamic-library-suffixes instead, it's already set up correctly. And the code should be ready for that variable having a nil value. (message "Cloning repository") ;; git clone xxx --depth 1 --quiet workdir (treesit--call-process-signal "git" nil t nil "clone" url "--depth" "1" "--quiet" workdir) Why "--depth 1"? This should be a defcustom, and the default should be to clone the full repository, IMO. Also, what about updating the library when it is already installed, and the Git repository already exists for it? Or are we going to clone anew each time and them remove the repository? that could make its cloning be slow in some cases. ;; cp "${grammardir}"/grammar.js "${sourcedir}" (copy-file (concat grammar-dir "/grammar.js") (concat source-dir "/grammar.js")) Why is this part needed? In any case, please don't use concat to produce file names, use expand-file-name instead. Also, we should call copy-file with 4th argument non-nil, I think. (treesit--call-process-signal cc nil t nil "-fPIC" "-c" "-I." "parser.c") I wonder why we don't use 'compile' here. That would show the compiler output without any extra efforts. ;; Copy out. (copy-file lib-name (concat out-dir "/") t) See above: don't use concat here. This command should also be mentioned in NEWS, where we describe how to install the grammar libraries. Bottom line: I think we need first to discuss how we want such a facility to work, and only then implement it.