From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Gregory Heytings Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter introduction documentation Date: Fri, 30 Dec 2022 15:23:27 +0000 Message-ID: <6c9d91cffcc2db9d5b78@heytings.org> References: <868rj6vfep.fsf@gmail.com> <4895891b-e5ea-9c37-f51b-df2e479ee758@yandex.ru> <83y1qt11xq.fsf@gnu.org> <9eb013da-d0fc-8e17-c6e3-1e8f913aebfa@yandex.ru> <83pmc50xxc.fsf@gnu.org> <71cfe4e8-3bb8-b0a6-9be5-8c0a6d92cfab@yandex.ru> <83h6xg29z3.fsf@gnu.org> <838ris22n4.fsf@gnu.org> <8335901zz3.fsf@gnu.org> <87cz84y5le.fsf@posteo.net> <3F91FDEA-881A-49DB-BB52-5A0D81C004CE@gmail.com> <87k02aihrz.fsf@posteo.net> <6c9d91cffc5a0ed2c6b9@heytings.org> <87358xb1gf.fsf@posteo.net> <6c9d91cffc0fa7da3033@heytings.org> <87a6359gvs.fsf@posteo.net> Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=us-ascii Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28147"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Yuan Fu , Stefan Monnier , Eli Zaretskii , Dmitry Gutov , Tim Cross , emacs-devel@gnu.org To: Philip Kaludercic Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 30 16:24:17 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pBHF3-0007A3-Ec for ged-emacs-devel@m.gmane-mx.org; Fri, 30 Dec 2022 16:24:17 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pBHEO-0000CY-27; Fri, 30 Dec 2022 10:23:36 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pBHEL-0000Am-Rz for emacs-devel@gnu.org; Fri, 30 Dec 2022 10:23:34 -0500 Original-Received: from heytings.org ([95.142.160.155]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pBHEK-0005Wm-1H; Fri, 30 Dec 2022 10:23:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heytings.org; s=20220101; t=1672413808; bh=XbNgE4zM8NNmpJudr/BPr0OIBpDrWDGfcY2ISk/M6FY=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References:From; b=3bXssyQlLpMXqY7ZNRajZMDF5b9RawMW4ZmGYpgaBFlnpa1IvRAjAjVCGWxjMBCWh lEhdKsNtmNYydx2TxW09LdLPe4QfTxisLYoPlQ2ceo25v7arcmSOgIbawdfLq6ocZm aujwo4tL13pKWNg9+dje7zfYfHTKArcrVGRr5YCKn5c6WX/y0rE1ps2SXa2h5hkzPA cX4UNUAx5ZVyIgSpH0AKtmrYqDzZsfYStOmq4rCSOLe9a0FuCiOs8z7sOJUygWX440 ZVf+EsUk3N7lzwVlPjXFZYI0p+5tpo+k2fPkFGh/CSA58u13BK8RxTG77bp5Q/rEuO wmRo6H0SoNYNg== In-Reply-To: <87a6359gvs.fsf@posteo.net> Received-SPF: pass client-ip=95.142.160.155; envelope-from=gregory@heytings.org; helo=heytings.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:302113 Archived-At: >> The script I sent is meant to clarify that point: to show how and to >> what extent it is possible, and what the complications (having to >> modify the source grammars manually) are. > > You are right, sorry about that. > No worries. I extended the hack a bit further: global = {} module = {} process = { env: { TREE_SITTER_GRAMMAR_PATH: 'grammar.js' } } function require(file) { const pref = [ "", "./", "../", "../../", "../../../" ]; const suff = [ "", ".js" ]; for (let i in pref) for (let j in suff) { const f = pref[i] + file + suff[j]; if (std.open(f, "r") !== null) { os.chdir(f.match(/.*\//)); eval(std.loadFile(f.replace(/.*\//, ''))); return module.exports; } } throw Error('File ' + file + ' not found'); } std.loadScript('/path/to/the/script/dsl.js') With that script, if you clone the 63 repositories listed on https://tree-sitter.github.io/tree-sitter/, and the tree-sitter-clojure repository (which is used by tree-sitter-commonlisp), in the same directory, you can generate 64 out of the 66 grammar.json files (two repositories contain two grammars: tree-sitter-typescript and tree-sitter-wasm) without any modification with: qjs --std /path/to/that/script.js > src/grammar.json The two exceptions are swift (in the the grammar.json file produced by qjs, two values seem to be misplaced) and toml (because it depends on regexp-util).