From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail
From: Yuan Fu <casouri@gmail.com>
Newsgroups: gmane.emacs.devel
Subject: Re: New package emacs-parser-generator
Date: Mon, 29 Nov 2021 11:22:49 -0800
Message-ID: <8ECC2C6B-9BB0-442F-98A5-39AB88E2AC12@gmail.com>
References: <838rx7w3cv.fsf@gnu.org>
 <B58DD8E6-5088-487E-934E-652D928493A1@cvj.se>
Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\))
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214";
	logging-data="12230"; mail-complaints-to="usenet@ciao.gmane.io"
Cc: Eli Zaretskii <eliz@gnu.org>,
 =?utf-8?Q?Daniel_Mart=C3=ADn?= <mardani29@yahoo.es>, monnier@iro.umontreal.ca,
 emacs-devel@gnu.org
To: Christian Johansson <christian@cvj.se>
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Nov 29 20:52:42 2021
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane-mx.org
Original-Received: from lists.gnu.org ([209.51.188.17])
	by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>)
	id 1mrmhe-00030q-GN
	for ged-emacs-devel@m.gmane-mx.org; Mon, 29 Nov 2021 20:52:42 +0100
Original-Received: from localhost ([::1]:45686 helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>)
	id 1mrmhc-0005UX-DV
	for ged-emacs-devel@m.gmane-mx.org; Mon, 29 Nov 2021 14:52:40 -0500
Original-Received: from eggs.gnu.org ([209.51.188.92]:33838)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <casouri@gmail.com>) id 1mrmEp-00089m-GN
 for emacs-devel@gnu.org; Mon, 29 Nov 2021 14:22:55 -0500
Original-Received: from [2607:f8b0:4864:20::82f] (port=35441
 helo=mail-qt1-x82f.google.com)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <casouri@gmail.com>)
 id 1mrmEn-00029Y-7N; Mon, 29 Nov 2021 14:22:55 -0500
Original-Received: by mail-qt1-x82f.google.com with SMTP id j17so17727095qtx.2;
 Mon, 29 Nov 2021 11:22:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:subject:from:in-reply-to:date:cc
 :content-transfer-encoding:message-id:references:to;
 bh=PrF3q5LaKeoaDcoJyxDqzBkBLnOS8jYmk5LkObe5pXQ=;
 b=nwKfHfc0Op9xaRzb6n8xQ62zLdadJLCjeWoG7R+GeHi61XzU8sa8u3Jdrf3WYdtSlM
 ygEuCOTe0mUSKGY6MiSYdr2ZYWLIJUxmC5L+H2MKzCO2oeAofQyxLP7CXQHH4AnPXzsJ
 7BZftOC0SN3f/fKU1L/ozOW/xaFUgy1KbZsyg70mLhSCfsd4s4Xl6N2m14h7rSYfO1ep
 +I0XS+7pvvWckPIj6jsKkddPqPkdY+TcLxlfF+OmdAz+rsmsnGijd1cONssHiTzJXbgs
 hFMEAAuf7bfkeWwJZpB49xt7tx1ncgSCcM8bkYg9idbEbGxLFxICd9yuyQwKBc0DFwEd
 UXOw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc
 :content-transfer-encoding:message-id:references:to;
 bh=PrF3q5LaKeoaDcoJyxDqzBkBLnOS8jYmk5LkObe5pXQ=;
 b=7+yYNVCD0+MlDXHiu9jsb2AmwvtPIpOhaoUdxlEPaSjMl5PgiviOu8LvLNuUheS0jy
 2PjwlU/mNK9sYzlS23DS61nwLSR/MMdzF9zGfzbdWoyWYS3bSY01oVML5SzO2LZUv5+2
 Mzmls9+na4QMCxOAgEf/+K1ZKCbajzgT9vBQZ+4iCX8M0gSdGAjC3HGOh1X1le4gszbN
 rWYJuhb+CPMWyOapRr9ZxKf20NNn4qBUXBoCJ8DyYd0H55r7YpDjLo9A0bcTwdUeB/RQ
 dsWlGaoc75mLRnjqHGC4eLtPP4KmJqZFXKwdDhYAR3MW+IYNljh3xO02JvkC6ZhrG56n
 Ph4g==
X-Gm-Message-State: AOAM531vSfmmDVA32lxdhHoV+oHywLJThnBUrG9pzY7Xei+sMjxnjGoj
 mdF+T04skQ7+TsZ/lnx5NOI=
X-Google-Smtp-Source: ABdhPJx+cSPbnc2C+dczu3+tUXCORBTgutGXrxB7305QMmRXWO+rt+qkoVJDg7oGefmslDzUHtr62A==
X-Received: by 2002:ac8:5fc2:: with SMTP id k2mr46109458qta.310.1638213771856; 
 Mon, 29 Nov 2021 11:22:51 -0800 (PST)
Original-Received: from smtpclient.apple ([2600:1700:2ec7:8c90:4513:869e:48de:4665])
 by smtp.gmail.com with ESMTPSA id w13sm9181603qko.20.2021.11.29.11.22.50
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 29 Nov 2021 11:22:51 -0800 (PST)
In-Reply-To: <B58DD8E6-5088-487E-934E-652D928493A1@cvj.se>
X-Mailer: Apple Mail (2.3693.20.0.1.32)
X-Host-Lookup-Failed: Reverse DNS lookup failed for 2607:f8b0:4864:20::82f
 (failed)
Received-SPF: pass client-ip=2607:f8b0:4864:20::82f;
 envelope-from=casouri@gmail.com; helo=mail-qt1-x82f.google.com
X-Spam_score_int: -12
X-Spam_score: -1.3
X-Spam_bar: -
X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
 <mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
 <mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org
Original-Sender: "Emacs-devel"
 <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>
Xref: news.gmane.io gmane.emacs.devel:280506
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/280506>

The tree-sitter integration doesn=E2=80=99t provide anything to define a =
grammar nor generating a parser. It merely uses pre-defined grammar =
(written by tree-sitter community) to parse buffers and produce an AST.

To write an tree-sitter grammar definition you need to write the grammar =
in JavaScript and pass it to tree-sitter=E2=80=99s parser generator. The =
generator spits out a grammar definition encoded in a C source file (in =
the form of a C struct). To use this grammar definition, we compile it =
to a library, load it at runtime, pass the struct to tree-sitter =
library, and tree-sitter can now parse according to this grammar =
definition.

The tree-sitter integration provides 1) Lisp wrappers for =
tree-sitter=E2=80=99s C API; 2) some convenient functions built on the C =
API; 3) integration with font-lock and indentation. I didn=E2=80=99t add =
any =E2=80=9Ccommon API=E2=80=9D, I simply used existing frameworks in =
Emacs: for font-lock I used font-lock-fontify-region-function, for =
indent I used indent-line-function. If in the future a common API for =
parses is desirable, tree-sitter can easily comply.

IOW, this is what tree-sitter integration currently does:

font-lock                                indent
      |                                     |
font-lock-fontify-region-function        indent-line-function
      |                                     |
      |                                     |
tree-sitter-font-lock-fontify-region     tree-sitter-indent-function

This is what could happen if we want a common API:

font-lock                                indent
      |                                     |
font-lock-fontify-region-function        indent-line-function
      |                                     |
      |                                     |
     common  API  --------------------------+
     /         \
    /           \
   /             \
tree-sitter    other parser

Yuan=