From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter api Date: Mon, 27 Sep 2021 22:33:17 -0700 Message-ID: <784947A6-6164-4158-993B-01E881690ADF@gmail.com> References: <83r1f7hydn.fsf@gnu.org> <83lf3zdh4z.fsf@gnu.org> <8965C4A0-79D3-4D77-A6BA-D07A6C93F7FE@gmail.com> <83ilz3cs4k.fsf@gnu.org> <04D19C1A-CD64-4156-B932-1C9FEEE4EC7B@gmail.com> <83zgsebc0r.fsf@gnu.org> <1F752923-F357-4A18-B6E2-0120F1B9BD37@gmail.com> <83fsu5bzem.fsf@gnu.org> <83zgsdad5j.fsf@gnu.org> <83sfy391ni.fsf@gnu.org> <03386E3C-A975-4ECD-BF89-6AC62F751725@gmail.com> <83ilyz8xdl.fsf@gnu.org> <86czp6ukpj.fsf@stephe-leake.org> <1AEED8A5-BF5E-45E5-AE22-9B405A710F9B@gmail.com> <83czp313tf.fsf@gnu.org> <7447C582-70CB-4599-BC76-AD5687D64250@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="19807"; mail-complaints-to="usenet@ciao.gmane.io" Cc: =?utf-8?B?VHXhuqVuLUFuaCBOZ3V54buFbg==?= , Theodor Thornhill , =?utf-8?Q?Cl=C3=A9ment_Pit-Claudel?= , Emacs developers , Eli Zaretskii , Stephen Leake , John Yates To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Sep 28 07:36:14 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mV5mn-0004rz-ML for ged-emacs-devel@m.gmane-mx.org; Tue, 28 Sep 2021 07:36:13 +0200 Original-Received: from localhost ([::1]:43782 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mV5mm-0002LC-DU for ged-emacs-devel@m.gmane-mx.org; Tue, 28 Sep 2021 01:36:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:56068) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mV5k3-0006IG-6X for emacs-devel@gnu.org; Tue, 28 Sep 2021 01:33:24 -0400 Original-Received: from mail-qk1-x72a.google.com ([2607:f8b0:4864:20::72a]:42871) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mV5k1-0005NM-Ak; Tue, 28 Sep 2021 01:33:22 -0400 Original-Received: by mail-qk1-x72a.google.com with SMTP id x12so6139685qkf.9; Mon, 27 Sep 2021 22:33:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=YxKbnfrY8EWOQQ5w9v/trXlo9Cy61nqmRj75VYLnI+g=; b=ZfK9jJJbQBCJObWdZkWFErUkMcvlaF0Zi5FLASVQV3sijwXibQlJRt5SWxYyjXf7Oq +Z50LwXhG1hPJy47zeWdQ8s/z67Z2A7jFbv1Ze8UWYFCEZusjIj6I24Xp37fcqlirTgs 4ayBcYKgFnbC6r/jODN9Z3lc3ZVOoeTZFFeurgbLMi+pf5FJO7sbcUU+5xfW+Q1q882e mZRev4t//qIY1Ijq5AFHfkY86XL6iQuR1sPcU+rLrEPYaJa+UziYLuOLNM5HoztYDOX0 88463kR0uUgwVKIO8xyXYhUk2ufIwLkQ+m6iz/BmL55qjdsKdWzGhPd251b4Yf2ESpOG PspA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=YxKbnfrY8EWOQQ5w9v/trXlo9Cy61nqmRj75VYLnI+g=; b=FVTzsiBa1Q2ayNjsGRQKdY81qn0Fhsqx9zJB9qoyUS5brzJbaakJOgzg1MUhwXRrS3 Y3NCOC/4S09uoQV8J8ZEs3pBrFUcsNLHhG7p82dKr/nFjyUBTjJaCxMsYmtsu14vdEQp 5wv/Y6wcb9YtI58mlujXMBNuKPdFU0EXpLSAVBEo96vm9g2vrkei3GeEwmQ2c6EqVbLN OeI4R2FYGTJHn0XYN84H0zPCH3MOHt2VFcmgt3EhUXeHFnFFSzhxHaD8TTbJDf9G/46e H6/nQbhR/z9uFObnqXFcQoxaxI9hVMXN0YD39udJIFBqOF2X6YtopPdw7c9LhHRbqezJ iAFQ== X-Gm-Message-State: AOAM532YB/NOV5rl1/NzfpO+slwUJ6y95PdkclKmUHPc7mn45qU4OLv0 izqT6govy9/SYggy9Q9DfTU= X-Google-Smtp-Source: ABdhPJzfplc39N7BWmoj0DDp1s6Vs+5W+UYkHN4qbZ4ezVFoI25ICjwcEoPhLit+ysPDhNsGTYFhgQ== X-Received: by 2002:a05:620a:14f:: with SMTP id e15mr3655152qkn.25.1632807199329; Mon, 27 Sep 2021 22:33:19 -0700 (PDT) Original-Received: from smtpclient.apple ([2600:1700:2ec7:8c9f:c95c:f98c:63f8:fcf4]) by smtp.gmail.com with ESMTPSA id 75sm14026011qkh.121.2021.09.27.22.33.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Sep 2021 22:33:18 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.3654.120.0.1.13) Received-SPF: pass client-ip=2607:f8b0:4864:20::72a; envelope-from=casouri@gmail.com; helo=mail-qk1-x72a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:275650 Archived-At: > On Sep 27, 2021, at 12:17 PM, Stefan Monnier = wrote: >=20 >> Currently, because font-lock.el uses functions and variables defined = in >> tree-sitter.el, >=20 > Why? I don't see any reason why you'd need to change font-lock.el to > add support for tree-sitter fontification. >=20 [Also in reply to Eli:] Before tree-sitter, font-lock roughly consists of two passes, the = syntactic pass (that uses the syntax table) and regex pass (that uses = regex matching). I added a three pass, tree-sitter pass, because I want = to add tree-sitter fontification on top of existing mechanisms, not = replacing it. This way we can still fontify keywords. Simply replacing = font-lock with tree-sitter font-lock would cause anything relying on the = existing fontification facility stop to work if I turn on tree-sitter. = For example, I use a package that fontify keywords like =E2=80=9CTODO=E2=80= =9D and =E2=80=9CFIXME=E2=80=9D, it would be a shame if it stops working = as soon as I turn on tree-sitter fontification. So it seemed natural to = me to augment font-lock.el instead of putting stuff in tree-sitter.el. = Now that I realized the dependency problem, I think I can move all the = font-lock integration code to tree-sitter.el and leave font-lock.el = untouched, but still maintain the augmentation nature. E.g., define a = tree-sitter-fontify-region-function that first calls = font-lock-fontify-region-function, then does tree-sitter fontification. = And user can turn on tree-sitter font-lock with, say = tree-sitter-font-lock-mode. With that said, I still have one thing not too sure. What should = tree-sitter.el do if libtree-sitter is not on the system, and = tree-sitter.c is not included in Emacs? Should we simply not include = tree-sitter.el? Is there existing build facility that can do that = (exclude tree-sitter.el when libtree-sitter is not found on system)? Yuan=