From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail
From: Ergus <spacibba@aol.com>
Newsgroups: gmane.emacs.devel
Subject: Re: cc-mode fontification feels random
Date: Sat, 12 Jun 2021 03:08:44 +0200
Message-ID: <20210612010844.45noqsg7wveeo3yw@Ergus>
References: <179f6e4fa40.2816.cc5b3318d7e9908e2c46732289705cb0@dancol.org>
 <83fsxpbpn9.fsf@gnu.org> <YMOLJ4xsdkt2alAg@ACM>
 <83k0n09tkp.fsf@gnu.org>
 <f1d3738b-b35d-b585-a596-147445784fd6@dancol.org>
 <jwvr1h8z2su.fsf-monnier+emacs@gnu.org> <837dj09p0e.fsf@gnu.org>
 <jwvbl8cyy5o.fsf-monnier+emacs@gnu.org>
 <20210611232535.b4dyu3a2yxvdixys@Ergus>
 <87a6nw6jtf.fsf@telefonica.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214";
	logging-data="2147"; mail-complaints-to="usenet@ciao.gmane.io"
Cc: emacs-devel@gnu.org
To: =?utf-8?B?w5NzY2Fy?= Fuentes <ofv@wanadoo.es>
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jun 12 03:09:44 2021
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane-mx.org
Original-Received: from lists.gnu.org ([209.51.188.17])
	by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>)
	id 1lrs9f-0000Pt-Uf
	for ged-emacs-devel@m.gmane-mx.org; Sat, 12 Jun 2021 03:09:44 +0200
Original-Received: from localhost ([::1]:38808 helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>)
	id 1lrs9f-0000oF-1b
	for ged-emacs-devel@m.gmane-mx.org; Fri, 11 Jun 2021 21:09:43 -0400
Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60342)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <spacibba@aol.com>) id 1lrs8v-00007N-GH
 for emacs-devel@gnu.org; Fri, 11 Jun 2021 21:08:57 -0400
Original-Received: from sonic304-10.consmr.mail.bf2.yahoo.com ([74.6.128.33]:44614)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <spacibba@aol.com>) id 1lrs8s-0006Gn-Sh
 for emacs-devel@gnu.org; Fri, 11 Jun 2021 21:08:57 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aol.com; s=a2048;
 t=1623460133; bh=QeYij9Uam4zXGO70uG9NvnDDgwaEdXVKnDZsdgj4kSw=;
 h=Date:From:To:Cc:Subject:References:In-Reply-To:From:Subject:Reply-To;
 b=I6BFHt3ej1G6tka6m6U5hV85/KFRUgiKn8eR0KVOO4X+Y1tuimWSWQPBrbTroX6CqF6A9Ng6KsOrNryCNkFLJPLMIupuPkQ+QEpHFoD7UwdIX5GqsdvNJJYeR224GbXcii75VS/+6n0/XRWwPvibYZsJ2XzkTbSVnDnio+QEr5RLp0FQLGnnTiIxBIlQvurySiusO6JsQBbT8ANITREI8c1eBpASQPvDB2Oh/PPtnXFiJwCQz6v09FbYqjzpgiheH8rQTrx8AkkdfPfgamVVNr4NgiCxxLXw8CnV6dBBq/swAYH9jzAN5KRiexHmTkWCLAJrxiIwIHW1014g5UZV+Q==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048;
 t=1623460133; bh=MvSZR2kPrxMLlWdHJ5NHEy1mXFM04YlWYXz0CFLNA2l=;
 h=X-Sonic-MF:Date:From:To:Subject:From:Subject;
 b=ARM1oHl/u7O4lyPjLC356d+zlbY5wp/HhntWUhW5cnetML4tHtMJnwYv2lBD1cRZNdiRHLNgGkZ+x+ycaO4g3vWkA5oDkA0q6EyjxQ+xRY/M6GFZwXd4U+xV65N8e07B1w8hqYuz50TbTPfY06eDeE2+fHwqy+yNcg6Hg9uQNvUd1mtQ2TRX2lIhs/+fmbrvnt/xStSjchz/DLP6L6/eDQWi1eXsHLrTxhwioEaLmvNosX1IyBlERx+hlMTeNSGDyZGw1qU/0oALNF72Mb05e0H0+oytYwbvADM6pBKHZc2ILtVWK/RdMWr9UR4KlG9rvU0EQnTBiNbuafGTh4nl9A==
X-YMail-OSG: 5SIH7_EVM1k4_KW2E5VnJcyUSaHVBijmQ3UXPEFhNFhpvBzti.H3IwXaAU9Z6UQ
 gIw.rQ.rMk0pAV_xM5BwHqybLKJG7ih.B0MowaX2FvBbvKYLq4yM0J.Eq6By4.pKKRwIn6jHK_IV
 vak8R2iH_g9eNfez3d1RtmsRFnpPiRXuOpaFyuC0IbMiXbhBjDYARclnwPmxH.5cLd8onOTUsm8G
 rUHVN3AgRq1kgCbDwAkvswNwA4atqeSF1Hmue4j6imJ9y.djYCQWaKhzZgMej6aEpxS975v21xZv
 vlZEsTN8kZoV2GPZi.3Gvh0pV.3fKYLmY_Tcz7QpPYOHKv6BChyd7ZuBh_piVqdjyEq88M1iAdxk
 kzOfwWr6xmkZrnTKv6Kr5lOjc.RHXPxQL87xjtE5McqdxUxI8XpYGsVtKb0anWRq4HsQK94UfUkr
 jXLLQlo2g6SyergvMoa3bLvS6xGLLvFWsMr9V4GZT0aj6eJaH8FURMXduR1m5UnmFIBWjVORZEqp
 WKuwB4OYyCg5SrU3.a6CUE2r.1O0v5gOV4Q4EpMJLWzjBkuB8KHmKeATXFaT_6IpCgidT_rEMCpS
 MygV9Z34.5KvGJW6P6aPt8h7ofATds2yGkSCZ5Hrt.evKozpO4H.JlHRHHCvZk3KB27poeuATrEf
 YztKTrNLbf.2H2ofmp9jAQoIfl4Q5O0GtxdyAwbjZg5DyGx3HefhIHkMDsLvv_dnm0FR0ZvM5Cg7
 jeiCeQWmNmfKEG0HgDq4y.uLf8AEgggTczd6nEJHyE_iOk7RlJFRoufBSCsLbmkPXWCYg5iPzdpb
 eor2e_wc0Gw4N.aOXnh9refuO9pxZRlMx0DxD4DaEg 
X-Sonic-MF: <spacibba@aol.com>
Original-Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic304.consmr.mail.bf2.yahoo.com with HTTP; Sat, 12 Jun 2021 01:08:53 +0000
Original-Received: by kubenode545.mail-prod1.omega.ir2.yahoo.com (VZM Hermes SMTP
 Server) with ESMTPA ID d25a85a4ab23a7be1df055eda32e46f2; 
 Sat, 12 Jun 2021 01:08:47 +0000 (UTC)
Content-Disposition: inline
In-Reply-To: <87a6nw6jtf.fsf@telefonica.net>
X-Mailer: WebService/1.1.18368
 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.aol
Received-SPF: pass client-ip=74.6.128.33; envelope-from=spacibba@aol.com;
 helo=sonic304-10.consmr.mail.bf2.yahoo.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
 <mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
 <mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org
Original-Sender: "Emacs-devel"
 <emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org>
Xref: news.gmane.io gmane.emacs.devel:270734
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/270734>

On Sat, Jun 12, 2021 at 01:52:12AM +0200, �scar Fuentes wrote:
>Ergus <spacibba@aol.com> writes:
>
>> Going a bit more into this. And reconsidering tree-sitter.
>>
>> As there is already a tree-sitter module package with some interesting
>> functionalities. (I know Eli didn't like some details in it's
>> implementation)
>>
>> But maybe it is a good time to try to disable the cc-mode font-locking
>> (I don't actually know if it is possible to do that), and repeat the
>> scrolling benchmark only with the tree-sitter-mode and
>> tree-sitter-hl-mode enabled?
>>
>> Just to see how it compares and how much of that approach is useful?
>
>More easily, you can use some of the editors that already use
>tree-sitter for fontification of C/C++ and do the PgDn test 

We have all the lisp machine overhead in the middle. So doing this will
be like comparing apples with pears.

>(which looks
>like a rather silly test to me, because who navigates large files by
>holding PgDn and why Emacs should support that terrible use case well?)

The scrolling test is because during scrolling we call re-display,
font-look and some hooks. So it is the easiest way to measure all the
syntax highlighting in action.

>This would provide a valuable comparison point for little effort.
>

I don't think so. The tree-sitter mode does not require special effort
to install. And comparing emacs vs emacs is more realistic and useful
IMHO (neovim redisplay is ridiculously fast).

But any way just to start: tree-sitter parses all the text in xdisp.c,
(in my machine), in 0.12 seconds from scratch and re-parses it (reusing
the tree) 10 times faster; in 0.008 ~ 0.01 seconds.

If we count that we don't need to re-parse the file (buffer), but only
the modified regions (that is possible to specify with the api). Then
the times are ridiculous small.

In this case the parse is mostly already done, so scrolling won't need
to parse the text to add the highlighting... so maybe we need something
else to measure the impact (maybe something that modifies the text)

BTW: Eli was concerned about the extra copy of the buffer text to send
it to tree-sitter. In this case the time to memcopy an array with all
xdisp text is ~0.00085 seconds.

Any way if we don't want the copy we can use
ts_parser_set_included_ranges to exclude the gap and pass the text
pointer directly without any copy.

>Although I'm more interested on accuracy, but it seems that the thread
>was effectively and hopelessly hijacked :-/
>
>
To improve accuracy we need to improve the parsing OR add more work to
cc-mode. So that's why we are looking for alternatives.

There are already some interesting information to see tree sitter in
action:

https://www.youtube.com/watch?v=ZwibVdNtFjs
https://www.youtube.com/watch?v=oSrXK8ovBfQ

Where you can actually see that the accuracy should also improve (and
probably some navigation commands)