From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Stephen Leake Newsgroups: gmane.emacs.devel Subject: Re: emacs-tree-sitter and Emacs Date: Wed, 01 Apr 2020 11:51:40 -0800 Message-ID: <864ku3htmb.fsf@stephe-leake.org> References: <83eeta3sa0.fsf@gnu.org> <86369ojbig.fsf@stephe-leake.org> <83lfnfz6jr.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="14525"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (windows-nt) To: emacs-devel Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Apr 01 21:52:35 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jJjPe-0003gy-TO for ged-emacs-devel@m.gmane-mx.org; Wed, 01 Apr 2020 21:52:35 +0200 Original-Received: from localhost ([::1]:36706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jJjPd-0006Kn-UL for ged-emacs-devel@m.gmane-mx.org; Wed, 01 Apr 2020 15:52:33 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:39488) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jJjOx-0005dy-WF for emacs-devel@gnu.org; Wed, 01 Apr 2020 15:51:55 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jJjOv-0007e4-NA for emacs-devel@gnu.org; Wed, 01 Apr 2020 15:51:51 -0400 Original-Received: from gateway20.websitewelcome.com ([192.185.66.3]:35962) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jJjOv-0007ZY-Cv for emacs-devel@gnu.org; Wed, 01 Apr 2020 15:51:49 -0400 Original-Received: from cm11.websitewelcome.com (cm11.websitewelcome.com [100.42.49.5]) by gateway20.websitewelcome.com (Postfix) with ESMTP id 6A953400D0D96 for ; Wed, 1 Apr 2020 13:35:36 -0500 (CDT) Original-Received: from host2007.hostmonster.com ([67.20.76.71]) by cmsmtp with SMTP id JjOqjvWMtSl8qJjOqjRI6k; Wed, 01 Apr 2020 14:51:45 -0500 X-Authority-Reason: nr=8 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=stephe-leake.org; s=default; h=Content-Type:MIME-Version:Message-ID: In-Reply-To:Date:References:Subject:To:From:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=hN2m/qAC1TfYZDvmcgMpMM9BdNIp+zsOk3v1kXRNgtY=; b=bo+HfU2Bl56sOvkmiY1VtWbMW 5Abl1YXRrQkTjQtfhF2yyQsCReN6fCHB1umGXvRAytJJ6+AkYehqCTxDfW1K/CeTSrCjWAfKDNAqA 34LIFZr/HgfHBppRSYNFyyuE8fku99em/FinEl04eB1qQAa7ywP2MCreyc7/8ZvT9TliKA8KVJaBN 4tXWIII5mErgAEcN9WzobXRLfksTT52CLU9nm7QeAQxaohWxhEdTPyOcHnf3A9tMeQ8PjdVBrYeoQ iu9Aj7rPVtIbtl3HFjSisAvZoC7qcJtVim4rAaUk9XOWVh5pFs4y0Xsv/NQ5u0u7fGzOUXpVF9l2W 3tzSgv6Xg==; Original-Received: from [76.77.182.20] (port=62575 helo=Takver4) by host2007.hostmonster.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.92) (envelope-from ) id 1jJjOq-003XCC-E2 for emacs-devel@gnu.org; Wed, 01 Apr 2020 13:51:44 -0600 In-Reply-To: <83lfnfz6jr.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 01 Apr 2020 16:20:24 +0300") X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - host2007.hostmonster.com X-AntiAbuse: Original Domain - gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - stephe-leake.org X-BWhitelist: no X-Source-IP: 76.77.182.20 X-Source-L: No X-Exim-ID: 1jJjOq-003XCC-E2 X-Source-Sender: (Takver4) [76.77.182.20]:62575 X-Source-Auth: stephen_leake@stephe-leake.org X-Email-Count: 1 X-Source-Cap: c3RlcGhlbGU7c3RlcGhlbGU7aG9zdDIwMDcuaG9zdG1vbnN0ZXIuY29t X-Local-Domain: yes X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 192.185.66.3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:246222 Archived-At: Eli Zaretskii writes: >> From: Stephen Leake >> Date: Tue, 31 Mar 2020 16:27:35 -0800 >> >> > OTOH, using an after-change hook has its downsides, even if disregard >> > slow-down (which I wouldn't). >> >> In wisi (used by ada-mode), the after-change hooks just record what >> regions have been changed; font-lock then triggers a parse if the region >> being fontified contains or is after a change region. Navigation and >> indent also trigger parses. > > Can you tell in more detail why you need to rely on these hooks? They > shouldn't be necessary, AFAIU. It is an optimization choice. In an unmodified buffer, that is smaller than 100,000 characters (default setting of wisi-partial-parse-threshold), the entire buffer is parsed once; that applies faces to all the Ada identifiers that need faces (standard font-lock regexp handles the reserved words). Then when font-lock fontifies a region, no parsing is needed. Indent is similar; the parse sets text properties holding the indent for each line; indent-region then applies them. When the user starts editing, and font-lock is requested, we need to know the changes before the font-lock region, because that can affect the interpretation of the code in the region. Worst case is adding an opening ". Adding/deleting "begin" or "end" can change indent (equivalent to adding/deleting { or } in C). If the default setting of jit-lock-defer-time (ie nil) is used, then font-lock runs immediately after each change, and the after-change hooks are not needed. But as I have mentioned, I always run with jit-lock-defer-time set to 1.0 (because parsing is not fast enough in some cases), so the change hooks are needed. In addition, indent-region is run when the user types return or tab (or otherwise invokes indent); there can easily be changes outside the line or region begin indented, again requiring change hooks. The alternative to not requiring after-change hooks is to always do a full parse, for ever call of fontify-region or indent-region. That is far too slow. Note that Tree-Sitter requires one full parse of the buffer to generate the parse tree that is later updated incrementally; in an unmodified buffer, only that one parse is needed. wisi can handle parsing only a small part of the file, but it produces incorrect results more often in that case, since it relies on error-correction to arrive at correct syntax. That's why partial parse is only used on very large files, where the parser is _always_ too slow; in most files, it is only too slow when there is a bad syntax error, and recover is slow. > And they cannot pick up every relevant change; for example, what > happens if some face used for font-lock is modified? Yes, that is a flaw. Not likely to occur in everyday use, and wisi provides wisi-parse-buffer to force a full parse for such situations. As I mentioned elsewhere, wisi provides wisi-inhibit-parse for use when an elisp author might be tempted to use inhibit-modification-hooks. >> By default font-lock runs after every character typed > > No, it only runs when redisplay kicks in. If you type very quickly, > it won't run for every character. At least AFAIR. What triggers redisplay? In practice, I and other ada-mode users notice font-lock running after each character, with the default setting of jit-lock-defer-time. There is a comment in jit-lock.el indicating that the default value may have been 0.25 at one point (I did not check the git history); perhaps you are remembering that behavior? For example, in Ada the comment-start is "--". No matter how fast I type the two chars, ada-mode reports a syntax error after the first one. Syntax errors are detected by a parse and reported via fringe marks, as in flymake; it blinks after the key is pressed twice, appearing after the first character is displayed, disappearing after the second is displayed. (I have just retested this in emacs from master). I don't think there's anything in ada-mode that forces a redisplay (except explicitly calling wisi-parse-buffer; that calls font-lock-ensure). But I'd be happy to investigate further if you are sure it should not work this way. The elisp manual section "Forcing redisplay" says "Emacs normally tries to redisplay the screen whenever it waits for input." After I type the first character, it is no longer waiting for input, it is processing that character. I assume here "process that char code" includes running after-change-functions, which is (small) elisp code. But I guess after processing that char, before calling redisplay, it checks if there is more input, which should be true if I type fast enough. Perhaps "process that char code" is faster than the combination of my fingers and the keyboard char send rate? Hmm. M-x (execute-kbd-macro "--") does not show a syntax-error fringe blink. I'm not sure if that is relevant here. >> which is often too slow in an ada-mode buffer; I always set >> jit-lock-defer-time to 1.0 seconds. > > That's too long to be pleasant on display, IMO. A second is a very > long time in this context. Other people have made the same complaint. I'm probably biased in accepting the slow parser behavior (I know how hard it would be to improve it :). Migrating from an external process to a module might help. Changing from partial parse to incremental parse might help. Setting jit-lock-defer-time to 0.25 eliminates the fringe blink when typing "--". If I watch very closely, I can just barely see the delay between displaying the last char and the change of color (from black to red). I'll run with 0.25 for a while; the parser has gotten better since the last time I changed that, so maybe that's good enough now. I mentioned above that the parser is only too slow when there is a bad syntax error, and recover is slow. However, that is the typical case while editing code. -- -- Stephe