From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#61369: Problem with keeping tree-sitter parse tree up-to-date Date: Sat, 18 Feb 2023 03:25:59 +0200 Message-ID: <7ee28606-18cc-ce4f-e601-3954489c4f4c@yandex.ru> References: <1AC63591-F4EF-411F-B554-7CD38B4B4888@gmail.com> <9c4e551b-42b3-8202-ccff-fb8170b616a6@yandex.ru> <7751EE35-F5FF-418B-AF28-F1FF5ECEF3AE@gmail.com> <52d15d7e-82e9-ca7b-be16-0ccf89d5053c@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2710"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Cc: Theodor Thornhill , 61369@debbugs.gnu.org To: Yuan Fu Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Feb 18 02:27:14 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pTC0P-0000Oo-PA for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 18 Feb 2023 02:27:13 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pTC0G-0000FG-2c; Fri, 17 Feb 2023 20:27:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pTC0F-0000Ex-4N for bug-gnu-emacs@gnu.org; Fri, 17 Feb 2023 20:27:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pTC0E-0000Px-Rn for bug-gnu-emacs@gnu.org; Fri, 17 Feb 2023 20:27:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pTC0E-0000ix-DY for bug-gnu-emacs@gnu.org; Fri, 17 Feb 2023 20:27:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 18 Feb 2023 01:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61369 X-GNU-PR-Package: emacs Original-Received: via spool by 61369-submit@debbugs.gnu.org id=B61369.16766835732712 (code B ref 61369); Sat, 18 Feb 2023 01:27:02 +0000 Original-Received: (at 61369) by debbugs.gnu.org; 18 Feb 2023 01:26:13 +0000 Original-Received: from localhost ([127.0.0.1]:42004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTBzR-0000hg-95 for submit@debbugs.gnu.org; Fri, 17 Feb 2023 20:26:13 -0500 Original-Received: from mail-wm1-f43.google.com ([209.85.128.43]:34700) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTBzL-0000hG-NB for 61369@debbugs.gnu.org; Fri, 17 Feb 2023 20:26:11 -0500 Original-Received: by mail-wm1-f43.google.com with SMTP id bg22-20020a05600c3c9600b003dff4480a17so1522100wmb.1 for <61369@debbugs.gnu.org>; Fri, 17 Feb 2023 17:26:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=OHoTsq5DDmCnJJ5XScuMc1tfGAMWhCAE7QlYlaXeAjg=; b=JHW2u/Oy9VftBe8DkRikTDUhAaUO7mRK9maW5+5P5IN89E9kkt7gsWmoSHZOfaQlMT XJZ/Yq77iuKeDqNMGEd+WURQcYorVKD4oEC0ixT5Zqpie9afx+OxBji+ltNi3yz1ZoJL tww2dLZqLQAN5ycx3mclWEIVsTUzoHcY59xMYxuE8IenO2jEw887y2E95ymMpU3+40ga 3A2b3q3SS7ZK6qNZxHohsXYQYSutPmspd3SxCQK2HAIFs1cGPjO0QqkhV9kIQLrB2Mzq p92ci35OMysLiRzu+gCA+eA+C1LW/8LmN87eXRd2ePhwtNC3xWbcsX+fcS176e+Z/+OV KJIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OHoTsq5DDmCnJJ5XScuMc1tfGAMWhCAE7QlYlaXeAjg=; b=xSVVLhipAYlDUbD7U7MkXipX5aqWwM7XL7LKPBwm4Yr1J0Ng32lm5/cDatxeth3WAW 0p5L39b9ouoPJNTGaSXvM0xyvMNcIYzsKOGHU0GCgwE1gwDOQi1Wj99vXscfEvx9RsA1 +iMWZPGlvsrVdUmekuf1hgpzMWu9aLFWQIt62eEGQ3qcj6fgbDyvIjOqgYoxjzsdBS1r 6XYDrPT7uCezELGiMYjvIBX25L2bcJ+gXjnjgA/n17V6FdSSnZTrDsnkZGccjxkiMJCx oItPx302LefeLdK6tSfoOyLm6vxLw++5PCC0Gm364PN4vMVVMEEQejRh7kGcQP232p2q NT+Q== X-Gm-Message-State: AO0yUKU68Y9fGwMxxzvM+zj1q8fmdYWpDQPZkxWijk6ZUujhU4Ol6VXP 73ZjC94/TG3F/VCYSCYl6G4= X-Google-Smtp-Source: AK7set/kkSQ8DfySm7OXvPC+OTrq2xdYLO02I7Nud+h3hV3eixz5a2u2RTnz7/OwaWYixs2bpIlSJg== X-Received: by 2002:a05:600c:746:b0:3de:d52:2cd2 with SMTP id j6-20020a05600c074600b003de0d522cd2mr1363160wmn.4.1676683561747; Fri, 17 Feb 2023 17:26:01 -0800 (PST) Original-Received: from [192.168.0.2] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id k37-20020a05600c1ca500b003dffe312925sm3201335wms.15.2023.02.17.17.26.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Feb 2023 17:26:01 -0800 (PST) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:255917 Archived-At: On 18/02/2023 03:14, Yuan Fu wrote: > > >> On Feb 17, 2023, at 4:11 PM, Dmitry Gutov wrote: >> >> On 18/02/2023 00:32, Yuan Fu wrote: >>> Thank you very much! I thought that clipping the change into the fixed visible range, and rely on treesit_sync_visible_region to add back the clipped “tail” when we extend the visible range would be equivalent to not clipping, but I guess clipping and re-adding affects how incremental parsing works inside tree-sitter. >> >> It seems like the "repairing" sync used a different range, one that didn't include the character number 68 inserted from the beginning. >> >> It just synced the 1 or 2 characters at the end of the buffer, the difference between the computed visible_end and the actual BUF_ZV_BYTE. > > That should be enough, no? Because other text didn’t change, they just moved. And tree-sitter should know that they moved. Or maybe I’m misunderstanding what you mean. But the "unsynced" character is at position 68. And we just tell tree-sitter to update positions 134-136. So it stays ignorant of the changed char in the middle of the buffer. It's not just about not knowing about the change either (the character in question is a newline, so its absence wouldn't lead to a syntax error), but about wrong offsets in the old parse tree, based on which the new tree is generated. That probably creates a wrong picture of the source text in the parser. >>> I don’t think this change would have any adverse effect, because if you think of it, inserting text in a narrowed region always extends the region, rather than pushing text at the end out of the narrowed region. So the right thing to do here is in fact not clipping new_end_offset. >> >> I figured it could be a problem if both old_end_byte and new_end_byte extend past the current restriction. > > That should be fine (ie, technically correct), since when we widen, the clipped text are reparsed by tree-sitter as new text. I guess the effect I was thinking of is that XTS_PARSER (lisp_parser)->visible_end would end up with a higher value than BUF_ZV_BYTE. Not sure if it's a problem. >> >> But I'm not sure whether that could actually happen in practice. The obvious attempts (undo a change outside of the narrowing, or revert the buffer when narrowing is in effect) didn't play out, but I'm not sure whether there is an actual hard limit on modifying the text outside of the current restriction. > > It is my impression that Emacs in general enforces the narrowing restriction strictly. And we are still correct when exceptions occur. Very good.