From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.bugs Subject: bug#67262: python-ts-mode cannot identify triple-quoted-strings Date: Tue, 12 Dec 2023 00:32:09 -0800 Message-ID: References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@gmail.com> <838r6vm3dj.fsf@gnu.org> <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@gutov.dev> <83v89qcfsb.fsf@gnu.org> <9B8C904A-3729-44AF-82F7-3BEA849F46D0@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1716"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: 67262@debbugs.gnu.org To: Dmitry Gutov , JD Smith , Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Dec 12 09:33:17 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rCyCa-0000I8-BI for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 12 Dec 2023 09:33:16 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rCyC8-0004Wq-VI; Tue, 12 Dec 2023 03:32:48 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rCyC7-0004WT-Fz for bug-gnu-emacs@gnu.org; Tue, 12 Dec 2023 03:32:47 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rCyC7-0004fI-88 for bug-gnu-emacs@gnu.org; Tue, 12 Dec 2023 03:32:47 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rCyCM-0007s2-EF for bug-gnu-emacs@gnu.org; Tue, 12 Dec 2023 03:33:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 12 Dec 2023 08:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67262 X-GNU-PR-Package: emacs Original-Received: via spool by 67262-submit@debbugs.gnu.org id=B67262.170236995630222 (code B ref 67262); Tue, 12 Dec 2023 08:33:02 +0000 Original-Received: (at 67262) by debbugs.gnu.org; 12 Dec 2023 08:32:36 +0000 Original-Received: from localhost ([127.0.0.1]:55228 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rCyBw-0007rO-5G for submit@debbugs.gnu.org; Tue, 12 Dec 2023 03:32:36 -0500 Original-Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]:44154) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rCyBs-0007r9-85 for 67262@debbugs.gnu.org; Tue, 12 Dec 2023 03:32:34 -0500 Original-Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6ce9c8c45a7so3315084b3a.0 for <67262@debbugs.gnu.org>; Tue, 12 Dec 2023 00:32:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702369931; x=1702974731; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=gJgzRFNGipNVfUJQbyZHG6I+vB4zLtfJxLkBUKmPFSc=; b=eNJpF4NJiYDmUHVfmiTPKza/2VOz2+0bTxO2OksZK8sRgomT0XJmyZ3JgYqLSOtfzm 0htKII/01MX5uZNGw9hCu0prz2Dfnbzo2Z3LChdChRry3uBiRTdr0ouPsTYQCF2J7+6F 44B52SO8vzk7ovUDxpx56fZrmaqcCRxqtUZjJime5f4EuuG5zYKZF9Hy7/xWsbUdQcX2 xglRp7E1HHBAbInFVgF0bkr4azYpNuKJYUeJMNGo6OQ04P8U386cWzIdZDN4DmfzPSdI wXWUKVRnFaRKhta6hDNZ3i72OvtWn3l+X83inBcbw4oknHyouEtzFS+I7m3jHif89If/ UEJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702369931; x=1702974731; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gJgzRFNGipNVfUJQbyZHG6I+vB4zLtfJxLkBUKmPFSc=; b=B7B7MEbGwgjwJ+Mi2dH1yrzbAVbWFGoChKYNMarPqc3MUDDN78qHwYT1y55ZjO6B4G 3Paq5Uff0LvDfC05LlKu4vMFpiAH1eoaoylNcwZ8ahoZII9mw+CVWvOhhbKLjpL5V/C9 3Ors1tx0bQKnyjydzWTE+ILPKMHBhknag0XdH0iXX3jRztMqTFepfbHpMVGf9wxCrb0w iVHTE0i1CYWrDSXN6OW9fP53ToOdIkA6mmg1Z6HZRvOoJICzMu/5s3lrJoQcptBYpMOa Lt9/BakR6jLfZ17ySsjesPvJ6ofqlXsvRxJ+Ia43XakcB84FYlFue1Ou3MZjLU4M14VC mtZw== X-Gm-Message-State: AOJu0Yy/xfjOWpWWeRrjecwp11taB6h9NX4/69TMgoDzs17S4CMutQXi qJcGsc1H1i5aQdLOQYaecko= X-Google-Smtp-Source: AGHT+IEagbP42RpaVPxjiC4rtiZBYE054Rq84ASHYx1d4AFmarwESJ+JmHDQ+BqLvuxYjXpVqy5nOQ== X-Received: by 2002:a05:6a20:f39c:b0:18c:92f0:69bd with SMTP id qr28-20020a056a20f39c00b0018c92f069bdmr2895210pzb.24.1702369931083; Tue, 12 Dec 2023 00:32:11 -0800 (PST) Original-Received: from [192.168.1.7] (172-117-161-177.res.spectrum.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id b1-20020a170902d30100b001cf96a0e4e6sm8013104plc.242.2023.12.12.00.32.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 12 Dec 2023 00:32:10 -0800 (PST) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276022 Archived-At: On 11/26/23 4:05 PM, Dmitry Gutov wrote: > On 27/11/2023 01:43, Yuan Fu wrote: >> >> On 11/26/23 6:58 AM, Dmitry Gutov wrote: >>> On 26/11/2023 04:04, Dmitry Gutov wrote: >>>> As for what to do about this one -- probably something involving >>>> syntax-propertize-extend-region-functions, adding an entry which >>>> would initialize the parser, but not call syntax-ppss-flush-cache >>>> directly (or at least not just that). It would signal the earlier >>>> position to extend to through some dynamic variable. This is >>>> getting tricky enough to move from the individual major modes into >>>> treesit.el proper, I think. >>> >>> Alternatively, we'd trigger updates eagerly from within >>> treesit_record_change -- that would make it slower, invalidating the >>> comment above it. Not sure by how much, though. >> >> It seems to me that what we need is to force a re-parse at the >> beginning of syntax-propertize or in syntax-ppss-flush-cache; the >> re-parse would cause the notifier to run, which runs >> python--treesit-parser-after-change. > > syntax-ppss-flush-cache is called by edits (and by the re-parse). It > seems like it will be odd to have execution the other way around > and/or add some hook into it which would call the re-parse and extend > the region to be invalidated. > > syntax-propertize could have another hook added, yes. Or an advice. > > But it seems better to reuse some of the existing hooks, such as > syntax-propertize-extend-region-functions. It treesit.c provided a way > to fetch the newly-invalidated region, the treesit-major-mode-setup > could add a new function to syntax-propertize-extend-region-functions > which would invoke that feature. But even now it can instantiate the > parse, which would call treesit-force-reparse internally, and then > collect the info from the callbacks. syntax-propertize-extend-region-functions looks perfect. We just need to force a reparse in it and the notifier will do the rest. > > And yet another way - is to extend the region to be propertized from > inside the major mode's syntax-propertize-function, invalidating some > earlier entries too. The main problem with that, I think, is that > every ts mode will have to repeat that trick. And that authors would > have to know to do that. How to make that easier and more obvious, is > a question. > > Finally, if I'm right that bug#66732 has a similar cause, then a > shared solution that can be reused by syntax and font-lock (or > preferably just fix both in the same place) would be ideal. > >> I'm not quite sure about how do we cause this re-parse. The >> straightforward approach would be calling treesit-force-reparse[1] in >> syntax-propertize/syntax-ppss-flush-cache. But ideally I'd like to >> keep tree-sitter transparent for syntax.el. Maybe we can add a hook >> in syntax-propertize/syntax-ppss-flush-cache. >> >> [1] This function doesn't exist yet, but it's easy to define in lisp. > > treesit-parser-root-node calls it anyway and does little else, so we > could get by with just using it. > Yep. Yuan