From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.bugs Subject: bug#39277: 26.3; Tcl font lock does not understand quoting Date: Tue, 27 Oct 2020 18:48:56 -0400 Message-ID: References: <20200125100009.33e3cpgmjszmpwzq@gentoo-zen2700x> <87lffs1zvw.fsf@cnu407c2zx.nsn-intra.net> <875z6v36lv.fsf@gnus.org> <87sg9ze6yj.fsf@cnu407c2zx.nsn-intra.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1968"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Lars Ingebrigtsen , 39277@debbugs.gnu.org, Hadrien Lacour To: mvar Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Oct 27 23:50:14 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kXXnB-0000NJ-Am for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 27 Oct 2020 23:50:13 +0100 Original-Received: from localhost ([::1]:40416 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kXXn9-0002XS-VJ for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 27 Oct 2020 18:50:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:38390) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kXXn1-0002Vh-DM for bug-gnu-emacs@gnu.org; Tue, 27 Oct 2020 18:50:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:34789) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kXXn1-0007jP-1R for bug-gnu-emacs@gnu.org; Tue, 27 Oct 2020 18:50:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kXXn0-0004CK-VT for bug-gnu-emacs@gnu.org; Tue, 27 Oct 2020 18:50:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 27 Oct 2020 22:50:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39277 X-GNU-PR-Package: emacs Original-Received: via spool by 39277-submit@debbugs.gnu.org id=B39277.160383895416033 (code B ref 39277); Tue, 27 Oct 2020 22:50:02 +0000 Original-Received: (at 39277) by debbugs.gnu.org; 27 Oct 2020 22:49:14 +0000 Original-Received: from localhost ([127.0.0.1]:46329 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXXmD-0004AX-RO for submit@debbugs.gnu.org; Tue, 27 Oct 2020 18:49:14 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:56176) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kXXmC-0004AJ-E5 for 39277@debbugs.gnu.org; Tue, 27 Oct 2020 18:49:13 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id A0EF4440822; Tue, 27 Oct 2020 18:49:06 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id DBB20440811; Tue, 27 Oct 2020 18:49:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1603838944; bh=/r6CfwJL9tcBjk7kv89rRznbFvxmKHf+C5gOLVHJVPQ=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=QUt4jTs3qltty5fVoo/AzYSxrDkXpo08jmVNo23B4v2NnXPkD3Y3CVnEnnC0Lsveg U+g813nIhHJpdN3AeNynSZ+E7vfxIwrYANpcC4LgeqeE0hQn0d9Z2wJz4OGMzy0E1O /Ep0wPLlcxj2clx2vBiKJXrNU473PNq0ThDs1tjD98yp3OHI6EGaxkgKVOfUBQwbtG 6C/JxxCD+tdIy1OK5QzOa3nZ4hXYJUxfQ6sOdIYzTeibleHAbHRVFmv/zvLiV3Uj8T 2TJbqSvv3MT6JTIG3ozo7hRD4ZEnqrupe3GSJUqeWLU8tmF8TJnkC1vCpQpeTKyhx4 0o7UUkOX0k/lA== Original-Received: from alfajor (unknown [157.52.9.240]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A9AF81203AD; Tue, 27 Oct 2020 18:49:04 -0400 (EDT) In-Reply-To: <87sg9ze6yj.fsf@cnu407c2zx.nsn-intra.net> (mvar's message of "Tue, 27 Oct 2020 22:42:44 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:191803 Archived-At: > thank you Lars for reverting, this didn't feel right anyway. I'll try to come up > with some more elegant solution or at least find some way to skip > breaking the other locks - for example moving the tcl-font-lock-keywords > regexp to the end of that list solves the problem Stefan mentioned but it > still doesn't address what Andreas pointed out, i.e. proc test (args) will > have args locked as a string. How 'bout the patch below? Stefan diff --git a/lisp/progmodes/tcl.el b/lisp/progmodes/tcl.el index 717008a0a2..d4d51e8b50 100644 --- a/lisp/progmodes/tcl.el +++ b/lisp/progmodes/tcl.el @@ -407,10 +407,64 @@ tcl-font-lock-keywords `tcl-typeword-list', and `tcl-keyword-list' by the function `tcl-set-font-lock-keywords'.") +(eval-and-compile + (defconst tcl--word-delimiters "[;{ \t\n")) + +(defun tcl--syntax-of-quote (pos) + "Decide whether a double quote opens a string or not." + ;; This is pretty tricky, because strings can be written as "..." + ;; or as {...} or without any quoting at all for some simple and not so + ;; simple cases (e.g. `abc' but also `a"b'). To make things more + ;; interesting, code is represented as strings, so the content of + ;; strings can be later re-lexed to find nested strings. + (save-excursion + (let ((ppss (syntax-ppss pos))) + (cond + ((nth 8 ppss) nil) ;; Within a string or a comment. + ((not (memq (char-before pos) + (cons nil + (eval-when-compile + (mapcar #'identity tcl--word-delimiters))))) + ;; The double quote appears within some other lexical entity. + ;; FIXME: Similar treatment should be used for `{' which can appear + ;; within non-delimited strings (but only at top-level, so + ;; maybe it's not worth worrying about). + (string-to-syntax ".")) + ((zerop (nth 0 ppss)) + ;; Not within a { ... }, so can't be truncated by a }. + ;; FIXME: The syntax-table also considers () and [] as paren + ;; delimiters just like {}, even though Tcl treats them differently. + ;; Tho I'm not sure it's worth worrying about, either. + nil) + (t + ;; A double quote within a {...}: leave it as a normal string + ;; delimiter only if we don't find a closing } before we + ;; find a closing ". + (let ((type nil) + (depth 0)) + (forward-char 1) + (while (and (not type) + (re-search-forward "[\"{}\\]" nil t)) + (pcase (char-after (match-beginning 0)) + (?\\ (forward-char 1)) + (?\" (setq type 'matched)) + (?\{ (cl-incf depth)) + (?\} (if (zerop depth) (setq type 'unmatched) + (cl-incf depth))))) + (when (> (line-beginning-position) pos) + ;; The quote is not on the same line as the deciding + ;; factor, so make sure we revisit this choice later. + (put-text-property pos (point) 'syntax-multiline t)) + (when (eq type 'unmatched) + ;; The quote has no matching close because a } closes the + ;; surrounding string before, so it doesn't really "open a string". + (string-to-syntax ".")))))))) + (defconst tcl-syntax-propertize-function (syntax-propertize-rules ;; Mark the few `#' that are not comment-markers. - ("[^;[{ \t\n][ \t]*\\(#\\)" (1 "."))) + ((concat "[^" tcl--word-delimiters "][ \t]*\\(#\\)") (1 ".")) + ("\"" (0 (tcl--syntax-of-quote (match-beginning 0))))) "Syntactic keywords for `tcl-mode'.") ;; FIXME need some way to recognize variables because array refs look @@ -593,6 +647,8 @@ tcl-mode '(tcl-font-lock-keywords nil nil nil beginning-of-defun)) (set (make-local-variable 'syntax-propertize-function) tcl-syntax-propertize-function) + (add-hook 'syntax-propertize-extend-region-functions + #'syntax-propertize-multiline 'append 'local) (set (make-local-variable 'imenu-generic-expression) tcl-imenu-generic-expression) diff --git a/test/manual/indent/tcl.tcl b/test/manual/indent/tcl.tcl new file mode 100644 index 0000000000..447b64cf1c --- /dev/null +++ b/test/manual/indent/tcl.tcl @@ -0,0 +1,19 @@ + +puts "hello}"; # Top-level strings can contain unescaped closing braces! + +puts a"b; # Non-delimited strings can contain quotes! +puts a""b; # Even several of them! + +proc foo1 {} { + puts "hello"; # Normal case! + puts "hello\}; # This will signal an error when `foo1` is called! +} + +proc foo1 {} { + puts "hello; # This will also signal an error when `foo1` is called! +} + +proc foo1 {} { + puts a"b; # This will not signal an error! + puts a""b"; # And that won't either! +}