From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#21449: Emacs lisp mode: incorrect fontification of symbols containing escaped characters. Date: Wed, 9 Sep 2015 20:10:42 +0000 Message-ID: <20150909201042.GA10205@acm.fritz.box> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1441829850 29016 80.91.229.3 (9 Sep 2015 20:17:30 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 9 Sep 2015 20:17:30 +0000 (UTC) To: 21449@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Sep 09 22:17:18 2015 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZZlnt-0003sK-1v for geb-bug-gnu-emacs@m.gmane.org; Wed, 09 Sep 2015 22:17:13 +0200 Original-Received: from localhost ([::1]:45541 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZZlns-0004KG-Ni for geb-bug-gnu-emacs@m.gmane.org; Wed, 09 Sep 2015 16:17:12 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46847) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZZlnn-0004K8-KC for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:17:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZZlni-0004FA-Fy for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:17:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:33781) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZZlni-0004F3-CE for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:17:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1ZZlnh-0008Nq-Ub for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:17:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 09 Sep 2015 20:17:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 21449 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.144182978732146 (code B ref -1); Wed, 09 Sep 2015 20:17:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 9 Sep 2015 20:16:27 +0000 Original-Received: from localhost ([127.0.0.1]:54224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZZln8-0008MO-47 for submit@debbugs.gnu.org; Wed, 09 Sep 2015 16:16:26 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:36477) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZZln6-0008ME-7M for submit@debbugs.gnu.org; Wed, 09 Sep 2015 16:16:24 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZZln4-0003Vm-Ia for submit@debbugs.gnu.org; Wed, 09 Sep 2015 16:16:23 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:41702) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZZln4-0003Vf-G4 for submit@debbugs.gnu.org; Wed, 09 Sep 2015 16:16:22 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46461) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZZln3-0004Fg-20 for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:16:22 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZZlmx-0003TL-U3 for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:16:20 -0400 Original-Received: from mail.muc.de ([193.149.48.3]:43489) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZZlmx-0003Sb-Hn for bug-gnu-emacs@gnu.org; Wed, 09 Sep 2015 16:16:15 -0400 Original-Received: (qmail 31047 invoked by uid 3782); 9 Sep 2015 20:09:32 -0000 Original-Received: from acm.muc.de (p579E9A15.dip0.t-ipconnect.de [87.158.154.21]) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 09 Sep 2015 22:09:31 +0200 Original-Received: (qmail 10482 invoked by uid 1000); 9 Sep 2015 20:10:42 -0000 Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:106325 Archived-At: Hello, Emacs. In Emacs lisp mode, write something like: (defun fix-re--RA|RB->R\(A|B\) (foo bar) ...) ^^^^^^^ . The part of the symbol indicated remains unfontified. This happens because the fontification patterns in .../lisp/emacs-lisp/lisp-mode.el dont't take account of escaped characters in symbol names. By replacing lots of "\\(?:\\sw\\|\\s_\\)" by "\\(?:\\sw\\|\\s_\\|\\\\.\\)", the fontification is repaired. As a bonus, this fix makes imenu work properly with these symbols too. Unfortunately, as yet the regular expression expressions "\\_<" and "\\_>" don't work properly with these symbols. To fix that would need an amendment to .../src/regex.c, with possibly .../src/syntax.c needing one too. It feels like there really ought to be some major mode dependent flag saying whether or not escaped characters are valid in identifiers. They are in Emacs lisp, but they're not in C. Anyhow, here's the patch: diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el index 8aa34c7..7be7cb3 100644 --- a/lisp/emacs-lisp/lisp-mode.el +++ b/lisp/emacs-lisp/lisp-mode.el @@ -110,7 +110,7 @@ ;; CLOS and EIEIO "defgeneric" "defmethod") t)) - "\\s-+\\(\\(\\sw\\|\\s_\\)+\\)")) + "\\s-+\\(\\(\\sw\\|\\s_\\|\\\\.\\)+\\)")) 2) (list (purecopy "Variables") (purecopy (concat "^\\s-*(" @@ -122,11 +122,11 @@ "defconstant" "defparameter" "define-symbol-macro") t)) - "\\s-+\\(\\(\\sw\\|\\s_\\)+\\)")) + "\\s-+\\(\\(\\sw\\|\\s_\\|\\\\.\\)+\\)")) 2) ;; For `defvar', we ignore (defvar FOO) constructs. (list (purecopy "Variables") - (purecopy (concat "^\\s-*(defvar\\s-+\\(\\(\\sw\\|\\s_\\)+\\)" + (purecopy (concat "^\\s-*(defvar\\s-+\\(\\(\\sw\\|\\s_\\|\\\\.\\)+\\)" "[[:space:]\n]+[^)]")) 1) (list (purecopy "Types") @@ -143,7 +143,7 @@ ;; CLOS and EIEIO "defclass") t)) - "\\s-+'?\\(\\(\\sw\\|\\s_\\)+\\)")) + "\\s-+'?\\(\\(\\sw\\|\\s_\\|\\\\.\\)+\\)")) 2)) "Imenu generic expression for Lisp mode. See `imenu-generic-expression'.") @@ -220,7 +220,7 @@ (defun lisp--el-match-keyword (limit) ;; FIXME: Move to elisp-mode.el. (catch 'found - (while (re-search-forward "(\\(\\(?:\\sw\\|\\s_\\)+\\)\\_>" limit t) + (while (re-search-forward "(\\(\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)\\_>" limit t) (let ((sym (intern-soft (match-string 1)))) (when (or (special-form-p sym) (and (macrop sym) @@ -349,7 +349,7 @@ ;; Any whitespace and defined object. "[ \t']*" "\\(([ \t']*\\)?" ;; An opening paren. - "\\(\\(setf\\)[ \t]+\\(?:\\sw\\|\\s_\\)+\\|\\(?:\\sw\\|\\s_\\)+\\)?") + "\\(\\(setf\\)[ \t]+\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\|\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)?") (1 font-lock-keyword-face) (3 (let ((type (get (intern-soft (match-string 1)) 'lisp-define-type))) (cond ((eq type 'var) font-lock-variable-name-face) @@ -373,7 +373,7 @@ ;; Any whitespace and defined object. "[ \t']*" "\\(([ \t']*\\)?" ;; An opening paren. - "\\(\\(setf\\)[ \t]+\\(?:\\sw\\|\\s_\\)+\\|\\(?:\\sw\\|\\s_\\)+\\)?") + "\\(\\(setf\\)[ \t]+\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\|\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)?") (1 font-lock-keyword-face) (3 (let ((type (get (intern-soft (match-string 1)) 'lisp-define-type))) (cond ((eq type 'var) font-lock-variable-name-face) @@ -395,22 +395,22 @@ (lisp--el-match-keyword . 1) ;; Exit/Feature symbols as constants. (,(concat "(\\(catch\\|throw\\|featurep\\|provide\\|require\\)\\_>" - "[ \t']*\\(\\(?:\\sw\\|\\s_\\)+\\)?") + "[ \t']*\\(\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)?") (1 font-lock-keyword-face) (2 font-lock-constant-face nil t)) ;; Erroneous structures. (,(concat "(" el-errs-re "\\_>") (1 font-lock-warning-face)) ;; Words inside \\[] tend to be for `substitute-command-keys'. - ("\\\\\\\\\\[\\(\\(?:\\sw\\|\\s_\\)+\\)\\]" + ("\\\\\\\\\\[\\(\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)\\]" (1 font-lock-constant-face prepend)) ;; Words inside ‘’ and '' and `' tend to be symbol names. - ("['`‘]\\(\\(?:\\sw\\|\\s_\\)\\(?:\\sw\\|\\s_\\)+\\)['’]" + ("['`‘]\\(\\(?:\\sw\\|\\s_\\|\\\\.\\)\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)['’]" (1 font-lock-constant-face prepend)) ;; Constant values. - ("\\_<:\\(?:\\sw\\|\\s_\\)+\\_>" 0 font-lock-builtin-face) + ("\\_<:\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\_>" 0 font-lock-builtin-face) ;; ELisp and CLisp `&' keywords as types. - ("\\_<\\&\\(?:\\sw\\|\\s_\\)+\\_>" . font-lock-type-face) + ("\\_<\\&\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\_>" . font-lock-type-face) ;; ELisp regexp grouping constructs (,(lambda (bound) (catch 'found @@ -447,19 +447,19 @@ (,(concat "(" cl-kws-re "\\_>") . 1) ;; Exit/Feature symbols as constants. (,(concat "(\\(catch\\|throw\\|provide\\|require\\)\\_>" - "[ \t']*\\(\\(?:\\sw\\|\\s_\\)+\\)?") + "[ \t']*\\(\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)?") (1 font-lock-keyword-face) (2 font-lock-constant-face nil t)) ;; Erroneous structures. (,(concat "(" cl-errs-re "\\_>") (1 font-lock-warning-face)) ;; Words inside ‘’ and '' and `' tend to be symbol names. - ("['`‘]\\(\\(?:\\sw\\|\\s_\\)\\(?:\\sw\\|\\s_\\)+\\)['’]" + ("['`‘]\\(\\(?:\\sw\\|\\s_\\|\\\\.\\)\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\)['’]" (1 font-lock-constant-face prepend)) ;; Constant values. - ("\\_<:\\(?:\\sw\\|\\s_\\)+\\_>" 0 font-lock-builtin-face) + ("\\_<:\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\_>" 0 font-lock-builtin-face) ;; ELisp and CLisp `&' keywords as types. - ("\\_<\\&\\(?:\\sw\\|\\s_\\)+\\_>" . font-lock-type-face) + ("\\_<\\&\\(?:\\sw\\|\\s_\\|\\\\.\\)+\\_>" . font-lock-type-face) ;; This is too general -- rms. ;; A user complained that he has functions whose names start with `do' ;; and that they get the wrong color. @@ -482,7 +482,7 @@ (let* ((firstsym (and listbeg (save-excursion (goto-char listbeg) - (and (looking-at "([ \t\n]*\\(\\(\\sw\\|\\s_\\)+\\)") + (and (looking-at "([ \t\n]*\\(\\(\\sw\\|\\s_\\|\\\\.\\)+\\)") (match-string 1))))) (docelt (and firstsym (function-get (intern-soft firstsym) -- Alan Mackenzie (Nuremberg, Germany).