From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Juri Linkov Newsgroups: gmane.emacs.bugs Subject: bug#33567: Syntactic fontification of diff hunks Date: Sat, 01 Dec 2018 23:55:40 +0200 Organization: LINKOV.NET Message-ID: <878t18j4is.fsf@mail.linkov.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: blaine.gmane.org 1543702275 9876 195.159.176.226 (1 Dec 2018 22:11:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 1 Dec 2018 22:11:15 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (x86_64-pc-linux-gnu) To: 33567@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Dec 01 23:11:11 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gTDTd-0002Ju-Hm for geb-bug-gnu-emacs@m.gmane.org; Sat, 01 Dec 2018 23:11:06 +0100 Original-Received: from localhost ([::1]:42697 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTDVj-0004YN-KV for geb-bug-gnu-emacs@m.gmane.org; Sat, 01 Dec 2018 17:13:15 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58533) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTDVa-0004YC-2I for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:13:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTDVW-0008GO-Rk for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:13:06 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:53086) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gTDVW-0008G1-NC for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:13:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gTDVW-0000nE-Gr for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:13:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Juri Linkov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 01 Dec 2018 22:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 33567 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.15437023372984 (code B ref -1); Sat, 01 Dec 2018 22:13:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 1 Dec 2018 22:12:17 +0000 Original-Received: from localhost ([127.0.0.1]:57344 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gTDUm-0000m1-4L for submit@debbugs.gnu.org; Sat, 01 Dec 2018 17:12:17 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:48701) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gTDUk-0000li-DF for submit@debbugs.gnu.org; Sat, 01 Dec 2018 17:12:15 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTDUd-00077B-Ed for submit@debbugs.gnu.org; Sat, 01 Dec 2018 17:12:09 -0500 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:44339) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gTDUc-00076Y-Qk for submit@debbugs.gnu.org; Sat, 01 Dec 2018 17:12:07 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58307) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gTDUb-00047R-62 for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:12:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gTDUW-000741-05 for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:12:03 -0500 Original-Received: from otter.birch.relay.mailchannels.net ([23.83.209.139]:19835) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gTDUV-00073A-AA for bug-gnu-emacs@gnu.org; Sat, 01 Dec 2018 17:11:59 -0500 X-Sender-Id: dreamhost|x-authsender|jurta@jurta.org Original-Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id C0E4B682852 for ; Sat, 1 Dec 2018 22:11:56 +0000 (UTC) Original-Received: from pdx1-sub0-mail-a3.g.dreamhost.com (unknown [100.96.30.62]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 7E0336829CE for ; Sat, 1 Dec 2018 22:11:56 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|jurta@jurta.org Original-Received: from pdx1-sub0-mail-a3.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.16.2); Sat, 01 Dec 2018 22:11:56 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|jurta@jurta.org X-MailChannels-Auth-Id: dreamhost X-Thread-Bitter: 37c3a45272aa7846_1543702316634_4153244178 X-MC-Loop-Signature: 1543702316634:558372839 X-MC-Ingress-Time: 1543702316634 Original-Received: from pdx1-sub0-mail-a3.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a3.g.dreamhost.com (Postfix) with ESMTP id 49C4280149 for ; Sat, 1 Dec 2018 14:11:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=linkov.net; h=from:to :subject:date:message-id:mime-version:content-type; s= linkov.net; bh=lB5Lj+JW/r/uVAhEclfmKSjPW8w=; b=fZ9+BAlXhrbZgX/d0 1eARWAgvJERMZWIuhYIWRV9Y7oaMonqjOTmYuBKm4Trkq3wil6nLkljjhvEvuOvZ +GIXnDdC7Vq/MgiVZaNPft8B8VZLPUYfklRR1h+c9vYPhxCW4oyO8ydGabKE2Ntd J/G12BIPrvleuazFHDo2NeU1fA= Original-Received: from mail.jurta.org (m91-129-107-242.cust.tele2.ee [91.129.107.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: jurta@jurta.org) by pdx1-sub0-mail-a3.g.dreamhost.com (Postfix) with ESMTPSA id D7E1080146 for ; Sat, 1 Dec 2018 14:11:54 -0800 (PST) X-DH-BACKEND: pdx1-sub0-mail-a3 X-VR-OUT-STATUS: OK X-VR-OUT-SCORE: 0 X-VR-OUT-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtkedruddvjedgudeiudcutefuodetggdotefrodftvfcurfhrohhfihhlvgemucggtfgfnhhsuhgsshgtrhhisggvpdfftffgtefojffquffvnecuuegrihhlohhuthemuceftddtnecunecujfgurhephffvufhofffkfgggtgesmhdtreertdertdenucfhrhhomheplfhurhhiucfnihhnkhhovhcuoehjuhhriheslhhinhhkohhvrdhnvghtqeenucffohhmrghinhepghhnuhdrohhrghenucfkphepledurdduvdelrddutdejrddvgedvnecurfgrrhgrmhepmhhouggvpehsmhhtphdphhgvlhhopehmrghilhdrjhhurhhtrgdrohhrghdpihhnvghtpeeluddruddvledruddtjedrvdegvddprhgvthhurhhnqdhprghthheplfhurhhiucfnihhnkhhovhcuoehjuhhriheslhhinhhkohhvrdhnvghtqedpmhgrihhlfhhrohhmpehjuhhriheslhhinhhkohhvrdhnvghtpdhnrhgtphhtthhopegsuhhgqdhgnhhuqdgvmhgrtghssehgnhhurdhorhhgnecuvehluhhsthgvrhfuihiivgeptd X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:152972 Archived-At: --=-=-= Content-Type: text/plain Tags: patch Severity: wishlist For a long time after announcing this feature in https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00537.html I received requests in private mails asking when I'll submit a complete patch. I'm sorry, it took much time addressing all concerns raised in that thread, and testing in many possible scenarios. Based on the feedback, I rewrote it several times, and now finally it's optimized to be fast and reliable. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=diff-font-lock-syntax.patch diff --git a/lisp/vc/diff-mode.el b/lisp/vc/diff-mode.el index 4adef02984..68b2f9e522 100644 --- a/lisp/vc/diff-mode.el +++ b/lisp/vc/diff-mode.el @@ -103,6 +103,21 @@ diff-font-lock-prettify :version "27.1" :type 'boolean) +(defcustom diff-font-lock-syntax 'vc + "If non-nil, diff hunk font-lock includes syntax highlighting. +If `vc', highlight syntax only in Diff buffers created by a version control +system that provides all necessary context for reliable highlighting. +If t, additionally try to get more context from existing files, or when +source files are not found, still try to highlight hunks without enough +context that sometimes might result in wrong fontification. +If `hunk-only', fontification is based on hunk alone, without full source. +This is the fastest, but less reliable." + :version "27.1" + :type '(choice (const :tag "Don't highlight syntax" nil) + (const :tag "Only under version control" vc) + (const :tag "Hunk-based only" hunk-only) + (const :tag "Without full source or get it from files" t))) + (defvar diff-vc-backend nil "The VC backend that created the current Diff buffer, if any.") @@ -406,6 +421,7 @@ diff-font-lock-keywords (1 font-lock-comment-delimiter-face) (2 font-lock-comment-face)) ("^[^-=+*!<>#].*\n" (0 'diff-context)) + (,#'diff--font-lock-syntax) (,#'diff--font-lock-prettify) (,#'diff--font-lock-refined))) @@ -2316,6 +2333,195 @@ diff--font-lock-prettify 'display ""))))) nil) +;;; Syntax highlighting from font-lock + +(defun diff--font-lock-syntax (max) + "Syntax highlighting from font-lock." + (when diff-font-lock-syntax + (when (get-char-property (point) 'diff--font-lock-syntax) + (goto-char (next-single-char-property-change + (point) 'diff--font-lock-syntax nil max))) + (let* ((min (point)) + (beg (or (ignore-errors (diff-beginning-of-hunk)) + (ignore-errors (diff-hunk-next) (point)) + max))) + (while (< beg max) + (let ((end + (save-excursion (goto-char beg) (diff-end-of-hunk) (point)))) + (if (< end min) (setq beg min)) + (unless (or (< end beg) + (get-char-property beg 'diff--font-lock-syntax)) + (diff-syntax-fontify beg end) + (let ((ol (make-overlay beg end))) + (overlay-put ol 'diff--font-lock-syntax t) + (overlay-put ol 'diff-mode 'syntax) + (overlay-put ol 'evaporate t) + (overlay-put ol 'modification-hooks + '(diff--font-lock-syntax--refresh)))) + (goto-char (max beg end)) + (setq beg (or (ignore-errors (diff-hunk-next) (point)) max)))))) + nil) + +(defun diff--font-lock-syntax--refresh (ol _after _beg _end &optional _len) + (delete-overlay ol)) + +(defun diff-syntax-fontify (start end) + (save-excursion + (diff-syntax-fontify-hunk start end t) + (diff-syntax-fontify-hunk start end nil))) + +(defvar diff-syntax-fontify-revisions (make-hash-table :test 'equal)) + +(defun diff-syntax-fontify-hunk (beg end old) + "Highlight language syntax in diff hunks." + (remove-overlays beg end 'diff-mode 'syntax) + (goto-char beg) + (let* ((hunk (buffer-substring-no-properties beg end)) + (text (or (ignore-errors (diff-hunk-text hunk (not old) nil)) "")) + (line (if (looking-at "\\(?:\\*\\{15\\}.*\n\\)?[-@* ]*\\([0-9,]+\\)\\([ acd+]+\\([0-9,]+\\)\\)?") + (if old (match-string 1) + (if (match-end 3) (match-string 3) (match-string 1))))) + (line-nb (and line (string-match "\\([0-9]+\\),\\([0-9]+\\)" line) + (list (string-to-number (match-string 1 line)) + (string-to-number (match-string 2 line))))) + props) + (cond + ((and diff-vc-backend (not (eq diff-font-lock-syntax 'hunk-only))) + (let* ((file (diff-find-file-name old t)) + (revision (and file (if (not old) (nth 1 diff-vc-revisions) + (or (nth 0 diff-vc-revisions) + (vc-working-revision file)))))) + (if file + (if (not revision) + ;; Get properties from the current working file + (when (and (not old) (file-exists-p file)) + ;; Try to reuse an existing buffer + (if (get-file-buffer (expand-file-name file)) + (with-current-buffer (get-file-buffer (expand-file-name file)) + (setq props (diff-syntax-fontify-props nil text line-nb t))) + ;; Get properties from the file + (with-temp-buffer + (insert-file-contents file t) + (setq props (diff-syntax-fontify-props file text line-nb))))) + ;; Get properties from a cached revision + (let* ((buffer-name (format " diff-syntax:%s.~%s~" + (expand-file-name file) revision)) + (buffer (gethash buffer-name diff-syntax-fontify-revisions)) + (no-init t)) + (unless (and buffer (buffer-live-p buffer)) + (let* ((vc-find-revision-no-save t) + (vc-buffer (save-window-excursion + ;; Restore restore previous window configuration + ;; because when vc-find-revision can't find a revision + ;; (e.g. for /dev/null), it jumps to another window + ;; using pop-to-buffer in vc-do-command when + ;; the buffer name doesn't begin with a space char. + (ignore-errors + (vc-find-revision (expand-file-name file) + revision diff-vc-backend))))) + (when vc-buffer + (with-current-buffer (get-buffer-create buffer-name) + (insert-buffer-substring-no-properties vc-buffer) + (setq buffer (current-buffer) no-init nil)) + (puthash buffer-name buffer diff-syntax-fontify-revisions) + (kill-buffer vc-buffer)))) + (when buffer + (with-current-buffer buffer + (setq props (diff-syntax-fontify-props file text line-nb no-init)))))) + ;; If file is unavailable, get properties from the hunk alone + (setq file (car (diff-hunk-file-names old))) + (with-temp-buffer + (insert text) + (setq props (diff-syntax-fontify-props file text line-nb nil t)))))) + ((eq diff-font-lock-syntax 'hunk-only) + (setq file (car (diff-hunk-file-names old))) + (with-temp-buffer + (insert text) + (setq props (diff-syntax-fontify-props file text line-nb nil t)))) + ((not (eq diff-font-lock-syntax 'vc)) + (let ((file (car (diff-hunk-file-names old)))) + (if (and file (file-exists-p file)) + ;; Try to get full text from the file + (with-temp-buffer + (insert-file-contents file t) + (setq props (diff-syntax-fontify-props file text line-nb))) + ;; Otherwise, get properties from the hunk alone + (with-temp-buffer + (insert text) + (setq props (diff-syntax-fontify-props file text line-nb nil t))))))) + + ;; Put properties over the hunk text + (when props + (goto-char beg) + (while (< (progn (forward-line 1) (point)) end) + (when (or (and (not old) (not (looking-at-p "[-<]"))) + (and old (not (looking-at-p "[+>]")))) + (if (and old (not (looking-at-p "[-<]"))) + ;; Fontify context lines only from new source, + ;; don't refontify context lines from old source. + (pop props) + (let ((line-props (pop props)) + (bol (1+ (point)))) + (dolist (prop line-props) + (let ((ol (make-overlay (+ bol (nth 0 prop)) + (+ bol (nth 1 prop)) + nil 'front-advance nil))) + (overlay-put ol 'evaporate t) + (overlay-put ol 'face (nth 2 prop))))))))))) + +(defun diff-syntax-fontify-props (file text line-nb &optional no-init hunk-only) + "Get font-lock properties from the source code." + (unless no-init + (buffer-disable-undo) + (font-lock-mode -1) + (let ((enable-local-variables :safe) ;; to find `mode:' + (buffer-file-name file)) + (set-auto-mode) + (generic-mode-find-file-hook))) + + (let ((font-lock-defaults (or font-lock-defaults '(nil t))) + props beg end) + (goto-char (point-min)) + (if hunk-only + (setq beg (point-min) end (point-max)) + (forward-line (1- (nth 0 line-nb))) + ;; non-regexp looking-at to compare hunk text for verification + (if (search-forward text (+ (point) (length text)) t) + (setq beg (- (point) (length text)) end (point)) + (goto-char (point-min)) + (if (search-forward text nil t) + (setq beg (- (point) (length text)) end (point))))) + + (when (and beg end) + (goto-char beg) + (when (text-property-not-all beg end 'fontified t) + (if file + ;; In a temporary or cached buffer + (save-excursion + (font-lock-fontify-region beg end) + (put-text-property beg end 'fontified t)) + ;; In an existing buffer + (font-lock-ensure beg end))) + + (while (< (point) end) + (let* ((bol (point)) + (eol (line-end-position)) + line-props + (searching t) + (from (point)) to + (val (get-text-property from 'face))) + (while searching + (setq to (next-single-property-change from 'face nil eol)) + (when val (push (list (- from bol) (- to bol) val) line-props)) + (setq val (get-text-property to 'face) from to) + (unless (< to eol) (setq searching nil))) + (when val (push (list from eol val) line-props)) + (push (nreverse line-props) props)) + (forward-line 1))) + (set-buffer-modified-p nil) + (nreverse props))) + + (defun diff--filter-substring (str) (when diff-font-lock-prettify ;; Strip the `display' properties added by diff-font-lock-prettify, --=-=-=--