From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: [patch] use font-lock Date: Fri, 23 May 2008 18:26:27 -0400 Organization: Merrill Press Message-ID: <200805231826.27371.danc@merrillpress.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_TS0NI4ljQr5YNzS" X-Trace: ger.gmane.org 1211581609 4460 80.91.229.12 (23 May 2008 22:26:49 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 23 May 2008 22:26:49 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat May 24 00:27:25 2008 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1JzfjP-0002mA-PH for ged-emacs-devel@m.gmane.org; Sat, 24 May 2008 00:27:24 +0200 Original-Received: from localhost ([127.0.0.1]:54710 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Jzfie-0005Fm-TK for ged-emacs-devel@m.gmane.org; Fri, 23 May 2008 18:26:36 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Jzfia-0005Fh-9E for emacs-devel@gnu.org; Fri, 23 May 2008 18:26:32 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1JzfiY-0005FR-5j for emacs-devel@gnu.org; Fri, 23 May 2008 18:26:31 -0400 Original-Received: from [199.232.76.173] (port=54227 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1JzfiY-0005FO-22 for emacs-devel@gnu.org; Fri, 23 May 2008 18:26:30 -0400 Original-Received: from vpn.merrillpress.com ([64.61.107.78]:58893) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1JzfiX-00056t-Q7 for emacs-devel@gnu.org; Fri, 23 May 2008 18:26:30 -0400 Original-Received: from pluto.merrillpress.net ([10.136.5.5]) by mars.merrillpress.net with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1JzfiW-0004rd-1z for emacs-devel@gnu.org; Fri, 23 May 2008 18:26:28 -0400 X-detected-kernel: by monty-python.gnu.org: Linux 2.6 (newer, 3) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:97623 Archived-At: --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline [Originally sent to the nxml mailing list] I've converted nxml to font-lock. I used the existing fontification machinery and put it inside a cc-mode-style matcher. Efficiency and output are the same, but: 1) hi-lock-mode works now 2) all conventional font-locking functions work as expected. For example, you can turn fontification on and off with M-x font-lock-mode. 3) font-lock-add-keywords DTRT 4) multiple-major-modes modes should be able to use nXML fontification now, though this remains untested I've also added a new function: nxml-debug-region: Interactive function. Activate the region and call nxml-debug-region. The new region is what nxml thinks should be re-fontified if the original region is changed. The new code probably works only on Emacs 22. Lightly tested, but it seems to handle corner highlighting cases fine. I removed a bunch of code that was "for redisplay", since I'm assuming font-lock handles those funky bits. In addition to the patch, I've attached a set of files that demonstrates extending nXML mode to work with the Genshi template engine. The examples ought to work with some slight modification of the embedded paths. (Does the Relax NG compact syntax offer a way to say "include the next file for this document type"?) --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/x-diff; charset="us-ascii"; name="nxml-font-lock.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="nxml-font-lock.patch" Index: rng-auto.el =================================================================== --- rng-auto.el (revision 44) +++ rng-auto.el (working copy) @@ -106,12 +106,9 @@ (autoload (quote nxml-mode) "nxml-mode" "\ Major mode for editing XML. -Syntax highlighting is performed unless the variable -`nxml-syntax-highlight-flag' is nil. - \\[nxml-finish-element] finishes the current element by inserting an end-tag. C-c C-i closes a start-tag with `>' and then inserts a balancing end-tag -leaving point between the start-tag and end-tag. +leaving point between the start-tag and end-tag. \\[nxml-balanced-close-start-tag-block] is similar but for block rather than inline elements: the start-tag, point, and end-tag are all left on separate lines. If `nxml-slash-auto-complete-flag' is non-nil, then inserting a `' and then inserts a balancing end-tag leaving point between the start-tag and end-tag. @@ -570,13 +557,9 @@ (nxml-clear-dependent-regions (point-min) (point-max)) (setq nxml-scan-end (copy-marker (point-min) nil)) (nxml-with-unmodifying-text-property-changes - (when nxml-syntax-highlight-flag - (nxml-clear-fontified (point-min) (point-max))) - (nxml-clear-inside (point-min) (point-max)) + (nxml-clear-inside (point-min) (point-max)) (nxml-with-invisible-motion (nxml-scan-prolog))))) - (when nxml-syntax-highlight-flag - (add-hook 'fontification-functions 'nxml-fontify nil t)) (add-hook 'after-change-functions 'nxml-after-change nil t) (add-hook 'write-contents-hooks 'nxml-prepare-to-save) (when (not (and (buffer-file-name) (file-exists-p (buffer-file-name)))) @@ -585,6 +568,18 @@ (setq buffer-file-coding-system nxml-default-buffer-file-coding-system)) (when nxml-auto-insert-xml-declaration-flag (nxml-insert-xml-declaration))) + + (setq font-lock-defaults + '(nxml-font-lock-keywords + t ; keywords-only; we highlight comments and strings here + nil ; font-lock-keywords-case-fold-search. XML is case sensitive + nil ; no special syntax table + nil ; no automatic syntactic fontification + (font-lock-extend-after-change-region-function + . nxml-extend-after-change-region) + (font-lock-extend-region-functions . (nxml-extend-region)) + (font-lock-unfontify-region-function . nxml-unfontify-region))) + (run-hooks 'nxml-mode-hook)) (defun nxml-degrade (context err) @@ -598,85 +593,91 @@ (save-restriction (widen) (nxml-with-unmodifying-text-property-changes - (nxml-clear-face (point-min) (point-max)) - (nxml-set-fontified (point-min) (point-max)) - (nxml-clear-inside (point-min) (point-max))) + (nxml-clear-inside (point-min) (point-max))) (setq mode-name "nXML/degraded")))) ;;; Change management +(defun nxml-debug-region (start end) + (interactive "r") + (let ((font-lock-beg start) + (font-lock-end end)) + (nxml-extend-region) + (goto-char font-lock-beg) + (set-mark font-lock-end))) + (defun nxml-after-change (start end pre-change-length) - ;; Work around bug in insert-file-contents. - (when (> end (1+ (buffer-size))) - (setq start 1) - (setq end (1+ (buffer-size)))) - (unless nxml-degraded + ; in font-lock mode, nxml-after-change1 is called via + ; nxml-extend-after-change-region instead so that the updated + ; book-keeping information is available for fontification. + + (unless (or font-lock-mode nxml-degraded) (condition-case err - (save-excursion - (save-restriction - (widen) - (save-match-data - (nxml-with-invisible-motion - (nxml-with-unmodifying-text-property-changes - (nxml-after-change1 start end pre-change-length)))))) + (save-excursion + (save-restriction + (widen) + (save-match-data + (nxml-with-invisible-motion + (nxml-with-unmodifying-text-property-changes + (nxml-after-change1 + start end pre-change-length)))))) + (error (nxml-degrade 'nxml-after-change err))))) (defun nxml-after-change1 (start end pre-change-length) - (setq nxml-last-fontify-end nil) + "after-change book-keeping. returns a cons containing a +possibly-enlarged change region. you must still call +nxml-extend-region on this expanded region to obtain the full +extent of the area needing refontification. + +For book-keeping, call this function even when fontification is +disabled." + ;; Work around bug in insert-file-contents, apparently + (when (> end (1+ (buffer-size))) + (setq start 1) + (setq end (1+ (buffer-size)))) + (let ((pre-change-end (+ start pre-change-length))) (setq start (nxml-adjust-start-for-dependent-regions start - end - pre-change-length)) + end + pre-change-length)) + + ;; If the prolog might have changed, rescan the prolog (when (<= start - ;; Add 2 so as to include the < and following char - ;; that start the instance, since changing these - ;; can change where the prolog ends. + ;; Add 2 so as to include the < and following char that + ;; start the instance (document element), since changing + ;; these can change where the prolog ends. (+ nxml-prolog-end 2)) - ;; end must be extended to at least the end of the old prolog + ;; end must be extended to at least the end of the old prolog in + ;; case the new prolog is shorter (when (< pre-change-end nxml-prolog-end) (setq end ;; don't let end get out of range even if pre-change-length ;; is bogus (min (point-max) (+ end (- nxml-prolog-end pre-change-end))))) + (nxml-scan-prolog))) - (cond ((<= end nxml-prolog-end) - (setq end nxml-prolog-end) - (goto-char start) - ;; This is so that Emacs redisplay works - (setq start (line-beginning-position))) - ((and (<= start nxml-scan-end) - (> start (point-min)) - (nxml-get-inside (1- start))) - ;; The closing delimiter might have been removed. - ;; So we may need to redisplay from the beginning - ;; of the token. - (goto-char (1- start)) - (nxml-move-outside-backwards) - ;; This is so that Emacs redisplay works - (setq start (line-beginning-position)) - (setq end (max (nxml-scan-after-change (point) end) - end))) - (t - (goto-char start) - ;; This is both for redisplay and to move back - ;; past any incomplete opening delimiters - (setq start (line-beginning-position)) - (setq end (max (nxml-scan-after-change start end) - end)))) - (when nxml-syntax-highlight-flag - (when (>= start end) - ;; Must clear at least one char so as to trigger redisplay. - (cond ((< start (point-max)) - (setq end (1+ start))) - (t - (setq end (point-max)) - (goto-char end) - (setq start (line-beginning-position))))) - (nxml-clear-fontified start end))) + (when (> end nxml-prolog-end) + (when (and (<= start nxml-scan-end) + (> start (point-min)) + (nxml-get-inside (1- start))) + ;; The closing delimiter might have been removed. + ;; So we may need to redisplay from the beginning + ;; of the token. + (goto-char (1- start)) + (nxml-move-outside-backwards) + (setq start (point))) + + (setq end (max (nxml-scan-after-change start end) + end))) + + (nxml-debug-change "nxml-after-change1" start end) + (cons start end)) + ;;; Encodings (defun nxml-insert-xml-declaration () @@ -862,59 +863,113 @@ ;;; Fontification -(defun nxml-fontify (start) - (condition-case err - (save-excursion - (save-restriction - (widen) - (save-match-data - (nxml-with-invisible-motion - (nxml-with-unmodifying-text-property-changes - (if (or nxml-degraded - ;; just in case we get called in the wrong buffer - (not nxml-prolog-end)) - (nxml-set-fontified start (point-max)) - (nxml-fontify1 start))))))) - (error - (nxml-degrade 'nxml-fontify err)))) +(defun nxml-unfontify-region (start end) + (font-lock-default-unfontify-region start end) + (nxml-clear-char-ref-extra-display start end)) -(defun nxml-fontify1 (start) - (cond ((< start nxml-prolog-end) - (nxml-fontify-prolog) - (nxml-set-fontified (point-min) - nxml-prolog-end)) - (t - (goto-char start) - (when (not (eq nxml-last-fontify-end start)) - (when (not (equal (char-after) ?\<)) - (search-backward "<" nxml-prolog-end t)) - (nxml-ensure-scan-up-to-date) - (nxml-move-outside-backwards)) - (let ((start (point))) - (nxml-do-fontify (min (point-max) - (+ start nxml-fontify-chunk-size))) - (setq nxml-last-fontify-end (point)) - (nxml-set-fontified start nxml-last-fontify-end))))) +(defun nxml-extend-region () + "Extend the region to hold the minimum area we can fontify with +nXML. Called with font-lock-beg and font-lock-end dynamically bound." + (let ((start font-lock-beg) + (end font-lock-end)) -(defun nxml-fontify-buffer () - (interactive) - (save-excursion - (save-restriction - (widen) - (nxml-with-invisible-motion - (goto-char (point-min)) - (nxml-with-unmodifying-text-property-changes - (nxml-fontify-prolog) - (goto-char nxml-prolog-end) - (nxml-do-fontify)))))) + (nxml-debug-change "nxml-extend-region(input)" start end) + (when (< start nxml-prolog-end) + (setq start (point-min))) + + (cond ((<= end nxml-prolog-end) + (setq end nxml-prolog-end)) + + (t + (goto-char start) + ;; some font-lock backends (like Emacs 22 jit-lock) snap + ;; the region to the beginning of the line no matter what + ;; we say here. To mitigate the resulting excess + ;; fontification, ignore leading whitespace. + (skip-syntax-forward " ") + + ;; find the beginning of the previous tag + (when (not (equal (char-after) ?\<)) + (search-backward "<" nxml-prolog-end t)) + (nxml-ensure-scan-up-to-date) + (nxml-move-outside-backwards) + (setq start (point)) + + (while (< (point) end) + (nxml-tokenize-forward)) + + (setq end (point)))) + + (when (or (< start font-lock-beg) + (> end font-lock-end)) + (setq font-lock-beg start + font-lock-end end) + (nxml-debug-change "nxml-extend-region" start end) + t))) + +(defun nxml-extend-after-change-region (start end pre-change-length) + (unless nxml-degraded + (setq nxml-last-fontify-end nil) + + (when (> end (1+ (buffer-size))) + (setq start 1) + (setq end (1+ (buffer-size)))) + + (condition-case err + (save-excursion + (save-restriction + (widen) + (save-match-data + (nxml-with-invisible-motion + (nxml-with-unmodifying-text-property-changes + (nxml-extend-after-change-region1 start end pre-change-length)))))) + (error + (nxml-degrade 'nxml-extend-after-change-region err))))) + +(defun nxml-extend-after-change-region1 (start end pre-change-length) + (let* ((region (nxml-after-change1 start end pre-change-length)) + (font-lock-beg (car region)) + (font-lock-end (cdr region))) + + (nxml-extend-region) + (cons font-lock-beg font-lock-end))) + +(defun nxml-fontify-matcher (bound) + "Called as font-lock keyword matcher." + + (unless nxml-degraded + (nxml-debug-change "nxml-fontify-matcher" (point) bound) + + (when (< (point) nxml-prolog-end) + (goto-char (point-min)) + (nxml-fontify-prolog) + (goto-char nxml-prolog-end)) + + (when (not (eq nxml-last-fontify-end (point))) + (when (not (equal (char-after) ?\<)) + (search-backward "<" nxml-prolog-end t)) + (nxml-ensure-scan-up-to-date) + (nxml-move-outside-backwards)) + + (let (xmltok-dependent-regions + xmltok-errors) + (while (and (< (point) bound) + (nxml-tokenize-forward)) + (nxml-apply-fontify-rule))) + + (setq nxml-last-fontify-end (point))) + + ;; Since we did the fontification internally, tell font-lock to not + ;; do anything itself. + nil) + (defun nxml-fontify-prolog () "Fontify the prolog. The buffer is assumed to be prepared for fontification. This does not set the fontified property, but it does clear faces appropriately." (let ((regions nxml-prolog-regions)) - (nxml-clear-face (point-min) nxml-prolog-end) (while regions (let ((region (car regions))) (nxml-apply-fontify-rule (aref region 0) @@ -922,17 +977,6 @@ (aref region 2))) (setq regions (cdr regions))))) -(defun nxml-do-fontify (&optional bound) - "Fontify at least as far as bound. -Leave point after last fontified position." - (unless bound (setq bound (point-max))) - (let (xmltok-dependent-regions - xmltok-errors) - (while (and (< (point) bound) - (nxml-tokenize-forward)) - (nxml-clear-face xmltok-start (point)) - (nxml-apply-fontify-rule)))) - ;; Vectors identify a substring of the token to be highlighted in some face. ;; Token types returned by xmltok-forward. @@ -2582,13 +2626,7 @@ (> (prefix-numeric-value arg) 0)))) (when (not (eq new nxml-char-ref-extra-display)) (setq nxml-char-ref-extra-display new) - (save-excursion - (save-restriction - (widen) - (if nxml-char-ref-extra-display - (nxml-with-unmodifying-text-property-changes - (nxml-clear-fontified (point-min) (point-max))) - (nxml-clear-char-ref-extra-display (point-min) (point-max)))))))) + (font-lock-fontify-buffer)))) (put 'nxml-char-ref 'evaporate t) --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/plain; charset="us-ascii"; name="qtmstr-nxml-mode.el" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="qtmstr-nxml-mode.el" ;;;; Customization for nXML mode (setq rng-schema-locating-files (append '("~/emacs/nxml-schema/schemas.xml") rng-schema-locating-files-default)) (defface nxml-template '((t (:bold t :foreground "blue" :background "#ddddff"))) "Face used to highlight embedded template constructs" :group 'nxml-highlighting-faces) (defun qtmstr-nxml-hook () (when (string-match "\\.html$" buffer-file-name) (let ((py-prefix-re (regexp-opt '( "py:if" "py:choose" "py:when" "py:otherwise" "py:for" "py:def" "py:match" "py:with" "py:attrs" "py:content" "py:replace" "py:strip")))) (font-lock-add-keywords nil `(("\\$\\([a-zA-Z_][a-zA-Z0-9_]*\\)" 1 'nxml-template prepend) ("\\${\\([^}\"]+\\)}" 1 'nxml-template prepend) (,(concat py-prefix-re "=\"\\([^\"]*\\)\"") 1 'nxml-template prepend)))))) (add-hook 'nxml-mode-hook #'qtmstr-nxml-hook) --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/plain; charset="us-ascii"; name="genshi.rnc" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="genshi.rnc" namespace py = "http://genshi.edgewall.org/" genshi.expr-type = xsd:string { minLength = "1" } genshi.with-type = xsd:string { minLength = "1" } genshi.choose-type = xsd:string genshi.def-type = xsd:string genshi.xpath-type = xsd:anyURI genshi.attrib = attribute py:if { genshi.expr-type }?, attribute py:choose { genshi.choose-type }?, attribute py:when { genshi.expr-type }?, attribute py:otherwise { genshi.expr-type }?, attribute py:for { genshi.expr-type }?, attribute py:def { genshi.def-type }?, attribute py:match { genshi.xpath-type }?, attribute py:with { genshi.with-type }?, attribute py:attrs { genshi.expr-type }?, attribute py:content { genshi.expr-type }?, attribute py:replace { genshi.expr-type }?, attribute py:strip { genshi.expr-type }? genshi.if.attlist = attribute expr { genshi.expr-type } genshi.for.attlist = attribute each { genshi.expr-type } genshi.def.attlist = attribute each { genshi.expr-type } genshi.with.attlist = attribute vars { genshi.with-type } --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/x-c++src; charset="us-ascii"; name="qtmstr-xhtml.rnc" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="qtmstr-xhtml.rnc" namespace py = "http://genshi.edgewall.org/" namespace xi = "http://www.w3.org/2001/XInclude" include "genshi.rnc" include "xinclude.rnc" include "../nxml/schema/xhtml.rnc" start |= head|body|p|\div|h1|h2|h3|h4|h5|h6|hr|pre|dl|ol|ul|table|form Common.attrib &= genshi.attrib head.attlist &= genshi.attrib html.attlist &= genshi.attrib Head.class = base | isindex | link | meta | script | title | style | if-head | for-head | def-head | with-head Head.model = Head.class* head.content &= Head.model* if-inline = element py:if { genshi.if.attlist, Inline.model } if-block = element py:if { genshi.if.attlist, Block.model } if-head = element py:if { genshi.if.attlist, Head.model } for-inline = element py:for { genshi.for.attlist, Inline.model } for-block = element py:for { genshi.for.attlist, Block.model } for-head = element py:for { genshi.for.attlist, Head.model } def-inline = element py:def { genshi.def.attlist, Inline.model } def-block = element py:def { genshi.def.attlist, Block.model } def-head = element py:def { genshi.def.attlist, Head.model } with-inline = element py:with { genshi.with.attlist, Inline.model } with-block = element py:with { genshi.with.attlist, Block.model } with-head = element py:with { genshi.with.attlist, Head.model } Inline.class |= if-inline | for-inline | def-inline | with-inline Block.class |= if-block | for-block | def-block | with-block xi-inline = element xi:include { xinclude.include.attlist, element xi:fallback { genshi.attrib, (xi-inline | Inline.model)* }? } xi-block = element xi:include { xinclude.include.attlist, element xi:fallback { genshi.attrib, (xi-block | Block.model)* }? } xi-head = element xi:include { xinclude.include.attlist, element xi:fallback { genshi.attrib, (xi-head | Head.model)* }? } Inline.class |= xi-inline Block.class |= xi-block Head.class |= xi-head --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/xml; charset="us-ascii"; name="schemas.xml" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="schemas.xml" --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/x-c++src; charset="us-ascii"; name="xinclude.rnc" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xinclude.rnc" namespace xi = "http://www.w3.org/2001/XInclude" namespace local = "" xinclude.include.attlist = attribute href { xsd:anyURI }?, attribute parse { xsd:string }?, attribute xpointer { xsd:string }?, attribute encoding { xsd:string }?, attribute accept { xsd:string }?, attribute accept-language { xsd:string }? --Boundary-00=_TS0NI4ljQr5YNzS Content-Type: text/html; charset="us-ascii"; name="HelloWorldPage.html" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="HelloWorldPage.html"
Hello from hello! Test ${name}
--Boundary-00=_TS0NI4ljQr5YNzS--