[ANN] Improved Flyspell check

[ANN] Improved Flyspell check

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

Hello,

I'd like to submit the following patch for review, testing and,
hopefully, inclusion.

From the user point of view, it improves checks when using Flyspell,
removing false-positive and allowing to check more areas.

More importantly, it doesn't rely anymore on fontification, which means
that activating checks is very demanding for the parser. So, it is
a good stress test for the newly integrated cache mechanism.


Feedback welcome.


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Improve-Flyspell-check.patch --]
[-- Type: text/x-diff, Size: 10394 bytes --]

From 56bbebbd926e04ad4382cc9f37fb9e23d548130a Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Sun, 3 Nov 2013 15:15:07 +0100
Subject: [PATCH] Improve Flyspell check

* lisp/org.el (org-mode-flyspell-verify): Rewrite function using
  "org-element.el".  In particular, it doesn't rely on fontification
  anymore.
(org-remove-flyspell-overlays-in): Remove function.
(org-do-emphasis-faces, org-activate-plain-links)
(org-fontify-meta-lines-and-blocks-1, org-activate-footnote-links)
(org-activate-target-links, org-activate-tags, org-activate-code)
(org-activate-angle-links): Don't call
`org-remove-flyspell-overlays-in'.
* contrib/lisp/org-wikinodes.el (org-wikinodes-activate-links): Don't
  call `org-remove-flyspell-overlays-in'.
---
 contrib/lisp/org-wikinodes.el |   5 --
 lisp/org.el                   | 111 ++++++++++++++++++++++++++----------------
 2 files changed, 70 insertions(+), 46 deletions(-)

diff --git a/contrib/lisp/org-wikinodes.el b/contrib/lisp/org-wikinodes.el
index 4efc373..c6f2006 100644
--- a/contrib/lisp/org-wikinodes.el
+++ b/contrib/lisp/org-wikinodes.el
@@ -79,15 +79,10 @@ to `directory'."
       (if (re-search-forward org-wikinodes-camel-regexp limit t)
 	  (if (equal (char-after (point-at-bol)) ?*)
 	      (progn
-		;; in  heading - deactivate flyspell
-		(org-remove-flyspell-overlays-in (match-beginning 0)
-						 (match-end 0))
 		(add-text-properties (match-beginning 0) (match-end 0)
 				     '(org-no-flyspell t))
 		t)
 	    ;; this is a wiki link
-	    (org-remove-flyspell-overlays-in (match-beginning 0)
-					     (match-end 0))
 	    (add-text-properties (match-beginning 0) (match-end 0)
 				 (list 'mouse-face 'highlight
 				       'face 'org-link
diff --git a/lisp/org.el b/lisp/org.el
index 2382a9c..a7ab90f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5459,8 +5459,6 @@ The following commands are available:
   (abbrev-table-put org-mode-abbrev-table
 		    :parents (list text-mode-abbrev-table)))
 
-(put 'org-mode 'flyspell-mode-predicate 'org-mode-flyspell-verify)
-
 (defsubst org-fix-ellipsis-at-bol ()
   (save-excursion (goto-char (window-start)) (recenter 0)))
 
@@ -5685,9 +5683,6 @@ The time stamps may be either active or inactive.")
 	    (font-lock-prepend-text-property (match-beginning 2) (match-end 2)
 					     'face
 					     (nth 1 a))
-	    (and (nth 2 a)
-		 (org-remove-flyspell-overlays-in
-		  (match-beginning 0) (match-end 0)))
 	    (add-text-properties (match-beginning 2) (match-end 2)
 				 '(font-lock-multiline t org-emphasis t))
 	    (when org-hide-emphasis-markers
@@ -5753,7 +5748,6 @@ prompted for."
   (let (f hl)
     (when (and (re-search-forward (concat org-plain-link-re) limit t)
 	       (not (org-in-src-block-p)))
-      (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
       (setq f (get-text-property (match-beginning 0) 'face))
       (setq hl (org-match-string-no-properties 0))
       (if (or (eq f 'org-tag)
@@ -5770,7 +5764,6 @@ prompted for."
 (defun org-activate-code (limit)
   (if (re-search-forward "^[ \t]*\\(:\\(?: .*\\|$\\)\n?\\)" limit t)
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(remove-text-properties (match-beginning 0) (match-end 0)
 				'(display t invisible t intangible t))
 	t)))
@@ -5812,7 +5805,6 @@ by a #."
 	  (cond
 	   ((member dc1 '("+html:" "+ascii:" "+latex:"))
 	    ;; a single line of backend-specific content
-	    (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	    (remove-text-properties (match-beginning 0) (match-end 0)
 				    '(display t invisible t intangible t))
 	    (add-text-properties (match-beginning 1) (match-end 3)
@@ -5901,7 +5893,6 @@ by a #."
   (if (and (re-search-forward org-angle-link-re limit t)
 	   (not (org-in-src-block-p)))
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(add-text-properties (match-beginning 0) (match-end 0)
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map))
@@ -5913,7 +5904,6 @@ by a #."
   (let ((fn (org-footnote-next-reference-or-definition limit)))
     (when fn
       (let ((beg (nth 1 fn)) (end (nth 2 fn)))
-	(org-remove-flyspell-overlays-in beg end)
 	(add-text-properties beg end
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map
@@ -5941,7 +5931,6 @@ by a #."
 		       'htmlize-link `(:uri ,hl))))
 	;; We need to remove the invisible property here.  Table narrowing
 	;; may have made some of this invisible.
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(remove-text-properties (match-beginning 0) (match-end 0)
 				'(invisible nil))
 	(if (match-end 3)
@@ -5965,7 +5954,6 @@ by a #."
   (if (and (re-search-forward org-tsr-regexp-both limit t)
 	   (not (equal (char-before (match-beginning 0)) 91)))
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(add-text-properties (match-beginning 0) (match-end 0)
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map))
@@ -5992,7 +5980,6 @@ by a #."
     (let ((case-fold-search t))
       (if (re-search-forward org-target-link-regexp limit t)
 	  (progn
-	    (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	    (add-text-properties (match-beginning 0) (match-end 0)
 				 (list 'mouse-face 'highlight
 				       'keymap org-mouse-map
@@ -6110,7 +6097,6 @@ between words."
 (defun org-activate-tags (limit)
   (if (re-search-forward (org-re "^\\*+.*[ \t]\\(:[[:alnum:]_@#%:]+:\\)[ \r\n]") limit t)
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 1) (match-end 1))
 	(add-text-properties (match-beginning 1) (match-end 1)
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map))
@@ -23861,34 +23847,77 @@ To get rid of the restriction, use \\[org-agenda-remove-restriction-lock]."
 
 ;;; Fixes and Hacks for problems with other packages
 
-;; Make flyspell not check words in links, to not mess up our keymap
-(defvar org-element-affiliated-keywords) ; From org-element.el
-(defvar org-element-block-name-alist)	 ; From org-element.el
 (defun org-mode-flyspell-verify ()
-  "Don't let flyspell put overlays at active buttons, or on
-   {todo,all-time,additional-option-like}-keywords."
-  (require 'org-element) ; For `org-element-affiliated-keywords'
-  (let ((pos (max (1- (point)) (point-min)))
-	(word (thing-at-point 'word)))
-    (and (not (get-text-property pos 'keymap))
-	 (not (get-text-property pos 'org-no-flyspell))
-	 (not (member word org-todo-keywords-1))
-	 (not (member word org-all-time-keywords))
-	 (not (member word org-options-keywords))
-	 (not (member word (mapcar 'car org-startup-options)))
-	 (not (member-ignore-case word org-element-affiliated-keywords))
-	 (not (member-ignore-case word (org-get-export-keywords)))
-	 (not (member-ignore-case
-	       word (mapcar 'car org-element-block-name-alist)))
-	 (not (member-ignore-case word '("BEGIN" "END" "ATTR")))
-	 (not (org-in-src-block-p)))))
-
-(defun org-remove-flyspell-overlays-in (beg end)
-  "Remove flyspell overlays in region."
-  (and (org-bound-and-true-p flyspell-mode)
-       (fboundp 'flyspell-delete-region-overlays)
-       (flyspell-delete-region-overlays beg end))
-  (add-text-properties beg end '(org-no-flyspell t)))
+  "Function used for `flyspell-generic-check-word-predicate'."
+  (if (org-at-heading-p)
+      ;; At a headline or an inlinetask, check title only.  This is
+      ;; faster than relying on `org-element-at-point'.
+      (and (save-excursion (beginning-of-line)
+			   (and (let ((case-fold-search t))
+				  (not (looking-at "\\*+ END[ \t]*$")))
+				(looking-at org-complex-heading-regexp)))
+	   (match-beginning 4)
+	   (>= (point) (match-beginning 4))
+	   (or (not (match-beginning 5))
+	       (< (point) (match-beginning 5))))
+    (let* ((element (org-element-at-point))
+	   (post-affiliated (org-element-property :post-affiliated element))
+	   (object-check
+	    (function
+	     (lambda (object)
+	       ;; Non-nil if flyspell checks can be done in OBJECT.
+	       (case (org-element-type object)
+		 ;; Prevent checks in links due to keybinding conflict
+		 ;; with Flyspell.
+		 ((code entity export-snippet inline-babel-call inline-src-block
+			line-break latex-fragment link macro statistics-cookie
+			target timestamp verbatim)
+		  nil)
+		 (footnote-reference
+		  ;; Only in inline footnotes, within the definition.
+		  (and (eq (org-element-property :type object) 'inline)
+		       (< (save-excursion
+			    (goto-char (org-element-property :begin object))
+			    (search-forward ":" nil t 2))
+			  (point))))
+		 (otherwise t))))))
+      (if (and post-affiliated (< (point) post-affiliated))
+	  (and (save-excursion
+		 (beginning-of-line)
+		 (let ((case-fold-search t)) (looking-at "[ \t]*#\\+CAPTION:")))
+	       (> (point) (match-end 0))
+	       (funcall object-check (org-element-context element)))
+	(case (org-element-type element)
+	  (comment
+	   (>= (point) (org-element-property :post-affiliated element)))
+	  (comment-block
+	   ;; Allow checks between block markers, not on them.
+	   (and (> (line-beginning-position)
+		   (org-element-property :post-affiliated element))
+		(save-excursion
+		  (end-of-line)
+		  (skip-chars-forward " \r\t\n")
+		  (< (point) (org-element-property :end element)))))
+	  ;; Arbitrary list of keywords where checks are meaningful.
+	  ;; Make sure point is on the value part of the element.
+	  (keyword
+	   (and (member (org-element-property :key element)
+			'("DESCRIPTION" "TITLE"))
+		(< (save-excursion
+		     (beginning-of-line) (search-forward ":") (point))
+		   (point))))
+	  ;; Check is globally allowed in paragraphs verse blocks and
+	  ;; table rows (after affiliated keywords) but some objects
+	  ;; must not be affected.
+	  ((paragraph table-row verse-block)
+	   (and (>= (point) (org-element-property :contents-begin element))
+		(< (point) (org-element-property :contents-end element))
+		(funcall object-check (org-element-context element))))
+	  (quote-section t)
+	  ;; No check everywhere else (including verbatim parts like
+	  ;; example blocks and fixed-width areas).
+	  (otherwise nil))))))
+(put 'org-mode 'flyspell-mode-predicate 'org-mode-flyspell-verify)
 
 ;; Make `bookmark-jump' shows the jump location if it was hidden.
 (eval-after-load "bookmark"
-- 
1.8.4.2

Re: [ANN] Improved Flyspell check

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 84 bytes --]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

Updated patch.


-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Improve-Flyspell-check.patch --]
[-- Type: text/x-diff, Size: 10287 bytes --]

From 13f68648a0ff468385668f6b141748607d3f8103 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Sun, 3 Nov 2013 15:15:07 +0100
Subject: [PATCH] Improve Flyspell check

* lisp/org.el (org-mode-flyspell-verify): Rewrite function using
  "org-element.el".  In particular, it doesn't rely on fontification
  anymore.
(org-remove-flyspell-overlays-in): Remove function.
(org-do-emphasis-faces, org-activate-plain-links)
(org-fontify-meta-lines-and-blocks-1, org-activate-footnote-links)
(org-activate-target-links, org-activate-tags, org-activate-code)
(org-activate-angle-links): Don't call
`org-remove-flyspell-overlays-in'.
* contrib/lisp/org-wikinodes.el (org-wikinodes-activate-links): Don't
  call `org-remove-flyspell-overlays-in'.
---
 contrib/lisp/org-wikinodes.el |   5 --
 lisp/org.el                   | 109 ++++++++++++++++++++++++++----------------
 2 files changed, 68 insertions(+), 46 deletions(-)

diff --git a/contrib/lisp/org-wikinodes.el b/contrib/lisp/org-wikinodes.el
index 4efc373..c6f2006 100644
--- a/contrib/lisp/org-wikinodes.el
+++ b/contrib/lisp/org-wikinodes.el
@@ -79,15 +79,10 @@ to `directory'."
       (if (re-search-forward org-wikinodes-camel-regexp limit t)
 	  (if (equal (char-after (point-at-bol)) ?*)
 	      (progn
-		;; in  heading - deactivate flyspell
-		(org-remove-flyspell-overlays-in (match-beginning 0)
-						 (match-end 0))
 		(add-text-properties (match-beginning 0) (match-end 0)
 				     '(org-no-flyspell t))
 		t)
 	    ;; this is a wiki link
-	    (org-remove-flyspell-overlays-in (match-beginning 0)
-					     (match-end 0))
 	    (add-text-properties (match-beginning 0) (match-end 0)
 				 (list 'mouse-face 'highlight
 				       'face 'org-link
diff --git a/lisp/org.el b/lisp/org.el
index 2382a9c..452babb 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5459,8 +5459,6 @@ The following commands are available:
   (abbrev-table-put org-mode-abbrev-table
 		    :parents (list text-mode-abbrev-table)))
 
-(put 'org-mode 'flyspell-mode-predicate 'org-mode-flyspell-verify)
-
 (defsubst org-fix-ellipsis-at-bol ()
   (save-excursion (goto-char (window-start)) (recenter 0)))
 
@@ -5685,9 +5683,6 @@ The time stamps may be either active or inactive.")
 	    (font-lock-prepend-text-property (match-beginning 2) (match-end 2)
 					     'face
 					     (nth 1 a))
-	    (and (nth 2 a)
-		 (org-remove-flyspell-overlays-in
-		  (match-beginning 0) (match-end 0)))
 	    (add-text-properties (match-beginning 2) (match-end 2)
 				 '(font-lock-multiline t org-emphasis t))
 	    (when org-hide-emphasis-markers
@@ -5753,7 +5748,6 @@ prompted for."
   (let (f hl)
     (when (and (re-search-forward (concat org-plain-link-re) limit t)
 	       (not (org-in-src-block-p)))
-      (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
       (setq f (get-text-property (match-beginning 0) 'face))
       (setq hl (org-match-string-no-properties 0))
       (if (or (eq f 'org-tag)
@@ -5770,7 +5764,6 @@ prompted for."
 (defun org-activate-code (limit)
   (if (re-search-forward "^[ \t]*\\(:\\(?: .*\\|$\\)\n?\\)" limit t)
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(remove-text-properties (match-beginning 0) (match-end 0)
 				'(display t invisible t intangible t))
 	t)))
@@ -5812,7 +5805,6 @@ by a #."
 	  (cond
 	   ((member dc1 '("+html:" "+ascii:" "+latex:"))
 	    ;; a single line of backend-specific content
-	    (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	    (remove-text-properties (match-beginning 0) (match-end 0)
 				    '(display t invisible t intangible t))
 	    (add-text-properties (match-beginning 1) (match-end 3)
@@ -5901,7 +5893,6 @@ by a #."
   (if (and (re-search-forward org-angle-link-re limit t)
 	   (not (org-in-src-block-p)))
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(add-text-properties (match-beginning 0) (match-end 0)
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map))
@@ -5913,7 +5904,6 @@ by a #."
   (let ((fn (org-footnote-next-reference-or-definition limit)))
     (when fn
       (let ((beg (nth 1 fn)) (end (nth 2 fn)))
-	(org-remove-flyspell-overlays-in beg end)
 	(add-text-properties beg end
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map
@@ -5941,7 +5931,6 @@ by a #."
 		       'htmlize-link `(:uri ,hl))))
 	;; We need to remove the invisible property here.  Table narrowing
 	;; may have made some of this invisible.
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(remove-text-properties (match-beginning 0) (match-end 0)
 				'(invisible nil))
 	(if (match-end 3)
@@ -5965,7 +5954,6 @@ by a #."
   (if (and (re-search-forward org-tsr-regexp-both limit t)
 	   (not (equal (char-before (match-beginning 0)) 91)))
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	(add-text-properties (match-beginning 0) (match-end 0)
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map))
@@ -5992,7 +5980,6 @@ by a #."
     (let ((case-fold-search t))
       (if (re-search-forward org-target-link-regexp limit t)
 	  (progn
-	    (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
 	    (add-text-properties (match-beginning 0) (match-end 0)
 				 (list 'mouse-face 'highlight
 				       'keymap org-mouse-map
@@ -6110,7 +6097,6 @@ between words."
 (defun org-activate-tags (limit)
   (if (re-search-forward (org-re "^\\*+.*[ \t]\\(:[[:alnum:]_@#%:]+:\\)[ \r\n]") limit t)
       (progn
-	(org-remove-flyspell-overlays-in (match-beginning 1) (match-end 1))
 	(add-text-properties (match-beginning 1) (match-end 1)
 			     (list 'mouse-face 'highlight
 				   'keymap org-mouse-map))
@@ -23861,34 +23847,75 @@ To get rid of the restriction, use \\[org-agenda-remove-restriction-lock]."
 
 ;;; Fixes and Hacks for problems with other packages
 
-;; Make flyspell not check words in links, to not mess up our keymap
-(defvar org-element-affiliated-keywords) ; From org-element.el
-(defvar org-element-block-name-alist)	 ; From org-element.el
 (defun org-mode-flyspell-verify ()
-  "Don't let flyspell put overlays at active buttons, or on
-   {todo,all-time,additional-option-like}-keywords."
-  (require 'org-element) ; For `org-element-affiliated-keywords'
-  (let ((pos (max (1- (point)) (point-min)))
-	(word (thing-at-point 'word)))
-    (and (not (get-text-property pos 'keymap))
-	 (not (get-text-property pos 'org-no-flyspell))
-	 (not (member word org-todo-keywords-1))
-	 (not (member word org-all-time-keywords))
-	 (not (member word org-options-keywords))
-	 (not (member word (mapcar 'car org-startup-options)))
-	 (not (member-ignore-case word org-element-affiliated-keywords))
-	 (not (member-ignore-case word (org-get-export-keywords)))
-	 (not (member-ignore-case
-	       word (mapcar 'car org-element-block-name-alist)))
-	 (not (member-ignore-case word '("BEGIN" "END" "ATTR")))
-	 (not (org-in-src-block-p)))))
-
-(defun org-remove-flyspell-overlays-in (beg end)
-  "Remove flyspell overlays in region."
-  (and (org-bound-and-true-p flyspell-mode)
-       (fboundp 'flyspell-delete-region-overlays)
-       (flyspell-delete-region-overlays beg end))
-  (add-text-properties beg end '(org-no-flyspell t)))
+  "Function used for `flyspell-generic-check-word-predicate'."
+  (if (org-at-heading-p)
+      ;; At a headline or an inlinetask, check title only.  This is
+      ;; faster than relying on `org-element-at-point'.
+      (and (save-excursion (beginning-of-line)
+			   (and (let ((case-fold-search t))
+				  (not (looking-at "\\*+ END[ \t]*$")))
+				(looking-at org-complex-heading-regexp)))
+	   (match-beginning 4)
+	   (>= (point) (match-beginning 4))
+	   (or (not (match-beginning 5))
+	       (< (point) (match-beginning 5))))
+    (let* ((element (org-element-at-point))
+	   (post-affiliated (org-element-property :post-affiliated element))
+	   (object-check
+	    (function
+	     ;; Non-nil if checks can be done for object at point.
+	     (lambda ()
+	       (let ((object (save-excursion
+			       (when (org-looking-at-p "\\>") (backward-char))
+			       (org-element-context element))))
+		 (case (org-element-type object)
+		   ;; Prevent checks in links due to keybinding conflict
+		   ;; with Flyspell.
+		   ((code entity export-snippet inline-babel-call
+			  inline-src-block line-break latex-fragment link macro
+			  statistics-cookie target timestamp verbatim)
+		    nil)
+		   (footnote-reference
+		    ;; Only in inline footnotes, within the definition.
+		    (and (eq (org-element-property :type object) 'inline)
+			 (< (save-excursion
+			      (goto-char (org-element-property :begin object))
+			      (search-forward ":" nil t 2))
+			    (point))))
+		   (otherwise t)))))))
+      (if (and post-affiliated (< (point) post-affiliated))
+	  (and (save-excursion
+		 (beginning-of-line)
+		 (let ((case-fold-search t)) (looking-at "[ \t]*#\\+CAPTION:")))
+	       (> (point) (match-end 0))
+	       (funcall object-check))
+	(case (org-element-type element)
+	  ((comment quote-section) t)
+	  (comment-block
+	   ;; Allow checks between block markers, not on them.
+	   (and (> (line-beginning-position)
+		   (org-element-property :post-affiliated element))
+		(save-excursion
+		  (end-of-line)
+		  (skip-chars-forward " \r\t\n")
+		  (< (point) (org-element-property :end element)))))
+	  ;; Arbitrary list of keywords where checks are meaningful.
+	  ;; Make sure point is on the value part of the element.
+	  (keyword
+	   (and (member (org-element-property :key element)
+			'("DESCRIPTION" "TITLE"))
+		(< (save-excursion
+		     (beginning-of-line) (search-forward ":") (point))
+		   (point))))
+	  ;; Check is globally allowed in paragraphs verse blocks and
+	  ;; table rows (after affiliated keywords) but some objects
+	  ;; must not be affected.
+	  ((paragraph table-row verse-block)
+	   (and (>= (point) (org-element-property :contents-begin element))
+		(< (point) (org-element-property :contents-end element))
+		(funcall object-check))))))))
+(put 'org-mode 'flyspell-mode-predicate 'org-mode-flyspell-verify)
 
 ;; Make `bookmark-jump' shows the jump location if it was hidden.
 (eval-after-load "bookmark"
-- 
1.8.4.2

Re: [ANN] Improved Flyspell check

[Eric Schulte]

I've been using this since it was sent, and I haven't noticed any bad
behavior.

Cheers,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
> Updated patch.
>
>
> -- 
> Nicolas Goaziou
>
> From 13f68648a0ff468385668f6b141748607d3f8103 Mon Sep 17 00:00:00 2001
> From: Nicolas Goaziou <n.goaziou@gmail.com>
> Date: Sun, 3 Nov 2013 15:15:07 +0100
> Subject: [PATCH] Improve Flyspell check
>
> * lisp/org.el (org-mode-flyspell-verify): Rewrite function using
>   "org-element.el".  In particular, it doesn't rely on fontification
>   anymore.
> (org-remove-flyspell-overlays-in): Remove function.
> (org-do-emphasis-faces, org-activate-plain-links)
> (org-fontify-meta-lines-and-blocks-1, org-activate-footnote-links)
> (org-activate-target-links, org-activate-tags, org-activate-code)
> (org-activate-angle-links): Don't call
> `org-remove-flyspell-overlays-in'.
> * contrib/lisp/org-wikinodes.el (org-wikinodes-activate-links): Don't
>   call `org-remove-flyspell-overlays-in'.
> ---
>  contrib/lisp/org-wikinodes.el |   5 --
>  lisp/org.el                   | 109 ++++++++++++++++++++++++++----------------
>  2 files changed, 68 insertions(+), 46 deletions(-)
>
> diff --git a/contrib/lisp/org-wikinodes.el b/contrib/lisp/org-wikinodes.el
> index 4efc373..c6f2006 100644
> --- a/contrib/lisp/org-wikinodes.el
> +++ b/contrib/lisp/org-wikinodes.el
> @@ -79,15 +79,10 @@ to `directory'."
>        (if (re-search-forward org-wikinodes-camel-regexp limit t)
>  	  (if (equal (char-after (point-at-bol)) ?*)
>  	      (progn
> -		;; in  heading - deactivate flyspell
> -		(org-remove-flyspell-overlays-in (match-beginning 0)
> -						 (match-end 0))
>  		(add-text-properties (match-beginning 0) (match-end 0)
>  				     '(org-no-flyspell t))
>  		t)
>  	    ;; this is a wiki link
> -	    (org-remove-flyspell-overlays-in (match-beginning 0)
> -					     (match-end 0))
>  	    (add-text-properties (match-beginning 0) (match-end 0)
>  				 (list 'mouse-face 'highlight
>  				       'face 'org-link
> diff --git a/lisp/org.el b/lisp/org.el
> index 2382a9c..452babb 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -5459,8 +5459,6 @@ The following commands are available:
>    (abbrev-table-put org-mode-abbrev-table
>  		    :parents (list text-mode-abbrev-table)))
>  
> -(put 'org-mode 'flyspell-mode-predicate 'org-mode-flyspell-verify)
> -
>  (defsubst org-fix-ellipsis-at-bol ()
>    (save-excursion (goto-char (window-start)) (recenter 0)))
>  
> @@ -5685,9 +5683,6 @@ The time stamps may be either active or inactive.")
>  	    (font-lock-prepend-text-property (match-beginning 2) (match-end 2)
>  					     'face
>  					     (nth 1 a))
> -	    (and (nth 2 a)
> -		 (org-remove-flyspell-overlays-in
> -		  (match-beginning 0) (match-end 0)))
>  	    (add-text-properties (match-beginning 2) (match-end 2)
>  				 '(font-lock-multiline t org-emphasis t))
>  	    (when org-hide-emphasis-markers
> @@ -5753,7 +5748,6 @@ prompted for."
>    (let (f hl)
>      (when (and (re-search-forward (concat org-plain-link-re) limit t)
>  	       (not (org-in-src-block-p)))
> -      (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>        (setq f (get-text-property (match-beginning 0) 'face))
>        (setq hl (org-match-string-no-properties 0))
>        (if (or (eq f 'org-tag)
> @@ -5770,7 +5764,6 @@ prompted for."
>  (defun org-activate-code (limit)
>    (if (re-search-forward "^[ \t]*\\(:\\(?: .*\\|$\\)\n?\\)" limit t)
>        (progn
> -	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>  	(remove-text-properties (match-beginning 0) (match-end 0)
>  				'(display t invisible t intangible t))
>  	t)))
> @@ -5812,7 +5805,6 @@ by a #."
>  	  (cond
>  	   ((member dc1 '("+html:" "+ascii:" "+latex:"))
>  	    ;; a single line of backend-specific content
> -	    (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>  	    (remove-text-properties (match-beginning 0) (match-end 0)
>  				    '(display t invisible t intangible t))
>  	    (add-text-properties (match-beginning 1) (match-end 3)
> @@ -5901,7 +5893,6 @@ by a #."
>    (if (and (re-search-forward org-angle-link-re limit t)
>  	   (not (org-in-src-block-p)))
>        (progn
> -	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>  	(add-text-properties (match-beginning 0) (match-end 0)
>  			     (list 'mouse-face 'highlight
>  				   'keymap org-mouse-map))
> @@ -5913,7 +5904,6 @@ by a #."
>    (let ((fn (org-footnote-next-reference-or-definition limit)))
>      (when fn
>        (let ((beg (nth 1 fn)) (end (nth 2 fn)))
> -	(org-remove-flyspell-overlays-in beg end)
>  	(add-text-properties beg end
>  			     (list 'mouse-face 'highlight
>  				   'keymap org-mouse-map
> @@ -5941,7 +5931,6 @@ by a #."
>  		       'htmlize-link `(:uri ,hl))))
>  	;; We need to remove the invisible property here.  Table narrowing
>  	;; may have made some of this invisible.
> -	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>  	(remove-text-properties (match-beginning 0) (match-end 0)
>  				'(invisible nil))
>  	(if (match-end 3)
> @@ -5965,7 +5954,6 @@ by a #."
>    (if (and (re-search-forward org-tsr-regexp-both limit t)
>  	   (not (equal (char-before (match-beginning 0)) 91)))
>        (progn
> -	(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>  	(add-text-properties (match-beginning 0) (match-end 0)
>  			     (list 'mouse-face 'highlight
>  				   'keymap org-mouse-map))
> @@ -5992,7 +5980,6 @@ by a #."
>      (let ((case-fold-search t))
>        (if (re-search-forward org-target-link-regexp limit t)
>  	  (progn
> -	    (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
>  	    (add-text-properties (match-beginning 0) (match-end 0)
>  				 (list 'mouse-face 'highlight
>  				       'keymap org-mouse-map
> @@ -6110,7 +6097,6 @@ between words."
>  (defun org-activate-tags (limit)
>    (if (re-search-forward (org-re "^\\*+.*[ \t]\\(:[[:alnum:]_@#%:]+:\\)[ \r\n]") limit t)
>        (progn
> -	(org-remove-flyspell-overlays-in (match-beginning 1) (match-end 1))
>  	(add-text-properties (match-beginning 1) (match-end 1)
>  			     (list 'mouse-face 'highlight
>  				   'keymap org-mouse-map))
> @@ -23861,34 +23847,75 @@ To get rid of the restriction, use \\[org-agenda-remove-restriction-lock]."
>  
>  ;;; Fixes and Hacks for problems with other packages
>  
> -;; Make flyspell not check words in links, to not mess up our keymap
> -(defvar org-element-affiliated-keywords) ; From org-element.el
> -(defvar org-element-block-name-alist)	 ; From org-element.el
>  (defun org-mode-flyspell-verify ()
> -  "Don't let flyspell put overlays at active buttons, or on
> -   {todo,all-time,additional-option-like}-keywords."
> -  (require 'org-element) ; For `org-element-affiliated-keywords'
> -  (let ((pos (max (1- (point)) (point-min)))
> -	(word (thing-at-point 'word)))
> -    (and (not (get-text-property pos 'keymap))
> -	 (not (get-text-property pos 'org-no-flyspell))
> -	 (not (member word org-todo-keywords-1))
> -	 (not (member word org-all-time-keywords))
> -	 (not (member word org-options-keywords))
> -	 (not (member word (mapcar 'car org-startup-options)))
> -	 (not (member-ignore-case word org-element-affiliated-keywords))
> -	 (not (member-ignore-case word (org-get-export-keywords)))
> -	 (not (member-ignore-case
> -	       word (mapcar 'car org-element-block-name-alist)))
> -	 (not (member-ignore-case word '("BEGIN" "END" "ATTR")))
> -	 (not (org-in-src-block-p)))))
> -
> -(defun org-remove-flyspell-overlays-in (beg end)
> -  "Remove flyspell overlays in region."
> -  (and (org-bound-and-true-p flyspell-mode)
> -       (fboundp 'flyspell-delete-region-overlays)
> -       (flyspell-delete-region-overlays beg end))
> -  (add-text-properties beg end '(org-no-flyspell t)))
> +  "Function used for `flyspell-generic-check-word-predicate'."
> +  (if (org-at-heading-p)
> +      ;; At a headline or an inlinetask, check title only.  This is
> +      ;; faster than relying on `org-element-at-point'.
> +      (and (save-excursion (beginning-of-line)
> +			   (and (let ((case-fold-search t))
> +				  (not (looking-at "\\*+ END[ \t]*$")))
> +				(looking-at org-complex-heading-regexp)))
> +	   (match-beginning 4)
> +	   (>= (point) (match-beginning 4))
> +	   (or (not (match-beginning 5))
> +	       (< (point) (match-beginning 5))))
> +    (let* ((element (org-element-at-point))
> +	   (post-affiliated (org-element-property :post-affiliated element))
> +	   (object-check
> +	    (function
> +	     ;; Non-nil if checks can be done for object at point.
> +	     (lambda ()
> +	       (let ((object (save-excursion
> +			       (when (org-looking-at-p "\\>") (backward-char))
> +			       (org-element-context element))))
> +		 (case (org-element-type object)
> +		   ;; Prevent checks in links due to keybinding conflict
> +		   ;; with Flyspell.
> +		   ((code entity export-snippet inline-babel-call
> +			  inline-src-block line-break latex-fragment link macro
> +			  statistics-cookie target timestamp verbatim)
> +		    nil)
> +		   (footnote-reference
> +		    ;; Only in inline footnotes, within the definition.
> +		    (and (eq (org-element-property :type object) 'inline)
> +			 (< (save-excursion
> +			      (goto-char (org-element-property :begin object))
> +			      (search-forward ":" nil t 2))
> +			    (point))))
> +		   (otherwise t)))))))
> +      (if (and post-affiliated (< (point) post-affiliated))
> +	  (and (save-excursion
> +		 (beginning-of-line)
> +		 (let ((case-fold-search t)) (looking-at "[ \t]*#\\+CAPTION:")))
> +	       (> (point) (match-end 0))
> +	       (funcall object-check))
> +	(case (org-element-type element)
> +	  ((comment quote-section) t)
> +	  (comment-block
> +	   ;; Allow checks between block markers, not on them.
> +	   (and (> (line-beginning-position)
> +		   (org-element-property :post-affiliated element))
> +		(save-excursion
> +		  (end-of-line)
> +		  (skip-chars-forward " \r\t\n")
> +		  (< (point) (org-element-property :end element)))))
> +	  ;; Arbitrary list of keywords where checks are meaningful.
> +	  ;; Make sure point is on the value part of the element.
> +	  (keyword
> +	   (and (member (org-element-property :key element)
> +			'("DESCRIPTION" "TITLE"))
> +		(< (save-excursion
> +		     (beginning-of-line) (search-forward ":") (point))
> +		   (point))))
> +	  ;; Check is globally allowed in paragraphs verse blocks and
> +	  ;; table rows (after affiliated keywords) but some objects
> +	  ;; must not be affected.
> +	  ((paragraph table-row verse-block)
> +	   (and (>= (point) (org-element-property :contents-begin element))
> +		(< (point) (org-element-property :contents-end element))
> +		(funcall object-check))))))))
> +(put 'org-mode 'flyspell-mode-predicate 'org-mode-flyspell-verify)
>  
>  ;; Make `bookmark-jump' shows the jump location if it was hidden.
>  (eval-after-load "bookmark"

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: [ANN] Improved Flyspell check

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> I've been using this since it was sent, and I haven't noticed any bad
> behavior.

Thanks for the feedback.

BTW, I tried to add `org-self-insert-command' to
`flyspell-delayed-commands', since `self-insert-command' belongs to
`flyspell-default-delayed-commands', but I didn't notice any
improvement. Does this change anything for you?


Regards,

-- 
Nicolas Goaziou

Re: [ANN] Improved Flyspell check

[Sebastien Vauban]

Hello Nicolas,

Nicolas Goaziou wrote:
> Eric Schulte <schulte.eric-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
>> I've been using this since it was sent, and I haven't noticed any bad
>> behavior.
>
> Thanks for the feedback.
>
> BTW, I tried to add `org-self-insert-command' to
> `flyspell-delayed-commands', since `self-insert-command' belongs to
> `flyspell-default-delayed-commands', but I didn't notice any
> improvement. Does this change anything for you?

Do I have to write something special for having the Org-enhanced Flyspelling?

I do have some configuration which I can reduce to this:

--8<---------------cut here---------------start------------->8---
;; Org-mode
(add-to-list 'load-path "~/Public/Repositories/org-mode/testing")
(add-to-list 'load-path "~/Public/Repositories/org-mode/contrib/lisp")
(add-to-list 'load-path "~/Public/Repositories/org-mode/lisp")

(setq ispell-dictionary "francais")

(setq ispell-program-name "aspell")

;; enable on-the-fly spell checking
(add-hook 'text-mode-hook
          (lambda ()
            (flyspell-mode 1)))

;; prevent flyspell from finding mistakes in the code
(add-hook 'prog-mode-hook
          (lambda ()
            ;; `ispell-comments-and-strings'
            (flyspell-prog-mode)))
--8<---------------cut here---------------end--------------->8---

Though, when opening the following Org file ...

--8<---------------cut here---------------start------------->8---
#+TITLE:     ECM pour Flyspell
#+Time-stamp: <2013-11-08>
#+DESCRIPTION:
#+KEYWORDS:
#+LANGUAGE:  en

* Développement
  :PROPERTIES:
  :CATEGORY: Dev
  :END:

** DONE Engager firme de relations publiques 			       :mail:
   DEADLINE: <2013-02-14 Thu>
   :LOGBOOK:
   CLOCK: [2013-02-05 Tue 09:00]--[2013-02-05 Tue 10:11] =>  1:11
   - State "TODO"        ->  "STRT"       [2013-02-05 Tue 17:48]
   - State "TODO"        ->  "STRT"       [2013-08-19 Mon 15:16]
   :END:

** TODO Appeler Laura pour bug dans le code shell 		      :phone:

Le bout de code suivant a un problème.

#+begin_src sh
wher command
#+end_src
--8<---------------cut here---------------end--------------->8---

... many Org keywords are still underlined (such as TITLE, KEYWORDS,
PROPERTIES, LOGBOOK, and code inside #+begin/end_src).

Demo at http://screencast.com/t/zRDTt99M3ey.

Best regards,
  Seb

PS- Environment:
- GNU Emacs 24.3.50.1 (i686-pc-mingw32) of 2013-10-19 on LEG570
- Org-mode version 8.2.1 (release_8.2.1-202-gea797e.dirty @
  ~/Public/Repositories/org-mode/lisp/)

-- 
Sebastien Vauban

Re: [ANN] Improved Flyspell check

[Nicolas Goaziou]

Hello,

"Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
writes:

> Nicolas Goaziou wrote:
>> Eric Schulte <schulte.eric-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>>> I've been using this since it was sent, and I haven't noticed any bad
>>> behavior.
>>
>> Thanks for the feedback.
>>
>> BTW, I tried to add `org-self-insert-command' to
>> `flyspell-delayed-commands', since `self-insert-command' belongs to
>> `flyspell-default-delayed-commands', but I didn't notice any
>> improvement. Does this change anything for you?
>
> Do I have to write something special for having the Org-enhanced
> Flyspelling?

You don't need anything special once the patch is applied (on master
branch). You may need to reload Org or restart Flyspell, though.


Regards,

-- 
Nicolas Goaziou

Re: [ANN] Improved Flyspell check

[Sebastien Vauban]

Hello,

Nicolas Goaziou wrote:
> "Sebastien Vauban" writes:
>> Nicolas Goaziou wrote:
>>> Eric Schulte <schulte.eric-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>>
>>>> I've been using this since it was sent, and I haven't noticed any bad
>>>> behavior.
>>>
>>> Thanks for the feedback.
>>>
>>> BTW, I tried to add `org-self-insert-command' to
>>> `flyspell-delayed-commands', since `self-insert-command' belongs to
>>> `flyspell-default-delayed-commands', but I didn't notice any
>>> improvement. Does this change anything for you?
>>
>> Do I have to write something special for having the Org-enhanced
>> Flyspelling?
>
> You don't need anything special once the patch is applied (on master
> branch). You may need to reload Org or restart Flyspell, though.

You mean it isn't applied yet?  Because I already reloaded Emacs a ton of times
today -- for different tests.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [ANN] Improved Flyspell check

[Nicolas Goaziou]

"Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
writes:

> You mean it isn't applied yet?  Because I already reloaded Emacs a ton of times
> today -- for different tests.

No it isn't applied yet. Meanwhile, you can test it by applying the
second patch in this thread.


Regards,

-- 
Nicolas Goaziou

Re: [ANN] Improved Flyspell check

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> I've been using this since it was sent, and I haven't noticed any bad
>> behavior.
>
> Thanks for the feedback.
>
> BTW, I tried to add `org-self-insert-command' to
> `flyspell-delayed-commands', since `self-insert-command' belongs to
> `flyspell-default-delayed-commands', but I didn't notice any
> improvement. Does this change anything for you?
>

I've tried this briefly but I don't see any effect.  It seems like a
safe default to add...

Best,

>
>
> Regards,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: [ANN] Improved Flyspell check

[Sebastien Vauban]

Nicolas Goaziou wrote:
> "Sebastien Vauban" writes:
>
>> You mean it isn't applied yet? Because I already reloaded Emacs a ton of
>> times today -- for different tests.
>
> No it isn't applied yet. Meanwhile, you can test it by applying the second
> patch in this thread.

Applied. Looks good so far. Thanks!

The only thing it does not ignore is the "state change" lines, such as:

   :LOGBOOK:
   CLOCK: [2013-02-05 Tue 09:00]--[2013-02-05 Tue 10:11] =>  1:11
   - State "TODO"        ->  "STRT"       [2013-02-05 Tue 17:48]   <<<<
   - State "TODO"        ->  "STRT"       [2013-08-19 Mon 15:16]   <<<<
   :END:

which I've customized.

Shouldn't they be purely ignored?

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [ANN] Improved Flyspell check

[Rasmus]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
> Updated patch.

I've been using this patch without issues for at least two week.  I
haven't experienced any slowdowns over wcheck other than that
associated with the design of Flyspell (e.g. flyspell-buffer moves the
cursor over all words).

Quick question: does ISpell/Flyspell feature options to:

  1. Move the ispell suggestion window to the bottom of the window
     (rather than the top)

  2. Select the first suggestion with 1 rather than 0?  The latter is
     harder to press on my keyboard.

Thanks,
Rasmus

-- 
Got mashed potatoes. Ain't got no T-Bone. No T-Bone

Re: [ANN] Improved Flyspell check

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> Hello,
>>
>> Eric Schulte <schulte.eric@gmail.com> writes:
>>
>>> I've been using this since it was sent, and I haven't noticed any bad
>>> behavior.
>>
>> Thanks for the feedback.
>>
>> BTW, I tried to add `org-self-insert-command' to
>> `flyspell-delayed-commands', since `self-insert-command' belongs to
>> `flyspell-default-delayed-commands', but I didn't notice any
>> improvement. Does this change anything for you?
>>
>
> I've tried this briefly but I don't see any effect.  It seems like a
> safe default to add...

Done.

I applied a variation of the original patch (so long).


Regards,

-- 
Nicolas Goaziou