[BUG] `org-fill-paragraph' doesn't use `fill-prefix'

[BUG] `org-fill-paragraph' doesn't use `fill-prefix'

[Daniel Hackney]

I proposed a fix [1] for this back in 2010, but it seems to have regressed
again. `org-fill-paragraph' no longer makes use of a `fill-prefix', so
filling things like email comments no longer works.

To test, launch =emacs -Q= and create a buffer with

#+BEGIN_EXAMPLE
> The
> cat
> in
> the
> hat
#+END_EXAMPLE

And press =M-q=. In `fundamental-mode', this results in

#+BEGIN_EXAMPLE
> The cat in the hat
#+END_EXAMPLE

In recent versions of org-mode, this results in

#+BEGIN_EXAMPLE
> The > cat > in > the > hat
#+END_EXAMPLE

I have tested the following versions of org-mode:

  | Mode                             | Working |
  |----------------------------------+---------|
  | fundamental-mode                 | Yes     |
  | org 7.9.3f (Emacs 24.3.1)        | No      |
  | org 8.0 (20130727.224 from MELPA | No      |
  | 7.8.11 (Emacs 24.2.1)            | Yes     |
  |----------------------------------+---------|

P.S. I'm not subscribed to the list, so please CC me in replies.

[1] http://permalink.gmane.org/gmane.emacs.orgmode/22705

--
Daniel Hackney

Re: [BUG] `org-fill-paragraph' doesn't use `fill-prefix'

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

Daniel Hackney <dan@haxney.org> writes:

> I proposed a fix [1] for this back in 2010, but it seems to have regressed
> again. `org-fill-paragraph' no longer makes use of a `fill-prefix', so
> filling things like email comments no longer works.

It has been discussed on this ML already. Org mode is not Message mode
and ">" prefix means nothing to it.

Also, it has its own set of special prefixes, which are not found in
Fundamental mode. For example, you can never have " : " as a fill prefix
since it creates a fixed-width area.

Therefore, I don't consider it to be a regression since it's not an
expected feature in the first place. But I admit it is still convenient.

Maybe we can introduce some support for `adaptive-fill-regexp' in
paragraphs and comments filling. Would you mind testing the following
patch?


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-support-for-adaptive-fill-regexp-in-paragraphs-a.patch --]
[-- Type: text/x-diff, Size: 5434 bytes --]

From d460b8048d7b3b308cd93794b1de46837438a8e6 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Sat, 27 Jul 2013 22:02:45 +0200
Subject: [PATCH] Add support for `adaptive-fill-regexp' in paragraphs and
 comments

* lisp/org.el (org-adaptive-fill-function, org-fill-paragraph): Add
  support for `adaptive-fill-regexp' in paragraphs and comments.
* testing/lisp/test-org.el: Add test.
---
 lisp/org.el              | 43 ++++++++++++++++++++++++++++++-------------
 testing/lisp/test-org.el | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+), 13 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 2f619cc..c852550 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -22179,7 +22179,15 @@ meant to be filled."
 	      (post-affiliated (org-element-property :post-affiliated element)))
 	 (unless (and post-affiliated (< p post-affiliated))
 	   (case type
-	     (comment (looking-at "[ \t]*# ?") (match-string 0))
+	     (comment
+	      (save-excursion
+		(beginning-of-line)
+		(looking-at "[ \t]*# ?")
+		(goto-char (match-end 0))
+		(let ((comment-prefix (match-string 0)))
+		  (if (looking-at adaptive-fill-regexp)
+		      (concat comment-prefix (match-string 0))
+		    comment-prefix))))
 	     (footnote-definition "")
 	     ((item plain-list)
 	      (make-string (org-list-item-body-column
@@ -22188,15 +22196,19 @@ meant to be filled."
 			   ? ))
 	     (paragraph
 	      ;; Fill prefix is usually the same as the current line,
-	      ;; except if the paragraph is at the beginning of an item.
+	      ;; except if the paragraph is at the beginning of an
+	      ;; item.  For convenience, if `adaptive-fill-regexp'
+	      ;; matches, use it.
 	      (let ((parent (org-element-property :parent element)))
-		(cond ((eq (org-element-type parent) 'item)
-		       (make-string (org-list-item-body-column
-				     (org-element-property :begin parent))
-				    ? ))
-		      ((save-excursion (beginning-of-line) (looking-at "[ \t]+"))
-		       (match-string 0))
-		      (t  ""))))
+		(save-excursion
+		  (beginning-of-line)
+		  (cond ((eq (org-element-type parent) 'item)
+			 (make-string (org-list-item-body-column
+				       (org-element-property :begin parent))
+				      ? ))
+			((looking-at adaptive-fill-regexp) (match-string 0))
+			((looking-at "[ \t]+") (match-string 0))
+			(t  "")))))
 	     (comment-block
 	      ;; Only fill contents if P is within block boundaries.
 	      (let* ((cbeg (save-excursion (goto-char post-affiliated)
@@ -22341,10 +22353,15 @@ a footnote definition, try to fill the first paragraph within."
 			      (line-end-position)))))
 		 ;; Do not fill comments when at a blank line or at
 		 ;; affiliated keywords.
-		 (let ((fill-prefix (save-excursion
-				      (beginning-of-line)
-				      (looking-at "[ \t]*#")
-				      (concat (match-string 0) " "))))
+		 (let ((fill-prefix
+			(save-excursion
+			  (beginning-of-line)
+			  (looking-at "[ \t]*#")
+			  (let ((comment-prefix (match-string 0)))
+			    (goto-char (match-end 0))
+			    (if (looking-at adaptive-fill-regexp)
+				(concat comment-prefix (match-string 0))
+			      (concat comment-prefix " "))))))
 		   (when (> end begin)
 		     (save-excursion
 		       (fill-region-as-paragraph begin end justify))))))
diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index 504defa..517d0d1 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -176,6 +176,14 @@
 	      (narrow-to-region 1 8)
 	      (org-fill-paragraph)
 	      (buffer-string)))))
+  ;; Handle `adaptive-fill-regexp' in paragraphs.
+  (should
+   (equal "> a b"
+	  (org-test-with-temp-text "> a\n> b"
+	    (let ((fill-column 5)
+		  (adaptive-fill-regexp "[ \t]*>+[ \t]*"))
+	      (org-fill-paragraph)
+	      (buffer-string)))))
   ;; Special case: Fill first paragraph when point is at an item or
   ;; a plain-list or a footnote reference.
   (should
@@ -225,6 +233,14 @@
 	    (let ((fill-column 20))
 	      (org-fill-paragraph)
 	      (buffer-string)))))
+  ;; Handle `adaptive-fill-regexp' in comments.
+  (should
+   (equal "# > a b"
+	  (org-test-with-temp-text "# > a\n# > b"
+	    (let ((fill-column 20)
+		  (adaptive-fill-regexp "[ \t]*>+[ \t]*"))
+	      (org-fill-paragraph)
+	      (buffer-string)))))
   ;; Do nothing at affiliated keywords.
   (org-test-with-temp-text "#+NAME: para\nSome\ntext."
     (let ((fill-column 20))
@@ -255,6 +271,15 @@
 	      (end-of-line)
 	      (org-auto-fill-function)
 	      (buffer-string)))))
+  ;; Auto fill paragraph when `adaptive-fill-regexp' matches.
+  (should
+   (equal "> 12345\n> 7890"
+	  (org-test-with-temp-text "> 12345 7890"
+	    (let ((fill-column 5)
+		  (adaptive-fill-regexp "[ \t]*>+[ \t]*"))
+	      (end-of-line)
+	      (org-auto-fill-function)
+	      (buffer-string)))))
   ;; Auto fill comments.
   (should
    (equal "  # 12345\n  # 7890"
@@ -263,6 +288,15 @@
 	      (end-of-line)
 	      (org-auto-fill-function)
 	      (buffer-string)))))
+  ;; Auto fill comments when `adaptive-fill-regexp' matches.
+  (should
+   (equal "  # > 12345\n  # > 7890"
+	  (org-test-with-temp-text "  # > 12345 7890"
+	    (let ((fill-column 10)
+		  (adaptive-fill-regexp "[ \t]*>+[ \t]*"))
+	      (end-of-line)
+	      (org-auto-fill-function)
+	      (buffer-string)))))
   ;; A hash within a line isn't a comment.
   (should-not
    (equal "12345 # 7890\n# 1"
-- 
1.8.3.4

Re: [BUG] `org-fill-paragraph' doesn't use `fill-prefix'

[Daniel Hackney]

Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> Daniel Hackney <dan@haxney.org> writes:
>
>> I proposed a fix [1] for this back in 2010, but it seems to have regressed
>> again. `org-fill-paragraph' no longer makes use of a `fill-prefix', so
>> filling things like email comments no longer works.
>
> It has been discussed on this ML already. Org mode is not Message mode
> and ">" prefix means nothing to it.

It isn't just ">" that behaves this way in `fundamental-mode' and older
versions of org-mode; there are a number of such characters. From a
quick test, in Emacs 24.2,

#+BEGIN_EXAMPLE
-- foo
-- bar
#+END_EXAMPLE

becomes

#+BEGIN_EXAMPLE
-- foo bar
#+END_EXAMPLE

whereas in the current org-mode, it is

#+BEGIN_EXAMPLE
-- foo -- bar
#+END_EXAMPLE

Same with

#+BEGIN_EXAMPLE
! foo
! bar
#+END_EXAMPLE

and

#+BEGIN_EXAMPLE
% foo
% bar
#+END_EXAMPLE

The following also fill to a single line:

#+BEGIN_EXAMPLE
% % foo
% % bar

! ! foo
! ! bar
#+END_EXAMPLE

From a brief look at `fill-paragraph', the relevant code is in
`fill-comment-paragraph'. I might take a closer look at it to see if I
can come up with my own fix.

> Also, it has its own set of special prefixes, which are not found in
> Fundamental mode. For example, you can never have " : " as a fill prefix
> since it creates a fixed-width area.

Of course. Fundamental mode will fill

#+BEGIN_EXAMPLE
- foo
- bar
#+END_EXAMPLE

into a single line, but org-mode never did. I'm not asking for new
behavior, simply the `fill-paragraph' style of org 7.8.11 back.

> Therefore, I don't consider it to be a regression since it's not an
> expected feature in the first place. But I admit it is still
> convenient.

Well, my patch from 2010 /was/ applied [1], so I would consider it an
expected feature :)

> Maybe we can introduce some support for `adaptive-fill-regexp' in
> paragraphs and comments filling. Would you mind testing the following
> patch?

I'll test it later today.

[1] http://article.gmane.org/gmane.emacs.orgmode/22954

--
Daniel Hackney

Re: [BUG] `org-fill-paragraph' doesn't use `fill-prefix'

[Daniel Hackney]

Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> Daniel Hackney <dan@haxney.org> writes:
>
>> I proposed a fix [1] for this back in 2010, but it seems to have regressed
>> again. `org-fill-paragraph' no longer makes use of a `fill-prefix', so
>> filling things like email comments no longer works.
>
> Maybe we can introduce some support for `adaptive-fill-regexp' in
> paragraphs and comments filling. Would you mind testing the following
> patch?

The patch works great for me. I'd love to see it in a released-to-ELPA
version of org... ;)

--
Daniel Hackney

Re: [BUG] `org-fill-paragraph' doesn't use `fill-prefix'

[Nicolas Goaziou]

Daniel Hackney <dan@haxney.org> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> wrote:
>> Daniel Hackney <dan@haxney.org> writes:
>>
>>> I proposed a fix [1] for this back in 2010, but it seems to have regressed
>>> again. `org-fill-paragraph' no longer makes use of a `fill-prefix', so
>>> filling things like email comments no longer works.
>>
>> Maybe we can introduce some support for `adaptive-fill-regexp' in
>> paragraphs and comments filling. Would you mind testing the following
>> patch?
>
> The patch works great for me. I'd love to see it in a released-to-ELPA
> version of org... ;)

I guess it can't hurt. Applied.

Thanks for bringing that (again) on the table.


Regards,

-- 
Nicolas Goaziou