bug#68418: refill-mode interferes with org-mode headlines

[Eli Zaretskii <eliz@gnu.org>]

merge 68418 52778
thanks

> From: Lars Rustand <rustand.lars@gmail.com>
> Date: Fri, 12 Jan 2024 21:18:51 +0100
> 
> 
> Enabling refill-mode in an org-mode buffer results in unwanted behaviour
> that interferes with headlines, list items, and possibly other types of
> line breaks.
> 
> I won't go too deeply into describing it here as it has been described
> well in the following bug report on the org-mode mailing list. In that
> thread it was suggested that the bug is actually not in org-mode, but
> instead in refill-mode.
> 
> See the original bug report on the org-mode list here:
> 
> https://list.orgmode.org/orgmode/3715730.BJ41AJB0qc@yksi/
> 
> 
> I also found a previous thread on this list where it has been discussed:
> 
> https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-12/msg01854.html
> 
> It seems that the bug was located and the cause was identified, but
> nothing more came out of it.

If a bug isn't picked up by someone who wants or is capable of working
on it, and has time for that, that bug will not be fixed, indeed.
That's one of the downsides of a community-driven project that is
based on volunteers.

> The thread derailed quickly and it seems the problem was ignored because
> of personal problems against the author. The fact that some of his
> previous bug reports were done to the wrong list does not mean that this
> bug should be ignored.
> 
> I hope the issue can be taken seriously this time, because it renders
> refill-mode unusable in org-mode buffers.

This is at least inaccurate, and quite a bit unfair, I'd say: the
thread didn't "derail".  Once the nature of the original report was
established and seemed to point to a problem in refill.el, the
discussion indeed went to a tangent, but no one said that the original
report was invalid or incorrect, or that we don't want to take it
seriously, let alone because the OP had some problems.  It's just a
bug that no one kept working on it.  Which is unfortunate, but it does
happen, since we don't have a way of assigning bugs to developers and
making sure they do fix them.

Anyway, with these tangents aside, coming back to the original report
and its initial analysis by Ihor in bug#52778, viz.:

> > I did some debugging for refill-mode in Org.
> > `refill-post-command-function' calls `refill-fill-paragraph-at', which
> > not only ignores `fill-paragraph-function', but even
> > `fill-forward-paragraph-function'. `forward-paragraph' is hard-coded,
> > I consider to be a bug.
> 
> Let me bump this bug report.
> AFAIU, it belongs to emacs.

I now have second thoughts about this.  fill-paragraph-function is
used by fill-paragraph, whereas refill.el calls
fill-region-as-paragraph.  fill-region-as-paragraph cannot possibly
call fill-paragraph-function, because the latter is not supposed to be
able to fill more than a single paragraph.  The doc string of
fill-paragraph-function actually says so:

  (defvar fill-paragraph-function nil
    "Mode-specific function to fill a paragraph, or nil if there is none.
  If the function returns nil, then `fill-paragraph' does its normal work.
  A value of t means explicitly \"do nothing special\".
  Note: This only affects `fill-paragraph' and not `fill-region'
  nor `auto-fill-mode', so it is often better to use some other hook,
  such as `fill-forward-paragraph-function'.")

Am I missing something?

If my analysis is correct, then to move forward with this bug, we
could either:

  . add a fill-region-function, make fill-region-as-paragraph call it,
    and then make Org set this function to something appropriate
  . modify refill.el to use fill-forward-paragraph-function instead of
    a literal forward-paragraph

I tried the latter (patch below), but I don't know if that gives
good-enough results, as I don't use Org intensively enough, and
certainly don't use refill-mode.  Can you try the patch below and
report back?

diff --git a/lisp/textmodes/refill.el b/lisp/textmodes/refill.el
index 937a8ed..22f67bc 100644
--- a/lisp/textmodes/refill.el
+++ b/lisp/textmodes/refill.el
@@ -106,10 +106,10 @@ refill-fill-paragraph-at
     ;; FIXME: forward-paragraph seems to disregard `use-hard-newlines',
     ;; leading to excessive refilling and wrong choice of fill-prefix.
     ;; might be a bug in my paragraphs.el.
-    (forward-paragraph)
+    (fill-forward-paragraph 1)
     (skip-syntax-backward "-")
     (let ((end (point))
-	  (beg (progn (backward-paragraph) (point)))
+	  (beg (progn (fill-forward-paragraph -1) (point)))
 	  (obeg (overlay-start refill-ignorable-overlay))
 	  (oend (overlay-end refill-ignorable-overlay)))
       (unless (> beg pos)      ;Don't fill if point is outside the paragraph.