bug#68418: refill-mode interferes with org-mode headlines

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> > I did some debugging for refill-mode in Org.
>> > `refill-post-command-function' calls `refill-fill-paragraph-at', which
>> > not only ignores `fill-paragraph-function', but even
>> > `fill-forward-paragraph-function'. `forward-paragraph' is hard-coded,
>> > I consider to be a bug.
>> 
>> Let me bump this bug report.
>> AFAIU, it belongs to emacs.
>
> I now have second thoughts about this.  fill-paragraph-function is
> used by fill-paragraph, whereas refill.el calls
> fill-region-as-paragraph.  fill-region-as-paragraph cannot possibly
> call fill-paragraph-function, because the latter is not supposed to be
> able to fill more than a single paragraph.  The doc string of
> fill-paragraph-function actually says so:
>
>   (defvar fill-paragraph-function nil
>     "Mode-specific function to fill a paragraph, or nil if there is none.
>   If the function returns nil, then `fill-paragraph' does its normal work.
>   A value of t means explicitly \"do nothing special\".
>   Note: This only affects `fill-paragraph' and not `fill-region'
>   nor `auto-fill-mode', so it is often better to use some other hook,
>   such as `fill-forward-paragraph-function'.")
>
> Am I missing something?

No. This indeed makes sense.
`fill-paragraph-function' is not even passed the region to fill, so it
has no way to know what it is supposed to fill and should determine the
paragraph boundaries by itself.

> If my analysis is correct, then to move forward with this bug, we
> could either:
>
>   . add a fill-region-function, make fill-region-as-paragraph call it,
>     and then make Org set this function to something appropriate

This is a bit tangent, but the current design of filling feels chaotic -
we have configurable fill-paragraph-function and
fill-forward-paragraph-function, while `fill-region-as-paragraph' is
hard-coded.  And forward-paragraph relies upon regexps...

It looks like the notion of paragraph when filling is designed to be
different compared to the notion of paragraph in `forward-paragraph'.
Is there any reason for such difference?

>   . modify refill.el to use fill-forward-paragraph-function instead of
>     a literal forward-paragraph
>
> I tried the latter (patch below), but I don't know if that gives
> good-enough results, as I don't use Org intensively enough, and
> certainly don't use refill-mode.  Can you try the patch below and
> report back?
>
> diff --git a/lisp/textmodes/refill.el b/lisp/textmodes/refill.el
> index 937a8ed..22f67bc 100644
> --- a/lisp/textmodes/refill.el
> +++ b/lisp/textmodes/refill.el
> @@ -106,10 +106,10 @@ refill-fill-paragraph-at
>      ;; FIXME: forward-paragraph seems to disregard `use-hard-newlines',
>      ;; leading to excessive refilling and wrong choice of fill-prefix.
>      ;; might be a bug in my paragraphs.el.
> -    (forward-paragraph)
> +    (fill-forward-paragraph 1)
>      (skip-syntax-backward "-")
>      (let ((end (point))
> -	  (beg (progn (backward-paragraph) (point)))
> +	  (beg (progn (fill-forward-paragraph -1) (point)))

This looks like an improvement. Also, it appears to fix the original
reproducer.

However, there will still be a problem, even with the proposed patch, in
the following scenario:

1. emacs -Q
2. Open a new org file
* Heading <point> :tag:
3. M-x refill-mode
4. Keep typing
* Heading alksjd alkjasd asldkj asdlkj aslldkj aslkdj asldkj
  asldkj <point>:tag:

See how the heading is split into multiple lines.

This happens because refill-mode uses `fill-region-as-paragraph', which
is impossible to configure by major modes.
As the above example demonstrates, not everything is safe to fill
without breaking the markup.

A similar problem appears in `fill-region' - it completely disregards
`fill-paragraph-function' and always calls `fill-region-as-paragraph'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>