bug#71017: [PATCH] Flow single-paragraph messages

[Sandra Snan via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks.  (I also see RFC 3676 mentioned -- what is this about?)

I mentioned RFC 3676 twice for different reasons.

First, it's introduced the delsp parameter which wasn't in 2646 so 
code related to that parameter shouldn't talk about 2646 but rather 
3676.

Second, I saw a comment referring to not tampering with the sig 
line as a hack. I happened to have the section of RFC 3676 open 
that mandates that behavior so I changed the comment, however, this 
might be in RFC 2646 also, I don't know that, so in that case that 
comment might better change to refer to 2646 instead.

>> First, the old code didn't refill or encode the last paragraph 
>> at all unless there was at least one hard newline EOF.
>
> Isn't this the documented behavior?

Re multi-paragraph messages:

No, it's not the documented behavior, it was an unrelated bug in 
fill-flowed-encode. It would refill all the other paragraphs, 
separated by hard newlines, just not the last one. That was a bug 
and broke documented behavior. I fixed that bug + another unrelated 
reflow bug. That was in fill-flowed-encode and that bugfix doesn't 
rely on the change in mml. With my fix in fill-flowed-encode, 
multi-paragraph-messages started working fine.

Re single-paragraph messages:

However, according to the old documented behavior, a message that 
contained no hard newlines should not be refilled. This documented 
behavior in mml-generate-mime-1 meant that single-paragraph 
messages would not be filled even with the fill-flowed-encode bug 
fixed. That is an unintended bad consequence of the documented 
behavior, a "bug in the design".

I did change that but I updated the documentation to match.

> The change seems to be an incompatible behavior change, so I 
> wonder whether we'd need some way for users to get back old 
> behavior.

There is still the (neglected) defcustom mml-enable-flowed which 
now becomes more relevant since it's a way to turn off all this 
meddling and reflowing in the first place.

The old documented behavior was bugged-by-design. It's not right 
that single paragraph messages are hardwrapped and not reflowed. In 
my day-to-day I write many messages in Emacs that I later see in 
threads in another MUA (Delta Chat) and these messages stand out in 
a way that something is wrong with them.

However, one intent behind the old behavior was, in spirit, good: 
It'd be good to detect whether or not users with mml-enable-flowed 
on have remembered to also turn on use-hard-newlines, which is 
important for users with that on to do, especially since 
mml-enable-flowed defaults to t. The old attempt at doing that was 
flawed since it only worked reliably for multi-paragraph messages.

Unfortunately there's currently no way to detect in a 
single-paragraph message whether or not use-hard-newlines have been 
turned on, since the variable it sets is buffer local. (One 
extremely klugy workaround would be to change the 
message-send-and-exit command to check whether use-hard-newlines is 
on and if it is, add an extra hard newline EOT just for detecting 
this. Not super into that solution so hopefully there are other 
ways.)

With this patch, the defcustom mml-enable-flowed becomes _the_ 
setting for this, which does match a lot of documentation on the 
books. Perhaps it shall no longer default to t though since it 
completely borks messages up if it's t but use-hard-newlines are 
not on!

So here we are:

The new behavior has a problem: messages will get reflowed if 
mml-enable-flowed is t (the default!) even when use-hard-newlines 
is off, meaning that even separate "\n\n" paragraphs will get 
flowed together which is not what people want. Use-hard-newlines 
should be mandatory whenever mml-enable-flowed is on.

The old behavior is not OK since single-paragraph messages will get 
messed up, hardwrapped even when those newlines were advertised as 
"soft", or not softwrapped even when the paragraph consists of just 
one single super long unbroken line.

I also have sent a patch to the messages-are-flowing project 
highlighting the importance of this variable:

https://github.com/legoscia/messages-are-flowing/pull/15/commits/ae432723c2565ceced5d01d9aa2d314bd42aaa3c

So how about this idea:

If mml-enable-flowed is on but fill-flowed-encode is asked to flow 
a message that doesn't have any hard newlines, assume Markdown 
semantics, i.e. special treatment for "\n\n+", "  $", and "^    ". 
I'll see if I have time to implement that this morning. I think I'd 
place that change in fill-flowed-encode.

That wouldn't affect people with mml-enable-flowed off, or people 
with both mml-enable-flowed and use-hard-newlines on, it'd just be 
a sort of DWIM fallback based on the guess that markdown semantics 
are somewhat widely known or expected in 2024, to prevent separate 
paragraphs to be flowed together for people with "incorrect" 
settings.

Again, if there were a cross-buffer way to reliably detect whether 
use-hard-newlines is on, that dwimmy fallback wouldn't be needed.

I thought about whether it'd have been better if it was instead 
soft newlines that were marked with a text property, not hard ones, 
but in the end that wouldn't properly softwrap messages with just 
one single overly long line.