org-(un)fill-buffer

org-(un)fill-buffer

[Psionic K]

I wrote up a small addition to the unfill package, which is very
convenient for switching hard newlines out in favor of tools like
visual-line-mode and adaptive-wrap.

The command unfilled every list and paragraph in the entire buffer.  PR is here:
https://github.com/purcell/unfill/pull/11#pullrequestreview-1812645481

Steve wants to consider it for the org package itself.  Questions from me:
1. will this be accepted?
2. where would it go?
3. because org is more general than unfill, I would instead name the
command org-fill-buffer and we can recommend that users run in a hook:
(setq-local fill-column most-positive-fixnum).  After that, every call
to `org-fill-buffer` will just do what they mean, just like how
`fill-paragraph` will respect fill-column.

I'm going to use the combination of `org-unindent-buffer` and
`org-fill-buffer` in my own personal org shortcuts, but such a command
is probably too much based on my personal taste.

I think I will be recommending this combination to users in an upcoming video:
1.  visual-fill-column
2.  visual-line-mode
3.  adaptive-wrap
4.  configuring an org mode hook for "unfill" behavior
5.  combining unindent with (un)fill in order to clean up old cruddy
hard-indented, hard-newline documents

Re: org-(un)fill-buffer

[Ihor Radchenko]

Psionic K <psionik@positron.solutions> writes:

> I wrote up a small addition to the unfill package, which is very
> convenient for switching hard newlines out in favor of tools like
> visual-line-mode and adaptive-wrap.
>
> The command unfilled every list and paragraph in the entire buffer.  PR is here:
> https://github.com/purcell/unfill/pull/11#pullrequestreview-1812645481

This PR introduces a function to "unfill" a region in Org mode buffer.

> Steve wants to consider it for the org package itself.  Questions from me:
> 1. will this be accepted?
> 2. where would it go?
> 3. because org is more general than unfill, I would instead name the
> command org-fill-buffer and we can recommend that users run in a hook:
> (setq-local fill-column most-positive-fixnum).  After that, every call
> to `org-fill-buffer` will just do what they mean, just like how
> `fill-paragraph` will respect fill-column.

You may instead just run
(let ((fill-column most-positive-fixnum)) (fill-region (point-min) (point-max)))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: org-(un)fill-buffer

[Psionic K]

> You may instead just run

No.  That will have to be run manually on every element and every line
of every list.  I suppose let's just not talk about it further and
I'll submit a patch so there's no confusion.

On Wed, Jan 10, 2024 at 9:31 PM Ihor Radchenko <yantar92@posteo.net> wrote:
>
> Psionic K <psionik@positron.solutions> writes:
>
> > I wrote up a small addition to the unfill package, which is very
> > convenient for switching hard newlines out in favor of tools like
> > visual-line-mode and adaptive-wrap.
> >
> > The command unfilled every list and paragraph in the entire buffer.  PR is here:
> > https://github.com/purcell/unfill/pull/11#pullrequestreview-1812645481
>
> This PR introduces a function to "unfill" a region in Org mode buffer.
>
> > Steve wants to consider it for the org package itself.  Questions from me:
> > 1. will this be accepted?
> > 2. where would it go?
> > 3. because org is more general than unfill, I would instead name the
> > command org-fill-buffer and we can recommend that users run in a hook:
> > (setq-local fill-column most-positive-fixnum).  After that, every call
> > to `org-fill-buffer` will just do what they mean, just like how
> > `fill-paragraph` will respect fill-column.
>
> You may instead just run
> (let ((fill-column most-positive-fixnum)) (fill-region (point-min) (point-max)))
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>



-- 

남백호
대표 겸 공동 창업자
포지트론

Re: org-(un)fill-buffer

[Psionic K]

[-- Attachment #1.1: Type: text/plain, Size: 256 bytes --]

This is the org-fill-buffer command, done generically for people who want
to fill or unfill the entire buffer, as is required when alternating
between hard newline filling and visual line mode filling.

See attached patch for docstring and commit message.

[-- Attachment #1.2: Type: text/html, Size: 503 bytes --]

[-- Attachment #2: 0001-org.el-introducing-org-fill-buffer.patch --]
[-- Type: text/x-patch, Size: 2452 bytes --]

From 706d5d71cdf1ed2528664bdaf714aad6bd15af6c Mon Sep 17 00:00:00 2001
From: Psionik K <73710933+psionic-k@users.noreply.github.com>
Date: Wed, 10 Jan 2024 23:15:26 +0900
Subject: [PATCH] org.el: introducing org-fill-buffer

* lisp/org.el: (org-fill-buffer) this command walks the tree and will
call fill-paragraph on every paragraph or plain-list element, enabling
the user to quickly cycle between hard newlines or visual-line-mode.
They can also adjust the fill, such as after removing indentation.
---
 lisp/org.el | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/lisp/org.el b/lisp/org.el
index 57379c26..1becc394 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -21901,6 +21901,38 @@ modified."
 		      (org-do-remove-indentation))))))))
     (funcall unindent-tree (org-element-contents parse-tree))))
 
+(defun org-fill-buffer ()
+  "Fill all paragraph and plain-list elements.
+This command can be used to add hard newlines or to remove them.
+To add hard newlines for text filling, set `fill-column' to the
+desired with.  To remove hard newlines, such as to prepare the
+contents for `visual-line-mode', set `fill-column' to
+`most-positive-fixnum'."
+  (interactive)
+  (unless (and  (eq major-mode 'org-mode))
+    (user-error "Cannot fill a buffer not in Org mode"))
+  (letrec ((parse-tree (org-element-parse-buffer 'greater-element nil 'defer))
+           (fill-tree
+	    (lambda (contents)
+	      (dolist (element (reverse contents))
+                (let ((type (org-element-type element)))
+		  (if (member type  '(headline section))
+		      (funcall fill-tree (org-element-contents element))
+		    (save-excursion
+		      (save-restriction
+		        (narrow-to-region
+		         (org-element-begin element)
+		         (org-element-end element))
+                        (goto-char (point-min))
+                        (pcase type
+                          (`paragraph (fill-paragraph))
+                          (`plain-list
+                           (mapc (lambda (i)
+                              (goto-char (car i))
+                              (fill-paragraph))
+                            (reverse (org-list-struct)))))))))))))
+    (funcall fill-tree (org-element-contents parse-tree))))
+
 (defun org-make-options-regexp (kwds &optional extra)
   "Make a regular expression for keyword lines.
 KWDS is a list of keywords, as strings.  Optional argument EXTRA,
-- 
2.42.0

Re: org-(un)fill-buffer

[Ihor Radchenko]

Psionic K <psionik@positron.solutions> writes:

>> You may instead just run
>> (let ((fill-column most-positive-fixnum)) (fill-region (point-min) (point-max)))
> No.  That will have to be run manually on every element and every line
> of every list.  I suppose let's just not talk about it further and
> I'll submit a patch so there's no confusion.

May you please elaborate what is wrong with `fill-region'?

> This is the org-fill-buffer command, done generically for people who want
> to fill or unfill the entire buffer, as is required when alternating
> between hard newline filling and visual line mode filling.
> ...
> * lisp/org.el: (org-fill-buffer) this command walks the tree and will
> call fill-paragraph on every paragraph or plain-list element, enabling
> the user to quickly cycle between hard newlines or visual-line-mode.
> They can also adjust the fill, such as after removing indentation.

I'd rather make use of the existing Emacs toggles that control filling,
so that `fill-region' works as expected.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: org-(un)fill-buffer

[Psionic K]

If I run fill-region on a buffer, there's a lot of errors where the
lack of element awareness means filling is attempted on text that does
not fill properly, such as property drawers, keywords, and even
src-blocks without newline separations.  The result requires way too
much cleanup.

It is critical to be able to unfill documents for people to migrate
off of hard newlines and onto visual line mode with variable pitched
fonts.

I could probably convert this to a region-based command and we could
shadow fill-region.  It might be slightly tricky to deal with the
region if it overlaps elements, but if I had to do it now, I would
make the inclusion of part of an element include all of that element.

On Thu, Jan 11, 2024 at 12:47 AM Ihor Radchenko <yantar92@posteo.net> wrote:
>
> Psionic K <psionik@positron.solutions> writes:
>
> >> You may instead just run
> >> (let ((fill-column most-positive-fixnum)) (fill-region (point-min) (point-max)))
> > No.  That will have to be run manually on every element and every line
> > of every list.  I suppose let's just not talk about it further and
> > I'll submit a patch so there's no confusion.
>
> May you please elaborate what is wrong with `fill-region'?
>
> > This is the org-fill-buffer command, done generically for people who want
> > to fill or unfill the entire buffer, as is required when alternating
> > between hard newline filling and visual line mode filling.
> > ...
> > * lisp/org.el: (org-fill-buffer) this command walks the tree and will
> > call fill-paragraph on every paragraph or plain-list element, enabling
> > the user to quickly cycle between hard newlines or visual-line-mode.
> > They can also adjust the fill, such as after removing indentation.
>
> I'd rather make use of the existing Emacs toggles that control filling,
> so that `fill-region' works as expected.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>



-- 

남백호
대표 겸 공동 창업자
포지트론

fill-region-as-paragraph does not respect fill-paragraph-function (was: org-(un)fill-buffer)

[Ihor Radchenko]

[ CCing emacs-devel ]

Psionic K <psionik@positron.solutions> writes:

> If I run fill-region on a buffer, there's a lot of errors where the
> lack of element awareness means filling is attempted on text that does
> not fill properly, such as property drawers, keywords, and even
> src-blocks without newline separations.  The result requires way too
> much cleanup.

It looks like `fill-region' sometimes calls `fill-region-as-paragraph'
that ignores the Org mode's setting of `fill-paragraph-function'.

May someone knowledgeable tell if this behaviour is intentional?
Or maybe it is a bug?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: org-(un)fill-buffer

[Samuel Wales]

i lost track of all the visual fill stuff vs. emacs native filling vs.
org filling vs. filladapt back before visual filling was able to fill
with both a fill column and a reasonably smart fill prefix reliably.
is that possible now?

also, if a new command is to be introduced, presumably it would work
on subtrees, paragraphs, lists, and regions, so that you could have it
not apply to informal non-org lists or code that is not in a source
block etc.

met with alex today
yesterday he wasn't available
  even thuogh he said he would be


On 1/10/24, Psionic K <psionik@positron.solutions> wrote:
> If I run fill-region on a buffer, there's a lot of errors where the
> lack of element awareness means filling is attempted on text that does
> not fill properly, such as property drawers, keywords, and even
> src-blocks without newline separations.  The result requires way too
> much cleanup.
>
> It is critical to be able to unfill documents for people to migrate
> off of hard newlines and onto visual line mode with variable pitched
> fonts.
>
> I could probably convert this to a region-based command and we could
> shadow fill-region.  It might be slightly tricky to deal with the
> region if it overlaps elements, but if I had to do it now, I would
> make the inclusion of part of an element include all of that element.
>
> On Thu, Jan 11, 2024 at 12:47 AM Ihor Radchenko <yantar92@posteo.net>
> wrote:
>>
>> Psionic K <psionik@positron.solutions> writes:
>>
>> >> You may instead just run
>> >> (let ((fill-column most-positive-fixnum)) (fill-region (point-min)
>> >> (point-max)))
>> > No.  That will have to be run manually on every element and every line
>> > of every list.  I suppose let's just not talk about it further and
>> > I'll submit a patch so there's no confusion.
>>
>> May you please elaborate what is wrong with `fill-region'?
>>
>> > This is the org-fill-buffer command, done generically for people who
>> > want
>> > to fill or unfill the entire buffer, as is required when alternating
>> > between hard newline filling and visual line mode filling.
>> > ...
>> > * lisp/org.el: (org-fill-buffer) this command walks the tree and will
>> > call fill-paragraph on every paragraph or plain-list element, enabling
>> > the user to quickly cycle between hard newlines or visual-line-mode.
>> > They can also adjust the fill, such as after removing indentation.
>>
>> I'd rather make use of the existing Emacs toggles that control filling,
>> so that `fill-region' works as expected.
>>
>> --
>> Ihor Radchenko // yantar92,
>> Org mode contributor,
>> Learn more about Org mode at <https://orgmode.org/>.
>> Support Org development at <https://liberapay.com/org-mode>,
>> or support my work at <https://liberapay.com/yantar92>
>
>
>
> --
>
> 남백호
> 대표 겸 공동 창업자
> 포지트론
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: org-(un)fill-buffer

[Psionic K]

> smart fill prefix reliably.  is that possible now?

It's reasonably complete for several documents I'm converting, such as
the transient showcase.  Visual fill requires two modes right now:

1. visual-line-mode (usually with visual-fill-column-mode as well)
2. adaptive-wrap-prefix-mode

Updating documents to this intended useage manner required an
"unfill."  I started off with the unfill package, which works by
setting fill-column to most-positive-fixnum.  Implementing an unfill
buffer command uncovered the issue with fill-region in org mode.


On Thu, Jan 11, 2024 at 10:12 AM Samuel Wales <samologist@gmail.com> wrote:
>
> i lost track of all the visual fill stuff vs. emacs native filling vs.
> org filling vs. filladapt back before visual filling was able to fill
> with both a fill column and a reasonably smart fill prefix reliably.
> is that possible now?
>
> also, if a new command is to be introduced, presumably it would work
> on subtrees, paragraphs, lists, and regions, so that you could have it
> not apply to informal non-org lists or code that is not in a source
> block etc.
>
> met with alex today
> yesterday he wasn't available
>   even thuogh he said he would be
>
>
> On 1/10/24, Psionic K <psionik@positron.solutions> wrote:
> > If I run fill-region on a buffer, there's a lot of errors where the
> > lack of element awareness means filling is attempted on text that does
> > not fill properly, such as property drawers, keywords, and even
> > src-blocks without newline separations.  The result requires way too
> > much cleanup.
> >
> > It is critical to be able to unfill documents for people to migrate
> > off of hard newlines and onto visual line mode with variable pitched
> > fonts.
> >
> > I could probably convert this to a region-based command and we could
> > shadow fill-region.  It might be slightly tricky to deal with the
> > region if it overlaps elements, but if I had to do it now, I would
> > make the inclusion of part of an element include all of that element.
> >
> > On Thu, Jan 11, 2024 at 12:47 AM Ihor Radchenko <yantar92@posteo.net>
> > wrote:
> >>
> >> Psionic K <psionik@positron.solutions> writes:
> >>
> >> >> You may instead just run
> >> >> (let ((fill-column most-positive-fixnum)) (fill-region (point-min)
> >> >> (point-max)))
> >> > No.  That will have to be run manually on every element and every line
> >> > of every list.  I suppose let's just not talk about it further and
> >> > I'll submit a patch so there's no confusion.
> >>
> >> May you please elaborate what is wrong with `fill-region'?
> >>
> >> > This is the org-fill-buffer command, done generically for people who
> >> > want
> >> > to fill or unfill the entire buffer, as is required when alternating
> >> > between hard newline filling and visual line mode filling.
> >> > ...
> >> > * lisp/org.el: (org-fill-buffer) this command walks the tree and will
> >> > call fill-paragraph on every paragraph or plain-list element, enabling
> >> > the user to quickly cycle between hard newlines or visual-line-mode.
> >> > They can also adjust the fill, such as after removing indentation.
> >>
> >> I'd rather make use of the existing Emacs toggles that control filling,
> >> so that `fill-region' works as expected.
> >>
> >> --
> >> Ihor Radchenko // yantar92,
> >> Org mode contributor,
> >> Learn more about Org mode at <https://orgmode.org/>.
> >> Support Org development at <https://liberapay.com/org-mode>,
> >> or support my work at <https://liberapay.com/yantar92>
> >
> >
> >
> > --
> >
> > 남백호
> > 대표 겸 공동 창업자
> > 포지트론
> >
> >
>
>
> --
> The Kafka Pandemic
>
> A blog about science, health, human rights, and misopathy:
> https://thekafkapandemic.blogspot.com



-- 

남백호
대표 겸 공동 창업자
포지트론