org-forward-paragraph doesn't work programatically (called from Lisp)

org-forward-paragraph doesn't work programatically (called from Lisp)

[Gerald Wildgruber]

Hi

I'm trying to unfill/unwrap Org mode files because I switched from auto-fill-mode to visual-line/visual-fill-column mode.

I'd like to harmonize = unfill my already existing (filled) Org mode files. I took the function in

https://www.emacswiki.org/emacs/UnfillParagraph)

and tried the following (from within an emacs lisp buffer):

(let ((fill-column most-positive-fixnum))
  (dolist (f (directory-files-recursively
              "~/directory/with/org/files/" (rx (or ".org" ".outl") eos)))
    (with-current-buffer (find-file-noselect f)
      (while (not (eobp))
        (fill-paragraph)
        (org-forward-paragraph))
      (save-buffer))))

I thought this would iterate over all paragraphs (or org-wise equivalent structures) until end of file is reached and unfill each one of them.

But it doesn't work. The iteration doesn't happen, it somehow doesn't move forward. It works though, if called interactively in an Org mode file.

Why is that, how can I use org-forward-paragraph programmatically?

Thanks

-- 
Sent with mu4e

Re: org-forward-paragraph doesn't work programatically (called from Lisp)

[Nicolas Goaziou]

Hello,

Gerald Wildgruber <wildgruber@tu-berlin.de> writes:

> I'm trying to unfill/unwrap Org mode files because I switched from auto-fill-mode to visual-line/visual-fill-column mode.
>
> I'd like to harmonize = unfill my already existing (filled) Org mode files. I took the function in
>
> https://www.emacswiki.org/emacs/UnfillParagraph)
>
> and tried the following (from within an emacs lisp buffer):
>
> (let ((fill-column most-positive-fixnum))
>   (dolist (f (directory-files-recursively
>               "~/directory/with/org/files/" (rx (or ".org" ".outl") eos)))
>     (with-current-buffer (find-file-noselect f)
>       (while (not (eobp))
>         (fill-paragraph)
>         (org-forward-paragraph))
>       (save-buffer))))
>
> I thought this would iterate over all paragraphs (or org-wise equivalent structures) until end of file is reached and unfill each one of them.
>
> But it doesn't work. The iteration doesn't happen, it somehow doesn't move forward. It works though, if called interactively in an Org mode file.
>
> Why is that, how can I use org-forward-paragraph programmatically?

I cannot reproduce your problem. What Org version are you using?

Regards,

-- 
Nicolas Goaziou

Re: org-forward-paragraph doesn't work programatically (called from Lisp)

[Gerald Wildgruber]

Hi,

I'm using git checkouts for both emacs and Org mode:

Org mode version 9.1.14 (release_9.1.14-921-g13626a)

GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30) of 2018-09-20

Yet, I can't get any result: upon evaluating the "let" in the scratch buffer, I just get "nil" in the echo area, and nothing else has happened, none of the files in the directory is touched.

@Nikolay: can you confirm that this worked for you?

Same thing with a single file:

(let ((fill-column most-positive-fixnum))
  (dolist (f (list "~/lorem.org"))
    (with-current-buffer (find-file-noselect f)
      (while (not (eobp))
        (fill-paragraph)
        (org-forward-paragraph))
      (save-buffer))))

Just "nil" and nothing else.

Thanks

Here's the single file lorem.org I tried to unfill:

==================================================

* Header 1

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
hendrerit tempor tellus. Donec pretium posuere tellus. Proin quam
nisl, tincidunt et, mattis eget, convallis nec, purus. Cum sociis
natoque penatibus et magnis dis parturient montes, nascetur ridiculus
mus. Nulla posuere. Donec vitae dolor. Nullam tristique diam non
turpis. Cras placerat accumsan nulla. Nullam rutrum. Nam vestibulum
accumsan nisl.

* Header 2

1. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Lorem
   ipsum dolor sit amet, consectetuer adipiscing elit.  Curabitur
   vulputate vestibulum lorem.  Praesent augue.  Aenean in sem ac leo
   mollis blandit.  Praesent fermentum tempor tellus.  Nullam libero
   mauris, consequat quis, varius et, dictum id, arcu.  Nulla posuere.
   Mauris mollis tincidunt felis.  Nullam eu ante vel est convallis
   dignissim.  Nullam libero mauris, consequat quis, varius et, dictum
   id, arcu.
2. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Aliquam
   posuere.  Fusce suscipit, wisi nec facilisis facilisis, est dui
   fermentum leo, quis tempor ligula erat quis odio.
3. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nam a
   sapien.  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
   Vivamus id enim.
4. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nam a
   sapien.  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
   Vivamus id enim.
5. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nam a
   sapien.  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
   Vivamus id enim.


* Header 3

- Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nullam
  rutrum.  Vivamus id enim.  Lorem ipsum dolor sit amet, consectetuer
  adipiscing elit.
- Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nunc
  eleifend leo vitae magna.
- Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nullam
  rutrum.  Donec at pede.  Curabitur vulputate vestibulum lorem.
  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
- Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nullam
  rutrum.  Donec at pede.  Curabitur vulputate vestibulum lorem.
  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
- Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Nullam
  rutrum.  Donec at pede.  Curabitur vulputate vestibulum lorem.
  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.


==================================================



On So, Okt 07 2018, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Hello,
>
> Gerald Wildgruber <wildgruber@tu-berlin.de> writes:
>
>> I'm trying to unfill/unwrap Org mode files because I switched from auto-fill-mode to visual-line/visual-fill-column mode.
>>
>> I'd like to harmonize = unfill my already existing (filled) Org mode files. I took the function in
>>
>> https://www.emacswiki.org/emacs/UnfillParagraph)
>>
>> and tried the following (from within an emacs lisp buffer):
>>
>> (let ((fill-column most-positive-fixnum))
>>   (dolist (f (directory-files-recursively
>>               "~/directory/with/org/files/" (rx (or ".org" ".outl") eos)))
>>     (with-current-buffer (find-file-noselect f)
>>       (while (not (eobp))
>>         (fill-paragraph)
>>         (org-forward-paragraph))
>>       (save-buffer))))
>>
>> I thought this would iterate over all paragraphs (or org-wise equivalent structures) until end of file is reached and unfill each one of them.
>>
>> But it doesn't work. The iteration doesn't happen, it somehow doesn't move forward. It works though, if called interactively in an Org mode file.
>>
>> Why is that, how can I use org-forward-paragraph programmatically?
>
> I cannot reproduce your problem. What Org version are you using?
>
> Regards,


-- 
Sent with mu4e

Re: org-forward-paragraph doesn't work programatically (called from Lisp)

[Kyle Meyer]

Gerald Wildgruber <wildgruber@tu-berlin.de> writes:

[...]

> Yet, I can't get any result: upon evaluating the "let" in the scratch
> buffer, I just get "nil" in the echo area, and nothing else has
> happened, none of the files in the directory is touched.
>
> @Nikolay: can you confirm that this worked for you?

I can confirm that it worked on my end.  But...

> Same thing with a single file:
>
> (let ((fill-column most-positive-fixnum))
>   (dolist (f (list "~/lorem.org"))
>     (with-current-buffer (find-file-noselect f)
>       (while (not (eobp))
>         (fill-paragraph)
>         (org-forward-paragraph))
>       (save-buffer))))

here are two ways I could make the above code fail:

  1) If the buffer for the file is already open and point is after the
     text.

     It seems your code should call `(goto-char (point-min))' and, if
     you care, restore it afterwards.

  2) Your code doesn't account for hidden text in the buffer.  You could
     call `(org-show-all)'.

I suspect #2 is why you're not seeing the results you expect.

-- 
Kyle

Re: org-forward-paragraph doesn't work programatically (called from Lisp)

[Gerald Wildgruber]

On Mo, Okt 08 2018, Kyle Meyer <kyle@kyleam.com> wrote:

>   2) Your code doesn't account for hidden text in the buffer.  You could
>      call `(org-show-all)'.
>
> I suspect #2 is why you're not seeing the results you expect.

Wow, thanks a lot Kylie, this was it! I never thought of having to unfold the Org mode entries in a buffer when opening in a non interactive way.

I do this now from a lisp buffer and both functions work exactly as expected:

single file:
============

(let ((fill-column most-positive-fixnum))
  (dolist (f (list "~/lorem.org"))
    (with-current-buffer (find-file-noselect f)
      (org-show-all)
      (while (not (eobp))
        ;; (mark-whole-buffer)
        (fill-paragraph)
        (org-forward-paragraph))
      (save-buffer))))


recursively through dir tree:
=============================

(let ((fill-column most-positive-fixnum))
  (dolist (f (directory-files-recursively
              "/dirs/with/org/files/" (rx (or ".org" ".outl") eos)))
    (with-current-buffer (find-file-noselect f)
      (org-show-all)
      (while (not (eobp))
        (fill-paragraph)
        (org-forward-paragraph))
      (save-buffer))))


That's great; I'm glad it works now from within emacs itself (and not via sed, awk or tr as I tried before), harnessing all the knowledge the editor has of its own constructs, especially with some of the more complicated list and enumeration structures, -- all of them are correctly unfilled now.

Thanks again

Gerald.

-- 
Sent with mu4e