How to get all paragraphs in list?

How to get all paragraphs in list?

[Jean Louis]

I would like to get all paragraphs in a list, even if paragraphs are
separated by two or more spaces, I would still like those spaces.

Purpose of doing it is joining lines of such paragraph as to paste it
into website pages or chats. Wrapped lines do not look nice when
pasted. 

Starting function to join lines is this one:

(defun join-lines ()
  "Joins lines of a paragraph of 100000 chars."
  (interactive)
  (let ((old fill-column))
    (setq fill-column 100000)
    (fill-paragraph nil)
    (setq fill-column old)))

My starting point to get all paragraphs could be in this function:

(defun sort-paragraphs (reverse beg end)
  "Sort paragraphs in region alphabetically; argument means descending order.
Called from a program, there are three arguments:
REVERSE (non-nil means reverse order), BEG and END (region to sort).
The variable `sort-fold-case' determines whether alphabetic case affects
the sort order."
  (interactive "P\nr")
  (save-excursion
    (save-restriction
      (narrow-to-region beg end)
      (goto-char (point-min))
      (sort-subr reverse
		 (lambda ()
		   (while (and (not (eobp)) (looking-at paragraph-separate))
		     (forward-line 1)))
		 (lambda ()
                   (forward-paragraph)
                   ;; If the buffer doesn't end with a newline, add a
                   ;; newline to avoid having paragraphs being
                   ;; concatenated after sorting.
                   (when (and (eobp)
                              (not (bolp)))
                     (insert "\n")))))))

Is there any existing way to fetch all paragraphs (separated by two or
more spaces) in the list?


Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: How to get all paragraphs in list?

[Teemu Likonen]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

* 2022-09-05 07:15:50+0300, Jean Louis wrote:

> Starting function to join lines is this one:
>
> (defun join-lines ()
>   "Joins lines of a paragraph of 100000 chars."
>   (interactive)
>   (let ((old fill-column))
>     (setq fill-column 100000)
>     (fill-paragraph nil)
>     (setq fill-column old)))

Not answering your actual question but I noticed that the above function
can be simplified because fill-column is a dynamic (special) variable:

    (defun join-lines ()
      "Joins lines of a paragraph of 100000 chars."
      (interactive)
      (let ((fill-column 100000))
        (fill-paragraph)))

-- 
/// Teemu Likonen - .-.. https://www.iki.fi/tlikonen/
// OpenPGP: 6965F03973F0D4CA22B9410F0F2CAE0E07608462

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 251 bytes --]

Re: How to get all paragraphs in list?

[Emanuel Berg]

Jean Louis wrote:

> (let ((old fill-column))
>   (setq fill-column 100000)
>   (fill-paragraph nil)
>   (setq fill-column old)))

Even if that's a lexical/static `let' it doesn't matter since
`fill-column' is dynamic/special anyway, try the below

;;; -*- lexical-binding: t -*-

(defun test-fill-column ()
  (message "%d" fill-column) )

(progn
  (let ((fill-column 1))
    (test-fill-column) )
  (test-fill-column) )

> (fill-paragraph nil)

(fill-paragraph)

> (forward-line 1)

(forward-line)

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to get all paragraphs in list?

[Alessandro Bertulli]

Jean Louis <bugs@gnu.support> writes:

> (defun join-lines ()
>   "Joins lines of a paragraph of 100000 chars."
>   (interactive)
>   (let ((old fill-column))
>     (setq fill-column 100000)
>     (fill-paragraph nil)
>     (setq fill-column old)))

Sorry I don't have an answer to your question, but I wondered: would it
be better to have fill-column set to its maximum value? Which, as far as
I found, is (point-max)? Here is a SE answer I found
https://emacs.stackexchange.com/a/8181/29817

Alessandro

RE: [External] : How to get all paragraphs in list?

[Drew Adams]

Without looking at the details of all you ask,
maybe something like this helps?

(defun paragraphs-in-region (&optional start end msgp)
  (interactive
   (if (use-region-p)
       (list (region-beginning) (region-end) t)
     (list (point-min) (point-max) t)))
  (let ((paras  ()))
    (save-excursion
      (goto-char start)
      (while (< (point) end)
        (push (buffer-substring-no-properties
               (point)
               (progn (forward-paragraph) (point)))
              paras))
      (setq paras  (nreverse paras))
      (when msgp (message "Paras: %S" paras))
      paras)))

Re: How to get all paragraphs in list?

[Jean Louis]

* Alessandro Bertulli <alessandro.bertulli96@gmail.com> [2022-09-05 12:16]:
> 
> Jean Louis <bugs@gnu.support> writes:
> 
> > (defun join-lines ()
> >   "Joins lines of a paragraph of 100000 chars."
> >   (interactive)
> >   (let ((old fill-column))
> >     (setq fill-column 100000)
> >     (fill-paragraph nil)
> >     (setq fill-column old)))
> 
> Sorry I don't have an answer to your question, but I wondered: would it
> be better to have fill-column set to its maximum value? Which, as far as
> I found, is (point-max)? Here is a SE answer I found
> https://emacs.stackexchange.com/a/8181/29817

I am adopting the two ideas and now I have this:

(defun join-lines ()
  "Joins lines of a paragraph."
  (interactive)
  (let ((fill-column (point-max)))
    (fill-paragraph nil)))

That function could be used when iterating interactively or in
temporary buffer over paragraphs.

Other solution could be when I convert paragraphs to list and then
simply remove new lines.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [External] : How to get all paragraphs in list?

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2022-09-05 19:03]:
> Without looking at the details of all you ask,
> maybe something like this helps?
> 
> (defun paragraphs-in-region (&optional start end msgp)
>   (interactive
>    (if (use-region-p)
>        (list (region-beginning) (region-end) t)
>      (list (point-min) (point-max) t)))
>   (let ((paras  ()))
>     (save-excursion
>       (goto-char start)
>       (while (< (point) end)
>         (push (buffer-substring-no-properties
>                (point)
>                (progn (forward-paragraph) (point)))
>               paras))
>       (setq paras  (nreverse paras))
>       (when msgp (message "Paras: %S" paras))
>       paras)))

Thank you, that gave me idea for these functions:

(defun rcd-paragraphs-iterate (function)
  "Iterate FUNCTION over paragraphs.

FUNCTION must accept string as single argument."
  (let ((start (if (use-region-p) (region-beginning) (point-min)))
	(end (if (use-region-p) (region-end) (point-max))))
    (save-excursion
      (goto-char start)
      (while (<= (point) end)
	(funcall function)
	(forward-paragraph)))))

(defun join-lines ()
  "Joins lines of a paragraph."
  (interactive)
  (let ((fill-column (point-max)))
    (fill-paragraph nil)))

(defun join-lines-buffer-or-region ()
  "Join lines on all buffer."
  (interactive)
  (rcd-paragraphs-iterate 'join-lines))

However, there is some problem as it blocks, never ends, when I
interrupt it with Ctrl-G then I see that lines have been joined in all
of the buffer, but `while' loop was I guess still running.

Do you maybe know why?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

RE: [External] : How to get all paragraphs in list?

[Drew Adams]

> However, there is some problem as it blocks, never ends, when I
> interrupt it with Ctrl-G then I see that lines have been joined in all
> of the buffer, but `while' loop was I guess still running.
> 
> Do you maybe know why?

You're repeatedly refilling/joining the same paragraphs.

`M-x debug-on-entry' shows you exactly what any of
your functions is doing, step by step.

Re: How to get all paragraphs in list?

[Emanuel Berg]

Jean Louis wrote:

> I am adopting the two ideas and now I have this:
>
> (defun join-lines ()
>   "Joins lines of a paragraph."
>   (interactive)
>   (let ((fill-column (point-max)))
>     (fill-paragraph nil)))

Adopt the third idea as well and remove that last nil ...
actually it is the only one.

-- 
underground experts united
https://dataswamp.org/~incal

RE: [External] : How to get all paragraphs in list?

[Jean Louis]

Give me clue how to avoid it.



On September 5, 2022 6:05:52 PM UTC, Drew Adams <drew.adams@oracle.com> wrote:
>> However, there is some problem as it blocks, never ends, when I
>> interrupt it with Ctrl-G then I see that lines have been joined in
>all
>> of the buffer, but `while' loop was I guess still running.
>> 
>> Do you maybe know why?
>
>You're repeatedly refilling/joining the same paragraphs.
>
>`M-x debug-on-entry' shows you exactly what any of
>your functions is doing, step by step.


Jean

Re: [External] : How to get all paragraphs in list?

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2022-09-05 21:07]:
> > However, there is some problem as it blocks, never ends, when I
> > interrupt it with Ctrl-G then I see that lines have been joined in all
> > of the buffer, but `while' loop was I guess still running.
> > 
> > Do you maybe know why?
> 
> You're repeatedly refilling/joining the same paragraphs.
> 
> `M-x debug-on-entry' shows you exactly what any of
> your functions is doing, step by step.

You have made this function:

(defun paragraphs-in-region (&optional start end msgp)
  "Return list of paragraphs in region.

By Drew Adams, Date: Mon, 5 Sep 2022 16:02:32 +0000, GNU Emacs Help."
  (interactive
   (if (use-region-p)
       (list (region-beginning) (region-end) t)
     (list (point-min) (point-max) t)))
  (let ((paras  ()))
    (save-excursion
      (goto-char start)
      (while (< (point) end)
        (push (buffer-substring-no-properties
               (point)
               (progn (forward-paragraph) (point)))
              paras))
      (setq paras  (nreverse paras))
      (when msgp (message "Paras: %S" paras))
      paras)))

Then I wanted to make general function to iterate over paragraphs this
way:

(defun rcd-paragraphs-iterate (function)
  "Iterate FUNCTION over paragraphs.

FUNCTION must accept string as single argument."
  (let ((start (if (use-region-p) (region-beginning) (point-min)))
	(end (if (use-region-p) (region-end) (point-max))))
    (save-excursion
      (goto-char start)
      (while (< (point) end)
	(funcall function)
	(forward-paragraph)))))

Then I use following functions to apply on the paragraph.

(defun join-lines ()
  "Joins lines of a paragraph."
  (interactive)
  (let ((fill-column (point-max)))
    (fill-paragraph nil)))

(defun join-lines-buffer-or-region ()
  "Join lines on all buffer."
  (interactive)
  (rcd-paragraphs-iterate 'join-lines))

That means `join-lines-buffer-or-region' applies on paragraphs in
buffer or region.

Problem is in:

      (while (< (point) end)
	(funcall function)
	(forward-paragraph)))))

I still cannot understand WHEN is that happening, I just assume that
it happens when there are some empty lines on the end. Sometimes it
works, sometimes not.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [External] : How to get all paragraphs in list?

[Emanuel Berg]

Jean Louis wrote:

> (defun rcd-paragraphs-iterate (function)
>   "Iterate FUNCTION over paragraphs.
> FUNCTION must accept string as single argument."
>   (let ((start (if (use-region-p) (region-beginning) (point-min)))
> 	(end (if (use-region-p) (region-end) (point-max))))

Should be an interactive function and this moved the the
spec ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : How to get all paragraphs in list?

[Yuri Khan]

On Wed, 7 Sept 2022 at 13:06, Jean Louis <bugs@gnu.support> wrote:

>       (while (< (point) end)
>         (funcall function)
>         (forward-paragraph)))))

This loop is fragile: if FUNCTION moves point backward in the buffer,
it may loop infinitely, or if it moves forward, it may skip some
paragraphs. I’d maybe wrap the funcall in a save-excursion.

More to the point[sic], you calculate and cache the END position as an
integer. So if your FUNCTION reduces the length of the buffer text,
you may never reach END. Should probably place a marker there, or
check for (region-end) or (point-max) every time.

RE: [External] : How to get all paragraphs in list?

[Drew Adams]

> From: Yuri Khan
>
> >       (while (< (point) end)
> >         (funcall function)
> >         (forward-paragraph)))))
> 
> This loop is fragile: if FUNCTION moves point backward in the buffer,
> it may loop infinitely, or if it moves forward, it may skip some
> paragraphs. I’d maybe wrap the funcall in a save-excursion.
> 
> More to the point[sic], you calculate and cache the END position as an
> integer. So if your FUNCTION reduces the length of the buffer text,
> you may never reach END. Should probably place a marker there, or
> check for (region-end) or (point-max) every time.

What Yuri said.

And I pointed you to using the debugger, if you
want to see and understand just what's going on
in your code.  You can also insert calls to
`message' in it at various places, to print values.

And you can insert calls to `debug' in it, as
breakpoints - e.g. (debug nil EVAL-ME1 EVAL-ME2...).

Re: [External] : How to get all paragraphs in list?

[Emanuel Berg]

Yuri Khan wrote:

> More to the point[sic], you calculate and cache the END
> position as an integer. So if your FUNCTION reduces the
> length of the buffer text, you may never reach END.
> Should probably place a marker there

Marker, do we have that? :O

How does that look/work?

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to get all paragraphs in list?

[Jean Louis]

* Emanuel Berg <incal@dataswamp.org> [2022-09-16 16:19]:
> Jean Louis wrote:
> 
> > I am adopting the two ideas and now I have this:
> >
> > (defun join-lines ()
> >   "Joins lines of a paragraph."
> >   (interactive)
> >   (let ((fill-column (point-max)))
> >     (fill-paragraph nil)))
> 
> Adopt the third idea as well and remove that last nil ...
> actually it is the only one.

Thanks sharp eye. I have improved it now.

(defun join-lines ()
  "Joins lines of a paragraph."
  (interactive)
  (let ((fill-column (point-max)))
    (fill-paragraph)))


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: How to get all paragraphs in list?

[Jean Louis]

* Emanuel Berg <incal@dataswamp.org> [2022-09-16 16:19]:
> Adopt the third idea as well and remove that last nil ...
> actually it is the only one.

(defun rcd-paragraphs-iterate (function)
  "Iterate FUNCTION over paragraphs.

FUNCTION must accept string as single argument."
  (let ((start (if (use-region-p) (region-beginning) (point-min)))
	(end (if (use-region-p) (region-end) (point-max))))
    (save-excursion
      (goto-char start)
      (while (< (point) end)
	(funcall function)
	(forward-paragraph)))))

Help me with this one. Problem is with (while (< (point) end) as it
will not accurately move every time.x


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [External] : How to get all paragraphs in list?

[Yuri Khan]

On Fri, 16 Sept 2022 at 20:29, Emanuel Berg <incal@dataswamp.org> wrote:

> > More to the point[sic], you calculate and cache the END
> > position as an integer. So if your FUNCTION reduces the
> > length of the buffer text, you may never reach END.
> > Should probably place a marker there
>
> Marker, do we have that? :O
>
> How does that look/work?

Will you forgive me if I say “(elisp) Markers”? Basically it’s a way
to refer to a position in the buffer in a way that remains valid and
somewhat sane under buffer text modification.

Re: [External] : How to get all paragraphs in list?

[Jean Louis]

* Emanuel Berg <incal@dataswamp.org> [2022-09-16 16:32]:
> Yuri Khan wrote:
> 
> > More to the point[sic], you calculate and cache the END
> > position as an integer. So if your FUNCTION reduces the
> > length of the buffer text, you may never reach END.
> > Should probably place a marker there
> 
> Marker, do we have that? :O
> 
> How does that look/work?

I guess, it was your correct guess.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [External] : How to get all paragraphs in list?

[Jean Louis]

* Yuri Khan <yuri.v.khan@gmail.com> [2022-09-16 19:26]:
> On Fri, 16 Sept 2022 at 20:29, Emanuel Berg <incal@dataswamp.org> wrote:
> 
> > > More to the point[sic], you calculate and cache the END
> > > position as an integer. So if your FUNCTION reduces the
> > > length of the buffer text, you may never reach END.
> > > Should probably place a marker there
> >
> > Marker, do we have that? :O
> >
> > How does that look/work?
> 
> Will you forgive me if I say “(elisp) Markers”? Basically it’s a way
> to refer to a position in the buffer in a way that remains valid and
> somewhat sane under buffer text modification.

That was alright, and I have changed (point-max) to (point-max-marker)
now and it seem to work. I have to test it many times until I become
sure.

This is important because I have many documents which I often quote,
and copy into other programs, where broken lines are making incorrect
representation.

(defun rcd-paragraphs-iterate (function)
  "Iterate FUNCTION over paragraphs.

FUNCTION must accept string as single argument."
  (let ((start (if (use-region-p) (region-beginning) (point-min)))
	(end (if (use-region-p) (region-end) (point-max-marker))))
    (save-excursion
      (goto-char start)
      (while (<= (point) end)
	(funcall function)
	(forward-paragraph)))))

(defun join-lines ()
  "Joins lines of a paragraph."
  (interactive)
  (let ((fill-column (point-max)))
    (fill-paragraph)))

(defun join-lines-buffer-or-region ()
  "Join lines on all buffer."
  (interactive)
  (rcd-paragraphs-iterate 'join-lines))

(defun join-lines-string (text)
  "Join lines within TEXT."
  (with-temp-buffer
    (insert text)
    (rcd-paragraphs-iterate 'join-lines)
    (buffer-string)))

(defun join-lines-1 (text)
  (with-temp-buffer 
    (let ((fill-column 100000))
      (insert text)
      (goto-char 0)
      (fill-paragraph)
      (buffer-string))))

(defun kill-region-join-lines (start end)
  "Kill region with lines joined"
  (interactive "r")
  (when (region-active-p)
    (let* ((text (buffer-substring-no-properties start end))
	   (joined (join-lines-1 text)))
      (deactivate-mark)
      (kill-new joined))))


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/