Alinea filling (hanlding of explicit line-breaks)

Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

Dear all,

I would like to suggest an evolution of fill.el. I can make the update
and provide the diff file.

The proposal is to add, in addition to paragraph separator, an alinea
separator. Accordingly there would be a forward-alinea function similar
to forward-paragraph.

An alinea is, in French regulatory jargon, (er, what is the English word
for alinea? ) a part in a paragraph that is separated by a line break,
and typically starting with an indentation. Not all pargraphs are
separated into alineas.

That evolution would be completely backward compatible: with a nil
alinea separator, the behaviour would be same as the current one, and
otherwise paragraph filling would fill each alineas within the current
paragraph as if they were paragraphs on their owns.

I felt the need for this in two occasions, the first one was for EUPP
(cf http://vincentbelaiche.pagesperso-orange.fr/#sec-1_2), and the
second one was with Org mode.

EUPP embeds lisp code into comments thanks to special markup added onto
the usual comment marks. With EUPP, I needed to prevent the paragraph
filling to mangle those comment extra marks, as otherwise they would be
erroneously separated from normal comment marks. One solution would be
that the EUPP mark that is at each end of line of embedded lisp code is
configured as an alinea separator.

I suggested an evolution on AUCTeX (as I use EUPP mostly in conjunction
with LaTeX code), but the answer from Ralf Angeli at that time was that
this should rather be an evolution of generic filling service in
fill.el, and I was convinced by him that this would be a better way.

Another occasion, is for Org mode. In Org-mode: the explicit line break
is a `\\' placed at end of line. If `\\' is not at end of line it loses
its meaning of line-break. Filling a paragraph that contains an explicit
line break is quite a disturbance because most often the line tailing \\
is moved and as after filling it is no longer at the end of line, it
loses its linebreaking power. The solution would be to configure `\\\\$'
into the alinea separator, so that filling a paragraph does not mix
alineas together.

Feedback to this proposal is most welcome, in case that there is a
consensus on the usefulness of alinea separation, I would provide the
updated code/bazaar diff.

VBR,
   Vincent.

Re: Alinea filling (hanlding of explicit line-breaks)

[Stefan Monnier]

> The proposal is to add, in addition to paragraph separator, an alinea
> separator. Accordingly there would be a forward-alinea function similar
> to forward-paragraph.

I'm not completely sure I understand the details, so let me try to
describe what I think you're suggesting: you're suggesting to
distinguish the notion of paragraph (as used by the paragraph-forward
movement command) from the notion of "unit of text to fill"m which you
call alinea.  And you also propose to complete this by adding
a corresponding forward-alinea command.

Is that right?

If that's the case, then I think we already have most of it in Emacs-24,
in the form of the fill-forward-paragraph-function, which decouples the
navigation command from the "unit of text to fill".  You can see it in
action in ChangeLog files, where M-q will only refill the current
"alinea" starting with a function or variable name whereas M-} will jump
over the text of the whole file (which is made of several alineas, each
describe one (or sometimes a set of) variables or functions).


        Stefan

Re: [Orgmode] Re: Alinea filling (hanlding of explicit line-breaks)

[Samuel Wales]

Vincent, examples?

-- 
The Kafka Pandemic:
http://thekafkapandemic.blogspot.com/2010/12/welcome-to-kafka-pandemic-two-forces_9182.html
I support the Whittemore-Peterson Institute (WPI)
===
I want to see the original (pre-hold) Lo et al. 2010 NIH/FDA/Harvard MLV paper.

Re: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

Hi,

Good to see that I am not the only one needing that sort of things.

Maybe an example is better to explain what I need, imagine that I have
the following two paragraphs:

-----------------------------------------------------------------------
aaa a aaa a aaa aaaaa aaaa a aaaa aaaa aaaaa aaaa aaa aaaaa aa aaaaa aaa aaaa aaa aa aaaa a aaaaa aaaaa aa aaaa aaaaa aaaa aaaaa aaaa aaa aaaa aaaaa aa a aaaaa aa aaaaa aaa aa aa aaa a aa aaaaa aaaa aaaa aaa aaaa aaaa a aaa aaa aaaaa aaaa aaa aaaaa aaaa aaaaa aaa aaa aaaa aaaaa a aaaa a aa aaaaa aaa aa a aaaa aaa a a aaa aaaa aaaaa aaaaa a aaaaa aaaa aaaaa aa aaa aaaa aaaa aaa aa a aaaaa aaaaa aaa aaa aaaaa aa aaaaa aa aaa aaaaa aaaaa\\
bbb b bbb b bbb bbbbb bbbb b bbbb bbbb bbbbb bbbb bbb bbbbb bb bbbbb bbb bbbb bbb bb bbbb b bbbbb bbbbb bb bbbb bbbbb bbbb bbbbb bbbb bbb bbbb bbbbb bb b bbbbb bb bbbbb bbb bb bb bbb b bb bbbbb bbbb bbbb bbb bbbb bbbb b bbb bbb bbbbb bbbb bbb bbbbb bbbb bbbbb bbb bbb bbbb bbbbb b bbbb b bb bbbbb bbb bb b bbbb bbb b b bbb bbbb bbbbb bbbbb b bbbbb bbbb bbbbb bb bbb bbbb bbbb bbb bb b bbbbb bbbbb bbb bbb bbbbb bb bbbbb bb bbb bbbbb bbbbb

ccc c ccc c ccc ccccc cccc c cccc cccc ccccc cccc ccc ccccc cc ccccc ccc cccc ccc cc cccc c ccccc ccccc cc cccc ccccc cccc ccccc cccc ccc cccc ccccc cc c ccccc cc ccccc ccc cc cc ccc c cc ccccc cccc cccc ccc cccc cccc c ccc ccc ccccc cccc ccc ccccc cccc ccccc ccc ccc cccc ccccc c cccc c cc ccccc ccc cc c cccc ccc c c ccc cccc ccccc ccccc c ccccc cccc ccccc cc ccc cccc cccc ccc cc c ccccc ccccc ccc ccc ccccc cc ccccc cc ccc ccccc ccccc\\
ddd d ddd d ddd ddddd dddd d dddd dddd ddddd dddd ddd ddddd dd ddddd ddd dddd ddd dd dddd d ddddd ddddd dd dddd ddddd dddd ddddd dddd ddd dddd ddddd dd d ddddd dd ddddd ddd dd dd ddd d dd ddddd dddd dddd ddd dddd dddd d ddd ddd ddddd dddd ddd ddddd dddd ddddd ddd ddd dddd ddddd d dddd d dd ddddd ddd dd d dddd ddd d d ddd dddd ddddd ddddd d ddddd dddd ddddd dd ddd dddd dddd ddd dd d ddddd ddddd ddd ddd ddddd dd ddddd dd ddd ddddd ddddd
-----------------------------------------------------------------------

after filling I wish that it would become

-----------------------------------------------------------------------
aaa a aaa a aaa aaaaa aaaa a aaaa aaaa aaaaa aaaa aaa aaaaa aa aaaaa aaa
aaaa aaa aa aaaa a aaaaa aaaaa aa aaaa aaaaa aaaa aaaaa aaaa aaa aaaa
aaaaa aa a aaaaa aa aaaaa aaa aa aa aaa a aa aaaaa aaaa aaaa aaa aaaa
aaaa a aaa aaa aaaaa aaaa aaa aaaaa aaaa aaaaa aaa aaa aaaa aaaaa a aaaa
a aa aaaaa aaa aa a aaaa aaa a a aaa aaaa aaaaa aaaaa a aaaaa aaaa aaaaa
aa aaa aaaa aaaa aaa aa a aaaaa aaaaa aaa aaa aaaaa aa aaaaa aa aaa
aaaaa aaaaa\\
bbb b bbb b bbb bbbbb bbbb b bbbb bbbb bbbbb bbbb bbb bbbbb bb bbbbb bbb
bbbb bbb bb bbbb b bbbbb bbbbb bb bbbb bbbbb bbbb bbbbb bbbb bbb bbbb
bbbbb bb b bbbbb bb bbbbb bbb bb bb bbb b bb bbbbb bbbb bbbb bbb bbbb
bbbb b bbb bbb bbbbb bbbb bbb bbbbb bbbb bbbbb bbb bbb bbbb bbbbb b bbbb
b bb bbbbb bbb bb b bbbb bbb b b bbb bbbb bbbbb bbbbb b bbbbb bbbb bbbbb
bb bbb bbbb bbbb bbb bb b bbbbb bbbbb bbb bbb bbbbb bb bbbbb bb bbb
bbbbb bbbbb


ccc c ccc c ccc ccccc cccc c cccc cccc ccccc cccc ccc ccccc cc ccccc ccc
cccc ccc cc cccc c ccccc ccccc cc cccc ccccc cccc ccccc cccc ccc cccc
ccccc cc c ccccc cc ccccc ccc cc cc ccc c cc ccccc cccc cccc ccc cccc
cccc c ccc ccc ccccc cccc ccc ccccc cccc ccccc ccc ccc cccc ccccc c cccc
c cc ccccc ccc cc c cccc ccc c c ccc cccc ccccc ccccc c ccccc cccc ccccc
cc ccc cccc cccc ccc cc c ccccc ccccc ccc ccc ccccc cc ccccc cc ccc
ccccc ccccc\\
ddd d ddd d ddd ddddd dddd d dddd dddd ddddd dddd ddd ddddd dd ddddd ddd
dddd ddd dd dddd d ddddd ddddd dd dddd ddddd dddd ddddd dddd ddd dddd
ddddd dd d ddddd dd ddddd ddd dd dd ddd d dd ddddd dddd dddd ddd dddd
dddd d ddd ddd ddddd dddd ddd ddddd dddd ddddd ddd ddd dddd ddddd d dddd
d dd ddddd ddd dd d dddd ddd d d ddd dddd ddddd ddddd d ddddd dddd ddddd
dd ddd dddd dddd ddd dd d ddddd ddddd ddd ddd ddddd dd ddddd dd ddd
ddddd ddddd
-----------------------------------------------------------------------

Where I assumed that alineas are separated by either an empty line or a
tailing `\\' while paragraphs are separated by just an empty line.

What you say (making distinct separation of paragraph for motion and of
fillable-region) is more than what I needed, because I assumed that
alineas and paragraphs are not completely separate things, as an alinea
is always a subset of a paragraph.

Concerning EUPP, after more thinking, I realized that alineas are not
the good approach: actually I don't need any special filling but rather
to disable filling in some occasions, and for that using the
fill-paragraph-function is a better approach.

So, for Org I will just install emacs-24 --- or at least the fill
package, is it backward compatible with an emacs-23 ?

Thanks for the feedback,
   Vincent.

Re: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

Hi,

Good to see that I am not the only one needing that sort of things.

Maybe an example is better to explain what I need, imagine that I have
the following two paragraphs:

-----------------------------------------------------------------------
aaa a aaa a aaa aaaaa aaaa a aaaa aaaa aaaaa aaaa aaa aaaaa aa aaaaa aaa aaaa aaa aa aaaa a aaaaa aaaaa aa aaaa aaaaa aaaa aaaaa aaaa aaa aaaa aaaaa aa a aaaaa aa aaaaa aaa aa aa aaa a aa aaaaa aaaa aaaa aaa aaaa aaaa a aaa aaa aaaaa aaaa aaa aaaaa aaaa aaaaa aaa aaa aaaa aaaaa a aaaa a aa aaaaa aaa aa a aaaa aaa a a aaa aaaa aaaaa aaaaa a aaaaa aaaa aaaaa aa aaa aaaa aaaa aaa aa a aaaaa aaaaa aaa aaa aaaaa aa aaaaa aa aaa aaaaa aaaaa\\
bbb b bbb b bbb bbbbb bbbb b bbbb bbbb bbbbb bbbb bbb bbbbb bb bbbbb bbb bbbb bbb bb bbbb b bbbbb bbbbb bb bbbb bbbbb bbbb bbbbb bbbb bbb bbbb bbbbb bb b bbbbb bb bbbbb bbb bb bb bbb b bb bbbbb bbbb bbbb bbb bbbb bbbb b bbb bbb bbbbb bbbb bbb bbbbb bbbb bbbbb bbb bbb bbbb bbbbb b bbbb b bb bbbbb bbb bb b bbbb bbb b b bbb bbbb bbbbb bbbbb b bbbbb bbbb bbbbb bb bbb bbbb bbbb bbb bb b bbbbb bbbbb bbb bbb bbbbb bb bbbbb bb bbb bbbbb bbbbb

ccc c ccc c ccc ccccc cccc c cccc cccc ccccc cccc ccc ccccc cc ccccc ccc cccc ccc cc cccc c ccccc ccccc cc cccc ccccc cccc ccccc cccc ccc cccc ccccc cc c ccccc cc ccccc ccc cc cc ccc c cc ccccc cccc cccc ccc cccc cccc c ccc ccc ccccc cccc ccc ccccc cccc ccccc ccc ccc cccc ccccc c cccc c cc ccccc ccc cc c cccc ccc c c ccc cccc ccccc ccccc c ccccc cccc ccccc cc ccc cccc cccc ccc cc c ccccc ccccc ccc ccc ccccc cc ccccc cc ccc ccccc ccccc\\
ddd d ddd d ddd ddddd dddd d dddd dddd ddddd dddd ddd ddddd dd ddddd ddd dddd ddd dd dddd d ddddd ddddd dd dddd ddddd dddd ddddd dddd ddd dddd ddddd dd d ddddd dd ddddd ddd dd dd ddd d dd ddddd dddd dddd ddd dddd dddd d ddd ddd ddddd dddd ddd ddddd dddd ddddd ddd ddd dddd ddddd d dddd d dd ddddd ddd dd d dddd ddd d d ddd dddd ddddd ddddd d ddddd dddd ddddd dd ddd dddd dddd ddd dd d ddddd ddddd ddd ddd ddddd dd ddddd dd ddd ddddd ddddd
-----------------------------------------------------------------------

after filling I wish that it would become

-----------------------------------------------------------------------
aaa a aaa a aaa aaaaa aaaa a aaaa aaaa aaaaa aaaa aaa aaaaa aa aaaaa aaa
aaaa aaa aa aaaa a aaaaa aaaaa aa aaaa aaaaa aaaa aaaaa aaaa aaa aaaa
aaaaa aa a aaaaa aa aaaaa aaa aa aa aaa a aa aaaaa aaaa aaaa aaa aaaa
aaaa a aaa aaa aaaaa aaaa aaa aaaaa aaaa aaaaa aaa aaa aaaa aaaaa a aaaa
a aa aaaaa aaa aa a aaaa aaa a a aaa aaaa aaaaa aaaaa a aaaaa aaaa aaaaa
aa aaa aaaa aaaa aaa aa a aaaaa aaaaa aaa aaa aaaaa aa aaaaa aa aaa
aaaaa aaaaa\\
bbb b bbb b bbb bbbbb bbbb b bbbb bbbb bbbbb bbbb bbb bbbbb bb bbbbb bbb
bbbb bbb bb bbbb b bbbbb bbbbb bb bbbb bbbbb bbbb bbbbb bbbb bbb bbbb
bbbbb bb b bbbbb bb bbbbb bbb bb bb bbb b bb bbbbb bbbb bbbb bbb bbbb
bbbb b bbb bbb bbbbb bbbb bbb bbbbb bbbb bbbbb bbb bbb bbbb bbbbb b bbbb
b bb bbbbb bbb bb b bbbb bbb b b bbb bbbb bbbbb bbbbb b bbbbb bbbb bbbbb
bb bbb bbbb bbbb bbb bb b bbbbb bbbbb bbb bbb bbbbb bb bbbbb bb bbb
bbbbb bbbbb


ccc c ccc c ccc ccccc cccc c cccc cccc ccccc cccc ccc ccccc cc ccccc ccc
cccc ccc cc cccc c ccccc ccccc cc cccc ccccc cccc ccccc cccc ccc cccc
ccccc cc c ccccc cc ccccc ccc cc cc ccc c cc ccccc cccc cccc ccc cccc
cccc c ccc ccc ccccc cccc ccc ccccc cccc ccccc ccc ccc cccc ccccc c cccc
c cc ccccc ccc cc c cccc ccc c c ccc cccc ccccc ccccc c ccccc cccc ccccc
cc ccc cccc cccc ccc cc c ccccc ccccc ccc ccc ccccc cc ccccc cc ccc
ccccc ccccc\\
ddd d ddd d ddd ddddd dddd d dddd dddd ddddd dddd ddd ddddd dd ddddd ddd
dddd ddd dd dddd d ddddd ddddd dd dddd ddddd dddd ddddd dddd ddd dddd
ddddd dd d ddddd dd ddddd ddd dd dd ddd d dd ddddd dddd dddd ddd dddd
dddd d ddd ddd ddddd dddd ddd ddddd dddd ddddd ddd ddd dddd ddddd d dddd
d dd ddddd ddd dd d dddd ddd d d ddd dddd ddddd ddddd d ddddd dddd ddddd
dd ddd dddd dddd ddd dd d ddddd ddddd ddd ddd ddddd dd ddddd dd ddd
ddddd ddddd
-----------------------------------------------------------------------

Where I assumed that alineas are separated by either an empty line or a
tailing `\\' while paragraphs are separated by just an empty line.

What you say (making distinct separation of paragraph for motion and of
fillable-region) is more than what I needed, because I assumed that
alineas and paragraphs are not completely separate things, as an alinea
is always a subset of a paragraph.

Concerning EUPP, after more thinking, I realized that alineas are not
the good approach: actually I don't need any special filling but rather
to disable filling in some occasions, and for that using the
fill-paragraph-function is a better approach.

So, for Org I will just install emacs-24 --- or at least the fill
package, is it backward compatible with an emacs-23 ?

Thanks for the feedback,
   Vincent.

Re: Alinea filling (hanlding of explicit line-breaks)

[Stefan Monnier]

> Maybe an example is better to explain what I need, imagine that I have
> the following two paragraphs:
[...]
> Where I assumed that alineas are separated by either an empty line or a
> tailing `\\' while paragraphs are separated by just an empty line.

An important question here is: is it important for M-} to ignore those \\?

> What you say (making distinct separation of paragraph for motion and of
> fillable-region) is more than what I needed, because I assumed that
> alineas and paragraphs are not completely separate things, as an alinea
> is always a subset of a paragraph.

Yes, that's usually the case.  But it's more trouble for the code to
enforce that fill-forward-paragraph-function only moves within the
current paragraph, so the extra power comes at no cost and you don't
have to use it.

> Concerning EUPP, after more thinking, I realized that alineas are not
> the good approach: actually I don't need any special filling but rather
> to disable filling in some occasions, and for that using the
> fill-paragraph-function is a better approach.

Of course fill-paragraph-function sucks because it only applies to
fill-paragraph and not to fill-region.

> So, for Org I will just install emacs-24 --- or at least the fill
> package, is it backward compatible with an emacs-23 ?

No idea.  It's likely, but I really can't guarantee it.


        Stefan

Re: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

>> Maybe an example is better to explain what I need, imagine that I have
>> the following two paragraphs:
>[...]
>> Where I assumed that alineas are separated by either an empty line or a
>> tailing `\\' while paragraphs are separated by just an empty line.
> 
>An important question here is: is it important for M-} to ignore those \\?
> 

I guess so, this is the current default Org behaviour anyhow. as far as
I understand, it would be anyhow possible to move point on an
alinea-by-alinea basis just by configuring the tailing \\ as anohter
paragraph separator.

I usually use alinea as follows:

-----------------------------------------------------------------------
John Foo (2011-02-16T07:36:02) said:\\
blah blah blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah

Mikaël Bar (2011-02-15T16:28:16) said:\\
gronk gronk gronk gronk gronk gronk gronk gronk gronk gronk gronk gronk
gronk gronk gronk gronk
-----------------------------------------------------------------------

I don't want M-} to recognize alineas as separate paragraphs because
the first alinea of each paragraph is just one line.

[...]

> Of course fill-paragraph-function sucks because it only applies to
> fill-paragraph and not to fill-region.

Do you mean that `fill-paragraph-function' is some kind of obsolete
feature and that people should use another kind of hook, and if so which
one?

[...]
 
> 
> 
>        Stefan

  Vincent.

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

>> Maybe an example is better to explain what I need, imagine that I have
>> the following two paragraphs:
>[...]
>> Where I assumed that alineas are separated by either an empty line or a
>> tailing `\\' while paragraphs are separated by just an empty line.
> 
>An important question here is: is it important for M-} to ignore those \\?
> 

I guess so, this is the current default Org behaviour anyhow. as far as
I understand, it would be anyhow possible to move point on an
alinea-by-alinea basis just by configuring the tailing \\ as anohter
paragraph separator.

I usually use alinea as follows:

-----------------------------------------------------------------------
John Foo (2011-02-16T07:36:02) said:\\
blah blah blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah blah blah blah blah blah
blah blah blah blah blah blah blah blah blah

Mikaël Bar (2011-02-15T16:28:16) said:\\
gronk gronk gronk gronk gronk gronk gronk gronk gronk gronk gronk gronk
gronk gronk gronk gronk
-----------------------------------------------------------------------

I don't want M-} to recognize alineas as separate paragraphs because
the first alinea of each paragraph is just one line.

[...]

> Of course fill-paragraph-function sucks because it only applies to
> fill-paragraph and not to fill-region.

Do you mean that `fill-paragraph-function' is some kind of obsolete
feature and that people should use another kind of hook, and if so which
one?

[...]
 
> 
> 
>        Stefan

  Vincent.

Re: Alinea filling (hanlding of explicit line-breaks)

[Stefan Monnier]

>> An important question here is: is it important for M-} to ignore those \\?
> I guess so, this is the current default Org behaviour anyhow.

Good.

> as far as I understand, it would be anyhow possible to move point on
> an alinea-by-alinea basis just by configuring the tailing \\ as
> anohter paragraph separator.

Actually, no, because paragraph-separate would cause the whole line that
ends with \\ to be treated as not being part of a paragraph, and
paragraph-start wouldn't be appropriate either.
Hence the "good" above :-(

>> Of course fill-paragraph-function sucks because it only applies to
>> fill-paragraph and not to fill-region.

> Do you mean that `fill-paragraph-function' is some kind of obsolete
> feature and that people should use another kind of hook, and if so
> which one?

Yes and no: there is no replacement for it.  There is
fill-forward-paragraph-function, which can handle some of the cases for
which fill-paragraph-function has been used, but there would need to be
something like a fill-region-as-paragraph-function and we don't have
that yet :-(


        Stefan

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Alinea filling (hanlding of explicit line-breaks)

[Stefan Monnier]

>> An important question here is: is it important for M-} to ignore those \\?
> I guess so, this is the current default Org behaviour anyhow.

Good.

> as far as I understand, it would be anyhow possible to move point on
> an alinea-by-alinea basis just by configuring the tailing \\ as
> anohter paragraph separator.

Actually, no, because paragraph-separate would cause the whole line that
ends with \\ to be treated as not being part of a paragraph, and
paragraph-start wouldn't be appropriate either.
Hence the "good" above :-(

>> Of course fill-paragraph-function sucks because it only applies to
>> fill-paragraph and not to fill-region.

> Do you mean that `fill-paragraph-function' is some kind of obsolete
> feature and that people should use another kind of hook, and if so
> which one?

Yes and no: there is no replacement for it.  There is
fill-forward-paragraph-function, which can handle some of the cases for
which fill-paragraph-function has been used, but there would need to be
something like a fill-region-as-paragraph-function and we don't have
that yet :-(


        Stefan

RE: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

Karl: I put you in the loop for info, because in Texinfo mode I think
that @* is used as an alinea separator similar to \\ in Org mode.

> From: monnier@iro.umontreal.ca
> To: vincent.b.1@hotmail.fr
> Date: Wed, 16 Feb 2011 10:02:37 -0500
> CC: emacs-orgmode@gnu.org; emacs-devel@gnu.org
> Subject: Re: Alinea filling (hanlding of explicit line-breaks)
>
> >> An important question here is: is it important for M-} to ignore those \\?
> > I guess so, this is the current default Org behaviour anyhow.
>
> Good.
>
> > as far as I understand, it would be anyhow possible to move point on
> > an alinea-by-alinea basis just by configuring the tailing \\ as
> > anohter paragraph separator.
>
> Actually, no, because paragraph-separate would cause the whole line
> that ends with \\ to be treated as not being part of a paragraph, and
> paragraph-start wouldn't be appropriate either.  Hence the "good"
> above :-(
>
> >> Of course fill-paragraph-function sucks because it only applies to
> >> fill-paragraph and not to fill-region.
>
> > Do you mean that `fill-paragraph-function' is some kind of obsolete
> > feature and that people should use another kind of hook, and if so
> > which one?
>
> Yes and no: there is no replacement for it. There is
> fill-forward-paragraph-function, which can handle some of the cases
> for which fill-paragraph-function has been used, but there would need
> to be something like a fill-region-as-paragraph-function and we don't
> have that yet :-(
>
>
> Stefan
>
Hello Stefan & al,

I have implemented the thing locally on my machine. It works well but
there is still something missing: the line containing the `\\' alinea
separtor is not filled. 

Here is what I did:

+ In org-set-autofill-regexps function I added
    (org-set-local 'fill-forward-paragraph-function 'org-fill-foward-paragraph)

+ I created the following function:

  (defun org-fill-foward-paragraph (&optional arg)
    (let ((paragraph-separate (concat ".*\\\\$\\|" paragraph-separate )))
      (forward-paragraph arg)    ))

Note that I my 7.01h version of org --- which is older than the latest
one --- org-fill-paragraph is roughly like this:

(defun org-fill-paragraph (&optional justify)
  "Re-align a table, pass through to fill-paragraph if no table."
  (let ((table-p (org-at-table-p))
	(table.el-p (org-at-table.el-p)))
    (cond ((and (equal (char-after (point-at-bol)) ?*)
		(save-excursion (goto-char (point-at-bol))
				(looking-at outline-regexp)))
	   t)					     ; skip headlines
	  (table.el-p t)			     ; skip table.el tables
	  (table-p (org-table-align) t)		     ; align org-mode tables
	  (t nil); let fill.el do the job
)))

That is to say there is a case where org lets fill.el do the job, while
in a later version (e.g. 7.4) org does some "\\\\$" detection on its
own without the use of the fill-forward-paragraph-function.

So, after more thinking about it the problem is the following: the
fill-forward-paragraph has only one parameter which is the paragraph
number --- with n = 0 => current --- but for finding the paragraph
boundary we need *two* parameter

=> 1st argument: paragraph number
=> 2nd argument: whether we want to point at the beginning the paragraph or to the end
   of the paragraph. 

=> maybe paragraph-separate could be a list of 3 items (REGEXP BEG END)
   where REGEXP is the usual regexp matching the separator, and BEG and
   END when non nil are function to go the the beginning of next or to
   the end of previous assuming that the match data corresponds to a
   match of REGEXP. This way would be really the most flexible.


VBR,
   Vincent

RE: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

Karl: I put you in the loop for info, because in Texinfo mode I think
that @* is used as an alinea separator similar to \\ in Org mode.

> From: monnier@iro.umontreal.ca
> To: vincent.b.1@hotmail.fr
> Date: Wed, 16 Feb 2011 10:02:37 -0500
> CC: emacs-orgmode@gnu.org; emacs-devel@gnu.org
> Subject: Re: Alinea filling (hanlding of explicit line-breaks)
>
> >> An important question here is: is it important for M-} to ignore those \\?
> > I guess so, this is the current default Org behaviour anyhow.
>
> Good.
>
> > as far as I understand, it would be anyhow possible to move point on
> > an alinea-by-alinea basis just by configuring the tailing \\ as
> > anohter paragraph separator.
>
> Actually, no, because paragraph-separate would cause the whole line
> that ends with \\ to be treated as not being part of a paragraph, and
> paragraph-start wouldn't be appropriate either.  Hence the "good"
> above :-(
>
> >> Of course fill-paragraph-function sucks because it only applies to
> >> fill-paragraph and not to fill-region.
>
> > Do you mean that `fill-paragraph-function' is some kind of obsolete
> > feature and that people should use another kind of hook, and if so
> > which one?
>
> Yes and no: there is no replacement for it. There is
> fill-forward-paragraph-function, which can handle some of the cases
> for which fill-paragraph-function has been used, but there would need
> to be something like a fill-region-as-paragraph-function and we don't
> have that yet :-(
>
>
> Stefan
>
Hello Stefan & al,

I have implemented the thing locally on my machine. It works well but
there is still something missing: the line containing the `\\' alinea
separtor is not filled. 

Here is what I did:

+ In org-set-autofill-regexps function I added
    (org-set-local 'fill-forward-paragraph-function 'org-fill-foward-paragraph)

+ I created the following function:

  (defun org-fill-foward-paragraph (&optional arg)
    (let ((paragraph-separate (concat ".*\\\\$\\|" paragraph-separate )))
      (forward-paragraph arg)    ))

Note that I my 7.01h version of org --- which is older than the latest
one --- org-fill-paragraph is roughly like this:

(defun org-fill-paragraph (&optional justify)
  "Re-align a table, pass through to fill-paragraph if no table."
  (let ((table-p (org-at-table-p))
	(table.el-p (org-at-table.el-p)))
    (cond ((and (equal (char-after (point-at-bol)) ?*)
		(save-excursion (goto-char (point-at-bol))
				(looking-at outline-regexp)))
	   t)					     ; skip headlines
	  (table.el-p t)			     ; skip table.el tables
	  (table-p (org-table-align) t)		     ; align org-mode tables
	  (t nil); let fill.el do the job
)))

That is to say there is a case where org lets fill.el do the job, while
in a later version (e.g. 7.4) org does some "\\\\$" detection on its
own without the use of the fill-forward-paragraph-function.

So, after more thinking about it the problem is the following: the
fill-forward-paragraph has only one parameter which is the paragraph
number --- with n = 0 => current --- but for finding the paragraph
boundary we need *two* parameter

=> 1st argument: paragraph number
=> 2nd argument: whether we want to point at the beginning the paragraph or to the end
   of the paragraph. 

=> maybe paragraph-separate could be a list of 3 items (REGEXP BEG END)
   where REGEXP is the usual regexp matching the separator, and BEG and
   END when non nil are function to go the the beginning of next or to
   the end of previous assuming that the match data corresponds to a
   match of REGEXP. This way would be really the most flexible.


VBR,
   Vincent

RE: Alinea filling (hanlding of explicit line-breaks)

[Karl Berry]

Hi Vincent and all,

    Karl: I put you in the loop for info, because in Texinfo mode I think
    that @* is used as an alinea separator similar to \\ in Org mode.

1. Sure, @* forces a line break in Texinfo.  How that technically
compares to \\ in org mode, I don't know.

2. The Texinfo support in Emacs, including info and texinfo-mode, etc.,
is maintained by the Emacs developers, not the Texinfo developers.
(After many years of skewed releases, etc., it became clear that was the
better approach.)

Happy hacking,
Karl

Re: Alinea filling (hanlding of explicit line-breaks)

[Stefan Monnier]

>> Actually, no, because paragraph-separate would cause the whole line
>> that ends with \\ to be treated as not being part of a paragraph, and
>> paragraph-start wouldn't be appropriate either.  Hence the "good"
>> above :-(
[...]
> I have implemented the thing locally on my machine.  It works well but
> there is still something missing: the line containing the `\\' alinea
> separtor is not filled.

As you can see above, I'm not surprised.
Just don't use paragraph-separate.  What I'd do is to use
a fill-forward-paragraph-function which calls forward-paragraph, then
searches for a "\\\\\\\\$" between the start and end point, and if found
adjust the end result accordingly.

If/when you come up with this function, please submit for inclusion in
tex-mode.el where it will come in handy as well.

> So, after more thinking about it the problem is the following: the
> fill-forward-paragraph has only one parameter which is the paragraph
> number --- with n = 0 => current --- but for finding the paragraph
> boundary we need *two* parameter

> => 1st argument: paragraph number
> => 2nd argument: whether we want to point at the beginning the
> paragraph or to the end of the paragraph. 

AFAIK the sign of the argument gives you this information.

> => maybe paragraph-separate could be a list of 3 items (REGEXP BEG END)
>    where REGEXP is the usual regexp matching the separator, and BEG and
>    END when non nil are function to go the the beginning of next or to
>    the end of previous assuming that the match data corresponds to a
>    match of REGEXP. This way would be really the most flexible.

Could be, but once you're in fill-forward-paragraph-function, you can do
it by hand with Elisp code, so it's not that important.  If/when we have
enough fill-forward-paragraph-functions we may revisit this opinion, but
I don't think we have enough experience yet to make a good design.


        Stefan

RE: Alinea filling (hanlding of explicit line-breaks)

[Vincent Belaïche]

> Date: Sat, 12 Mar 2011 22:41:02 +0000
> From: karl@freefriends.org
> To: vincent.b.1@hotmail.fr
> CC: monnier@iro.umontreal.ca; emacs-devel@gnu.org
> Subject: RE: Alinea filling (hanlding of explicit line-breaks)
>
[...]
> 1. Sure, @* forces a line break in Texinfo. How that technically
> compares to \\ in org mode, I don't know.
>
[...]

FYI, the only --- but significative --- difference is that in Org \\
needs to be placed at end of line to be active. This is the reason why
it is quite disturbing in Org if the paragraph filling moves \\ from the
end of of line. 

In Texinfo it is rather a matter of taste whether you like it or not
that in the source code @* are also at end of line to look like the
output --- I personally like it this way.

VBR,
   Vincent.

RE: Alinea filling (hanlding of explicit line-breaks)‏

[Vincent Belaïche]

Salut Stéfan,

>>> Actually, no, because paragraph-separate would cause the whole line
>>> that ends with \\ to be treated as not being part of a paragraph, and
>>> paragraph-start wouldn't be appropriate either.  Hence the "good"
>>> above :-(
>[...]
>> I have implemented the thing locally on my machine.  It works well but
>> there is still something missing: the line containing the `\\' alinea
>> separtor is not filled.
> 
>As you can see above, I'm not surprised.
>Just don't use paragraph-separate.  What I'd do is to use
>a fill-forward-paragraph-function which calls forward-paragraph, then
>searches for a "\\\\\\\\$" between the start and end point, and if found
>adjust the end result accordingly.
> 
>If/when you come up with this function, please submit for inclusion in
>tex-mode.el where it will come in handy as well.
> 
>> So, after more thinking about it the problem is the following: the
>> fill-forward-paragraph has only one parameter which is the paragraph
>> number --- with n = 0 => current --- but for finding the paragraph
>> boundary we need *two* parameter
> 
>> => 1st argument: paragraph number
>> => 2nd argument: whether we want to point at the beginning the
>> paragraph or to the end of the paragraph. 
> 
>AFAIK the sign of the argument gives you this information.

Oh, I had missed that. The docstring of forward-paragraph is not very
clear about that.

So, I understand you as follows: basically when you are in the middle of
a paragraph, then -1 and +1 both mean current paragraph, with -1 meaning
beginning of paragraph and and +1 meaning end of paragraph.

But it seems that it does not exactly does the job this way: if the
argument sign is <0 and you are in the middle of a paragraph, then,
unless the paragraph is at beginning of buffer you are going to the
character before the 1st one, rather than to the 1st character of
paragraph.

So, it seems that you are indeed going to and end of paragraph with
arg > 0, but that with arg < 0, the what happens is more fuzzy.

> 
>> => maybe paragraph-separate could be a list of 3 items (REGEXP BEG END)
>>    where REGEXP is the usual regexp matching the separator, and BEG and
>>    END when non nil are function to go the the beginning of next or to
>>    the end of previous assuming that the match data corresponds to a
>>    match of REGEXP. This way would be really the most flexible.
> 
>Could be, but once you're in fill-forward-paragraph-function, you can do
>it by hand with Elisp code, so it's not that important.  If/when we have
>enough fill-forward-paragraph-functions we may revisit this opinion, but
>I don't think we have enough experience yet to make a good design.
> 

You are right, this can be done by implementing
fill-forward-paragraph-function accordingly. My idea was just to use the
same forward-paragraph engine for paragraph motion and for paragraph
filling, so the fill-forward-paragraph-function would just have been
some wrapper to call forward-paragraph with the correct parameters for
filling. 

Now, I am a bit confused about what should be the correct behaviour of
the fill-forward-paragraph-function: is that the following:

- arg < 0: goto beginning of paragraph arg+1, with paragraph 0 = current

- arg > 0 : goto beginning of paragraph arg-1, with paragraph 0 =
            current

Where:

- beginning = point to 1st character of paragraph,
- end = point to next character to last character of paragraph
      (typically the `\n' at the end of paragraph is part of this
      paragraph).

> 
>        Stefan

  Vincent.

Re: Alinea filling (hanlding of explicit line-breaks)‏

[Stefan Monnier]

> But it seems that it does not exactly does the job this way: if the
> argument sign is <0 and you are in the middle of a paragraph, then,
> unless the paragraph is at beginning of buffer you are going to the
> character before the 1st one, rather than to the 1st character of
> paragraph.

Actually, IIRC, it only goes to the previous line if that line is empty.

> So, it seems that you are indeed going to and end of paragraph with
> arg > 0, but that with arg < 0, the what happens is more fuzzy.

Yes, indeed.  But I don't think fill.el depends on this detail of
forward-paragraph's behavior.

> Now, I am a bit confused about what should be the correct behaviour of
> the fill-forward-paragraph-function: is that the following:

> - arg < 0: goto beginning of paragraph arg+1, with paragraph 0 = current

> - arg > 0 : goto beginning of paragraph arg-1, with paragraph 0 =
>             current

I can't remember what forward-paragraph does for a 0 argument, but for
positive arguments it goes to "the next Nth paragraph end" and for
negative argument it goes to the next "Nth paragraph start".
And fill.el only calls that function with values +1 and -1.

More specifically, fill.el mostly does something like:

  (fill-region-as-paragraph (progn (forward-paragraph -1) (point))
                            (progn (forward-paragraph 1) (point)))


-- Stefan