bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Geoff Shannon]

[-- Attachment #1.1: Type: text/plain, Size: 3218 bytes --]

When the final character on a line is a period, auto filling both with
do-auto-fill and with fill-paragraph incorrectly moves the entire word
preceding the period to the next line.  This is not the case when it is
a letter in the fill-column.

To reproduce, `emacs -Q` Then load up this text in a buffer, and turn
auto-fill-mode on, and assuming that fill-column is set to 70.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusm

Both lines are the same except for the final character.  If we type a
space at the end of both lines, the one word with a trailing period is
(incorrectly) moved to the next line, while the other does nothing.

This behaviour doesn't seem to be tied to the value of fill-column (I
tried it with a few 5 and 70).

I've attached a dribble file, starting from 'emacs -Q' that illustrates the
filling issues I'm talking about.

Also, it appears that setting sentence-end-double-space changes the
behaviour to what I would expect.  However, in terms of the aesthetic
result of the fill, I think the current behaviour is still incorrect.

In GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.6.4)
 of 2014-01-01 on minty-dark-tower
Bzr revision: 115828 eggert@cs.ucla.edu-20140101231359-pzcy6lepssiaboye
Windowing system distributor `The X.Org Foundation', version 11.0.11303000
System Description:    Linux Mint 15 Olivia

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x b <return> C-e C-a C-x b <return> C-SPC C-e M-w
M-x r e p o r t <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils time-date tooltip electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

[-- Attachment #1.2: Type: text/html, Size: 3673 bytes --]

[-- Attachment #2: filldribble --]
[-- Type: application/octet-stream, Size: 166 bytes --]

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Glenn Morris]

Geoff Shannon wrote:

> When the final character on a line is a period, auto filling both with
> do-auto-fill and with fill-paragraph incorrectly moves the entire word
> preceding the period to the next line.  This is not the case when it is
> a letter in the fill-column.
>
> To reproduce, `emacs -Q` Then load up this text in a buffer, and turn
> auto-fill-mode on, and assuming that fill-column is set to 70.
>
> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.
>
> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusm
>
> Both lines are the same except for the final character.  If we type a
> space at the end of both lines, the one word with a trailing period is
> (incorrectly) moved to the next line, while the other does nothing.
[...]
> Also, it appears that setting sentence-end-double-space changes the
> behaviour to what I would expect.  However, in terms of the aesthetic
> result of the fill, I think the current behaviour is still incorrect.

I believe the reason for this is explained in a comment in the
definition of fill-nobreak-p:

     ;; Don't break after a period followed by just one space.
     ;; Move back to the previous place to break.
     ;; The reason is that if a period ends up at the end of a
     ;; line, further fills will assume it ends a sentence.
     ;; If we now know it does not end a sentence, avoid putting
     ;; it at the end of the line.

So it sounds like it is not possible to get the behaviour that you would
prefer.

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Glenn Morris]

PS don't you think saying that fill-paragraph "fails" is a bit strong? :)

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Geoff Shannon]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]

Definitely too strong, but I couldn't think of a better and equally concise
way to phrase it. :)

Thanks for the pointer to that function. Unless I'm misunderstanding
though, when the period does end a sentence it should *not* go back a word,
correct?  The original context I noticed this behavior in is writing text
in markdown mode.

It's definitely not the end of the world but it is sort of unsightly when
one line is ten characters shorter than the rest of the paragraph...  I can
live with simply customizing sentence-end-double-space to get the behavior
I want.
 On May 2, 2014 5:49 PM, "Glenn Morris" <rgm@gnu.org> wrote:

>
> PS don't you think saying that fill-paragraph "fails" is a bit strong? :)
>

[-- Attachment #2: Type: text/html, Size: 1073 bytes --]

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Stefan Monnier]

>      ;; Don't break after a period followed by just one space.
>      ;; Move back to the previous place to break.
>      ;; The reason is that if a period ends up at the end of a
>      ;; line, further fills will assume it ends a sentence.
>      ;; If we now know it does not end a sentence, avoid putting
>      ;; it at the end of the line.

It's likely that this is causing the problem, but I'm not convinced that
it justifies the current behavior.  IOW, I think it's just a bug.


        Stefan

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>      ;; Don't break after a period followed by just one space.
>>      ;; Move back to the previous place to break.
>>      ;; The reason is that if a period ends up at the end of a
>>      ;; line, further fills will assume it ends a sentence.
>>      ;; If we now know it does not end a sentence, avoid putting
>>      ;; it at the end of the line.
>
> It's likely that this is causing the problem, but I'm not convinced that
> it justifies the current behavior.  IOW, I think it's just a bug.

Ah, this explains the weird behaviour I see from time to time.

If I have

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.|

then his a space, I get

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
eius. |

Then I type on

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
eius.  Ut enim ad|

And then hitting `M-q' gives me

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.
Ut enim ad|

Anybody against changing this odd behaviour in `fill-nobreak-p'?  It's
not quite clear what the overall repercussions would be, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>>      ;; Don't break after a period followed by just one space.
>>>      ;; Move back to the previous place to break.
>>>      ;; The reason is that if a period ends up at the end of a
>>>      ;; line, further fills will assume it ends a sentence.
>>>      ;; If we now know it does not end a sentence, avoid putting
>>>      ;; it at the end of the line.
>>
>> It's likely that this is causing the problem, but I'm not convinced that
>> it justifies the current behavior.  IOW, I think it's just a bug.
>
> Ah, this explains the weird behaviour I see from time to time.
>
> If I have
>
> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.|
>
> then his a space, I get
>
> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
> eius. |
>
> Then I type on
>
> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
> eius.  Ut enim ad|
>
> And then hitting `M-q' gives me
>
> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.
> Ut enim ad|
>
> Anybody against changing this odd behaviour in `fill-nobreak-p'?  It's
> not quite clear what the overall repercussions would be, though.

No complaints here.  :-)

The most annoying thing is that it does this:

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
    eius.  Ut enim ad|

Pressing M-q leads to:

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.
    Ut enim ad|

Pressing M-q again leads to:

    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
    eius. Ut enim ad|

It would IMO be really good if we could avoid this.

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> The most annoying thing is that it does this:
>
>     Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
>     eius.  Ut enim ad|
>
> Pressing M-q leads to:
>
>     Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.
>     Ut enim ad|
>
> Pressing M-q again leads to:
>
>     Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
>     eius. Ut enim ad|

I'm not able to reproduce that -- `M-q' is "stable" for me (i.e., it
remains filled the first way).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Kangas <stefan@marxist.se> writes:
>
>> The most annoying thing is that it does this:
>>
>>     Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
>>     eius.  Ut enim ad|
>>
>> Pressing M-q leads to:
>>
>>     Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eius.
>>     Ut enim ad|
>>
>> Pressing M-q again leads to:
>>
>>     Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do
>>     eius. Ut enim ad|
>
> I'm not able to reproduce that -- `M-q' is "stable" for me (i.e., it
> remains filled the first way).

It turns out that if you are looking for this buggy behavior, you really
do need to go out of your way:

    (setq sentence-end "\\.  ?")

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

>> Anybody against changing this odd behaviour in `fill-nobreak-p'?  It's
>> not quite clear what the overall repercussions would be, though.
>
> No complaints here.  :-)

I've now done this on the trunk.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Kangas <stefan@marxist.se> writes:
>
>>> Anybody against changing this odd behaviour in `fill-nobreak-p'?  It's
>>> not quite clear what the overall repercussions would be, though.
>>
>> No complaints here.  :-)
>
> I've now done this on the trunk.

I guess this can be closed?

bug#17321: 24.3.50; Fill paragraph fails with period in fill-column

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

>> I've now done this on the trunk.
>
> I guess this can be closed?

Hm, I guess I hit the wrong button -- "fixed" instead of "close"?  So
I'm now closing the report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no