bug#47857: fill-paragraph vs. FULLWIDTH DIGITs: bug exposed

bug#47857: fill-paragraph vs. FULLWIDTH DIGITs: bug exposed

[積丹尼 Dan Jacobson]

Disaster! M-q runs the command fill-paragraph

And turns:

Today's lucky number(s)!:
９
１０
１１

Into:

Today's lucky number(s)!:９１０１１

OK, if it is so smart, then why doesn't it also turn:

Today's lucky number(s)!: ９ １０ １１

into:

Today's lucky number(s)!:９１０１１

Well, surprise, it was actually very happy with ９ １０ １１ all along.

OK, let's add some space:
Today's lucky number(s)!: ９         １０ １１
becomes:
Today's lucky number(s)!: ９ １０ １１
perfect.

Anyway if it were stable in its output, then one would always get the same
results. Bug exposed!

(Yes, even LC_ALL isn't going to help. And yes, all this works fine if
all this was good old ASCII. emacs-version "27.1".)

bug#47857: fill-paragraph vs. FULLWIDTH DIGITs: bug exposed

[Eli Zaretskii]

> From: 積丹尼 Dan Jacobson
>  <jidanni@jidanni.org>
> Date: Sun, 18 Apr 2021 09:06:55 +0800
> 
> Disaster! M-q runs the command fill-paragraph
> 
> And turns:
> 
> Today's lucky number(s)!:
> ９
> １０
> １１
> 
> Into:
> 
> Today's lucky number(s)!:９１０１１
> 
> OK, if it is so smart, then why doesn't it also turn:
> 
> Today's lucky number(s)!: ９ １０ １１
> 
> into:
> 
> Today's lucky number(s)!:９１０１１
> 
> Well, surprise, it was actually very happy with ９ １０ １１ all along.
> 
> OK, let's add some space:
> Today's lucky number(s)!: ９         １０ １１
> becomes:
> Today's lucky number(s)!: ９ １０ １１
> perfect.
> 
> Anyway if it were stable in its output, then one would always get the same
> results. Bug exposed!

I don't see any bug here.  This is the documented behavior: filling
deletes any excess whitespace, so 2 or more consecutive SPC characters
are squeezed to a single SPC.  All the rest is a typical jidanni-style
hysteria.

bug#47857: fill-paragraph vs. FULLWIDTH DIGITs: bug exposed

[Stefan Kangas]

tags 47857 notabug
close 47857
thanks

Eli Zaretskii <eliz@gnu.org> writes:

> I don't see any bug here.  This is the documented behavior: filling
> deletes any excess whitespace, so 2 or more consecutive SPC characters
> are squeezed to a single SPC.  All the rest is a typical jidanni-style
> hysteria.

I'm therefore closing this bug.

bug#47857: fill-paragraph vs. FULLWIDTH DIGITs: bug exposed

[積丹尼 Dan Jacobson]

EZ> I don't see any bug here.  This is the documented behavior: filling
EZ> deletes any excess whitespace, so 2 or more consecutive SPC characters
EZ> are squeezed to a single SPC.

Ah, but one needs

Today's lucky number(s)!: <-added space here
９ <-added space here
１０ <-added space here
１１

to keep the whitespace in with fill-paragraph, whereas no spaces are
required for

Today's lucky number(s)!:
9
10
11

To safely become:

Today's lucky number(s)!: 9 10 11

(Also the value of enable-kinsoku has no effect here or in #47856,
which makes sense.)

bug#47857: fill-paragraph vs. FULLWIDTH DIGITs: bug exposed

[Eli Zaretskii]

> From: 積丹尼 Dan Jacobson <jidanni@jidanni.org>
> Cc: 47857@debbugs.gnu.org
> Date: Tue, 20 Apr 2021 07:27:05 +0800
> 
> EZ> I don't see any bug here.  This is the documented behavior: filling
> EZ> deletes any excess whitespace, so 2 or more consecutive SPC characters
> EZ> are squeezed to a single SPC.
> 
> Ah, but one needs
> 
> Today's lucky number(s)!: <-added space here
> ９ <-added space here
> １０ <-added space here
> １１
> 
> to keep the whitespace in with fill-paragraph, whereas no spaces are
> required for
> 
> Today's lucky number(s)!:
> 9
> 10
> 11
> 
> To safely become:
> 
> Today's lucky number(s)!: 9 10 11

This is a feature: full-width Latin characters, such as ０, １, and ９,
are exempted from converting newlines into spaces during filling.
They follow the rules of other CJK characters, which are processed the
same.  I think it's because of line-breaking rules: we can break the
line between any 2 CJK characters, including those above, and thus
adding spaces where there were newlines is superfluous, as it would
artificially create separate words where only one has been before.