Auto-fill-mode bug: unbreakable space lost

Auto-fill-mode bug: unbreakable space lost

[Gaetan Leurent]

- Steps to reproduce with a recent CVS emacs:

start emacs -Q

switch some buffer to text-mode: M-x text-mode
activate auto-fill-mode: M-x auto-fill-mode

insert the following text:
blahblah blahblah blahblahblah blahblah blahblahblah blahblah blahblah blah
(note that the last space is an unbreakable one)

put the point before the unbreakable space, and insert a normal space

the unbreakable space disappears when the text warps (fill-column should
be 70)


- This behavior seems wrong to me: I think the unbreakable space should
be kept. Besides, it was kept in emacs 21.3.

- With a CVS version from 2005-02-21 with the multi-tty patch, emacs
even crashes after trying a couple of time in a Gnus buffer (but not in
a regular text-mode buffer). This no longer happens with a recent CVS,
but I suspect there is (or was) something really wrong somewhere.

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan Monnier]

> - Steps to reproduce with a recent CVS emacs:
> start emacs -Q

> switch some buffer to text-mode: M-x text-mode
> activate auto-fill-mode: M-x auto-fill-mode

> insert the following text:
> blahblah blahblah blahblahblah blahblah blahblahblah blahblah blahblah blah
> (note that the last space is an unbreakable one)

> put the point before the unbreakable space, and insert a normal space

> the unbreakable space disappears when the text warps (fill-column should
> be 70)


> - This behavior seems wrong to me: I think the unbreakable space should
> be kept. Besides, it was kept in emacs 21.3.

It seems the problem is due to `back-to-indentation' which skips chars with
a "whitespace" syntax and NBSP does have whitespace syntax.

Maybe NBSP shouldn't have whitespace syntax.


        Stefan

Re: Auto-fill-mode bug: unbreakable space lost

[Kevin Rodgers]

Stefan Monnier wrote:
> It seems the problem is due to `back-to-indentation' which skips chars with
> a "whitespace" syntax and NBSP does have whitespace syntax.
> 
> Maybe NBSP shouldn't have whitespace syntax.

Whaaa?  What syntax should it have instead?

Perhaps whitespace is actually a category consisting of
horizontal (space, tab, NBSP) and vertical subcategories
(linefeed, return, formfeed).

-- 
Kevin Rodgers

Re: Auto-fill-mode bug: unbreakable space lost

[David Kastrup]

Kevin Rodgers <ihs_4664@yahoo.com> writes:

> Stefan Monnier wrote:
>> It seems the problem is due to `back-to-indentation' which skips
>> chars with a "whitespace" syntax and NBSP does have whitespace
>> syntax.  Maybe NBSP shouldn't have whitespace syntax.
>
> Whaaa?  What syntax should it have instead?

Word constituent or symbol?  Whitespace is, with regard to filling,
interchangeable, and NBSP certainly does not fit that category.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan]

>> It seems the problem is due to `back-to-indentation' which skips chars with
>> a "whitespace" syntax and NBSP does have whitespace syntax.
>> Maybe NBSP shouldn't have whitespace syntax.
> Whaaa?  What syntax should it have instead?

I guess punctuation.  An NBSP *looks like* whitespace but its intended
behavior is to not be treated like whitespace (at leasst according to this
bug-report).


        Stefan

Re: Auto-fill-mode bug: unbreakable space lost

[Gaetan Leurent]

Stefan wrote on 23 Mar 2005 14:34:51 +0100:

>>> Maybe NBSP shouldn't have whitespace syntax.
>> Whaaa?  What syntax should it have instead?
>
> I guess punctuation.  An NBSP *looks like* whitespace but its intended
> behavior is to not be treated like whitespace (at leasst according to this
> bug-report).

Well, I don't know exactly how NBSP should be handled, and in fact both
solutions could make sense.

The current behavior annoys me because I make the French quotation mark
'«' insert '«  »' and put the point between the NBSP, because french
quotations should be « like this », and if auto-fill mode gets in my way
while I fill the quotation, I loose the NBSP.

Concerning whitespace, a quick 'fgrep '" \t"' -r .' in the emacs source
shows quite lot of place where this is still used to match whitespace
instead of using the whitespace syntax, even in fill.el, so it's a bit
confusing...

-- 
Gaëtan LEURENT

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan Monnier]

> The current behavior annoys me because I make the French quotation mark
> '«' insert '«  »' and put the point between the NBSP, because french
> quotations should be « like this », and if auto-fill mode gets in my way
> while I fill the quotation, I loose the NBSP.

I find NBSP to be a source of too many problems, so I think it should be
kept for those rare cases where the unbreakability of a space isn't obvious
from context.

Try (add-hook 'fill-nobreak-predicate 'fill-french-nobreak-p)
Note that fill-french-nobreak-p is new in Emacs-CVS.
   
> Concerning whitespace, a quick 'fgrep '" \t"' -r .' in the emacs source
> shows quite lot of place where this is still used to match whitespace
> instead of using the whitespace syntax, even in fill.el, so it's a bit
> confusing...

Indeed, it's messy.


        Stefan

Re: Auto-fill-mode bug: unbreakable space lost

[Gaetan Leurent]

Stefan Monnier wrote on 23 Mar 2005 20:57:46 +0100:

> I find NBSP to be a source of too many problems, so I think it should be
> kept for those rare cases where the unbreakability of a space isn't obvious
> from context.
>
> Try (add-hook 'fill-nobreak-predicate 'fill-french-nobreak-p)
> Note that fill-french-nobreak-p is new in Emacs-CVS.

This is not a solution, I can't just put a normal space and rely on
emacs to DTRT. For instance, if I am writing an HTML page, even if a
space is obviously unbreakable, I must put a NBSP. If I am writing some
text together with someone else, I can't assume that his text editor
will DTRT. And even when writing an email, if someone quotes my text and
re-warp it, I don't want those obviously unbreakable spaces to be
broken.

So NBSP is useful, and emacs should handle it properly.

-- 
Gaëtan LEURENT

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan]

> So NBSP is useful, and Emacs should handle it properly.

I had no intention to claim that Emacs's treatment of NBSP shouldn't be
improved.  I was just pointing out some other solution which I find superior
in many cases.  I'd even argue that other tools should do the same.


        Stefan

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan]

> So NBSP is useful, and Emacs should handle it properly.

BTW, could you try:

  (modify-syntax-entry (make-char 'latin-iso8859-1 160) "."
                       (standard-syntax-table))

and/or

  (modify-syntax-entry (make-char 'latin-iso8859-15 160) "."
                       (standard-syntax-table))

and see if it fixes your problem (it should) and also if it has other
undesirable side-effects?


        Stefan

Re: Auto-fill-mode bug: unbreakable space lost

[Juri Linkov]

BTW, now Emacs highlights non-breaking space and soft hyphen only for
two charsets.  Perhaps support for all other charsets should be added
to xdisp.c? I.e. to test for the following characters:

iso8859-2:  0x920 0x92d
iso8859-3:  0x9a0 0x9ad
iso8859-4:  0xa20 0xa2d
iso8859-5:  0xe20 0xe2d
iso8859-6:  0xba0 0xbad
iso8859-7:  0xb20 0xb2d
iso8859-8:  0xc20 0xc2d
iso8859-9:  0xea0 0xead
iso8859-14: 0xfa0 0xfad
iso8859-15: 0xf20 0xf2d

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Auto-fill-mode bug: unbreakable space lost

[Ehud Karni]

On Thu, 24 Mar 2005 23:33:48 +0200, Juri Linkov <juri@jurta.org> wrote:
>
> BTW, now Emacs highlights non-breaking space and soft hyphen only for
> two charsets.  Perhaps support for all other charsets should be added
> to xdisp.c? I.e. to test for the following characters:
>
> iso8859-2:  0x920 0x92d
> iso8859-3:  0x9a0 0x9ad
> iso8859-4:  0xa20 0xa2d
> iso8859-5:  0xe20 0xe2d
> iso8859-6:  0xba0 0xbad
> iso8859-7:  0xb20 0xb2d
> iso8859-8:  0xc20 0xc2d
> iso8859-9:  0xea0 0xead
> iso8859-14: 0xfa0 0xfad
> iso8859-15: 0xf20 0xf2d

I second that. May be we should define the shift space (S-SPC) key to
produce the non-breaking space in the current charset (I did it for
Hebrew - iso8859-8). The definition for non-breaking hyphen (minus)
can not be done this way, because shift hyphen produce underscore (_)
(at least in US/Hebrew keyboard).

Ehud.


--
 Ehud Karni           Tel: +972-3-7966-561  /"\
 Mivtach - Simon      Fax: +972-3-7966-667  \ /  ASCII Ribbon Campaign
 Insurance agencies   (USA) voice mail and   X   Against   HTML   Mail
 http://www.mvs.co.il  FAX:  1-815-5509341  / \
 GnuPG: 98EA398D <http://www.keyserver.net/>    Better Safe Than Sorry

Re: Auto-fill-mode bug: unbreakable space lost

[Andreas Schwab]

"Ehud Karni" <ehud@unix.mvs.co.il> writes:

> I second that. May be we should define the shift space (S-SPC) key to
> produce the non-breaking space in the current charset (I did it for
> Hebrew - iso8859-8). The definition for non-breaking hyphen (minus)
> can not be done this way, because shift hyphen produce underscore (_)
> (at least in US/Hebrew keyboard).

You can use C-x 8 SPC and C-x 8 - to produce these characters, which
together with unify-8859-on-encoding-mode (which is the default) gives you
the right characters for your charset.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan Monnier]

> I second that. May be we should define the shift space (S-SPC) key to
> produce the non-breaking space in the current charset (I did it for
> Hebrew - iso8859-8).

Please don't make it too easy to insert a NBSP, except in text modes.
Subreptitious insertion of NBSP in code is a pain the <beep>.

I just helped someone who suffered from "in C++ every once in a while, the
SPC after << turns into a char \240 which I can't see in Emacs but on which
gcc burps".  It turns out the problem is that to insert a < on his keyboard
he needs to hit Alt-, and he often wasn't careful enough to release the Alt
before hitting the subsequent SPC and Alt-SPC on his keyboard sends an NBSP.

So you can understand I'm not looking forward to Shift-SPC inserting
an NBSP.


        Stefan

Re: Auto-fill-mode bug: unbreakable space lost

[Richard Stallman]

    I second that. May be we should define the shift space (S-SPC) key to
    produce the non-breaking space in the current charset (I did it for
    Hebrew - iso8859-8).

That could be a good idea, but it could also conceivably cause
problems.  Maybe some people type spaces while holding down the shift
key and expect it to be an ordinary space (such as, between words that
are all caps).

I don't know whether this will turn out to be a real problem or not.
I just see that it could be one.

    You can use C-x 8 SPC and C-x 8 - to produce these characters, which
    together with unify-8859-on-encoding-mode (which is the default) gives you
    the right characters for your charset.

I was thinking that C-x 8 is obsolete, but if this is the best way to
do that, maybe it means C-x 8 is not obsolete.  What do people think?

Re: Auto-fill-mode bug: unbreakable space lost

[Miles Bader]

On Sat, 26 Mar 2005 22:52:47 -0500, Richard Stallman <rms@gnu.org> wrote:
>     I second that. May be we should define the shift space (S-SPC) key to
>     produce the non-breaking space in the current charset (I did it for
>     Hebrew - iso8859-8).
> 
> That could be a good idea, but it could also conceivably cause
> problems.  Maybe some people type spaces while holding down the shift
> key and expect it to be an ordinary space (such as, between words that
> are all caps).

Yes, I think it would be a real problem.

I use S-SPC to toggle the X input method, and am _constantly_
triggering it by mistake, even though I'm (obviously) very aware of
the problem and try not to.  As you can imagine this usually happens
when typing a space after entering a capitalized word or shifted
symbol -- my shift-key release reflex is often just a little too slow.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: Auto-fill-mode bug: unbreakable space lost

[Richard Stallman]

    I use S-SPC to toggle the X input method, and am _constantly_
    triggering it by mistake, even though I'm (obviously) very aware of
    the problem and try not to.  As you can imagine this usually happens
    when typing a space after entering a capitalized word or shifted
    symbol -- my shift-key release reflex is often just a little too slow.

Ok, we will not redefine S-SPC.

Re: Auto-fill-mode bug: unbreakable space lost

[David Kastrup]

Miles Bader <snogglethorpe@gmail.com> writes:

> I use S-SPC to toggle the X input method, and am _constantly_
> triggering it by mistake, even though I'm (obviously) very aware of
> the problem and try not to.

And you are English.  In German, _every_ noun is capitalized.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Auto-fill-mode bug: unbreakable space lost

[Miles Bader]

On Sat, 26 Mar 2005 22:52:47 -0500, Richard Stallman <rms@gnu.org> wrote:
> I was thinking that C-x 8 is obsolete, but if this is the best way to
> do that, maybe it means C-x 8 is not obsolete.  What do people think?

Why would C-x 8 be obsolete?  What has replaced it?

The various latin-x input methods are certainly not good replacements,
as they're quite a bit more heavyweight and intrusive (my standard
input method is set to Japanese, and I often use C-x 8 to enter the
odd accented letter or symbol into english text).

-miles
-- 
Do not taunt Happy Fun Ball.

Re: Auto-fill-mode bug: unbreakable space lost

[Richard Stallman]

    Why would C-x 8 be obsolete?  What has replaced it?

Input methods, I would have thought.  I am surprised by the idea that
C-x 8 is better.

Re: Auto-fill-mode bug: unbreakable space lost

[Miles Bader]

On Mon, 28 Mar 2005 11:25:51 -0500, Richard Stallman <rms@gnu.org> wrote:
>     Why would C-x 8 be obsolete?  What has replaced it?
> 
> Input methods, I would have thought.  I am surprised by the idea that
> C-x 8 is better.

Um, quoting the part of my mail that you omitted:

   The various latin-x input methods are certainly not good replacements,
   as they're quite a bit more heavyweight and intrusive (my standard
   input method is set to Japanese, and I often use C-x 8 to enter the
   odd accented letter or symbol into english text).

IOW:  C-x 8 is very nice for _occasional_ use; input-methods are too
heavy-weight if you don't use a latin input method as your regular
input method.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: Auto-fill-mode bug: unbreakable space lost

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sat, 26 Mar 2005 22:52:47 -0500
> Cc: juri@jurta.org, monnier@iro.umontreal.ca, gaetan.leurent@ens.fr,
> 	emacs-devel@gnu.org
> 
> Maybe some people type spaces while holding down the shift key and
> expect it to be an ordinary space (such as, between words that are
> all caps).

Yes, I do this very often, and am annoyed by some programs that
interpret S-SPC differently from SPC.  Please, let's not add Emacs to
the list of these annoying programs.

Re: Auto-fill-mode bug: unbreakable space lost

[Gaetan Leurent]

Richard Stallman wrote on 27 Mar 2005 04:52:47 +0100:

> I was thinking that C-x 8 is obsolete, but if this is the best way to
> do that, maybe it means C-x 8 is not obsolete.  What do people think?

I think C-x 8 is usefull if your keyboard allows you to type most of the
characters you need, but not all. For instance, with a French keyboard
you can type 'à' but not 'á' and this is not worth using an input-method
in emacs.

Well, we don't use 'á' in French, but I sometimes really use C-x 8 to
enter a NBSP or a 'þ' in a smiley when I don't have a Compose Key.

-- 
Gaëtan LEURENT

Re: Auto-fill-mode bug: unbreakable space lost

[Evil Boris]

Richard Stallman <rms@gnu.org> writes:

> I was thinking that C-x 8 is obsolete, but if this is the best way to
> do that, maybe it means C-x 8 is not obsolete.  What do people think?

This is just my opinion, but I find C-x 8 extremely useful.  Most
frequently I use no input method at all (I like the characters I type
to match the characters that I appear on the screen :), and
occasionally have use to cyrillic-translit, say for e-mails.  However,
to insert an occasional ' into a é I use C-x 8.  Is there any nice
alternative to doing this?

--Boris

PS. By the way, should there be some documentation that describes C-x
8 commands, as a group [C-h k only responds to, say "C-x 8 ' a" with
the somewhat useless "á [translated from ...] runs
self-insert-command"]?  Does it exist already?  Where and how would I
find it?  In particular, what encoding does it put the characters
into?  Is it covered by unify-on-{en,de}coding machinery?  How do I
find out?

Re: Auto-fill-mode bug: unbreakable space lost

[Gaetan Leurent]

Evil Boris wrote on 29 Mar 2005 16:51:39 +0200:

> PS. By the way, should there be some documentation that describes C-x
> 8 commands, as a group ?

C-x 8 C-h lists the key translations starting with C-x 8.
It doesn't answer your other questions, though.

-- 
Gaëtan LEURENT

Re: Auto-fill-mode bug: unbreakable space lost

[Richard Stallman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

    PS. By the way, should there be some documentation that describes C-x
    8 commands, as a group [C-h k only responds to, say "C-x 8 ' a" with
    the somewhat useless "á [translated from ...] runs
    self-insert-command"]?

I don't understand those words.

If you mean you want a list of the various possible C-x 8 commands,
you can get that with C-x 8 C-h.

Re: Auto-fill-mode bug: unbreakable space lost

[Richard Stallman]

    BTW, now Emacs highlights non-breaking space and soft hyphen only for
    two charsets.  Perhaps support for all other charsets should be added
    to xdisp.c? I.e. to test for the following characters:

This seems like the right way to go.  Could you do it in some
reasonably clean way?

Re: Auto-fill-mode bug: unbreakable space lost

[EvilborisNet]

Richard Stallman <rms@gnu.org> wrote:

>    PS. By the way, should there be some documentation that describes C-x
>    8 commands, as a group [C-h k only responds to, say "C-x 8 ' a" with
>    the somewhat useless "á [translated from ...] runs
>    self-insert-command"]?
>
>I don't understand those words.
>
>If you mean you want a list of the various possible C-x 8 commands,
>you can get that with C-x 8 C-h.

Ooops. My apologies.  I did not realize that after a long list like this:

key             binding
---             -------

C-x 8 ~     Prefix Command
C-x 8 |     ¦
C-x 8 x     ×
C-x 8 m     µ
C-x 8 u     µ
C-x 8 o     °
C-x 8 c     ¢
C-x 8 `     Prefix Command
...

there is also a table for all the bindings of the form C-x 8 ~ XX then C-x 8 ` XX etc.  Come to think of it, I knew of this feature from using gnus, but have forgotten it.  Sorry.

--Boris

__________________________________________________________________
Switch to Netscape Internet Service.
As low as $9.95 a month -- Sign up today at http://isp.netscape.com/register

Netscape. Just the Net You Need.

New! Netscape Toolbar for Internet Explorer
Search from anywhere on the Web and block those annoying pop-ups.
Download now at http://channels.netscape.com/ns/search/install.jsp

Re: Auto-fill-mode bug: unbreakable space lost

[Stefan Monnier]

Any objection against the patch below?
(if accepted, I'll also apply a similar patch to the other latin-N.el files)


        Stefan


--- latin-1.el	10 mar 2005 16:19:43 -0500	1.5
+++ latin-1.el	04 avr 2005 14:55:53 -0400	
@@ -40,7 +40,7 @@
        (if set-case-syntax-set-multibyte
 	   (- (make-char 'latin-iso8859-1) 128)
 	 0)))
-  (set-case-syntax 160 " " tbl)		;no-break space
+  (set-case-syntax 160 "." tbl)		;no-break space
   (set-case-syntax 161 "." tbl)		;inverted exclamation mark
   (set-case-syntax 162 "w" tbl)		;cent sign
   (set-case-syntax 163 "w" tbl)		;pound sign

Re: Auto-fill-mode bug: unbreakable space lost

[Richard Stallman]

    Any objection against the patch below?

I think it is probably right.