Char 2208 mixed with char 32

Char 2208 mixed with char 32

[Andreas Röhler]

Hi all,

bug happens _not_ with Emacs -Q:

copying indented textes from KMail into Emacs, it
inserts sometimes non-breakable space (2208, #o4240,
#x8a0) where Emacs -Q inserts SPACE.

So a line with three spaces is inserted as
 
Char:   (2208, #o4240, #x8a0, file ...) 
Char: SPC (32, #o40, #x20)
Char:   (2208, #o4240, #x8a0, file ...)

Behaviour was affirmed by Peter Dyballa
<Peter_Dyballa@web.de> ...  Kopie:
help-gnu-emacs@gnu.org

who brought the following to consideration:

,----
| So in future you might have problems finding words when  
| searching for SPC<some word> or when tabifying.
`----

Indeed if we have

   foo
   foo

where the first is translated as reported, whereas the
second is indented by three SPC (32, #o40, #x20)

^[ ]+foo

matches only the second.

That may cause disturbances.

Thanks

Andreas Röhler

In GNU Emacs 23.0.50.1 (i686-pc-linux-gnu, GTK+ Version 2.10.6)
 of 2007-12-09 
Windowing system distributor `The X.Org Foundation', version 11.0.70199902
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: de_DE.UTF-8
  locale-coding-system: utf-8
  default-enable-multibyte-characters: t

Major mode: Englisch

Minor modes in effect:
  auto-completion-mode: t
  csv-field-index-mode: t
  shell-dirtrack-mode: t
  recentf-mode: t
  partial-completion-mode: t
  iswitchb-mode: t
  auto-image-file-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Recent input:

Re: Char 2208 mixed with char 32

[Peter Dyballa]

Am 19.12.2007 um 21:27 schrieb Andreas Röhler:

> bug happens _not_ with Emacs -Q:


That's real bad! Because this indicates that your customisation is  
causing the "translation." And: the right address is given when you  
select the bug report entry from the Help menu ...

Binary partitioning (?) (binäres Teilen) can help to isolate the  
statement. I.e. comment half of ~/.emacs and save. Launch. Same  
behaviour? Then the bug is in the other half. Cure? Then the bug is  
in the commented half. Uncomment one half of the commented half.  
Save. Launch again and decide anew. Until the bug is found.

--
Mit friedvollen Grüßen

   Pete

Alle reden vom Wetter - die Bahn fährt nicht.

Re: Char 2208 mixed with char 32

[Jason Rumney]

Peter Dyballa wrote:
>
> Am 19.12.2007 um 21:27 schrieb Andreas Röhler:
>
>> bug happens _not_ with Emacs -Q:
>
>
> That's real bad! Because this indicates that your customisation is
> causing the "translation." And: the right address is given when you
> select the bug report entry from the Help menu ...
>
> Binary partitioning (?) (binäres Teilen) can help to isolate the
> statement. I.e. comment half of ~/.emacs and save. Launch. Same
> behaviour? Then the bug is in the other half. Cure? Then the bug is in
> the commented half. Uncomment one half of the commented half. Save.
> Launch again and decide anew. Until the bug is found.
Usually (unless the bug is timing sensitive, and happens only when
.emacs is loaded at startup), it is not necessary to comment anything
out, just start emacs with -Q and use eval-region to evaluate parts of
your .emacs.

Re: Char 2208 mixed with char 32

[Andreas Röhler]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

Am Donnerstag, 20. Dezember 2007 22:35 schrieb Jason Rumney:
> Peter Dyballa wrote:
> > Am 19.12.2007 um 21:27 schrieb Andreas Röhler:
> >> bug happens _not_ with Emacs -Q:
> >
> > That's real bad! Because this indicates that your customisation is
> > causing the "translation." And: the right address is given when you
> > select the bug report entry from the Help menu ...
> >
> > Binary partitioning (?) (binäres Teilen) can help to isolate the
> > statement. I.e. comment half of ~/.emacs and save. Launch. Same
> > behaviour? Then the bug is in the other half. Cure? Then the bug is in
> > the commented half. Uncomment one half of the commented half. Save.
> > Launch again and decide anew. Until the bug is found.
>
> Usually (unless the bug is timing sensitive, and happens only when
> .emacs is loaded at startup), it is not necessary to comment anything
> out, just start emacs with -Q and use eval-region to evaluate parts of
> your .emacs.


Thanks all.

AFAIS error occurs if

 '(x-select-enable-clipboard t)

in custom-file is evaluated.

Made some check-functions to watch execution. Code
attached. 

Alternating output of `custom-test' displayed switching from
correct to wrong behaviour:

Char: SPC (32, #o40, #x20) point=1 of 7 (0%) column=0
Char: C-j (10, #o12, #xa) point=2525 of 3883 (65%) <1-2618> column=27
Mark set
Char:   (2208, #o4240, #x8a0, file ...) point=1 of 7 (0%) column=0
Char: C-j (10, #o12, #xa) point=2557 of 3883 (66%) <1-2618> column=32
Mark set
Char:   (2208, #o4240, #x8a0, file ...) point=1 of 7 (0%) column=0
Char: C-j (10, #o12, #xa) point=2608 of 3883 (67%) <1-2618> column=51
Quit


Andreas Röhler

[-- Attachment #2: init-custom-test.el.tgz --]
[-- Type: application/x-tgz, Size: 1487 bytes --]

Re: Char 2208 mixed with char 32

[Andreas Röhler]

Am Donnerstag, 20. Dezember 2007 22:35 schrieb Jason Rumney:
> Peter Dyballa wrote:
> > Am 19.12.2007 um 21:27 schrieb Andreas Röhler:
> >> bug happens _not_ with Emacs -Q:
> >
> > That's real bad! Because this indicates that your customisation is
> > causing the "translation." And: the right address is given when you
> > select the bug report entry from the Help menu ...
> >
> > Binary partitioning (?) (binäres Teilen) can help to isolate the
> > statement. I.e. comment half of ~/.emacs and save. Launch. Same
> > behaviour? Then the bug is in the other half. Cure? Then the bug is in
> > the commented half. Uncomment one half of the commented half. Save.
> > Launch again and decide anew. Until the bug is found.
>
> Usually (unless the bug is timing sensitive, and happens only when
> .emacs is loaded at startup), it is not necessary to comment anything
> out, just start emacs with -Q and use eval-region to evaluate parts of
> your .emacs.

BTW: Bug vanished after toggling off x-select-enable-clipboard; however 
afterwards I can't longer copy text from Emacs into here. So we are 
encircled? :)


Andreas Röhler

Re: Char 2208 mixed with char 32

[Jason Rumney]

Andreas Röhler wrote:

> Thanks all.
>
> AFAIS error occurs if
>
>  '(x-select-enable-clipboard t)
>
> in custom-file is evaluated.
>   

What appears to be happening is that KMail is using KHTML to render
message content (regardless of content-type). To get indenting to work
within KHTML, plain text needs to have some spaces translated to
non-break spaces. A web search shows bug reports for KHTML in 2003 about
this problem - specifically that non-breaking space is translated to
space for the primary selection but not the clipboard, but it was
apparently fixed back then. What version of KMail and KHTML/KDE
(whichever you can get version numbers for) are you using?

Re: Char 2208 mixed with char 32

[Andreas Röhler]

Am Freitag, 21. Dezember 2007 16:11 schrieb Jason Rumney:
> Andreas Röhler wrote:
> > Thanks all.
> >
> > AFAIS error occurs if
> >
> >  '(x-select-enable-clipboard t)
> >
> > in custom-file is evaluated.
>
> What appears to be happening is that KMail is using KHTML to render
> message content (regardless of content-type). To get indenting to work
> within KHTML, plain text needs to have some spaces translated to
> non-break spaces. A web search shows bug reports for KHTML in 2003 about
> this problem - specifically that non-breaking space is translated to
> space for the primary selection but not the clipboard, but it was
> apparently fixed back then. What version of KMail and KHTML/KDE
> (whichever you can get version numbers for) are you using?

KMail 1.9.5 (KDE 3.5.5 "release 45" openSUSE 10.2)

Re: Char 2208 mixed with char 32

[Andreas Röhler]

Am Freitag, 21. Dezember 2007 16:11 schrieb Jason Rumney:
> Andreas Röhler wrote:
> > Thanks all.
> >
> > AFAIS error occurs if
> >
> >  '(x-select-enable-clipboard t)
> >
> > in custom-file is evaluated.
>
> What appears to be happening is that KMail is using KHTML to render
> message content (regardless of content-type). To get indenting to work
> within KHTML, plain text needs to have some spaces translated to
> non-break spaces. A web search shows bug reports for KHTML in 2003 about
> this problem - specifically that non-breaking space is translated to
> space for the primary selection but not the clipboard, but it was
> apparently fixed back then. What version of KMail and KHTML/KDE
> (whichever you can get version numbers for) are you using?

      
It's gone after I deleted entry of 

`x-select-enable-clipboard' from custom-file.

Without that: if it was enabled, inserting from X was
wrong, if disabled, copied string didn't go to
clipboard any more.

From this I wouldn't conclude KDE the reason...

Andreas Röhler

Re: Char 2208 mixed with char 32

[Jason Rumney]

Andreas Röhler wrote:      
> It's gone after I deleted entry of 
>
> `x-select-enable-clipboard' from custom-file.
>
> Without that: if it was enabled, inserting from X was
> wrong, if disabled, copied string didn't go to
> clipboard any more.
>
> From this I wouldn't conclude KDE the reason...
>   

The only thing that changes when you unset x-select-enable-clipboard, is
that copy and yank between Emacs and other applications use the primary
selection mechanism of X rather than the clipboard. So if the results of
yanking differ, then the other application must be putting different
text on the clipboard than what it has in its primary selection. If you
search for [KHTML clipboard 0xA0], you'll find reports to that effect.

Looking at the code for 3.5.5
(http://websvn.kde.org/tags/KDE/3.5.5/kdelibs/khtml/khtml_ext.cpp), the
bug has been fixed in KHTML, but there is an "extensionProxy" mechanism
that allows applications to override KHTML's behaviour, so I wonder if
KMail uses that and therefore does not get the benefit of the bug fix.

Re: Char 2208 mixed with char 32

[Andreas Röhler]

Am Freitag, 21. Dezember 2007 23:08 schrieben Sie:
> Andreas Röhler wrote:
> > It's gone after I deleted entry of
> >
> > `x-select-enable-clipboard' from custom-file.
> >
> > Without that: if it was enabled, inserting from X was
> > wrong, if disabled, copied string didn't go to
> > clipboard any more.
> >
> > From this I wouldn't conclude KDE the reason...
>
> The only thing that changes when you unset x-select-enable-clipboard, is
> that copy and yank between Emacs and other applications use the primary
> selection mechanism of X rather than the clipboard. So if the results of
> yanking differ, then the other application must be putting different
> text on the clipboard than what it has in its primary selection. If you
> search for [KHTML clipboard 0xA0], you'll find reports to that effect.
>
> Looking at the code for 3.5.5
> (http://websvn.kde.org/tags/KDE/3.5.5/kdelibs/khtml/khtml_ext.cpp), the
> bug has been fixed in KHTML, but there is an "extensionProxy" mechanism
> that allows applications to override KHTML's behaviour, so I wonder if
> KMail uses that and therefore does not get the benefit of the bug fix.

Thanks for the hint.

As it works now for me (somehow...), I won't dig further into that matter now.

Just for the curious this link about recent discussions:

http://lists.gnu.org/archive/html/emacs-devel/2007-10/msg00768.html

Should someone else report that kind of problem, we
should go on certainly.

Thanks again

Andreas Röhler