Fwd: Line endings bug

Fwd: Line endings bug

[Seiji Zenitani]

Hi,

I received the following fix to the mac port.  Although I have not  
yet met troubles regarding CR/LF, I think it makes sense to modify  
utf-16be/16le-mac to utf-16be/16le in the last two lines in mac- 
string-to-utxt, term/mac-win.el.  The default eol character seems to  
be 'unix and I hear OS X's internal code is utf-16 + LF (unix).   
Could you please examine it?

==
Seiji Zenitani
zenitani@gmail.com


Begin forwarded message:
>
> The function mac-string-to-utxt (inside term/mac-win.el) is used to  
> encode text which is then placed onto the system clipboard, It  
> currently uses the encodings utf-16be-mac and/or utf-16le-mac which  
> replace all \n characters with \r (0x0A with 0x0D), You can add the  
> following to .emacs to fix this and preserve the correct newlines  
> in the buffer/file:
>
> (defun mac-string-to-utxt (string &optional coding-system)
>  (or coding-system (setq coding-system mac-system-coding-system))
>  (let (data encoding)
>    (when (and (fboundp 'mac-code-convert-string)
>               (memq (coding-system-base coding-system)
>                     (find-coding-systems-string string)))
>      (setq coding-system
>            (coding-system-change-eol-conversion coding-system 'unix))
>      (let ((str string))
>        (when (and (eq system-type 'darwin)
>                   (eq coding-system 'japanese-shift-jis-mac))
>          (setq encoding mac-text-encoding-mac-japanese-basic-variant)
>          (setq str (subst-char-in-string ?\\ ?\x80 str))
>          (subst-char-in-string ?\\ ?\x5c str t)
>          ;; ASCII-only?
>          (if (string-match "\\`[\x00-\x7f]*\\'" str)
>              (setq str nil)))
>        (and str
>             (setq data (mac-code-convert-string
>                         (encode-coding-string str coding-system)
>                         (or encoding coding-system) nil)))))
>    (or data (encode-coding-string string (if (eq (byteorder) ?B)
>                                              'utf-16be
>                                            'utf-16le)))))
>
> The only change is in the last two lines, removal of -mac in each.

Re: Fwd: Line endings bug

[YAMAMOTO Mitsuharu]

>>>>> On Wed, 16 May 2007 21:01:25 -0400, Seiji Zenitani <zenitani@gmail.com> said:

> I received the following fix to the mac port.  Although I have not
> yet met troubles regarding CR/LF, I think it makes sense to modify
> utf-16be/16le-mac to utf-16be/16le in the last two lines in
> mac-string-to-utxt, term/mac-win.el.

I'm not quite sure what bug is fixed by that change.  What was OP's
concrete problem?

> The default eol character seems to be 'unix and I hear OS X's
> internal code is utf-16 + LF (unix).  Could you please examine it?

At least, Safari and Apple Mail use CR when they put multi-line
data to the clipboard.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Line endings bug

[Seiji Zenitani]

On 2007/05/16, at 22:54, YAMAMOTO Mitsuharu wrote:

>>>>>> On Wed, 16 May 2007 21:01:25 -0400, Seiji Zenitani  
>>>>>> <zenitani@gmail.com> said:
>
>> I received the following fix to the mac port.  Although I have not
>> yet met troubles regarding CR/LF, I think it makes sense to modify
>> utf-16be/16le-mac to utf-16be/16le in the last two lines in
>> mac-string-to-utxt, term/mac-win.el.
>
> I'm not quite sure what bug is fixed by that change.  What was OP's
> concrete problem?
>
I'm sorry but I lost contact with him.

>> The default eol character seems to be 'unix and I hear OS X's
>> internal code is utf-16 + LF (unix).  Could you please examine it?
>
> At least, Safari and Apple Mail use CR when they put multi-line
> data to the clipboard.

How to confirm this?

When I copy text from the Safari/Mail.app and then type the following  
command,
the text file "sample.txt" has LF, not CR.

  $ pbpaste > sample.txt
  $ hexdump sample.txt

0000000 0a41 0a42 0a43
0000006

In this case, the text is as follows:

A
B
C

I use OSX 10.4.9 on Intel Mac Book Pro.


Seiji

Re: Line endings bug

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 21 May 2007 22:34:30 -0400, Seiji Zenitani <zenitani@gmail.com> said:

>> I'm not quite sure what bug is fixed by that change.  What was OP's
>> concrete problem?
>> 
> I'm sorry but I lost contact with him.

Hmm, how can we incorporate that code in the case that it is correct?

>>> The default eol character seems to be 'unix and I hear OS X's
>>> internal code is utf-16 + LF (unix).  Could you please examine it?
>> 
>> At least, Safari and Apple Mail use CR when they put multi-line
>> data to the clipboard.

> How to confirm this?

(x-get-selection-internal 'CLIPBOARD 'public.utf16-plain-text)

This gives us raw byte data without any conversions.

> When I copy text from the Safari/Mail.app and then type the
> following command, the text file "sample.txt" has LF, not CR.

>   $ pbpaste > sample.txt
>   $ hexdump sample.txt

I can see the difference in the results of pbpaste between Safari and
Emacs.  As both Safari and pbpaste are Cocoa apps, some hidden
information might be used to determine the actual EOL type.  But I
think that the current behavior of Emacs is correct as a Carbon app.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp