Re: [HELP] (bug?) Saving a buffer without any conversion?

[Kenichi Handa <handa@m17n.org>]

In article <87fzrxszs8.fsf@lexx.delysid.org>, Mario Lang <mlang@delysid.org> writes:
> We're receiving binary content via a network process.  After the
> transfer is complete, this buffer should be saved to a file.

> The effect I'm having is that we receive 1372422 bytes via the process
> filter function STRING argument, and after insertion into a buffer,
> we have a buffer with buffer-size 1372422, but after calling (save-buffer)
> we get this:

> -rw-r--r--    1 root     root      1865264 Jan 13 18:35 blah28.mp3

> I'm using:

>       (set-process-coding-system proc 'binary 'binary)
>       (set-buffer-file-coding-system 'no-conversion t)

storm@cua.dk (Kim F. Storm) writes:
> I have looked at Mario's data before sending it to emacs and after
> emacs has written it to a file.

> It seems that every byte in the range 0xa0 .. 0xff that were in the
> original file is prefixed with an 0x81 byte in the file containing the
> received data.  To me, that looks like the internal multi-byte
> representation for the binary data.

No.  0x81 means that 0xA0..0xFF are decoded as Latin-1
chars.  That's why raw-text and no-conversion write out 0x81
as is to a file.  And that means that somehow:
	(set-process-coding-system proc 'binary 'binary)
didn't take effect.  When did you execute this function?  It
should be before accepting any data from the process
(usually just after start-process or open-network-stream).

I tried the follwoing code and the written file "temp" was
the same as "temp.png".

(defun temp-sentinel (proc str)
  (if (string= str "finished\n")
      (save-excursion
	(set-buffer (process-buffer proc))
	(write-file "~/temp"))))

(let (proc)
  (save-excursion
    (set-buffer (get-buffer-create "temp"))
    (set-buffer-file-coding-system 'binary)
    (erase-buffer))
  (setq proc (start-process "cat" "temp" "cat" "/home/handa/temp.png"))
  (set-process-sentinel proc 'temp-sentinel)
  (set-process-coding-system proc 'binary 'binary))

Eli Zaretskii <eliz@is.elta.co.il> writes:
>>  It seems that every byte in the range 0xa0 .. 0xff that were in the
>>  original file is prefixed with an 0x81 byte in the file containing the
>>  received data.  To me, that looks like the internal multi-byte
>>  representation for the binary data.

> Yes.  That's what no-conversion does: it prevents encoding of the 
> internal buffer's contents.

> I suggest to use raw-text for both coding systems above, and see if that 
> helps.

The difference of no-conversion and raw-text is only in
handling of EOL format.  He should surely use no-conversion
because raw-text will convert both CRLF and LF into LF.

> An alternative approach is to (set-buffer-multibyte nil) before reading 
> the data into it and before saving it.

Yes.  For instance, by slightly modifying the above code as below:

(let (proc)
  (save-excursion
    (set-buffer (get-buffer-create "temp"))
    (set-buffer-file-coding-system 'binary)
    (erase-buffer)
    (set-buffer-multibyte nil))
  (setq proc (start-process "cat" "temp" "cat" "/home/handa/temp.png"))
  (set-process-sentinel proc 'temp-sentinel)
  (set-process-coding-system proc 'binary 'binary))

we get the same result more efficiently.

>>  The buffer's coding system for save is no-conversion.  How did
>>  that internal data end up in the file?

> Probably because the buffer was a multibyte buffer, in which case 
> no-conversion writes out the internal representation.  That's why I 
> suggested using raw-text to save the buffer.

Please note that the internal representation for raw-bytes
(eight-bit-control and eight-bit-graphic) are never exposed
in a file even by no-conversion.  As I wrote above, 0x81 is
not a leading-byte for raw-bytes but for Latin-1.

---
Ken'ichi HANDA
handa@m17n.org