Characters saved mismatch?

Characters saved mismatch?

[Angelo Graziosi]

I notice this in my Emacs master build on MSYS2/MinGW64:

   emacs -Q &
   C-x C-f foo.text RET

Now in foo.text type

foo RET
bar RET

and then save: C-x C-s. In the mini-buffer (and in Messages) there is 
the message:

   Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)

but

   ls -l foo.text
   -rw-r--r-- 1 bingo bongo 10  7 mag 01.07 foo.text

10 bytes! This is right because on Windows, by default, Emacs uses the 
DOS style at the end of a line: CR+LF.

Why then it writes that has written 8 characters? It seems it is 
remembering what by default occurs on GNU/Linux (only LF).. Or not?


Ciao,
   Angelo.

Re: Characters saved mismatch?

[Paul Eggert]

Angelo Graziosi wrote:
>
> Why then it writes that has written 8 characters?

Because it's counting buffer characters, not file system characters. This 
reminds me, these newly-added "(N characters)" messages are unpopular so I just 
now turned them off by default; see Bug#26796.

Re: Characters saved mismatch?

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 428 bytes --]

On Sat, May 6, 2017 at 9:06 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

> Angelo Graziosi wrote:
> >
> > Why then it writes that has written 8 characters?
>
> Because it's counting buffer characters, not file system characters. This
> reminds me, these newly-added "(N characters)" messages are unpopular so I
> just
> now turned them off by default; see Bug#26796.
>

Thanks. I also did not find them useful.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 813 bytes --]

Re: Characters saved mismatch?

[Angelo Graziosi]

Il 07/05/2017 03:06, Paul Eggert ha scritto:
> Angelo Graziosi wrote:
>>
>> Why then it writes that has written 8 characters?
> 
> Because it's counting buffer characters, not file system characters. 

Hmm... but my example (foo.text) has 6 characters if we do not count end 
line:

foo
bar


> This reminds me, these newly-added "(N characters)" messages are 
> unpopular so I just now turned them off by default; see Bug#26796.

Maybe you have to remove them at all..

Re: Characters saved mismatch?

[Stephen Berman]

On Sun, 7 May 2017 10:06:48 +0200 Angelo Graziosi <angelo.graziosi@alice.it> wrote:

> Il 07/05/2017 03:06, Paul Eggert ha scritto:
>> Angelo Graziosi wrote:
>>>
>>> Why then it writes that has written 8 characters?
>>
>> Because it's counting buffer characters, not file system characters. 
>
> Hmm... but my example (foo.text) has 6 characters if we do not count end line:
>
> foo
> bar

That's probably six alphabetic characters plus 2 newlines = 8 characters.

Steve Berman

Re: Characters saved mismatch?

[Yuri Khan]

On Sun, May 7, 2017 at 6:27 AM, Angelo Graziosi
<angelo.graziosi@alice.it> wrote:

>   Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)
>
> but
>
>   ls -l foo.text
>   -rw-r--r-- 1 bingo bongo 10  7 mag 01.07 foo.text
>
> 10 bytes!

You are comparing a character count against a byte count. Well *of
course* they are going to be different. The assumption “one character
is encoded by one byte” has been false for quite a while now.

Re: Characters saved mismatch?

[Angelo Graziosi]

Il 07/05/2017 10:52, Stephen Berman ha scritto:
> On Sun, 7 May 2017 10:06:48 +0200 Angelo Graziosi <angelo.graziosi@alice.it> wrote:
> 
>> Il 07/05/2017 03:06, Paul Eggert ha scritto:
>>> Angelo Graziosi wrote:
>>>>
>>>> Why then it writes that has written 8 characters?
>>>
>>> Because it's counting buffer characters, not file system characters.
>>
>> Hmm... but my example (foo.text) has 6 characters if we do not count end line:
>>
>> foo
>> bar
> 
> That's probably six alphabetic characters plus 2 newlines = 8 characters.

You should reread the thread...

Re: Characters saved mismatch?

[Angelo Graziosi]

Il 07/05/2017 10:56, Yuri Khan ha scritto:
> On Sun, May 7, 2017 at 6:27 AM, Angelo Graziosi
> <angelo.graziosi@alice.it> wrote:
> 
>>    Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)
>>
>> but
>>
>>    ls -l foo.text
>>    -rw-r--r-- 1 bingo bongo 10  7 mag 01.07 foo.text
>>
>> 10 bytes!
> 
> You are comparing a character count against a byte count. Well *of
> course* they are going to be different. The assumption “one character
> is encoded by one byte” has been false for quite a while now.
> 

Someone should explain the meaning of "Wrote ‘c:/msys64/tmp/foo.text’ (8 
characters)".

If it refers to the number of characters, my example contains 6 
characters: f-o-o-b-a-r and not 8.

As I wrote, in Windows Emacs uses DOS style, more precisely 'utf-8-dos'. 
That should mean 1 byte/ch and CR+LF for end line (RET). This mean that

foo RET
bar RET

should contain (3+2) * 2 = 10 bytes as, 'ls' shows..

Then, where does "Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)" came 
from, on Windows?

Re: Characters saved mismatch?

[Yuri Khan]

On Sun, May 7, 2017 at 4:15 PM, Angelo Graziosi
<angelo.graziosi@alice.it> wrote:

> Then, where does "Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)" came from,
> on Windows?

A line separator counts as a single character. In the utf-8-dos
encoding, it is encoded as two bytes.

You’re not going to suggest that spaces between words shouldn’t count
as characters, or are you?

Re: Characters saved mismatch?

[Angelo Graziosi]

Il 07/05/2017 11:25, Yuri Khan ha scritto:
> On Sun, May 7, 2017 at 4:15 PM, Angelo Graziosi
> <angelo.graziosi@alice.it> wrote:
> 
>> Then, where does "Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)" came from,
>> on Windows?
> 
> A line separator counts as a single character. In the utf-8-dos
> encoding, it is encoded as two bytes.

You are just repeating what I write...

> 
> You’re not going to suggest that spaces between words shouldn’t count
> as characters, or are you?

Where I wrote about spaces?

BTW, why spaces should not count? Why Emacs has the command 'M-x 
delete-trailing-whitespace'? or the hook

(add-hook 'before-save-hook 'delete-trailing-whitespace)

?

Try this in a empty buffer and then save:

foo SPC SPC bar RET

both in DOS and in UNIX style (utf-8-dos, utf-8-unix), and then count...

Re: Characters saved mismatch?

[Yuri Khan]

On Sun, May 7, 2017 at 5:50 PM, Angelo Graziosi
<angelo.graziosi@alice.it> wrote:
>
> Il 07/05/2017 11:25, Yuri Khan ha scritto:
>>
>> On Sun, May 7, 2017 at 4:15 PM, Angelo Graziosi
>> <angelo.graziosi@alice.it> wrote:
>>
>>> Then, where does "Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)" came
>>> from, on Windows?
>>
>> A line separator counts as a single character. In the utf-8-dos
>> encoding, it is encoded as two bytes.
>
> You are just repeating what I write...

So why does it surprise you that the count of characters differs from
the count of bytes?

>> You’re not going to suggest that spaces between words shouldn’t count
>> as characters, or are you?
>
> Where I wrote about spaces?

You didn’t. But I got the impression you are suggesting line breaks do
not count as characters, and spaces are one step further in that
direction.

Re: Characters saved mismatch?

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On Sun, May 7, 2017, 5:16 AM Angelo Graziosi <angelo.graziosi@alice.it>
wrote:

>
>
> Someone should explain the meaning of "Wrote ‘c:/msys64/tmp/foo.text’ (8
> characters)".
>

As others stated, that's because each newline is counted as 1 character
too.

If it refers to the number of characters, my example contains 6
> characters: f-o-o-b-a-r and not 8.
>

No, it contains 8 characters:

1. f
2. o
3. o
4. Newline (Just 1 character, does not matter if it is 1 byte on unix or 2
bytes on Windows. This is character count, not byte count.)
5. b
6. a
7. r
8. Newline

As I wrote, in Windows Emacs uses DOS style, more precisely 'utf-8-dos'.
> That should mean 1 byte/ch and CR+LF for end line (RET). This mean that
>
> foo RET
> bar RET
>
> should contain (3+2) * 2 = 10 bytes as, 'ls' shows..
>

As written about emacs sees the newline as just 1 character. Emacs is
printing character count, while ls is printing byte count, and thus the
difference.

Visualize that newline character as just 1 character as the "\n" used in
regexps to match newlines.

Then, where does "Wrote ‘c:/msys64/tmp/foo.text’ (8 characters)" came
> from, on Windows?
>

As explained above.

> --

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 2437 bytes --]

Re: Characters saved mismatch?

[Angelo Graziosi]

Il 07/05/2017 14:46, Kaushal Modi ha scritto:
> 
> As written about emacs sees the newline as just 1 character. 

But all this is confusing when the end line is utf-8-dos...

Re: Characters saved mismatch?

[Stefan Monnier]

>> As written about emacs sees the newline as just 1 character. 
> But all this is confusing when the end line is utf-8-dos...

What's confusing about it?  The line separator is a character, just like
the non-breaking space is a character.  The coding system then encodes
those as some number of bytes, typically 1 or 2 (e.g. using utf-8-dos,
they will both use up 2 bytes, IIRC, whereas with latin-1-unix they'd
both use up a single byte, and of course you get other combinations
with latin-1-dos and utf-8-unix, and I'll let you check what can happen
with things like iso-2022-7bit).


        Stefan