numbers instead of accented letters...

numbers instead of accented letters...

[Jean-Christophe Helary]

What have I possibly done to have my .emacs.el display number sequences instead of accented letters ?

I just noticed that a few days ago...

for ex:

;; pour h\303\251riter du $PATH quand Emacs est lanc\303\251 hors de la ligne de commande

Also, it is not only a display issue, when I type such accented sequences I can only see the number sequences...

Other files (normal text files, etc.) are not affected by this behavior.

To make sure that I use UTF-8 everywhere I have this code:

;; UTF-8
(setq locale-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(prefer-coding-system 'utf-8)
(when (display-graphic-p)
  (setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING)))

It must be something trivial but I have no idea where to start.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: numbers instead of accented letters...

[Eli Zaretskii]

> From: Jean-Christophe Helary <brandelune@gmail.com>
> Date: Sun, 25 Nov 2018 11:55:02 +0900
> 
> What have I possibly done to have my .emacs.el display number sequences instead of accented letters ?
> 
> I just noticed that a few days ago...
> 
> for ex:
> 
> ;; pour h\303\251riter du $PATH quand Emacs est lanc\303\251 hors de la ligne de commande
> 
> Also, it is not only a display issue, when I type such accented sequences I can only see the number sequences...
> 
> Other files (normal text files, etc.) are not affected by this behavior.
> 
> To make sure that I use UTF-8 everywhere I have this code:
> 
> ;; UTF-8
> (setq locale-coding-system 'utf-8)
> (set-terminal-coding-system 'utf-8)
> (set-keyboard-coding-system 'utf-8)
> (set-selection-coding-system 'utf-8)
> (prefer-coding-system 'utf-8)
> (when (display-graphic-p)
>   (setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING)))

On what OS is that?

Also, what other non-ASCII characters do you have in your .emacs.el?
(You can search for them with "N-: (skip-chars-forward "\000-\177") RET.)

And if you remove all of the above settings, does the problem go away?

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

> On Nov 25, 2018, at 12:38, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Jean-Christophe Helary <brandelune@gmail.com>
>> Date: Sun, 25 Nov 2018 11:55:02 +0900
>> 
>> What have I possibly done to have my .emacs.el display number sequences instead of accented letters ?
>> 
>> I just noticed that a few days ago...
>> 
>> for ex:
>> 
>> ;; pour h\303\251riter du $PATH quand Emacs est lanc\303\251 hors de la ligne de commande
>> 
>> Also, it is not only a display issue, when I type such accented sequences I can only see the number sequences...
>> 
>> Other files (normal text files, etc.) are not affected by this behavior.
>> 
>> To make sure that I use UTF-8 everywhere I have this code:
>> 
>> ;; UTF-8
>> (setq locale-coding-system 'utf-8)
>> (set-terminal-coding-system 'utf-8)
>> (set-keyboard-coding-system 'utf-8)
>> (set-selection-coding-system 'utf-8)
>> (prefer-coding-system 'utf-8)
>> (when (display-graphic-p)
>>  (setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING)))
> 
> On what OS is that?

macOS

> Also, what other non-ASCII characters do you have in your .emacs.el?

Most of my comments are written in French so there are a few others.

> (You can search for them with "N-: (skip-chars-forward "\000-\177") RET.)
> 
> And if you remove all of the above settings, does the problem go away?

I commented out the code and relaunched emacs. I get the same weird behavior in .emacs.el but not in other buffers (not even *scratch*)

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: numbers instead of accented letters...

[Eli Zaretskii]

> From: Jean-Christophe Helary <brandelune@gmail.com>
> Date: Sun, 25 Nov 2018 13:22:41 +0900
> 
> >> (setq locale-coding-system 'utf-8)
> >> (set-terminal-coding-system 'utf-8)
> >> (set-keyboard-coding-system 'utf-8)
> >> (set-selection-coding-system 'utf-8)
> >> (prefer-coding-system 'utf-8)
> >> (when (display-graphic-p)
> >>  (setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING)))
> > 
> > On what OS is that?
> 
> macOS

Then I'm not sure the above is right, and you should remove that.
Although it doesn't seem to be the root cause of your problem.

> > Also, what other non-ASCII characters do you have in your .emacs.el?
> 
> Most of my comments are written in French so there are a few others.

I think you will have to review all of them.  Look for those which are
shown as a single octal escape, not 2 octal escapes one after the
other.

If nothing else help, bisect your .emacs.el to find the part which
causes this, and if looking at it doesn't provide a clue, post that
part here.

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

>> Most of my comments are written in French so there are a few others.
> 
> I think you will have to review all of them.  Look for those which are
> shown as a single octal escape, not 2 octal escapes one after the
> other.
> 
> If nothing else help, bisect your .emacs.el to find the part which
> causes this, and if looking at it doesn't provide a clue, post that
> part here.

I will.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

> On Nov 26, 2018, at 1:21, Eli Zaretskii <eliz@gnu.org <mailto:eliz@gnu.org>> wrote:
> 
>>> Also, what other non-ASCII characters do you have in your .emacs.el?
>> 
>> Most of my comments are written in French so there are a few others.
> 
> I think you will have to review all of them.  Look for those which are
> shown as a single octal escape, not 2 octal escapes one after the
> other.

Now that you mention that, yesterday, when I received your mail, the 2 octal escapes that I used to see where transformed into 1 octal escape *and* I was able to enter accented letters. Then everything reverted to the messy state it was when I wrote (2 octal escapes + no input possible).

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com <http://mac4translators.blogspot.com/> @brandelune

Re: numbers instead of accented letters...

[Stefan Monnier]

> ;; pour h\303\251riter du $PATH quand Emacs est lanc\303\251 hors de la ligne de commande

Do you also see it if you open the file with `emacs -Q`?
If so, try the following:

    emacs -Q ~/.emacs.el
    C-x RET r utf-8 RET

this should re-read the file, but forcing Emacs to interpret it as utf-8,
so presumably, "hériter" should now appear correctly.  Also the left part of
the mode-line should say "U" to mean.
If that's indeed, the case, then try to save the file (after making
a trivial modification to it like `SPC DEL`).

If this works without complaint, check the "U" in the mode-line: is it
still there?


        Stefan

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

   On Nov 26, 2018, at 1:56, Stefan Monnier <[1]monnier@iro.umontreal.ca>
   wrote:

     ;; pour h\303\251riter du $PATH quand Emacs est lanc\303\251 hors de
     la ligne de commande

   Do you also see it if you open the file with `emacs -Q`?
   If so, try the following:
      emacs -Q ~/.emacs.el
      C-x RET r utf-8 RET
   this should re-read the file, but forcing Emacs to interpret it as
   utf-8,
   so presumably, "heriter" should now appear correctly.  Also the left
   part of
   the mode-line should say "U" to mean.
   If that's indeed, the case, then try to save the file (after making
   a trivial modification to it like `SPC DEL`).
   If this works without complaint, check the "U" in the mode-line: is it
   still there?

   Everything worked as you said and indeed the U is still there.
   When I first open the file I get an = where the U should be.
   (emacs manual: "`=' means no conversion whatsoever, and is usually used
   for files containing non-textual data"...)
   So, should I interpret that as emacs is, for some reason, not using
   UTF-8 to read the file ?
   Jean-Christophe Helary
   -----------------------------------------------
   [2]http://mac4translators.blogspot.com @brandelune

References

   1. mailto:monnier@iro.umontreal.ca
   2. http://mac4translators.blogspot.com/

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

> On Nov 26, 2018, at 1:56, Stefan Monnier <monnier@iro.umontreal.ca <mailto:monnier@iro.umontreal.ca>> wrote:
> 
>> ;; pour h\303\251riter du $PATH quand Emacs est lanc\303\251 hors de la ligne de commande
> 
> Do you also see it if you open the file with `emacs -Q`?
> If so, try the following:
> 
>    emacs -Q ~/.emacs.el
>    C-x RET r utf-8 RET
> 
> this should re-read the file, but forcing Emacs to interpret it as utf-8,
> so presumably, "hériter" should now appear correctly.  Also the left part of
> the mode-line should say "U" to mean.
> If that's indeed, the case, then try to save the file (after making
> a trivial modification to it like `SPC DEL`).
> 
> If this works without complaint, check the "U" in the mode-line: is it
> still there?

Everything worked as you said and indeed the U is still there.

When I first open the file I get an = where the U should be.

(emacs manual: "‘=’ means no conversion whatsoever, and is usually used for files containing non-textual data"...)

So, should I interpret that as emacs is, for some reason, not using UTF-8 to read the file ?


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com <http://mac4translators.blogspot.com/> @brandelune

Re: numbers instead of accented letters...

[Stefan Monnier]

>> If this works without complaint, check the "U" in the mode-line: is it
>> still there?
> Everything worked as you said and indeed the U is still there.

And if you now try `emacs -Q ~/.emacs.el` anew, you again see the \NNN
and the "=" in the modeline?

Does the file have file-local variables as in

    -*- ... -*-

on the first line or

    Local Variables:
    ...
    End:

on the last few lines?

> So, should I interpret that as Emacs is, for some reason, not using
> UTF-8 to read the file ?

Yes, the `=` means that Emacs decided that this file should be treated
as binary data rather than utf-8 text, basically.

Could you also try to search for a 0-byte?  E.g. with

    C-s C-q 0 C-s C-s

?


        Stefan

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

> On Nov 26, 2018, at 22:20, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>>> If this works without complaint, check the "U" in the mode-line: is it
>>> still there?
>> Everything worked as you said and indeed the U is still there.
> 
> And if you now try `emacs -Q ~/.emacs.el` anew, you again see the \NNN
> and the "=" in the modeline?

That's correct.

> Does the file have file-local variables as in
> 
>    -*- ... -*-
> 
> on the first line or

I have the blurb added by Package.el at the top:

; Added by Package.el.  This must come before configurations of
;; installed packages.  Don't delete this line.  If you don't want it,
;; just comment it out by adding a semicolon to the start of the line.
;; You may delete these explanatory comments.


>    Local Variables:
>    ...
>    End:
> 
> on the last few lines?

And custom-set-faces at the end.

>> So, should I interpret that as Emacs is, for some reason, not using
>> UTF-8 to read the file ?
> 
> Yes, the `=` means that Emacs decided that this file should be treated
> as binary data rather than utf-8 text, basically.
> 
> Could you also try to search for a 0-byte?  E.g. with
> 
>    C-s C-q 0 C-s C-s
> 
> ?

I have one. I created a keyboard macro the other day to have Cmd+A do a (home made) select all, because I did not know that emacs had a select-all command already. The macro is:

;;; Select All : Cmd+A (Meta+A)
(fset 'selectAll
   (lambda (&optional arg) "Keyboard macro." (interactive "p") (kmacro-exec-ring-item '("\274^@\0\276" 0 "%d") arg)))
(global-set-key (kbd "M-a") 'selectAll)

If I remove that code, the file behaves properly...


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: numbers instead of accented letters...

[Stefan Monnier]

> ;;; Select All : Cmd+A (Meta+A)
> (fset 'selectAll
>    (lambda (&optional arg) "Keyboard macro." (interactive "p")
> (kmacro-exec-ring-item '("\274^@\276" 0 "%d") arg)))
> (global-set-key (kbd "M-a") 'selectAll)
>
> If I remove that code, the file behaves properly...

Aha!

Then the fix for you is to write \0 instead of the 0 byte:

    (kmacro-exec-ring-item '("\274^@\0\276" 0 "%d") arg)))

Now, could you open a bug report about that: creating a keyboard macro
shouldn't cause that kind of trouble!


        Stefan

Re: numbers instead of accented letters...

[Jean-Christophe Helary]

> On Nov 26, 2018, at 23:20, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> ;;; Select All : Cmd+A (Meta+A)
>> (fset 'selectAll
>>   (lambda (&optional arg) "Keyboard macro." (interactive "p")
>> (kmacro-exec-ring-item '("\274^@\276" 0 "%d") arg)))
>> (global-set-key (kbd "M-a") 'selectAll)
>> 
>> If I remove that code, the file behaves properly...
> 
> Aha!
> 
> Then the fix for you is to write \0 instead of the 0 byte:
> 
>    (kmacro-exec-ring-item '("\274^@\0\276" 0 "%d") arg)))
> 
> Now, could you open a bug report about that: creating a keyboard macro
> shouldn't cause that kind of trouble!

:) Yep, that's what I thought too. Although I'll also probably just remap mark-whole-buffer to Cmd-A.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune