bidi, hebrew: files may need to be utf-8 with a byte order mark

bidi, hebrew: files may need to be utf-8 with a byte order mark

[Uwe Brauer]

Hi

I am trying to contribute to the keyman project and provide a phonetic
hebrew keyboard layout. It was pointed out to me by the maintainers,
that I should use 

,----
| One issue is that some of the files may need to be utf-8 with a byte
| order mark. I would suggest using Visual Studio Code as the editor. It
| will reduce grief on Mac OS or Linux.
`----

I would like to know whether GNU emacs (master) satisfies these
requirements. I am pretty sure it does, but would like to get a
confirmation, best by Eli.

Regards

Uwe Brauer 

Re: bidi, hebrew: files may need to be utf-8 with a byte order mark

[Eli Zaretskii]

> From: Uwe Brauer <oub@mat.ucm.es>
> Date: Fri, 04 Jan 2019 18:13:30 +0100
> 
> I am trying to contribute to the keyman project and provide a phonetic
> hebrew keyboard layout. It was pointed out to me by the maintainers,
> that I should use 
> 
> ,----
> | One issue is that some of the files may need to be utf-8 with a byte
> | order mark. I would suggest using Visual Studio Code as the editor. It
> | will reduce grief on Mac OS or Linux.
> `----
> 
> I would like to know whether GNU emacs (master) satisfies these
> requirements. I am pretty sure it does, but would like to get a
> confirmation, best by Eli.

I don't think I understand the question.  If you ask whether Emacs
supports UTF-8 encoding with BOM, then the answer is yes: see
coding-system utf-8-with-signature.  Emacs has supported this since
long ago, not just on master.  But this has no direct relation to
bidi.

Re: bidi, hebrew: files may need to be utf-8 with a byte order mark

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1654 bytes --]

>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

   >> From: Uwe Brauer <oub@mat.ucm.es>
   >> Date: Fri, 04 Jan 2019 18:13:30 +0100
   >> 
   >> I am trying to contribute to the keyman project and provide a phonetic
   >> hebrew keyboard layout. It was pointed out to me by the maintainers,
   >> that I should use 
   >> 
   >> ,----
   >> | One issue is that some of the files may need to be utf-8 with a byte
   >> | order mark. I would suggest using Visual Studio Code as the editor. It
   >> | will reduce grief on Mac OS or Linux.
   >> `----
   >> 
   >> I would like to know whether GNU emacs (master) satisfies these
   >> requirements. I am pretty sure it does, but would like to get a
   >> confirmation, best by Eli.

   > I don't think I understand the question.  If you ask whether Emacs
   > supports UTF-8 encoding with BOM, then the answer is yes: see
   > coding-system utf-8-with-signature.  Emacs has supported this since
   > long ago, not just on master.  But this has no direct relation to
   > bidi.

Well I just included the remark from the keyman project. So BOM means
byte order mark, I presume and as I understand GNU emacs supports it. 

I recall some time ago some specific issue of the hebrew or better bidi
implementation, like some missing feature which was only included
recently (by this I mean ~25 or later. I call it recently since Ubuntu
still ships officially GNU emacs 24).

I also thought that BOM is related to bidi, but since I have no much
idea about the implantation, I see that I was wrong. 

Thanks for the answer.

BTW is that a BOM ‏?
(insert-char 8207 1 t)


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5025 bytes --]

Re: bidi, hebrew: files may need to be utf-8 with a byte order mark

[Eli Zaretskii]

> From: Uwe Brauer <oub@mat.ucm.es>
> Date: Fri, 04 Jan 2019 20:49:40 +0100
> 
>    > I don't think I understand the question.  If you ask whether Emacs
>    > supports UTF-8 encoding with BOM, then the answer is yes: see
>    > coding-system utf-8-with-signature.  Emacs has supported this since
>    > long ago, not just on master.  But this has no direct relation to
>    > bidi.
> 
> Well I just included the remark from the keyman project. So BOM means
> byte order mark, I presume and as I understand GNU emacs supports it. 
> 
> I recall some time ago some specific issue of the hebrew or better bidi
> implementation, like some missing feature which was only included
> recently (by this I mean ~25 or later. I call it recently since Ubuntu
> still ships officially GNU emacs 24).

If you mean support for changes introduced in Unicode 6.3, then Emacs
supports that since v25.1.  But again, what does that have to do with
keyboard layouts?

> I also thought that BOM is related to bidi

No, it is related to automatic detection of the platform's endianness
i.e. big-endian vs little-endian.

> BTW is that a BOM ‏?
> (insert-char 8207 1 t)

No, the BOM is 0xFEFF, which is 65279 decimal.

Re: bidi, hebrew: files may need to be utf-8 with a byte order mark

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 788 bytes --]



   > If you mean support for changes introduced in Unicode 6.3, then Emacs
   > supports that since v25.1.  But again, what does that have to do with
   > keyboard layouts?

Well as I said keyman software provides (not sure about the license)
free keyboard layouts for different platforms including iOS. 

So in order to have my favorite phonetic hebrew layout for keyman, I
need to modify certain files, (like for example
/usr/share/X11/xkb/symbols/il as  in unix based systems) 
and submit these as patch. 

These modified files need that specific UTF8 support I mentioned and
that is why the keyman maintainers proposed to use visual editor.

However I prefer GNU emacs, if I can.
Thanks to your information I can. 

Did I explain the issue now detailed enough?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5025 bytes --]

Re: bidi, hebrew: files may need to be utf-8 with a byte order mark

[Eli Zaretskii]

> From: Uwe Brauer <oub@mat.ucm.es>
> Date: Fri, 04 Jan 2019 21:48:25 +0100
> 
>    > If you mean support for changes introduced in Unicode 6.3, then Emacs
>    > supports that since v25.1.  But again, what does that have to do with
>    > keyboard layouts?
> 
> Well as I said keyman software provides (not sure about the license)
> free keyboard layouts for different platforms including iOS. 
> 
> So in order to have my favorite phonetic hebrew layout for keyman, I
> need to modify certain files, (like for example
> /usr/share/X11/xkb/symbols/il as  in unix based systems) 
> and submit these as patch. 
> 
> These modified files need that specific UTF8 support I mentioned and
> that is why the keyman maintainers proposed to use visual editor.

I understand that, but the changes in bidi processing that were
introduced in Unicode 6.3 should have no effect on any keyboard layout
files.

Re: bidi, hebrew: files may need to be utf-8 with a byte order mark

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 255 bytes --]



   > I understand that, but the changes in bidi processing that were
   > introduced in Unicode 6.3 should have no effect on any keyboard layout
   > files.

I can only tell you what the keyman maintainers told me. I really don't
understand their code.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5025 bytes --]