Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Mark H. David]

Emacs 25 just started correctly allowing input and display of Hebrew with nikud.  Thank you Emacs 25 developers, and congratulations!

I'm trying to type Yiddish with Emacs 25 on the Mac, which for purposes of this discussion essentially means entering Hebrew characters from one of the built-in Hebrew keyboards on the Mac.  I do not want to use any kind of custom keyboard, since I use a lot of different Macs and use the Hebrew keyboard with a lot of different applications, not just Emacs, so I do not want to have to customize beyond the built-in keyboard.

This relies heavily on using the Mac keyboard's option key, since that's used for typing Hebrew character combinations, which involves inserting Unicode characters from the Hebrew range as a sequence of two characters: a base Hebrew character plus a diacritic character, known as nikud, the combination of which I'm calling here collectively "Hebrew with nikud".  

Here's a reference showing Hebrew keyboard for Mac OS (and other OS's):

http://www.ivritype.com/hebrew/kbd/

Here are the option key assignments on the Mac Hebrew keyboard used for Yiddish:

pasekh: Option+6
komets: Option+7
dogesh: Option+,
khirek: Option+4
sin: Option+A
paseky-tsvey-yudn: Option+H

For example, typing this email in my browser with the Hebrew keyboard installed, to type the character combination

שׂ

which is called "Sin", and looks like a normal "Shin" character with a "Sin-dot" diacritic in the upper left corner, I switch to the Hebrew keyboard, then type 

Option+A

which inserts the base character for the letter shin followed by the diacritic sin-dot.

Doing this in emacs is really awkward, at least how I'm doing it.  Maybe someone has a better way?

First of all, it's necessary to at some time have changed the variable 

mac-option-modifier

to none.  That let's you use the Option modifier key. But what that does is makes you not have a META key.  How can you use Emacs without a meta key?  I cannot really see having to use ESC instead of Meta in this day and age.

Second, after you switch the keyboard to Hebrew, and start typing a few characters, all the standard combinations used with the "English" keyboard go away. Like CTRL+A becomes "CTRL+ש" (control + shin).  I noticed this does not take effect immediately after switching keyboards, only after I've typed some Hebrew text using that keyboard.  Anyhow, how can you use Emacs with every normal binding of control and meta keystroke combinations unavailable to you?

I figured if anyone is using Emacs on Mac for Hebrew with Nikud (i.e., Hebrew with vowels, as used for extra pronunciation guidance, children's books, and liturgy), they would have to have worked out a better approach.

Can anyone provide specific solutions or approaches, i.e., for the Mac?

Thanks,

Mark

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

On 12/6/16 12:32 PM, Mark H. David wrote:
> First of all, it's necessary to at some time have changed the variable
>
> mac-option-modifier
>
> to none.  That let's you use the Option modifier key. But what that
does is makes you not have a META key.  How can you use Emacs without a
meta key?  I cannot really see having to use ESC instead of Meta in this
day and age.

You might set ns-command-modifier to Meta. The default is Super and maps
to standard Mac copy/paste/etc. keys. I do know that many people like
that better than option (the command key (⌘) is also physically closer
to where Alt is on non-mac keyboards, which I think is why it's popular
to use for Meta).

You can also set left and right modifiers to something different. Maybe
that is something that will work in your case.

To see the whole list, `M-x customize-group RET ns RET`.

-David



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3819 bytes --]

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Mark H. David]

Thanks for info about moving the meta key.

How about just the behavior of Emacs that it interprets, e.g., ctrl+a as ctrl+sin if you happen to be in another keyboard "input source", i.e., Hebrew. I think there should be a switch to preserve default (US) keyboard for control- and meta- (and probably other-) modified keystrokes, and I think this switch should be on by default.  

Also, I'd like to know why it seems to work this "preserve mode" right after you switch input sources but before you type some characters, and then switches to the other "non-preserve mode".

----- Original message -----
From: David Caldwell <david@porkrind.org>
To: "Mark H. David" <mhd@yv.org>, emacs-devel@gnu.org
Subject: Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?
Date: Tue, 6 Dec 2016 13:10:36 -0800

On 12/6/16 12:32 PM, Mark H. David wrote:
> First of all, it's necessary to at some time have changed the variable
>
> mac-option-modifier
>
> to none.  That let's you use the Option modifier key. But what that
does is makes you not have a META key.  How can you use Emacs without a
meta key?  I cannot really see having to use ESC instead of Meta in this
day and age.

You might set ns-command-modifier to Meta. The default is Super and maps
to standard Mac copy/paste/etc. keys. I do know that many people like
that better than option (the command key (⌘) is also physically closer
to where Alt is on non-mac keyboards, which I think is why it's popular
to use for Meta).

You can also set left and right modifiers to something different. Maybe
that is something that will work in your case.

To see the whole list, `M-x customize-group RET ns RET`.

-David


Email had 1 attachment:
+ smime.p7s
  5k (application/pkcs7-signature)

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This relies heavily on using the Mac keyboard's option key, since
  > that's used for typing Hebrew character combinations, which
  > involves inserting Unicode characters from the Hebrew range as a
  > sequence of two characters: a base Hebrew character plus a
  > diacritic character, known as nikud, the combination of which I'm
  > calling here collectively "Hebrew with nikud".

What about when not using a Mac keyboard?  How good is our support
for this feature on ordinary PC keyboards with the GNU/Linux system?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Mark H. David]

It's similarly a bummer on GNU/Linux with a PC keyboard.

Straightforward way to try it:
  Add Hebrew language keyboard in your modern GUI in the GNU/Linux-y way.
  Fire up emacs, go to scratch buffer
  Switch keyboard to "he" (Hebrew keyboard)
  Type keys with caps: ASDF - 4 Hebrew letters show up: שדגכ
  Type ctrl+b, meaning backward-char. Doesn't work, you get: "C-נ is undefined".

נ is the Hebrew character you get when you type the keycap B.

I tried this on Emacs 24. I'm sure it's the same on Emacs 25 in this respect.

By the way, my keyboard switching key is assigned to Windows+Space [hold down "Windows" key while typing the space bar], so when I type Windows+Space, Emacs says "s-SPC is undefined".  The system correctly intercepts my typing Windows+Space and switches the keyboard, but something is wrong there: it should not even be seeing that key. Hopefully that's fixed upstream. I'll check another day.

Thanks,

Mark

----- Original message -----
From: Richard Stallman <rms@gnu.org>
To: "Mark H. David" <mhd@yv.org>
Cc: emacs-devel@gnu.org
Subject: Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?
Date: Wed, 07 Dec 2016 15:54:55 -0500

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This relies heavily on using the Mac keyboard's option key, since
  > that's used for typing Hebrew character combinations, which
  > involves inserting Unicode characters from the Hebrew range as a
  > sequence of two characters: a base Hebrew character plus a
  > diacritic character, known as nikud, the combination of which I'm
  > calling here collectively "Hebrew with nikud".

What about when not using a Mac keyboard?  How good is our support
for this feature on ordinary PC keyboards with the GNU/Linux system?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Eli Zaretskii]

> From: "Mark H. David" <mhd@yv.org>
> Date: Wed, 07 Dec 2016 15:02:30 -0800
> Cc: emacs-devel@gnu.org
> 
> It's similarly a bummer on GNU/Linux with a PC keyboard.
> 
> Straightforward way to try it:
>   Add Hebrew language keyboard in your modern GUI in the GNU/Linux-y way.
>   Fire up emacs, go to scratch buffer
>   Switch keyboard to "he" (Hebrew keyboard)
>   Type keys with caps: ASDF - 4 Hebrew letters show up: שדגכ
>   Type ctrl+b, meaning backward-char. Doesn't work, you get: "C-נ is undefined".
> 
> נ is the Hebrew character you get when you type the keycap B.
> 
> I tried this on Emacs 24. I'm sure it's the same on Emacs 25 in this respect.

It's not an Emacs issue, it's how keyboard works on X.  (Ironically,
MS-Windows gets it right, so keys with Ctrl modifier are still ASCII
when I switch the keyboard to Hebrew.)

May I suggest to try one of the Emacs's own built-in Hebrew input
methods instead?

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Mark H. David]

No, using an Emacs input method is not acceptable in this case. 

The goal of this exercise is to allow one to use the built-in input method
(here, Hebrew keyboard MacOS (or Windows or GNU/Linux)), without needing
Emacs's input methods.  Nowadays, built-in input methods involve very little
setup work on all the platforms. And then you have a uniform way to enter your
text in many apps, e.g., firefox, chrome, safari, terminal, mail, what have you.
So, just want Emacs to behave like these apps.  It kind of does, as far as
self-inserting characters are concerned, but having to switch back to US keyboard
just to type C-x, C-c, C-s, et al, is "insane". How to fix it?

First approximation: when a key K comes in to Emacs, and not using US keyboard
and K is not defined in Emacs, reinterpret K with corresponding character in US keyboard
combined with modifier bits of of K.

Example:

User types C-<keycap for S> when in Hebrew input method. I.e.,
it looks like they typed C-s, but since the Hebrew input method
is on, Emacs is going to get a different key event.

So, Emacs gets a key whose modifier bits are for the control key,
and character is ד (daled), a Hebrew character that is assigned
to key cap S.  So, Emacs initially sees that as C-ד.

So, Emacs decides that C-ד is not defined. So, Emacs
looks up the corresponding character in the US keyboard, which
is "s".  So, reinterpret the keystroke as C-s.

I think this should be an optional switch, although I would want it to be the default.
In any case, if this does not work out for you -- e.g., you're really winning
with Emacs input methods, or whatever, you can just turn it off.

Thanks,

Mark



----- Original message -----
From: Eli Zaretskii <eliz@gnu.org>
To: "Mark H. David" <mhd@yv.org>
Cc: rms@gnu.org, emacs-devel@gnu.org
Subject: Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?
Date: Thu, 08 Dec 2016 05:42:18 +0200

> From: "Mark H. David" <mhd@yv.org>
> Date: Wed, 07 Dec 2016 15:02:30 -0800
> Cc: emacs-devel@gnu.org
> 
> It's similarly a bummer on GNU/Linux with a PC keyboard.
> 
> Straightforward way to try it:
>   Add Hebrew language keyboard in your modern GUI in the GNU/Linux-y way.
>   Fire up emacs, go to scratch buffer
>   Switch keyboard to "he" (Hebrew keyboard)
>   Type keys with caps: ASDF - 4 Hebrew letters show up: שדגכ
>   Type ctrl+b, meaning backward-char. Doesn't work, you get: "C-נ is undefined".
> 
> נ is the Hebrew character you get when you type the keycap B.
> 
> I tried this on Emacs 24. I'm sure it's the same on Emacs 25 in this respect.

It's not an Emacs issue, it's how keyboard works on X.  (Ironically,
MS-Windows gets it right, so keys with Ctrl modifier are still ASCII
when I switch the keyboard to Hebrew.)

May I suggest to try one of the Emacs's own built-in Hebrew input
methods instead?

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Eli Zaretskii]

> From: "Mark H. David" <mhd@yv.org>
> Cc: rms@gnu.org, emacs-devel@gnu.org
> Date: Mon, 12 Dec 2016 13:09:58 -0800
> 
> First approximation: when a key K comes in to Emacs, and not using US keyboard
> and K is not defined in Emacs, reinterpret K with corresponding character in US keyboard
> combined with modifier bits of of K.

The problem is, the US keyboard key that corresponds to a given
character is not easily deduced, AFAIK, because different localized
keyboards have slight variations in their layout.

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Juri Linkov]

> The goal of this exercise is to allow one to use the built-in input method
> (here, Hebrew keyboard MacOS (or Windows or GNU/Linux)), without needing
> Emacs's input methods.  Nowadays, built-in input methods involve very little
> setup work on all the platforms. And then you have a uniform way to enter your
> text in many apps, e.g., firefox, chrome, safari, terminal, mail, what have you.
> So, just want Emacs to behave like these apps.  It kind of does, as far as
> self-inserting characters are concerned, but having to switch back to US keyboard
> just to type C-x, C-c, C-s, et al, is "insane". How to fix it?

We already solved the same problem for Cyrillic:
https://debbugs.gnu.org/9751#85

You can use the same solution for Hebrew as well.
Give to this command the name of the Emacs input method
that works correctly for your keyboard, and it will create
a reverse mapping converting all entered keys with modifiers
to ASCII without the need to activate the Emacs input method:

(defun reverse-input-method (input-method)
  "Build the reverse mapping of single letters from INPUT-METHOD."
  (interactive
   (list (read-input-method-name "Use input method (default current): ")))
  (if (and input-method (symbolp input-method))
      (setq input-method (symbol-name input-method)))
  (let ((current current-input-method)
        (modifiers '(nil (control) (meta) (control meta))))
    (when input-method
      (activate-input-method input-method))
    (when (and current-input-method quail-keyboard-layout)
      (dolist (map (cdr (quail-map)))
        (let* ((to (car map))
               (from (quail-get-translation
                      (cadr map) (char-to-string to) 1)))
          (when (and (characterp from) (characterp to))
            (dolist (mod modifiers)
              (define-key local-function-key-map
                (vector (append mod (list from)))
                (vector (append mod (list to)))))))))
    (when input-method
      (activate-input-method current))))

If a list of modifiers '(control) (meta) (control meta)' is not enough,
then you can add more combinations of modifiers with:

(defun powerset (S)
    (let ((x (car S)))
      (if (cdr S)
          (let ((y (powerset (remove x S))))
            (append (list (list x))
                    (mapcar (lambda (e) (cons x e)) y)
                    y))
        (list (list x)))))

and

(powerset '(control meta super hyper))

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Mark H. David]

But Juri, you say

> Give to this command the name of the Emacs input method
> that works correctly for your keyboard

so I don't see how this applies to the case I'm describing,
that is, using the built-in input method (built into the OS)
rather than Emacs's input method, not using any input method
of Emacs's.  Can you help me understand?

Thanks,

Mark

----- Original message -----
From: Juri Linkov <juri@linkov.net>
To: "Mark H. David" <mhd@yv.org>
Cc: Eli Zaretskii <eliz@gnu.org>, rms@gnu.org, emacs-devel@gnu.org
Subject: Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?
Date: Tue, 13 Dec 2016 23:27:15 +0200

> The goal of this exercise is to allow one to use the built-in input method
> (here, Hebrew keyboard MacOS (or Windows or GNU/Linux)), without needing
> Emacs's input methods.  Nowadays, built-in input methods involve very little
> setup work on all the platforms. And then you have a uniform way to enter your
> text in many apps, e.g., firefox, chrome, safari, terminal, mail, what have you.
> So, just want Emacs to behave like these apps.  It kind of does, as far as
> self-inserting characters are concerned, but having to switch back to US keyboard
> just to type C-x, C-c, C-s, et al, is "insane". How to fix it?

We already solved the same problem for Cyrillic:
https://debbugs.gnu.org/9751#85

You can use the same solution for Hebrew as well.
Give to this command the name of the Emacs input method
that works correctly for your keyboard, and it will create
a reverse mapping converting all entered keys with modifiers
to ASCII without the need to activate the Emacs input method:

(defun reverse-input-method (input-method)
  "Build the reverse mapping of single letters from INPUT-METHOD."
  (interactive
   (list (read-input-method-name "Use input method (default current): ")))
  (if (and input-method (symbolp input-method))
      (setq input-method (symbol-name input-method)))
  (let ((current current-input-method)
        (modifiers '(nil (control) (meta) (control meta))))
    (when input-method
      (activate-input-method input-method))
    (when (and current-input-method quail-keyboard-layout)
      (dolist (map (cdr (quail-map)))
        (let* ((to (car map))
               (from (quail-get-translation
                      (cadr map) (char-to-string to) 1)))
          (when (and (characterp from) (characterp to))
            (dolist (mod modifiers)
              (define-key local-function-key-map
                (vector (append mod (list from)))
                (vector (append mod (list to)))))))))
    (when input-method
      (activate-input-method current))))

If a list of modifiers '(control) (meta) (control meta)' is not enough,
then you can add more combinations of modifiers with:

(defun powerset (S)
    (let ((x (car S)))
      (if (cdr S)
          (let ((y (powerset (remove x S))))
            (append (list (list x))
                    (mapcar (lambda (e) (cons x e)) y)
                    y))
        (list (list x)))))

and

(powerset '(control meta super hyper))

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Juri Linkov]

>> Give to this command the name of the Emacs input method
>> that works correctly for your keyboard
>
> so I don't see how this applies to the case I'm describing,
> that is, using the built-in input method (built into the OS)
> rather than Emacs's input method, not using any input method
> of Emacs's.  Can you help me understand?

Emacs doesn't know about key mappings of the OS built-in input method,
so one way to get some mappings is from the input methods in the
Quail package.  After you review and confirm that one of available
Quail input methods corresponds to your OS input method,
you can use it to create an inverse mapping.  So you only
get the information about key mappings from the input method,
not directly using it.

Have you tried the proposed command?  What happens?

> ----- Original message -----
> From: Juri Linkov <juri@linkov.net>
> To: "Mark H. David" <mhd@yv.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, rms@gnu.org, emacs-devel@gnu.org
> Subject: Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?
> Date: Tue, 13 Dec 2016 23:27:15 +0200
>
> We already solved the same problem for Cyrillic:
> https://debbugs.gnu.org/9751#85
>
> You can use the same solution for Hebrew as well.
> Give to this command the name of the Emacs input method
> that works correctly for your keyboard, and it will create
> a reverse mapping converting all entered keys with modifiers
> to ASCII without the need to activate the Emacs input method:
>
> (defun reverse-input-method (input-method)
>   "Build the reverse mapping of single letters from INPUT-METHOD."
>   (interactive
>    (list (read-input-method-name "Use input method (default current): ")))
>   (if (and input-method (symbolp input-method))
>       (setq input-method (symbol-name input-method)))
>   (let ((current current-input-method)
>         (modifiers '(nil (control) (meta) (control meta))))
>     (when input-method
>       (activate-input-method input-method))
>     (when (and current-input-method quail-keyboard-layout)
>       (dolist (map (cdr (quail-map)))
>         (let* ((to (car map))
>                (from (quail-get-translation
>                       (cadr map) (char-to-string to) 1)))
>           (when (and (characterp from) (characterp to))
>             (dolist (mod modifiers)
>               (define-key local-function-key-map
>                 (vector (append mod (list from)))
>                 (vector (append mod (list to)))))))))
>     (when input-method
>       (activate-input-method current))))
>
> If a list of modifiers '(control) (meta) (control meta)' is not enough,
> then you can add more combinations of modifiers with:
>
> (defun powerset (S)
>     (let ((x (car S)))
>       (if (cdr S)
>           (let ((y (powerset (remove x S))))
>             (append (list (list x))
>                     (mapcar (lambda (e) (cons x e)) y)
>                     y))
>         (list (list x)))))
>
> and
>
> (powerset '(control meta super hyper))

Re: Is there a sane way to type Hebrew with nikud with Emacs 25/Mac?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's similarly a bummer on GNU/Linux with a PC keyboard.

If we do work to improve this, let's please do it in a way that 
helps on GNU/Linux also -- not solely on MacOS.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.