How do you write input methods?

How do you write input methods?

[Perry E. Metzger]

The elisp manual says:

"How to define input methods is not yet documented in this manual, but 
here we describe how to use them."

Do any documents exist at all on how to define a new input method, or do 
I have to read the code?

Perry

Re: How do you write input methods?

[Stephen Berman]

On Sun, 29 Aug 2021 21:10:25 -0400 "Perry E. Metzger" <perry@piermont.com> wrote:

> The elisp manual says:
>
> "How to define input methods is not yet documented in this manual, but here we
> describe how to use them."
>
> Do any documents exist at all on how to define a new input method, or do I
> have to read the code?

I don't know of any such documentation, but at least for relatively
simple input methods it's pretty straightforward to define them.  You
need to use quail-define-package to define the package containing the
input method and quail-define-rules to define translation rules used by
the input method; many examples (of both simple and more complicated
input methods) are in the Emacs lisp/leim/quail directory.  And you need
to use register-input-method to enable invoking the input method via
`C-\'; examples are in the lisp/leim/leim-list.el file.

Steve Berman

Re: How do you write input methods?

[Joost Kremers]

On Mon, Aug 30 2021, Stephen Berman wrote:
> And you need
> to use register-input-method to enable invoking the input method via
> `C-\'; examples are in the lisp/leim/leim-list.el file.

That's interesting... I define a couple of personal input methods in my init
files and they do show up when I do `M-x set-input-method`, even though I don't
use `register-input-method`.

Hmm, looking at the doc string of `register-input-method`, I see the following
comment:

"
The commands ‘describe-input-method’ and ‘list-input-methods’ need
these duplicated values to show some information about input methods
**without loading the relevant Quail packages.** (emphasis mine)
"

Does that mean that `register-input-method` is basically just a way to tell
Emacs about available input methods without loading them? And since my input
methods get loaded during startup, there's no need to register them anymore?


-- 
Joost Kremers
Life has its moments

Re: How do you write input methods?

[Stephen Berman]

On Mon, 30 Aug 2021 13:24:07 +0200 Joost Kremers <joostkremers@fastmail.fm> wrote:

> On Mon, Aug 30 2021, Stephen Berman wrote:
>> And you need
>> to use register-input-method to enable invoking the input method via
>> `C-\'; examples are in the lisp/leim/leim-list.el file.
>
> That's interesting... I define a couple of personal input methods in my init
> files and they do show up when I do `M-x set-input-method`, even though I don't
> use `register-input-method`.
>
> Hmm, looking at the doc string of `register-input-method`, I see the following
> comment:
>
> "
> The commands ‘describe-input-method’ and ‘list-input-methods’ need
> these duplicated values to show some information about input methods
> **without loading the relevant Quail packages.** (emphasis mine)
> "
>
> Does that mean that `register-input-method` is basically just a way to tell
> Emacs about available input methods without loading them? And since my input
> methods get loaded during startup, there's no need to register them anymore?

I thought I had found it was necessary to use register-input-method but
I just tested omitting that and it seems you're right.  Either I
misremembered or misunderstood.  I guess that's an argument for
documenting how to write input methods ;-)

Steve Berman

Re: How do you write input methods?

[Perry E. Metzger]

On 8/30/21 07:48, Stephen Berman wrote:
>
> I thought I had found it was necessary to use register-input-method but
> I just tested omitting that and it seems you're right.  Either I
> misremembered or misunderstood.  I guess that's an argument for
> documenting how to write input methods ;-)
>
If you would be willing to write something very primitive, I'll 
volunteer to turn it into more readable texinfo. If we get started, 
almost certainly people will get angry about misstatements and help to 
fix them. :)

Perry

Re: How do you write input methods?

[Stephen Berman]

On Mon, 30 Aug 2021 12:23:40 -0400 "Perry E. Metzger" <perry@piermont.com> wrote:

> On 8/30/21 07:48, Stephen Berman wrote:
>>
>> I thought I had found it was necessary to use register-input-method but
>> I just tested omitting that and it seems you're right.  Either I
>> misremembered or misunderstood.  I guess that's an argument for
>> documenting how to write input methods ;-)
>>
> If you would be willing to write something very primitive, I'll volunteer to
> turn it into more readable texinfo. If we get started, almost certainly people
> will get angry about misstatements and help to fix them. :)

I'm afraid that even just coming up with a semi-coherent summary of the
two required functions I mentioned based on their doc strings would take
more time and effort than I can afford now.  But judging by the related
thread you started, your level of understanding of Emacs input methods
already surpasses mine, so if you have the time, I think you're in a
better position than me to get the doc ball rolling.  Otherwise, and if
nobody else jumps in, when I do have more time, I can try to cook
something up.

Steve Berman

Re: How do you write input methods?

[Eli Zaretskii]

> Date: Sun, 29 Aug 2021 21:10:25 -0400
> From: "Perry E. Metzger" <perry@piermont.com>
> 
> The elisp manual says:
> 
> "How to define input methods is not yet documented in this manual, but 
> here we describe how to use them."
> 
> Do any documents exist at all on how to define a new input method

No.

> or do I have to read the code?

Yes.

Re: How do you write input methods?

[Perry E. Metzger]

On 8/30/21 07:46, Eli Zaretskii wrote:
>> Do any documents exist at all on how to define a new input method
> No.
>
>> or do I have to read the code?
> Yes.
>
That's rather unfortunate. I hope someone who knows it well might write 
some documentation at some point.

Perry

Re: How do you write input methods?

[Eli Zaretskii]

> Date: Mon, 30 Aug 2021 09:40:26 -0400
> Cc: emacs-devel@gnu.org
> From: "Perry E. Metzger" <perry@piermont.com>
> 
> On 8/30/21 07:46, Eli Zaretskii wrote:
> >> Do any documents exist at all on how to define a new input method
> > No.
> >
> >> or do I have to read the code?
> > Yes.
> >
> That's rather unfortunate. I hope someone who knows it well might write 
> some documentation at some point.

Alas, we don't have anyone like that on board.  However, adding simple
input methods, like those where you type a small number of ASCII
characters to produce a non-ASCII character, is easy enough by looking
at existing input methods for various Latin languages.  It's pretty
much boilerplate code.

Re: How do you write input methods?

[Perry E. Metzger]

On 8/30/21 10:00, Eli Zaretskii wrote:
>> or do I have to read the code?
>>> Yes.
>>>
>> That's rather unfortunate. I hope someone who knows it well might write
>> some documentation at some point.
> Alas, we don't have anyone like that on board.  However, adding simple
> input methods, like those where you type a small number of ASCII
> characters to produce a non-ASCII character, is easy enough by looking
> at existing input methods for various Latin languages.  It's pretty
> much boilerplate code.

Can you suggest an input method that follows exactly that pattern as a 
model?

Perry

Re: How do you write input methods?

[Eli Zaretskii]

> Date: Mon, 30 Aug 2021 10:22:13 -0400
> Cc: emacs-devel@gnu.org
> From: "Perry E. Metzger" <perry@piermont.com>
> 
> > Alas, we don't have anyone like that on board.  However, adding simple
> > input methods, like those where you type a small number of ASCII
> > characters to produce a non-ASCII character, is easy enough by looking
> > at existing input methods for various Latin languages.  It's pretty
> > much boilerplate code.
> 
> Can you suggest an input method that follows exactly that pattern as a 
> model?

Any input method defined in latin-post.el or latin-pre.el, for
example.

If these don't fit your needs for some reason, perhaps you could tell
more about what you are trying to do?

Re: How do you write input methods?

[Perry E. Metzger]

On 8/30/21 11:43, Eli Zaretskii wrote:
>>> Alas, we don't have anyone like that on board.  However, adding simple
>>> input methods, like those where you type a small number of ASCII
>>> characters to produce a non-ASCII character, is easy enough by looking
>>> at existing input methods for various Latin languages.  It's pretty
>>> much boilerplate code.
>> Can you suggest an input method that follows exactly that pattern as a
>> model?
> Any input method defined in latin-post.el or latin-pre.el, for
> example.

Those seem to fit the bill nicely, thank you.


Perry

Re: How do you write input methods?

[André A. Gomes]

"Perry E. Metzger" <perry@piermont.com> writes:

> Do any documents exist at all on how to define a new input method, or
> do I have to read the code?

As you've been told, the key aspects are the quail package and the leim
folder.  Let me share smth that wasn't yet addressed.

Some time ago, I wanted to write a short emacs package that would
transform a string as if it had been typed by another keyboard layout.
Trivial example: "yes" on QWERTY maps to "zes" on QWERTZ.

You can take a look, but the quality is rather poor.  It was the first
time I wrote elisp.

https://github.com/aadcg/emacs-yeis

I found out there is a small built-in package called robin.  Quail
supports CJK input-methods, while robin doesn't.  Sadly, even though
there are many input methods that aren't CJK, robin is nowhere used in
emacs.

I thought I could share this idea, but I'm not sure the maintainers will
like the idea to move non-CJK input methods from quail to robin.  From
the user perspective, it would bring new interesting features.

Russian-speaking people might know about a proprietary program called
Punto Switcher.  Or you might know about a free program called xneur
(https://xneur.ru/).

We could have those features in Emacs :)


--
André A. Gomes
"Free Thought, Free World"

Re: How do you write input methods?

[Eli Zaretskii]

> From: André A. Gomes <andremegafone@gmail.com>
> Date: Mon, 30 Aug 2021 17:29:15 +0300
> Cc: emacs-devel@gnu.org
> 
> I thought I could share this idea, but I'm not sure the maintainers will
> like the idea to move non-CJK input methods from quail to robin.  From
> the user perspective, it would bring new interesting features.

I don't think I understand what kind of "move" you are proposing here,
and what features would that enable.  Please elaborate.

Re: How do you write input methods?

[André A. Gomes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: André A. Gomes <andremegafone@gmail.com>
>> Date: Mon, 30 Aug 2021 17:29:15 +0300
>> Cc: emacs-devel@gnu.org
>>
>> I thought I could share this idea, but I'm not sure the maintainers will
>> like the idea to move non-CJK input methods from quail to robin.  From
>> the user perspective, it would bring new interesting features.
>
> I don't think I understand what kind of "move" you are proposing here,
> and what features would that enable.  Please elaborate.

There's a feature regarding input methods (IMs) that Emacs doesn't have,
but it's relatively easy to implement.

Let me describe it.  Let's think about non-CJK IMs, i.e. IMs for which
there is a bijection between the keyboard layouts.  The simplest example
would be the bijection between QWERTY and QWERTZ.  Examples of
"interesting" bijections would be mapping QWERTY to either ЙЦУКЕН
(Russian layout) or the Greek layout.

It's handy to toggle IMs, but here's what often happens: you type a
string; realise that the wrong IM was selected; toggle the IM; delete
the string and retype.  This is tiresome.  This is the itch I'm
scratching.

Robin can be leveraged to implement such a feature.  There are actually
2 features here.

One is to be able to manually transform a string as if it has been typed
with the another IM selected.  Example: take the QWERTY <-> QWERTZ
bijection; "yes" maps to "zes", and vice-versa.

The second is a minor mode that implements a "smarter" way to
automatically toggle IMs.  Example: take the US QWERTY <-> ЙЦУКЕН
bijection; type "ghb" while the US IM is selected; it automatically
changes the IM to russian and transforms it into "при".  There's no word
in english starting with "ghb", whereas "при" is the beginning of the
word "привет".  Indeed, this mode requires some work.

For a more visual explanation and a low quality proof of concept, please
refer to https://github.com/aadcg/emacs-yeis.

When I worked on this, I tried to imagine how it could potentially be
integrated into Emacs.  I see two alternatives.

Here's how I think it should be done.  It makes little sense that all
IMs are defined with quail, expect for CJK and other "complex" ones.
But the "simple" ones should use always robin.  Nothing would change for
the user, and the above mentioned features would be available "for
free".

The alternative is to keep all IMs definition, and add the new ones.
Say that those will have "-robin" as suffix.  In this scenario, you'll
have, for instance, "russian-computer" (defined with quail) and
"russian-computer-robin" (defined with robin).  But they're actually the
same, so it makes little sense.

Hopefully that was clear.


--
André A. Gomes
"Free Thought, Free World"

Re: How do you write input methods?

[Eli Zaretskii]

> From: André A. Gomes <andremegafone@gmail.com>
> Cc: emacs-devel@gnu.org,  perry@piermont.com
> Date: Mon, 30 Aug 2021 23:12:07 +0300
> 
> When I worked on this, I tried to imagine how it could potentially be
> integrated into Emacs.  I see two alternatives.
> 
> Here's how I think it should be done.  It makes little sense that all
> IMs are defined with quail, expect for CJK and other "complex" ones.
> But the "simple" ones should use always robin.  Nothing would change for
> the user, and the above mentioned features would be available "for
> free".
> 
> The alternative is to keep all IMs definition, and add the new ones.
> Say that those will have "-robin" as suffix.  In this scenario, you'll
> have, for instance, "russian-computer" (defined with quail) and
> "russian-computer-robin" (defined with robin).  But they're actually the
> same, so it makes little sense.

Thanks.  I think adding robin-based input methods is the better way of
supporting this feature.  We cannot easily discontinue support for
quail-based input methods, even for non-CJK languages, because people
are using them, and the robin-based method will most probably not be
100% compatible.

Re: How do you write input methods?

[Filipp Gunbin]

FWIW, I use this (hack?) to employ available input methods for
transliteration.

Filipp

--8<---------------cut here---------------start------------->8---
(defun fg-dotemacs-translit (str input-method)
  (with-temp-buffer
    (let ((input-method-verbose-flag nil)
          res)
      (activate-input-method input-method)
      (discard-input)
      (setq unread-command-events
            ;; add newline so input method doesn't wait for more input
            ;; in case of multiple translation possibilities
            (listify-key-sequence (concat str "\n")))
      (while (input-pending-p)
        (push (read-char nil t) res))
      (concat (reverse
               ;; don't forget to remove newline
               (cdr res))))))


(ert-deftest fg-dotemacs-translit-test()
  (should (equal "русский" (fg-dotemacs-translit "russkij" "cyrillic-translit")))
  ;; `u' / `sh' may have different completions
  (should (equal "гу" (fg-dotemacs-translit "gu" "cyrillic-translit")))
  (should (equal "ш" (fg-dotemacs-translit "sh" "cyrillic-translit"))))
--8<---------------cut here---------------end--------------->8---

Re: How do you write input methods?

[Eli Zaretskii]

> From: Filipp Gunbin <fgunbin@fastmail.fm>
> Cc: Eli Zaretskii <eliz@gnu.org>,  perry@piermont.com,  emacs-devel@gnu.org
> Date: Wed, 01 Sep 2021 20:50:04 +0300
> 
> FWIW, I use this (hack?) to employ available input methods for
> transliteration.

Out of curiosity: why do you need something like that? what's the use
case?

Re: How do you write input methods?

[Filipp Gunbin]

On 01/09/2021 20:57 +0300, Eli Zaretskii wrote:

>> From: Filipp Gunbin <fgunbin@fastmail.fm>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  perry@piermont.com,  emacs-devel@gnu.org
>> Date: Wed, 01 Sep 2021 20:50:04 +0300
>>
>> FWIW, I use this (hack?) to employ available input methods for
>> transliteration.
>
> Out of curiosity: why do you need something like that? what's the use
> case?

It's because our LDAP server at work is a bit messy, attributes like
"cn" and "displayname" sometimes contain first then last name, sometimes
vice versa, sometimes in Russian, sometimes in English.  I use email
expansion in message-mode:

--8<---------------cut here---------------start------------->8---
(eval-after-load 'message
  (lambda ()
    (define-key message-mode-map (kbd "C-c TAB") #'eudc-expand-inline)))
--8<---------------cut here---------------end--------------->8---

I want to type first letters of surname in English, press C-c TAB and
get email autocompleted.  But the only "stable", predictable attribute
seems to be "sn", which is unfortunately in Russian.  So I do:

--8<---------------cut here---------------start------------->8---
(eudc-protocol-set 'eudc-inline-query-format '((sn) (email)) 'ldap)
(advice-add #'eudc-ldap-simple-query-internal
            :filter-args #'fg-dotemacs-ldap-advice)

(defun fg-dotemacs-ldap-advice (args)
  (cons (mapcar (lambda (attr)
                  (if (and (eq (car attr) 'sn)
                           (stringp (cdr attr))
                           (seq-every-p (lambda (c)
                                          (eq (aref char-script-table c) 'latin))
                                        (cdr attr)))
                      (cons (car attr)
                            (fg-dotemacs-translit (cdr attr) "cyrillic-translit"))
                    attr))
                (car args))
        (cdr args)))
--8<---------------cut here---------------end--------------->8---

As ugly as it is, but it works.

Filipp

Re: How do you write input methods?

[Eli Zaretskii]

> From: Filipp Gunbin <fgunbin@fastmail.fm>
> Cc: andremegafone@gmail.com,  perry@piermont.com,  emacs-devel@gnu.org
> Date: Wed, 01 Sep 2021 23:33:51 +0300
> 
> > Out of curiosity: why do you need something like that? what's the use
> > case?
> 
> It's because our LDAP server at work is a bit messy, attributes like
> "cn" and "displayname" sometimes contain first then last name, sometimes
> vice versa, sometimes in Russian, sometimes in English.  I use email
> expansion in message-mode:
> 
> --8<---------------cut here---------------start------------->8---
> (eval-after-load 'message
>   (lambda ()
>     (define-key message-mode-map (kbd "C-c TAB") #'eudc-expand-inline)))
> --8<---------------cut here---------------end--------------->8---
> 
> I want to type first letters of surname in English, press C-c TAB and
> get email autocompleted.  But the only "stable", predictable attribute
> seems to be "sn", which is unfortunately in Russian.  So I do:
> 
> --8<---------------cut here---------------start------------->8---
> (eudc-protocol-set 'eudc-inline-query-format '((sn) (email)) 'ldap)
> (advice-add #'eudc-ldap-simple-query-internal
>             :filter-args #'fg-dotemacs-ldap-advice)
> 
> (defun fg-dotemacs-ldap-advice (args)
>   (cons (mapcar (lambda (attr)
>                   (if (and (eq (car attr) 'sn)
>                            (stringp (cdr attr))
>                            (seq-every-p (lambda (c)
>                                           (eq (aref char-script-table c) 'latin))
>                                         (cdr attr)))
>                       (cons (car attr)
>                             (fg-dotemacs-translit (cdr attr) "cyrillic-translit"))
>                     attr))
>                 (car args))
>         (cdr args)))
> --8<---------------cut here---------------end--------------->8---
> 
> As ugly as it is, but it works.

So what you need is to apply the cyrillic-translit input method, just
not in an interactive context, is that true?

Re: How do you write input methods?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 376 bytes --]

On Thu, Sep 02, 2021 at 09:23:14AM +0300, Eli Zaretskii wrote:

[...]

> So what you need is to apply the cyrillic-translit input method, just
> not in an interactive context, is that true?

That's how I understood it: transliterate a given string [1], instead of
a "stream of input events". Sounds useful to me :)

Cheers

[1] according to a given input method's rules

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How do you write input methods?

[Michael Welsh Duggan]

[-- Attachment #1: Type: text/plain, Size: 516 bytes --]

<tomas@tuxteam.de> writes:

> On Thu, Sep 02, 2021 at 09:23:14AM +0300, Eli Zaretskii wrote:
>
> [...]
>
>> So what you need is to apply the cyrillic-translit input method, just
>> not in an interactive context, is that true?
>
> That's how I understood it: transliterate a given string [1], instead of
> a "stream of input events". Sounds useful to me :)
>
> Cheers
>
> [1] according to a given input method's rules
>
>  - t

I wrote something like this a while back as an experiment.  Here's what
I came up with:


[-- Attachment #2: quail-transform.el --]
[-- Type: application/emacs-lisp, Size: 1869 bytes --]

[-- Attachment #3: Type: text/plain, Size: 42 bytes --]


-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: How do you write input methods?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

On Thu, Sep 02, 2021 at 08:59:54PM -0400, Michael Welsh Duggan wrote:
> <tomas@tuxteam.de> writes:

[...]

> > That's how I understood it: transliterate a given string [1], instead of
> > a "stream of input events". Sounds useful to me :)
> >
> > Cheers
> >
> > [1] according to a given input method's rules
> >
> >  - t
> 
> I wrote something like this a while back as an experiment.  Here's what
> I came up with:

Nice :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How do you write input methods?

[Filipp Gunbin]

On 02/09/2021 09:23 +0300, Eli Zaretskii wrote:

>> From: Filipp Gunbin <fgunbin@fastmail.fm>
>> Cc: andremegafone@gmail.com,  perry@piermont.com,  emacs-devel@gnu.org
>> Date: Wed, 01 Sep 2021 23:33:51 +0300
>>
>> > Out of curiosity: why do you need something like that? what's the use
>> > case?
>>
>> It's because our LDAP server at work is a bit messy, attributes like
>> "cn" and "displayname" sometimes contain first then last name, sometimes
>> vice versa, sometimes in Russian, sometimes in English.  I use email
>> expansion in message-mode:
>>
>> --8<---------------cut here---------------start------------->8---
>> (eval-after-load 'message
>>   (lambda ()
>>     (define-key message-mode-map (kbd "C-c TAB") #'eudc-expand-inline)))
>> --8<---------------cut here---------------end--------------->8---
>>
>> I want to type first letters of surname in English, press C-c TAB and
>> get email autocompleted.  But the only "stable", predictable attribute
>> seems to be "sn", which is unfortunately in Russian.  So I do:
>>
>> --8<---------------cut here---------------start------------->8---
>> (eudc-protocol-set 'eudc-inline-query-format '((sn) (email)) 'ldap)
>> (advice-add #'eudc-ldap-simple-query-internal
>>             :filter-args #'fg-dotemacs-ldap-advice)
>>
>> (defun fg-dotemacs-ldap-advice (args)
>>   (cons (mapcar (lambda (attr)
>>                   (if (and (eq (car attr) 'sn)
>>                            (stringp (cdr attr))
>>                            (seq-every-p (lambda (c)
>>                                           (eq (aref char-script-table c) 'latin))
>>                                         (cdr attr)))
>>                       (cons (car attr)
>>                             (fg-dotemacs-translit (cdr attr) "cyrillic-translit"))
>>                     attr))
>>                 (car args))
>>         (cdr args)))
>> --8<---------------cut here---------------end--------------->8---
>>
>> As ugly as it is, but it works.
>
> So what you need is to apply the cyrillic-translit input method, just
> not in an interactive context, is that true?

Yes, right.