textconv.c

textconv.c

[Eli Zaretskii]

Please move the code which manipulates buffer text and the gap to
insdel.c.  And I don't really understand why you needed to add
copy_buffer, since we already have insert_from_buffer and copy_text.

The rest (textconv_query) should go to some existing file(fns.c,
perhaps?).  There's no justification for a new file for such a little
code.

Thanks.

P.S. I wonder why you couldn't ask about this before doing this stuff.
This list is supposed to be used for such design discussions.

Re: textconv.c

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> Please move the code which manipulates buffer text and the gap to
> insdel.c.  And I don't really understand why you needed to add
> copy_buffer, since we already have insert_from_buffer and copy_text.
>
> The rest (textconv_query) should go to some existing file(fns.c,
> perhaps?).  There's no justification for a new file for such a little
> code.
>
> Thanks.

That code is going to be significantly expanded in the feature/android
branch, so I expect it will become much larger (and definitely worth its
own file.)

`insert_from_buffer' inserts text into a buffer, instead of copying out
of a buffer, and `copy_text' seems to not handle the gap nor work on
character positions.  The work on the feature/android branch will later
be useful for PGTK as well.

I implemented this feature on X in order to have an implementation of
text conversion on a free operating system.

> P.S. I wonder why you couldn't ask about this before doing this stuff.
> This list is supposed to be used for such design discussions.

I didn't think the small amount of code that was installed amounted to a
significant design change.

There will be many more changes on the feature/android branch, since
Android input methods more or less can't work without the ability to
perform arbitrary modifications to buffer text, and the ability to
obtain buffer contents and the position of the point in the buffer being
displayed.

My idea is that on each redisplay after the point in a frame's selected
window changes, and each change to buffer contents in the selected
window around point, 600 characters (which is the absolute minimum
required to satisfy the input method-side buffer content analysis)
around point will be transferred to the thread which runs the Android
GUI and be made available to input methods, and any editing requests
from input methods will be sent back to Emacs, translated into ``text
conversion'' requests, and then performed in the selected window's
buffer.

textconv.c will contain the code that figures out what text to report to
the input method, and how to carry out its editing requests
(i.e. whether or not to convert text insertion and deletion into key
events, or to insert that text directly into the buffer.)

I have a sinking suspicion that this is going to be non trivial.

I tried five or six variations of using a standalone buffer for
character compositions from Android input methods before concluding that
such an approach can not work, because input methods demand text they
insert to later appear inside a query for the contents of the buffer,
and to be able to later delete the text themselves.  Basically, the
problem is this: if you type

  h e l l o SPC

and the input method inserts ``hello '', it will remember that it
inserted ``hello '' at the current position of point.  When you later
press backspace, instead of sending a delete key event, the input method
just asks you to delete the text that was where it thinks ``hello '' is.

In effect, this means that when you type DEL, Emacs actually receives:

  ``replace text from 0 to 6 with the preconversion text hello''

and if you keep pressing DEL after that, Emacs doesn't get any delete
key events, but rather:

  ``replace the preconversion text with hell''
  ``replace the preconversion text with hel''
  ``replace the preconversion text he''
  ``replace the preconversion text h''
  ``delete the preconversion text''

until the word ``hello'' disappears from the buffer.  Then, instead of
sending more delete key events, the input method asks for 600 characters
before the current position of the cursor, analyzes the text there, and
performs the same process on each character, until it cannot find any
more text, at which point it stops sending any events altogether.

Likewise, when inserting text, the input method does not send any key
events.  Instead, when you type ``hello'' and finally SPC, the input
method simply tells Emacs to:

  ``tell me where the point is''

suppose Emacs responds with the number ``42''.  The input method will
then ask:

  ``give me 100 before 42, and then 5 characters after 42''

at which point it performs analysis on both segments of extracted text
in order to determine whether or not to punctuate the word, or even to
insert any text at all.  Finally, it asks:

  ``insert the text hello after 42''

which Emacs is supposed to obey.

The same applies in a lesser degree to newer input methods on Wayland
and X11 systems.  The current support we have there is quite buggy and
does not work with input methods designed for handhelds, and input
methods which perform any kind of analysis on text surrounding a buffer
did not work at all before the change I installed.

I stayed up very late last night trying several different approaches to
the problem before concluding that for input methods to work on Android
(and probably Wayland based systems after several years), Emacs must
provide a 100% by-the-book implementation of what input methods expect.
Here are two examples of what approaches I tried, and how it blew up.

If you provide an editing buffer that is always completely empty, the
input method refuses to send any events at all.  The input method which
comes with the Android emulator complains loudly about Emacs
``misbehaving'' and not reporting back changes made by the input method
itself.  The third party input method AnySoftKeyboard doesn't do that,
but insteads assumes Emacs is buggy and proceeds to operate on the text
it thinks it inserted, so when you hit DEL it ends up sending the word
you previously inserted with a single letter chopped off the end,
instead of a DEL key event.

If you provide an editing buffer that contains any preconversion text,
but clears itself and sends its contents to Emacs every time
preconversion ends, then AnySoftKeyboard simply ignores the buffer being
cleared, while the Android emulator's input method locks up for a while
to re-initialize itself.  The emulator's input method always inserts
spaces after the word itself, and without signalling any kind of
preconversion, and it does not insert spaces after it locks up, in
effect making it impossible to insert any spaces at all.

I tried more approaches, but I can't remember what exactly they were or
why they didn't work.

Re: textconv.c

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Sun, 12 Feb 2023 22:05:27 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Please move the code which manipulates buffer text and the gap to
> > insdel.c.  And I don't really understand why you needed to add
> > copy_buffer, since we already have insert_from_buffer and copy_text.
> >
> > The rest (textconv_query) should go to some existing file(fns.c,
> > perhaps?).  There's no justification for a new file for such a little
> > code.
> >
> > Thanks.
> 
> That code is going to be significantly expanded in the feature/android
> branch, so I expect it will become much larger (and definitely worth its
> own file.)

I don't see how this changes the situation in any significant way.  If
and when we see that the added stuff is too large, we can then decide
to split some of it.  But not in advance, and we are not there yet.

> `insert_from_buffer' inserts text into a buffer, instead of copying out
> of a buffer

I don't see why this is important, since you can switch the buffer
temporarily.  We do this all over the place, since insdel.c always
works on the current buffer.

> and `copy_text' seems to not handle the gap nor work on character
> positions.

Look at its callers, and you will see how that part is done.  In a
nutshell, you prepare the gap in advance, and then call copy_text.

> There will be many more changes on the feature/android branch, since
> Android input methods more or less can't work without the ability to
> perform arbitrary modifications to buffer text, and the ability to
> obtain buffer contents and the position of the point in the buffer being
> displayed.

[lots of details omitted]

And you intended to produce code which supports this without any
discussions of the architecture and design?  I'm surprised, to say the
least.  This has to be discussed, with the participation of everyone
on board who knows about the Emacs internals related to these issues.
We have here a significant amount of knowledge, expertise, and past
experience with similar issues, and disregarding that and trying to
solve this by your lone self is at the very least unwise.

My suggestion is that you describe the problem(s), i.e. what these
input methods expect from the client application, in enough detail
that will allow people here think about it and suggest solutions.
Please don't write even a single line of code before such a
description is posted and people have enough time to respond with
suggestions, ideas, and questions.  (I have already a couple of ideas,
but will withhold them until I'm convinced that I understand the
problems to be solved.)

P.S. And please start a thread with a new, more meaningful name when
you post those details.

Thanks.

Re: textconv.c

[Eli Zaretskii]

> Date: Sun, 12 Feb 2023 16:32:16 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> My suggestion is that you describe the problem(s), i.e. what these
> input methods expect from the client application, in enough detail
> that will allow people here think about it and suggest solutions.

Btw, if this stuff is described somewhere, like descriptions of some
protocols and/or standards, pointing to that might save you a lot of
writing.

Re: textconv.c

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 12 Feb 2023 16:32:16 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: emacs-devel@gnu.org
>> 
>> My suggestion is that you describe the problem(s), i.e. what these
>> input methods expect from the client application, in enough detail
>> that will allow people here think about it and suggest solutions.
>
> Btw, if this stuff is described somewhere, like descriptions of some
> protocols and/or standards, pointing to that might save you a lot of
> writing.

Here is the InputConnection protocol that Emacs must implement:

  https://developer.android.com/reference/android/view/inputmethod/InputConnection

and here is the InputMethodManager that Emacs must call:

  https://developer.android.com/reference/android/view/inputmethod/InputMethodManager

and here is the `text-input-unstable-v3' protocol currently used by most
Wayland compositors.  It is soon to be replaced by
`text-input-unstable-v4', which I have not yet investigated:

  https://wayland.app/protocols/text-input-unstable-v3

The requests of interest in the Android InputConnection protocol are
commitCompletion, commitCorrection, commitText, deleteSurroundingText,
finishComposingText, getExtractedText, getSelectedText,
getSurroundingText, getTextBeforeCursor, getTextAfterCursor,
replaceText, setSelection, requestCursorUpdates, requestTextBoundsInfo,
setComposingRegion, setComposingText, setSelection, and takeSnapshot.

These must be implemented by Emacs for input methods to work correctly.
Emacs must also call the following APIs in InputMethodManager:

  restartInput
  invalidateInput
  updateCursor
  updateCursorAnchorInfo
  updateExtractedText
  updateSelection.

The requests of interest in the Wayland text-input-unstable-v3 protocol
are preedit_string, commit_string, delete_surrounding_text,
set_surrounding_text; the rest are optional (but this is supposed to
change in the V4 protocol.)

Android input methods (was: Re: textconv.c)

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't see why this is important, since you can switch the buffer
> temporarily.  We do this all over the place, since insdel.c always
> works on the current buffer.

The text is copied into a C ``char *'', not into another buffer.

> [lots of details omitted]
>
> And you intended to produce code which supports this without any
> discussions of the architecture and design?  I'm surprised, to say the
> least.  This has to be discussed, with the participation of everyone
> on board who knows about the Emacs internals related to these issues.
> We have here a significant amount of knowledge, expertise, and past
> experience with similar issues, and disregarding that and trying to
> solve this by your lone self is at the very least unwise.

[...]

> My suggestion is that you describe the problem(s), i.e. what these
> input methods expect from the client application, in enough detail
> that will allow people here think about it and suggest solutions.
> Please don't write even a single line of code before such a
> description is posted and people have enough time to respond with
> suggestions, ideas, and questions.  (I have already a couple of ideas,
> but will withhold them until I'm convinced that I understand the
> problems to be solved.)
>
> P.S. And please start a thread with a new, more meaningful name when
> you post those details.

Ok, now done.

Basically, today, on Android (but also on other platforms), input
methods desire fine grained control over buffer contents, as they start
to provide more and more features aside from simply composing text.

This is mainly seen on Android, but they have appeared in other systems
as well, most notably GNOME on handheld devices.  What is being said
below applies to those input methods as well.

In the past, input methods more or less worked like this: when a key
press arrives, the input method receives it first, performs
transformations, and either returns it to the application or inserts it
into a composition buffer.  Once the composition completes (through the
user pressing enter, or some other similar key), the text is sent to
Emacs, which converts each character inside into a key press, to be
inserted by self-insert-command.

On Android, input methods work the other way around.  They do the text
insertion and deletion themselves, all whilst querying the text editor
about the position of the caret (point) and the text around it for
reference.  Emacs is only told to insert or delete text at a specific
position in its buffer, and is obligated to inform the input method
about changes around the caret.

If Emacs makes a change to the buffer outside the area in which the
input method expresses interest, then it is obligated to ``restart'' the
input method.  This takes a significant amount of time to complete.

Sometimes, the input method will also tell Emacs to mark a portion of
the buffer as ``preconversion text'' (or a ``composing span''), which is
an ephemeral region which may be replaced by the input method by some
other text, or deleted altogether.  The intention is that the input
method will display temporary edits to the buffer used to display
the contents of any on-going composition to the user within that
ephemeral region.

Input methods on Android make extensive use of this functionality, even
for input in languages that utilize Latin or Cyrillic script.  Consider
a user who wants to delete the words ``tomorrow afternoon'' and replace
them with ``next Thursday'' in the following buffer:

  Why don't we both look through all the television channels tomorrow
  afternoon for offensive content we can complain about?

on a desktop, this would be simple; assume that point is already after
the word ``tomorrow afternoon''.  The user will press the delete key
enough times to delete ``tomorrow afternoon'', and then type in ``next
Thursday''.

On Android, this is completely different.  Once the input method (and on
screen keyboard) is displayed, it looks at the text surrounding the
point.  It sees the word:

  ``afternoon''

immediately before point, and the text:

  `` for''

immediately after.  Since the caret (point) is closer to the word
``afternoon'' than it is to the word ``for'', it now considers itself to
be editing the word ``for''.

The input method then tells Emacs that ``afternoon'' is now the
ephemeral region, by issuing a request along the lines of:

  ``set the preconversion region to 69-78''

Emacs is now expected to indicate, by displaying an underline, that the
IME is now editing the word ``afternoon''.  As the user starts to press
delete in the input method, the input method starts to issue requests to
replace the contents of the preconversion region with something else:

  ``replace the preconversion region contents with afternoo''
  ``replace the preconversion region contents with afterno''
  ``replace the preconversion region contents with aftern''
  ``replace the preconversion region contents with after''
  ``replace the preconversion region contents with afte''
  ``replace the preconversion region contents with aft''
  ``replace the preconversion region contents with af''
  ``replace the preconversion region contents with a''
  ``remove the preconversion region entirely''

at that point, the input method asks for the contents of the buffer
before point again, and repeats the whole process.  Point is now 69,
immediately after a newline character, which cannot be meaningfully
composed.  Input methods have been observed to do one of two things:
either the input method will issue a request:

  ``delete one character before 69''

or it will say:

  ``set the preconversion region to 68-69''
  ``remove the preconversion region''

sometimes, the input method will start to delete entire words at a time.
When that happens, the input method will look backwards and ask for the
text:

  ``tomorrow\n''

and simply ask Emacs:

  ``delete 9 characters after the position 60''

or perhaps

  ``set the preconversion region to 60-69''
  ``remove the preconversion region''

or perhaps some other combination that I have yet to see in practice.
Now assume that the user changes his mind in the middle of the
operation, say immediately after ``afternoon'' has become ``aftern''.
The input method may display the text ``afternoon'' in a button, to
allow him to undo the change immediately.  If that is pressed, Emacs
might receive:

  ``replace the preconversion region contents with afternoon''
  ``stop preconverting text''

or alternatively:

  ``stop preconverting text''
  ``insert the text oon after 75''

or some other request.

All of this is behavior I have observed CJK and English input methods
perform.  An input method is not obligated to behave in any way like
what I have described above, as long as it constrains its edits to some
reasonable position (600 characters) around the caret; if it makes edits
any further away from the caret than that, the behavior of the
application is undefined.  i.e. it might also be valid for the input
method to say:

  ``replace 0-123 with <random string>''
  ``replace 0-123 with <random string>''
  ``replace 0-123 with <random string>''
  ``replace 0-123 with <random string>''
  ``replace 0-123 with <random string>''
  ``replace 0-123 with <random string>''
  ``replace 0-123 with <random string>''

over and over again, though I don't see the utility in that.  But the
input method will stop working properly until the next time it is reset
if it doesn't see the replacement reflected in Emacs's own buffer
contents.

Sometimes, an input method will also monitor changes to the caret
position.  At this point, Emacs is obligated to report any changes to
the on screen caret to the input method, so it knows where it should
begin to make edits from.

An input method might also ask for a region of text to be ``extracted'',
which means Emacs must report each change to the buffer that modifies
said region to the input method, but is relieved of the obligation to
reset the input method as long as a ``major change'' (whatever that
means) has not happened to the buffer contents, or outside the extracted
text.  What I have observed is that the region of extracted text is wide
enough to perform actions such as refilling a paragraph or indenting a
line without resetting the input method, but not much more than that.

In any case, the conclusion is that Emacs must present a completely
correct view of the buffer contents of the selected window and the
location of its point to the input method, correctly report edits made
by the input method to the buffer contents and any edits made by Emacs
after that, and dilligently report changes to extracted text and/or
reset the input method on ``major changes'' such as the selected buffer
or window changing, or edits happening outside extracted text.

Otherwise, the behavior of the input method becomes undefined (and
nasty.)

Now, it is sometimes possible to disable the input method and to simply
work with an on screen keyboard (which is what the Android port
currently does), but that precludes entering any non-ASCII text, and is
a luxury which is only affored by several input methods.  Also, it
wouldn't be out of character for GNOME to demand applications implement
input method support their ``right way'' either, at some point in the
future, so we will have to implement this properly, if not now, then at
some point in the future.

Thanks.

Re: Android input methods (was: Re: textconv.c)

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Sun, 12 Feb 2023 23:27:00 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I don't see why this is important, since you can switch the buffer
> > temporarily.  We do this all over the place, since insdel.c always
> > works on the current buffer.
> 
> The text is copied into a C ``char *'', not into another buffer.

A buffer in Emacs is just a linear array of bytes, so the above
supports copying into a buffer and from a buffer as well.

Once again, look at the callers of copy_text, and you will see it.

> On Android, input methods work the other way around.  They do the text
> insertion and deletion themselves, all whilst querying the text editor
> about the position of the caret (point) and the text around it for
> reference.  Emacs is only told to insert or delete text at a specific
> position in its buffer, and is obligated to inform the input method
> about changes around the caret.

Maybe I'm missing something, but where's the problem in this?  We have
all this implemented in insdel.c, and a command can tell Emacs to
insert, delete, or replace text.  IOW, so far you describe the
communications between Emacs and the input method as a stream of
insert/delete/replace commands, for which we already have all the
infrastructure, and the only thing that's missing is the low-level
communication protocol with the input method, which should issue those
commands.

> If Emacs makes a change to the buffer outside the area in which the
> input method expresses interest, then it is obligated to ``restart'' the
> input method.  This takes a significant amount of time to complete.

What does Emacs have to do to "restart" an input method? what does
this mean in practice?

And how much time is "a significant amount of time" in this case?

> Sometimes, the input method will also tell Emacs to mark a portion of
> the buffer as ``preconversion text'' (or a ``composing span''), which is
> an ephemeral region which may be replaced by the input method by some
> other text, or deleted altogether.

Again, I see no problems with this: markers will handle such a region.

> at that point, the input method asks for the contents of the buffer
> before point again, and repeats the whole process.

What does "asks for the contents of the buffer" mean in practice? what
does the input method tell Emacs, and what does it expect from Emacs
in response? if it expects us to send it the text it requested, then
how (by which medium and protocol) is that text sent by Emacs to the
input method?

> All of this is behavior I have observed CJK and English input methods
> perform.  An input method is not obligated to behave in any way like
> what I have described above, as long as it constrains its edits to some
> reasonable position (600 characters) around the caret; if it makes edits
> any further away from the caret than that, the behavior of the
> application is undefined.

"Undefined" meaning that input methods will not usually do this?  Or
what does it mean?

> But the input method will stop working properly until the next time
> it is reset if it doesn't see the replacement reflected in Emacs's
> own buffer contents.

How can the input method know that the replacement is reflected in the
Emacs buffer?

> Sometimes, an input method will also monitor changes to the caret
> position.  At this point, Emacs is obligated to report any changes to
> the on screen caret to the input method, so it knows where it should
> begin to make edits from.

Again no problem for us: "we have the technology", in the form of
buffer-modification hooks.

> An input method might also ask for a region of text to be ``extracted'',
> which means Emacs must report each change to the buffer that modifies
> said region to the input method

Same.

> In any case, the conclusion is that Emacs must present a completely
> correct view of the buffer contents of the selected window and the
> location of its point to the input method, correctly report edits made
> by the input method to the buffer contents and any edits made by Emacs
> after that, and dilligently report changes to extracted text and/or
> reset the input method on ``major changes'' such as the selected buffer
> or window changing, or edits happening outside extracted text.

And the problem with doing what the input method expects is...?

> Now, it is sometimes possible to disable the input method and to simply
> work with an on screen keyboard (which is what the Android port
> currently does), but that precludes entering any non-ASCII text, and is
> a luxury which is only affored by several input methods.

You mean, if I select, say, a Cyrillic keyboard and start typing, what
Emacs will see is the above complex insert/delete/replace commands
instead of a series of single Cyrillic characters?  And this only
happens with non-ASCII characters, but not with ASCII?

Re: Android input methods

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> Maybe I'm missing something, but where's the problem in this?  We have
> all this implemented in insdel.c, and a command can tell Emacs to
> insert, delete, or replace text.  IOW, so far you describe the
> communications between Emacs and the input method as a stream of
> insert/delete/replace commands, for which we already have all the
> infrastructure, and the only thing that's missing is the low-level
> communication protocol with the input method, which should issue those
> commands.

My problem is that modes such as `electric-indent-mode' expect, for
example, newline characters to be inserted by the return key, and do not
indent if the change is made directly by the input method.

So at least until the other major modes are adjusted to work with these
input methods, there will need to be complicated conversion between
input method editing commands and the key events which will normally
perform the same thing.

>> If Emacs makes a change to the buffer outside the area in which the
>> input method expresses interest, then it is obligated to ``restart'' the
>> input method.  This takes a significant amount of time to complete.
>
> What does Emacs have to do to "restart" an input method? what does
> this mean in practice?

In practice, this means the input method will hide and reshow its
editing window.

> And how much time is "a significant amount of time" in this case?

Up to a second.

>> Sometimes, the input method will also tell Emacs to mark a portion of
>> the buffer as ``preconversion text'' (or a ``composing span''), which is
>> an ephemeral region which may be replaced by the input method by some
>> other text, or deleted altogether.
>
> Again, I see no problems with this: markers will handle such a region.
>
>> at that point, the input method asks for the contents of the buffer
>> before point again, and repeats the whole process.
>
> What does "asks for the contents of the buffer" mean in practice? what
> does the input method tell Emacs, and what does it expect from Emacs
> in response? if it expects us to send it the text it requested, then
> how (by which medium and protocol) is that text sent by Emacs to the
> input method?

When this happens, the input method specifies an offset around the caret
or absolute buffer positions, and calls a callback.  Emacs is then
supposed to reply by returning the text from that callback.

>> All of this is behavior I have observed CJK and English input methods
>> perform.  An input method is not obligated to behave in any way like
>> what I have described above, as long as it constrains its edits to some
>> reasonable position (600 characters) around the caret; if it makes edits
>> any further away from the caret than that, the behavior of the
>> application is undefined.
>
> "Undefined" meaning that input methods will not usually do this?  Or
> what does it mean?

It means that input methods are not allowed to do this, so applications
don't have to allow it to happen.

> How can the input method know that the replacement is reflected in the
> Emacs buffer?

It will either query for the buffer contents after some time, or monitor
for the change.

>> Sometimes, an input method will also monitor changes to the caret
>> position.  At this point, Emacs is obligated to report any changes to
>> the on screen caret to the input method, so it knows where it should
>> begin to make edits from.
>
> Again no problem for us: "we have the technology", in the form of
> buffer-modification hooks.
>
>> An input method might also ask for a region of text to be ``extracted'',
>> which means Emacs must report each change to the buffer that modifies
>> said region to the input method
>
> Same.
>
>> In any case, the conclusion is that Emacs must present a completely
>> correct view of the buffer contents of the selected window and the
>> location of its point to the input method, correctly report edits made
>> by the input method to the buffer contents and any edits made by Emacs
>> after that, and dilligently report changes to extracted text and/or
>> reset the input method on ``major changes'' such as the selected buffer
>> or window changing, or edits happening outside extracted text.
>
> And the problem with doing what the input method expects is...?

That most of Emacs expects character input events, and not for text to
be changed underneath its nose.  So, Emacs will have to implement (at
least optional) translation between these complex editing commands and
simple character input events.

> You mean, if I select, say, a Cyrillic keyboard and start typing, what
> Emacs will see is the above complex insert/delete/replace commands
> instead of a series of single Cyrillic characters?  And this only
> happens with non-ASCII characters, but not with ASCII?

Yes, that's what will happen right now.

Re: Android input methods

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Mon, 13 Feb 2023 10:21:24 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Maybe I'm missing something, but where's the problem in this?  We have
> > all this implemented in insdel.c, and a command can tell Emacs to
> > insert, delete, or replace text.  IOW, so far you describe the
> > communications between Emacs and the input method as a stream of
> > insert/delete/replace commands, for which we already have all the
> > infrastructure, and the only thing that's missing is the low-level
> > communication protocol with the input method, which should issue those
> > commands.
> 
> My problem is that modes such as `electric-indent-mode' expect, for
> example, newline characters to be inserted by the return key, and do not
> indent if the change is made directly by the input method.

is the only problem with such modes?  Or are there other issues?

> So at least until the other major modes are adjusted to work with these
> input methods, there will need to be complicated conversion between
> input method editing commands and the key events which will normally
> perform the same thing.

OK, so let's talk about the details in those case where we would like
to produce single-key events to Emacs.

How large are the "preconversion text" regions in practice?  You
described quite large regions, but I'm not sure I understand why an
input method would want to manipulate such a large portion of text.
Surely, inserting a single non-ASCII character never needs to
manipulate more than a few character positions?  I mean, to type a
character one usually needs to enter a small number of keystrokes;
anything else would make typing extremely inconvenient and slow.

If input methods are actually modifying relatively small portions of
text (even if they request much larger regions to do that), producing
single-key events from that should not be too hard: all you need is
compare two almost identical stretches of text.  Am I missing
something?

> >> If Emacs makes a change to the buffer outside the area in which the
> >> input method expresses interest, then it is obligated to ``restart'' the
> >> input method.  This takes a significant amount of time to complete.
> >
> > What does Emacs have to do to "restart" an input method? what does
> > this mean in practice?
> 
> In practice, this means the input method will hide and reshow its
> editing window.

The editing window is shown by the input method, not by Emacs, right?
I asked about what Emacs needs to do for this "restarting" action.

> > And how much time is "a significant amount of time" in this case?
> 
> Up to a second.

So the user types something and the response is after 1 sec?

> > What does "asks for the contents of the buffer" mean in practice? what
> > does the input method tell Emacs, and what does it expect from Emacs
> > in response? if it expects us to send it the text it requested, then
> > how (by which medium and protocol) is that text sent by Emacs to the
> > input method?
> 
> When this happens, the input method specifies an offset around the caret
> or absolute buffer positions, and calls a callback.  Emacs is then
> supposed to reply by returning the text from that callback.

No problems here, except the usual issue with our superset of UTF-8.
We'd need to encode the text.  However, for relatively short stretches
of text this should be fast enough.  And the other direction already
exists: decode_coding_gap.

> > How can the input method know that the replacement is reflected in the
> > Emacs buffer?
> 
> It will either query for the buffer contents after some time, or monitor
> for the change.

No problem here, then.

> >> In any case, the conclusion is that Emacs must present a completely
> >> correct view of the buffer contents of the selected window and the
> >> location of its point to the input method, correctly report edits made
> >> by the input method to the buffer contents and any edits made by Emacs
> >> after that, and dilligently report changes to extracted text and/or
> >> reset the input method on ``major changes'' such as the selected buffer
> >> or window changing, or edits happening outside extracted text.
> >
> > And the problem with doing what the input method expects is...?
> 
> That most of Emacs expects character input events, and not for text to
> be changed underneath its nose.

That's no accurate: a process filter can (and many do) insert text
directly into the buffer, in relatively large chunks.

> So, Emacs will have to implement (at least optional) translation
> between these complex editing commands and simple character input
> events.

Yes, but AFAIU only to placate the electric-input features and
similar.

> > You mean, if I select, say, a Cyrillic keyboard and start typing, what
> > Emacs will see is the above complex insert/delete/replace commands
> > instead of a series of single Cyrillic characters?  And this only
> > happens with non-ASCII characters, but not with ASCII?
> 
> Yes, that's what will happen right now.

Strange design.  Any idea why non-ASCII characters get such complex
treatment?

Re: Android input methods

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

>> My problem is that modes such as `electric-indent-mode' expect, for
>> example, newline characters to be inserted by the return key, and do not
>> indent if the change is made directly by the input method.
>
> is the only problem with such modes?  Or are there other issues?

There are many others: consider the vc-dir buffer: when you type ``mmmm
i'', it expects individual key presses, each one of which marks a file
or registers it with the VC system.

> OK, so let's talk about the details in those case where we would like
> to produce single-key events to Emacs.
>
> How large are the "preconversion text" regions in practice?  You
> described quite large regions, but I'm not sure I understand why an
> input method would want to manipulate such a large portion of text.
> Surely, inserting a single non-ASCII character never needs to
> manipulate more than a few character positions?  I mean, to type a
> character one usually needs to enter a small number of keystrokes;
> anything else would make typing extremely inconvenient and slow.
>
> If input methods are actually modifying relatively small portions of
> text (even if they request much larger regions to do that), producing
> single-key events from that should not be too hard: all you need is
> compare two almost identical stretches of text.  Am I missing
> something?

If you insert text, the input method might choose to suggest
replacements for the text, typically of the surrounding word, but also
sometimes up to entire sentences in length.

In addition, there is a mode where the input method displays extracted
text in a window of its own, and only sends the resulting changes back
to Emacs after it finishes.  Such changes can be almost arbitrary in
many cases.

However, I think I've found an easier solution that doesn't involve any
text comparison: we can enable input methods only for editing modes that
derive from `text-mode', and perhaps prog-mode as well, whilst utilizing
the ASCII keyboard fallback on modes which derive from special-mode.

It should be easy to fix electric indentation and other
prog-mode-features to handle text insertion by the input method.

Apparently (and this is not written in any documentation), if you set
the input type to TYPE_CLASS_TEXT without providing any ``input method
connection'', even input methods that normally don't provide the ASCII
keyboard fallback can be convinced to do so.

>> >> If Emacs makes a change to the buffer outside the area in which the
>> >> input method expresses interest, then it is obligated to ``restart'' the
>> >> input method.  This takes a significant amount of time to complete.
>> >
>> > What does Emacs have to do to "restart" an input method? what does
>> > this mean in practice?
>> 
>> In practice, this means the input method will hide and reshow its
>> editing window.
>
> The editing window is shown by the input method, not by Emacs, right?
> I asked about what Emacs needs to do for this "restarting" action.
>
>> > And how much time is "a significant amount of time" in this case?
>> 
>> Up to a second.
>
> So the user types something and the response is after 1 sec?

Yes, if the editor makes an edit that the input method wasn't expecting,
you will see the input method window disappear and reappear again up to
a second later.  It works in practice because editors do not do that.

The period is apparently shorter if you turn off animations, but only
the user can do that in the system settings.

> No problems here, except the usual issue with our superset of UTF-8.
> We'd need to encode the text.  However, for relatively short stretches
> of text this should be fast enough.  And the other direction already
> exists: decode_coding_gap.

This should be easy: we provide character positions and Unicode
characters to the input method, but use the NULL byte for characters
that are not representable in UTF-16 (including those which need to be
represented by surrogate pairs.)

>> So, Emacs will have to implement (at least optional) translation
>> between these complex editing commands and simple character input
>> events.
>
> Yes, but AFAIU only to placate the electric-input features and
> similar.

Please see what I said above about toggling the input method behavior
depending on the major mode of the selected buffer.

>> > You mean, if I select, say, a Cyrillic keyboard and start typing, what
>> > Emacs will see is the above complex insert/delete/replace commands
>> > instead of a series of single Cyrillic characters?  And this only
>> > happens with non-ASCII characters, but not with ASCII?
>> 
>> Yes, that's what will happen right now.
>
> Strange design.  Any idea why non-ASCII characters get such complex
> treatment?

I don't know.  It seems to be an initial oversight that had to be kept
for backwards compatibility reasons, because applications do not expect
key events with (a definite misnomer) the `unicode_char' field set to
some value greater than 127.  I can't find this written down anywhere,
however, except there are simply no keymaps that map keys to larger
characters.

Re: Android input methods

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Mon, 13 Feb 2023 22:37:19 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> My problem is that modes such as `electric-indent-mode' expect, for
> >> example, newline characters to be inserted by the return key, and do not
> >> indent if the change is made directly by the input method.
> >
> > is the only problem with such modes?  Or are there other issues?
> 
> There are many others: consider the vc-dir buffer: when you type ``mmmm
> i'', it expects individual key presses, each one of which marks a file
> or registers it with the VC system.

These commands don't insert text, so I'm unsure how they are
relevant.  The "mmm" is not inserted into a buffer, it is a series of
3 commands.

> > If input methods are actually modifying relatively small portions of
> > text (even if they request much larger regions to do that), producing
> > single-key events from that should not be too hard: all you need is
> > compare two almost identical stretches of text.  Am I missing
> > something?
> 
> If you insert text, the input method might choose to suggest
> replacements for the text, typically of the surrounding word, but also
> sometimes up to entire sentences in length.

This feature should be turned off.  It is incompatible with Emacs.  We
request users to turn off bidi reordering of terminal emulators for
similar reasons.  There's no way we can or should allow external
features do stuff like that, because they will never be as flexible as
Emacs features.

At the very least we should disable them now.  Maybe later we will
find less drastic solutions (or maybe the input methods will grow up
and become friendlier to Emacs).

> In addition, there is a mode where the input method displays extracted
> text in a window of its own, and only sends the resulting changes back
> to Emacs after it finishes.  Such changes can be almost arbitrary in
> many cases.

Turn this off.

> However, I think I've found an easier solution that doesn't involve any
> text comparison: we can enable input methods only for editing modes that
> derive from `text-mode', and perhaps prog-mode as well, whilst utilizing
> the ASCII keyboard fallback on modes which derive from special-mode.

I don't believe this is so easy.  We'd need a more flexible control on
when the input method is enabled and disabled.  Just the major mode is
not fine-grained enough.

> > No problems here, except the usual issue with our superset of UTF-8.
> > We'd need to encode the text.  However, for relatively short stretches
> > of text this should be fast enough.  And the other direction already
> > exists: decode_coding_gap.
> 
> This should be easy: we provide character positions and Unicode
> characters to the input method, but use the NULL byte for characters
> that are not representable in UTF-16 (including those which need to be
> represented by surrogate pairs.)

We already have the machinery to replace un-encodable characters with
a fixed character while encoding, but my point is that we will need to
encode; we cannot just memcpy.  So this will be slower than just
copying, but not terribly so.

Btw, are you saying that the text should be encoded in UTF-16?  Is
that because it's Java?

> > Strange design.  Any idea why non-ASCII characters get such complex
> > treatment?
> 
> I don't know.  It seems to be an initial oversight that had to be kept
> for backwards compatibility reasons, because applications do not expect
> key events with (a definite misnomer) the `unicode_char' field set to
> some value greater than 127.  I can't find this written down anywhere,
> however, except there are simply no keymaps that map keys to larger
> characters.

<Shrug> Even MS-Windows is capable of accepting and processing UTF-16
encoded characters in its character input routines.  So I'm still
puzzled.

Re: Android input methods

[Eli Zaretskii]

> Date: Mon, 13 Feb 2023 17:17:14 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Po Lu <luangruo@yahoo.com>
> > Cc: emacs-devel@gnu.org
> > Date: Mon, 13 Feb 2023 22:37:19 +0800
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > >> My problem is that modes such as `electric-indent-mode' expect, for
> > >> example, newline characters to be inserted by the return key, and do not
> > >> indent if the change is made directly by the input method.
> > >
> > > is the only problem with such modes?  Or are there other issues?
> > 
> > There are many others: consider the vc-dir buffer: when you type ``mmmm
> > i'', it expects individual key presses, each one of which marks a file
> > or registers it with the VC system.
> 
> These commands don't insert text, so I'm unsure how they are
> relevant.  The "mmm" is not inserted into a buffer, it is a series of
> 3 commands.

Come to think of this, we should simply turn off the input method in
these situations.  Basically, all special-modes don't want input
methods to interfere -- except when those modes prompt the user for
some text.  So I think we need a way for Emacs to turn the input
method on and off as needed.

Re: Android input methods

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> These commands don't insert text, so I'm unsure how they are
> relevant.  The "mmm" is not inserted into a buffer, it is a series of
> 3 commands.

Right, but if you type ``mmm'' while the IME is active, then the IME
will try to insert ``mmm'' as text.

> This feature should be turned off.  It is incompatible with Emacs.  We
> request users to turn off bidi reordering of terminal emulators for
> similar reasons.  There's no way we can or should allow external
> features do stuff like that, because they will never be as flexible as
> Emacs features.
>
> At the very least we should disable them now.  Maybe later we will
> find less drastic solutions (or maybe the input methods will grow up
> and become friendlier to Emacs).

That is possible, but we will have to ask users to do that.

> Turn this off.

OK, we already try, but there is no guarantee that this will work.

> I don't believe this is so easy.  We'd need a more flexible control on
> when the input method is enabled and disabled.  Just the major mode is
> not fine-grained enough.

Any ideas there?  I mean, under what precise circumstances should Emacs
enable and/or disable the input method?

> We already have the machinery to replace un-encodable characters with
> a fixed character while encoding, but my point is that we will need to
> encode; we cannot just memcpy.  So this will be slower than just
> copying, but not terribly so.
>
> Btw, are you saying that the text should be encoded in UTF-16?  Is
> that because it's Java?

Yes.  And instead of code points or bytes, the positions given to Emacs
are in 16-bit short units, so to convert them to multibyte character
positions in Emacs without stripping out both unencodable characters and
those that require surrogate pairs will be nasty.

Re: Android input methods

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 14 Feb 2023 09:57:29 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > These commands don't insert text, so I'm unsure how they are
> > relevant.  The "mmm" is not inserted into a buffer, it is a series of
> > 3 commands.
> 
> Right, but if you type ``mmm'' while the IME is active, then the IME
> will try to insert ``mmm'' as text.

Which is why IME should be turned off in that case.  For comparison,
no one in their right mind will type "mmm" in that case while some
Leim input method is active, right?

> > This feature should be turned off.  It is incompatible with Emacs.  We
> > request users to turn off bidi reordering of terminal emulators for
> > similar reasons.  There's no way we can or should allow external
> > features do stuff like that, because they will never be as flexible as
> > Emacs features.
> >
> > At the very least we should disable them now.  Maybe later we will
> > find less drastic solutions (or maybe the input methods will grow up
> > and become friendlier to Emacs).
> 
> That is possible, but we will have to ask users to do that.

If there's no way to do that programmatically, yes.

> > I don't believe this is so easy.  We'd need a more flexible control on
> > when the input method is enabled and disabled.  Just the major mode is
> > not fine-grained enough.
> 
> Any ideas there?  I mean, under what precise circumstances should Emacs
> enable and/or disable the input method?

Don't know yet.  I suggest to have some variable that Lisp can bind to
control this.  Then we'll find the cases where it is needed as we go.

> > Btw, are you saying that the text should be encoded in UTF-16?  Is
> > that because it's Java?
> 
> Yes.  And instead of code points or bytes, the positions given to Emacs
> are in 16-bit short units, so to convert them to multibyte character
> positions in Emacs without stripping out both unencodable characters and
> those that require surrogate pairs will be nasty.

I wouldn't call that "nasty".  AFAIU, we will need a UTF-16 variant of
BYTES_BY_CHAR_HEAD (called, say, UNITS_BY_CHAR_HEAD), and the rest is
relatively simple, like what we do with our internal encoding.

Re: Android input methods

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> Which is why IME should be turned off in that case.  For comparison,
> no one in their right mind will type "mmm" in that case while some
> Leim input method is active, right?

Yes.  I've not had the time to check, but all of these modes derive from
`special-mode' now, right?

> Don't know yet.  I suggest to have some variable that Lisp can bind to
> control this.  Then we'll find the cases where it is needed as we go.

OK, I will implement input method support along the lines of what was
discussed here.

Re: Android input methods

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 14 Feb 2023 23:14:45 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Which is why IME should be turned off in that case.  For comparison,
> > no one in their right mind will type "mmm" in that case while some
> > Leim input method is active, right?
> 
> Yes.  I've not had the time to check, but all of these modes derive from
> `special-mode' now, right?

They should.  But note that those modes could enter recursive-edit in
the minibuffer or elsewhere, and let users edit parts of the display,
in which case we do want the IME back on.

> > Don't know yet.  I suggest to have some variable that Lisp can bind to
> > control this.  Then we'll find the cases where it is needed as we go.
> 
> OK, I will implement input method support along the lines of what was
> discussed here.

Thanks.  I understand that, if single-key input is expected, we are
talking about some intermediate hidden buffer, used to compare what
the IME produces with what was there before, and emit synthetic
keystrokes, is that right?

Re: Android input methods

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> They should.  But note that those modes could enter recursive-edit in
> the minibuffer or elsewhere, and let users edit parts of the display,
> in which case we do want the IME back on.

OK, I will keep that in mind.

>> > Don't know yet.  I suggest to have some variable that Lisp can bind to
>> > control this.  Then we'll find the cases where it is needed as we go.
>> 
>> OK, I will implement input method support along the lines of what was
>> discussed here.
>
> Thanks.  I understand that, if single-key input is expected, we are
> talking about some intermediate hidden buffer, used to compare what
> the IME produces with what was there before, and emit synthetic
> keystrokes, is that right?

Right, if that is all the input method supports.  But Emacs will try
hard to ask the input method to send plain key events first.

Re: Android input methods (was: Re: textconv.c)

[Lynn Winebarger]

On Sun, Feb 12, 2023 at 10:30 AM Po Lu <luangruo@yahoo.com> wrote:
>
> Now, it is sometimes possible to disable the input method and to simply
> work with an on screen keyboard (which is what the Android port
> currently does), but that precludes entering any non-ASCII text, and is
> a luxury which is only affored by several input methods.  Also, it
> wouldn't be out of character for GNOME to demand applications implement
> input method support their ``right way'' either, at some point in the
> future, so we will have to implement this properly, if not now, then at
> some point in the future.

Interesting.  As a user, it sounds like Emacs ought to make itself
available as one of those input methods, rather than delegating its
editing functions to other "input methods".

Lynn