bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Filipp Gunbin]

emacs -Q
C-u C-\ russian-computer
C-x (
<type any letter>
C-x )
C-x e

The last command inserts two chars instead of one.


In GNU Emacs 27.0.50 (build 2, x86_64-apple-darwin17.6.0)
Repository revision: 6ffc6a698f5d425e402c35010394cdb17d8888ce
System Description:  Mac OS X 10.13.5

Configured using:
 'configure CC=/usr/bin/gcc --without-all --without-ns --with-dbus
 --with-file-notification=no --with-gnutls --with-modules --with-xml2
 --with-zlib'

Configured features:
GNUTLS LIBXML2 ZLIB MODULES

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: ru_RU.UTF-8
  locale-coding-system: utf-8-unix

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> From: Filipp Gunbin <fgunbin@fastmail.fm>
> Date: Tue, 10 Jul 2018 01:41:01 +0300
> 
> 
> emacs -Q
> C-u C-\ russian-computer
> C-x (
> <type any letter>
> C-x )
> C-x e
> 
> The last command inserts two chars instead of one.
> 
> 
> In GNU Emacs 27.0.50 (build 2, x86_64-apple-darwin17.6.0)
> Repository revision: 6ffc6a698f5d425e402c35010394cdb17d8888ce
> System Description:  Mac OS X 10.13.5

Thanks.  I see this in the current emacs-26 branch as well.  If you
can afford it, would you please bisect this bug?  I'd like to at least
know how long ago did we break this (I'm afraid not too long).

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Filipp Gunbin]

On 10/07/2018 19:18 +0300, Eli Zaretskii wrote:

>> From: Filipp Gunbin <fgunbin@fastmail.fm>
>> Date: Tue, 10 Jul 2018 01:41:01 +0300
>>
>>
>> emacs -Q
>> C-u C-\ russian-computer
>> C-x (
>> <type any letter>
>> C-x )
>> C-x e
>>
>> The last command inserts two chars instead of one.
>>
>>
>> In GNU Emacs 27.0.50 (build 2, x86_64-apple-darwin17.6.0)
>> Repository revision: 6ffc6a698f5d425e402c35010394cdb17d8888ce
>> System Description:  Mac OS X 10.13.5
>
> Thanks.  I see this in the current emacs-26 branch as well.  If you
> can afford it, would you please bisect this bug?  I'd like to at least
> know how long ago did we break this (I'm afraid not too long).

I was trying to find the "good" commit to start bisect with, got past
25.1 (bug present there), but cannot build any of 24.x:

Loading /Users/fgunbin/src/emacs-bisect/lisp/button.el (source)...
Loading /Users/fgunbin/src/emacs-bisect/lisp/startup.el (source)...
Wrong type argument: listp, 35183230188054
make[1]: *** [Makefile:757: bootstrap-emacs] Error 1
make[1]: Leaving directory '/Users/fgunbin/src/emacs-bisect/src'
make: *** [Makefile:382: src] Error 2

I'm not sure what to do with this error.

Thanks,
Filipp

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Glenn Morris]

Filipp Gunbin wrote:

> I was trying to find the "good" commit to start bisect with, got past
> 25.1 (bug present there), but cannot build any of 24.x:

Maybe your toolchain is tool new/old/something. Anyway, builds for me,
and bisects to 30a6b1f81412044aa7dda5573b0142a0a03c4fd3.

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  32108@debbugs.gnu.org
> Date: Tue, 10 Jul 2018 20:54:31 -0400
> 
> Filipp Gunbin wrote:
> 
> > I was trying to find the "good" commit to start bisect with, got past
> > 25.1 (bug present there), but cannot build any of 24.x:
> 
> Maybe your toolchain is tool new/old/something. Anyway, builds for me,
> and bisects to 30a6b1f81412044aa7dda5573b0142a0a03c4fd3.

Thanks!

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  32108@debbugs.gnu.org
> Date: Tue, 10 Jul 2018 20:54:31 -0400
> 
> bisects to 30a6b1f81412044aa7dda5573b0142a0a03c4fd3.

AFAICS, this happens because when an input method is active, read_char
is called recursively from quail-start-translation, and that recursive
call records the same character for the second time.

Any ideas except introduction of a new variable, inhibit-record-char,
which quail-start-translation would bind to prevent recording of the
same key more than once?

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Stefan Monnier]

> AFAICS, this happens because when an input method is active, read_char
> is called recursively from quail-start-translation, and that recursive
> call records the same character for the second time.
>
> Any ideas except introduction of a new variable, inhibit-record-char,
> which quail-start-translation would bind to prevent recording of the
> same key more than once?

I haven't taken the time to look at the details of why read_char sees
the character twice, but maybe we can use the (t . EVENT) format of
events which says that EVENT shouldn't be recorded (tho I'm not sure
it's the same form of "recording" and I'm not sure either that we can
apply this trick to the right "copy" of the two).


        Stefan

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Glenn Morris <rgm@gnu.org>, fgunbin@fastmail.fm, 32108@debbugs.gnu.org
> Date: Wed, 11 Jul 2018 08:58:25 -0400
> 
> I haven't taken the time to look at the details of why read_char sees
> the character twice, but maybe we can use the (t . EVENT) format of
> events which says that EVENT shouldn't be recorded (tho I'm not sure
> it's the same form of "recording" and I'm not sure either that we can
> apply this trick to the right "copy" of the two).

I think I'm missing something: how does (t . EVENT) help us in this
case?  What we need is a way to distinguish between a character that
gets re-processed by quail.el from the use case described in the
reference mentioned in the log message of commit 30a6b1f, which was
the reason why we added the second call to record_char to read_char,
where previously there was none.  Don't both of these use cases use
the same machinery?

Thanks.

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> Date: Wed, 11 Jul 2018 12:29:03 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: fgunbin@fastmail.fm, 32108@debbugs.gnu.org
> 
> > From: Glenn Morris <rgm@gnu.org>
> > Cc: Eli Zaretskii <eliz@gnu.org>,  32108@debbugs.gnu.org
> > Date: Tue, 10 Jul 2018 20:54:31 -0400
> > 
> > bisects to 30a6b1f81412044aa7dda5573b0142a0a03c4fd3.
> 
> AFAICS, this happens because when an input method is active, read_char
> is called recursively from quail-start-translation, and that recursive
> call records the same character for the second time.
> 
> Any ideas except introduction of a new variable, inhibit-record-char,
> which quail-start-translation would bind to prevent recording of the
> same key more than once?

No other ideas, so I pushed to the master branch a fix along the above
lines (except it turned out to be slightly more tricky due to input
methods that translate sequences of two or more keys).  Please test.

Btw, it turned out the offending commit also caused us in this
situation to output keys twice to the dribble file and insert them
twice into recent-keys, so this fix resolves those problems as well.

Let's see how much I broke this time...

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Filipp Gunbin]

On 21/07/2018 12:15 +0300, Eli Zaretskii wrote:

> No other ideas, so I pushed to the master branch a fix along the above
> lines (except it turned out to be slightly more tricky due to input
> methods that translate sequences of two or more keys).  Please test.

LGTM, it indeed fixes the problem, thanks!

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> From: Filipp Gunbin <fgunbin@fastmail.fm>
> Cc: rgm@gnu.org,  monnier@iro.umontreal.ca,  32108@debbugs.gnu.org
> Date: Mon, 23 Jul 2018 21:38:29 +0300
> 
> On 21/07/2018 12:15 +0300, Eli Zaretskii wrote:
> 
> > No other ideas, so I pushed to the master branch a fix along the above
> > lines (except it turned out to be slightly more tricky due to input
> > methods that translate sequences of two or more keys).  Please test.
> 
> LGTM, it indeed fixes the problem, thanks!

Thanks for testing.  I'm closing the bug, feel free to reopen if some
issues arise.

bug#32108: 27.0.50; Insertion of char from input method during macro invocation duplicates the char

[Eli Zaretskii]

> From: Filipp Gunbin <fgunbin@fastmail.fm>
> Cc: 32108@debbugs.gnu.org
> Date: Wed, 11 Jul 2018 01:38:23 +0300
> 
> > Thanks.  I see this in the current emacs-26 branch as well.  If you
> > can afford it, would you please bisect this bug?  I'd like to at least
> > know how long ago did we break this (I'm afraid not too long).
> 
> I was trying to find the "good" commit to start bisect with, got past
> 25.1 (bug present there), but cannot build any of 24.x:

OK, thanks.  I see that Emacs 24.5 doesn't have the problem, so the
bug was introduced in development of Emacs 25.