bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Yikai Zhao]

I encountered this bug while testing the mps (scratch/igc) branch. I
cannot reproduce this with the current master branch.

I'm running Linux, X11, fcitx chinese input method.

Fcitx input method is popular among CJK users. When it's enabled, all
character inputs should be displayed in the "fcitx preedit box"; until a
confirmation key (e.g. space) is pressed, the composed characters should
then be inserted into the application (e.g. emacs).

Now, with the mps branch, occasinoally, some key input would NOT go to
the fcitx preedit box; instead, it goes into emacs directly. It happens
regardless of whether the fcitx preedit box is currently active. (aka,
both first-chars and non-first-chars may have this problem). If the
fcitx preedit box is active when is happens, it would remain active.

For example, when I type "niha", it starts at this:

+----------emacs buffer---------------+
| xxxxxx|                             |
|       +-----fcitx box-------+       |
|       | niha                |       |
|       | 你好 你哈 你害 ..   |       |
|       +---------------------+       |
|                                     |
+-------------------------------------+

Then I type "o", the expected behavior is:

+----------emacs buffer---------------+
| xxxxxx|                             |
|       +-----fcitx box-------+       |
|       | nihao               |       |
|       | 你好 你哈 你害 ..   |       |
|       +---------------------+       |
|                                     |
+-------------------------------------+

But instead, what I get is:

+----------emacs buffer---------------+
| xxxxxxo|                             |
|       +-----fcitx box-------+       |
|       | niha                |       |
|       | 你好 你哈 你害 ..   |       |
|       +---------------------+       |
|                                     |
+-------------------------------------+


In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.22.30, cairo version 1.15.10) of 2024-11-24 built on f2908c960c38
Repository revision: 0756b1f2f5452d715396f66d887c137776e360ca
Repository branch: scratch/igc
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Ubuntu 22.04.5 LTS

Configured using:
 'configure --prefix=/work/dist/AppDir --disable-locallisppath
 --with-native-compilation=aot --with-json --with-threads --with-sqlite3
 --with-tree-sitter --with-dbus --with-xml2 --with-modules --with-libgmp
 --with-gpm --with-lcms2 --with-mps --with-x --without-pgtk
 --without-gconf --with-x-toolkit=gtk3 --with-xft --without-tiff
 --without-imagemagick --with-gif --with-png --with-rsvg --with-webp
 --with-harfbuzz --with-cairo --with-libotf --without-m17n-flt
 --with-jpeg emacs_cv_jpeglib=/usr/lib/x86_64-linux-gnu/libjpeg.a
 CPPFLAGS=-I/work/dist/AppDir/include LDFLAGS=-L/work/dist/AppDir/lib'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LCMS2
LIBOTF LIBXML2 MODULES MPS NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11
XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $EMACSDATA: /tmp/.mount_emacsCDP179/share/emacs/31.0.50/etc
  value of $EMACSDOC: /tmp/.mount_emacsCDP179/share/emacs/31.0.50/etc
  value of $EMACSLOADPATH: /tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp
  value of $EMACSPATH:
/tmp/.mount_emacsCDP179/libexec/emacs/31.0.50/x86_64-pc-linux-gnu
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=fcitx
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  evil-vimish-fold-mode: t
  vimish-fold-mode: t
  diff-hl-mode: t
  flycheck-posframe-mode: t
  flycheck-mode: t
  ligature-mode: t
  whitespace-mode: t
  electric-pair-mode: t
  hl-todo-mode: t
  dtrt-indent-mode: t
  projectile-mode: t
  tempel-abbrev-mode: t
  company-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  hl-line-mode: t
  display-line-numbers-mode: t
  windmove-mode: t
  recentf-mode: t
  pixel-scroll-precision-mode: t
  server-mode: t
  winner-mode: t
  global-auto-revert-mode: t
  save-place-mode: t
  vertico-mode: t
  which-key-mode: t
  global-evil-visualstar-mode: t
  evil-visualstar-mode: t
  evil-snipe-override-mode: t
  evil-snipe-override-local-mode: t
  evil-owl-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  evil-commentary-mode: t
  evil-mode: t
  evil-local-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  minibuffer-regexp-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/yikai/.emacs.d/lib/which-key/which-key hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/which-key
/home/yikai/.emacs.d/lib/transient/lisp/transient hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/transient
/home/yikai/.emacs.d/lib/editorconfig/editorconfig hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/editorconfig
/home/yikai/.emacs.d/lib/editorconfig/editorconfig-tools hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/editorconfig-tools
/home/yikai/.emacs.d/lib/editorconfig/editorconfig-fnmatch hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/editorconfig-fnmatch
/home/yikai/.emacs.d/lib/editorconfig/editorconfig-core hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/editorconfig-core
/home/yikai/.emacs.d/lib/editorconfig/editorconfig-core-handle hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/editorconfig-core-handle
/home/yikai/.emacs.d/lib/editorconfig/editorconfig-conf-mode hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/editorconfig-conf-mode
/home/yikai/.emacs.d/lib/compat/compat hides
/tmp/.mount_emacsCDP179/share/emacs/31.0.50/lisp/emacs-lisp/compat

Features:
(shadow sort mail-extr emacsbug evil-vimish-fold vimish-fold f s
git-gutter-fringe fringe-helper git-gutter evil-collection-diff-hl
diff-hl evil-collection-log-view log-view evil-collection-vc-dir vc-dir
ewoc vc vc-dispatcher flycheck-posframe posframe flycheck-google-cpplint
evil-collection-flycheck flycheck ligature whitespace elec-pair hl-todo
dtrt-indent company-keywords company-dabbrev-code company-dabbrev
company-files projectile evil-collection-grep grep ibuf-ext
evil-collection-ibuffer ibuffer ibuffer-loaddefs url-queue
pr-review-search tempel company-abbrev company-emoji company-emoji-list
company-capf company bazel evil-collection-xref xref which-func
testcover evil-collection-edebug edebug evil-collection-debug debug
backtrace evil-collection-python python treesit project
evil-collection-imenu imenu ffap cc-mode cc-fonts cc-guess cc-menus
cc-cmds textsec uni-scripts idna-mapping ucs-normalize uni-confusable
textsec-check pr-review pr-review-render shr pixel-fill kinsoku url-file
svg dom pr-review-action magit-diff smerge-mode diff
evil-collection-diff-mode diff-mode track-changes git-commit
evil-collection-log-edit log-edit message sendmail yank-media
evil-collection-dired dired dired-loaddefs rfc822 mml mml-sec
evil-collection-epa epa derived epg rfc6068 epg-config gnus-util
mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader pcvs-util
add-log magit-core magit-autorevert magit-margin magit-transient
magit-process with-editor magit-mode transient browse-url benchmark
magit-git magit-base crm pr-review-input evil-collection-markdown-mode
markdown-mode evil-collection-outline noutline outline mule-util pulse
mail-utils network-stream url-cache hl-line display-line-numbers
pr-review-notification pr-review-listview pr-review-api ghub-graphql
treepy gsexp ghub url-http mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr url-gw nsm url-auth url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap let-alist gnutls puny pr-review-common
evil-collection-magit-section magit-section dash windmove cl-print igc
vertico-directory orderless recentf tree-widget wid-edit
evil-collection-consult consult cursor-sensor help-fns time pixel-scroll
cua-base auth-source-pass url-parse url-vars server fcitx dbus xml
winner evil-collection-vterm vterm evil-collection-bookmark bookmark pp
face-remap evil-collection-compile compile text-property-search
evil-collection-term term disp-table ehelp find-func vterm-module
term/xterm xterm cc-styles cc-align cc-engine cc-vars cc-defs
google-c-style midnight autorevert filenotify saveplace tramp-cache
time-stamp tramp-sh tramp trampver tramp-integration files-x
tramp-message tramp-compat xdg shell pcomplete evil-collection-comint
comint ansi-osc parse-time iso8601 time-date auth-source eieio
eieio-core password-cache json map ansi-color tramp-loaddefs cus-load
evil-collection-vertico vertico compat solarized-light-theme
solarized-theme solarized solarized-faces color
evil-collection-which-key which-key fringe-scale switch-buffer-functions
evil-visualstar evil-snipe evil-owl format-spec evil-surround
evil-commentary evil-commentary-integration
evil-collection-tabulated-list evil-collection-tab-bar
evil-collection-simple evil-collection-replace
evil-collection-process-menu evil-collection-kmacro evil-collection-info
evil-collection-indent evil-collection-help evil-collection-elisp-mode
evil-collection-eldoc evil-collection-buff-menu evil-collection annalist
evil evil-integration evil-maps evil-commands evil-digraphs reveal
evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core advice evil-common
thingatpt rect evil-vars ring edmacro kmacro byte-opt delight comp-run
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core
yaml-mode-autoloads xonsh-mode-autoloads with-editor-autoloads
which-key-autoloads wgrep-autoloads vterm-autoloads vimrc-mode-autoloads
vimish-fold-autoloads vertico-autoloads treesit-auto-autoloads
treepy-autoloads transient-autoloads tempel-autoloads
switch-buffer-functions-autoloads suggest-autoloads sudo-edit-autoloads
spinner-autoloads solarized-theme-autoloads s-autoloads
rust-mode-autoloads rg-autoloads rainbow-mode-autoloads pydoc-autoloads
protobuf-mode-autoloads projectile-autoloads pr-review-autoloads
posframe-autoloads popup-autoloads pkg-info-autoloads php-mode-autoloads
package-lint-autoloads org2elcomment-autoloads org-tree-slide-autoloads
orderless-autoloads markdown-mode-autoloads magit-autoloads lv-autoloads
lua-mode-autoloads lsp-pyright-autoloads lsp-mode-autoloads
lsp-haskell-autoloads loop-autoloads llama-autoloads ligature-autoloads
kotlin-mode-autoloads just-mode-autoloads jsonnet-mode-autoloads
jinx-autoloads jinja2-mode-autoloads ht-autoloads hl-todo-autoloads
haskell-mode-autoloads groovy-mode-autoloads gptel-autoloads
goto-chg-autoloads google-c-style-autoloads go-mode-autoloads
gn-mode-autoloads git-link-autoloads git-gutter-fringe-autoloads
git-gutter-autoloads ghub-autoloads fringe-helper-autoloads
flycheck-posframe-autoloads flycheck-package-autoloads
flycheck-google-cpplint-autoloads flycheck-autoloads fish-mode-autoloads
fcitx-autoloads f-autoloads explain-pause-mode-autoloads
expand-region-autoloads exec-path-from-shell-autoloads
evil-visualstar-autoloads evil-vimish-fold-autoloads
evil-surround-autoloads evil-snipe-autoloads evil-owl-autoloads
evil-commentary-autoloads evil-collection-autoloads evil-autoloads
epl-autoloads epkg-autoloads embark-autoloads emacsql-autoloads
emacs-fringe-scale-autoloads editorconfig-autoloads
ebuild-mode-autoloads dumb-jump-autoloads dtrt-indent-autoloads
dockerfile-mode-autoloads diff-hl-autoloads devdocs-browser-autoloads
delight-autoloads dash-autoloads cuda-mode-autoloads copilot-autoloads
consult-flycheck-autoloads consult-autoloads compat-autoloads
company-emoji-autoloads company-autoloads codeium-autoloads cl-macs
cmake-mode-autoloads closql-autoloads bpftrace-mode-autoloads
borg-autoloads bazel-autoloads avy-autoloads annalist-autoloads
add-node-modules-path-autoloads borg loaddefs-gen generate-lisp-file
lisp-mnt radix-tree pcase info comp cl-seq comp-cstr cl-extra help-mode
comp-common warnings icons subr-x rx gv cl-loaddefs cl-lib bytecomp
byte-compile rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd touch-screen tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo gtk x-toolkit xinput2 x multi-tty move-toolbar
make-network-process native-compile mps emacs)

Memory information:
((conses 24 0 0) (symbols 56 0 0) (strings 40 0 0) (string-bytes 1 0)
 (vectors 24 0) (vector-slots 8 0 0) (floats 24 0 0)
 (intervals 64 0 0) (buffers 1000 0))

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Pip Cet via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Thursday, November 28th, 2024 at 13:18, Yikai Zhao <yikai@z1k.dev> wrote:
> I encountered this bug while testing the mps (scratch/igc) branch. I
> cannot reproduce this with the current master branch.

Can you reproduce it on the scratch/igc branch, but compiled without mps support?

That might help us narow it down to the MPS code or some unrelated change on that branch.

Pip

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Yikai Zhao]

I can confirm that the issue does not happen on the scratch/igc branch
without mps support
(or at least much less frequent)

On Thu, Nov 28, 2024 at 9:33 PM Pip Cet <pipcet@protonmail.com> wrote:
>
> On Thursday, November 28th, 2024 at 13:18, Yikai Zhao <yikai@z1k.dev> wrote:
> > I encountered this bug while testing the mps (scratch/igc) branch. I
> > cannot reproduce this with the current master branch.
>
> Can you reproduce it on the scratch/igc branch, but compiled without mps support?
>
> That might help us narow it down to the MPS code or some unrelated change on that branch.
>
> Pip

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Gerd Möllmann]

Yikai Zhao <yikai@z1k.dev> writes:

> I can confirm that the issue does not happen on the scratch/igc branch
> without mps support
> (or at least much less frequent)
>
> On Thu, Nov 28, 2024 at 9:33 PM Pip Cet <pipcet@protonmail.com> wrote:
>>
>> On Thursday, November 28th, 2024 at 13:18, Yikai Zhao <yikai@z1k.dev> wrote:
>> > I encountered this bug while testing the mps (scratch/igc) branch. I
>> > cannot reproduce this with the current master branch.
>>
>> Can you reproduce it on the scratch/igc branch, but compiled without mps support?
>>
>> That might help us narow it down to the MPS code or some unrelated change on that branch.
>>
>> Pip

Not sure if that is used in your build, but in x_display_info (xterm.h)
I see a number of struct frame pointers that are not fixed in fix_frame,
starting with

  struct frame *x_focus_frame;

And if it's not that display info that is being used, I'd bet a small
amount that whatever is actually used (pgtk_display_info?) has a similar
problems.

(Can't fix this myself, sorry, I only have macOS.)

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

Wanted to get Helmut onboard, in case he's interested, but forgot to add
him in CC. Now he is.

> Yikai Zhao <yikai@z1k.dev> writes:
>
>> I can confirm that the issue does not happen on the scratch/igc branch
>> without mps support
>> (or at least much less frequent)
>>
>> On Thu, Nov 28, 2024 at 9:33 PM Pip Cet <pipcet@protonmail.com> wrote:
>>>
>>> On Thursday, November 28th, 2024 at 13:18, Yikai Zhao <yikai@z1k.dev> wrote:
>>> > I encountered this bug while testing the mps (scratch/igc) branch. I
>>> > cannot reproduce this with the current master branch.
>>>
>>> Can you reproduce it on the scratch/igc branch, but compiled without mps support?
>>>
>>> That might help us narow it down to the MPS code or some unrelated change on that branch.
>>>
>>> Pip
>
> Not sure if that is used in your build, but in x_display_info (xterm.h)
> I see a number of struct frame pointers that are not fixed in fix_frame,
> starting with
>
>   struct frame *x_focus_frame;
>
> And if it's not that display info that is being used, I'd bet a small
> amount that whatever is actually used (pgtk_display_info?) has a similar
> problems.
>
> (Can't fix this myself, sorry, I only have macOS.)

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Helmut Eller]

On Fri, Nov 29 2024, Gerd Möllmann wrote:

>> Not sure if that is used in your build, but in x_display_info (xterm.h)
>> I see a number of struct frame pointers that are not fixed in fix_frame,
>> starting with
>>
>>   struct frame *x_focus_frame;
>>
>> And if it's not that display info that is being used, I'd bet a small
>> amount that whatever is actually used (pgtk_display_info?) has a similar
>> problems.
>>
>> (Can't fix this myself, sorry, I only have macOS.)

I think the x_display_info struct (I guess usually only one exists) is
allocated in x_term_init (or pgtk_term_init) with igc_xzalloc_ambig.  So
theoretically it doesn't need to be traced.

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Fri, Nov 29 2024, Gerd Möllmann wrote:
>
>>> Not sure if that is used in your build, but in x_display_info (xterm.h)
>>> I see a number of struct frame pointers that are not fixed in fix_frame,
>>> starting with
>>>
>>>   struct frame *x_focus_frame;
>>>
>>> And if it's not that display info that is being used, I'd bet a small
>>> amount that whatever is actually used (pgtk_display_info?) has a similar
>>> problems.
>>>
>>> (Can't fix this myself, sorry, I only have macOS.)
>
> I think the x_display_info struct (I guess usually only one exists) is
> allocated in x_term_init (or pgtk_term_init) with igc_xzalloc_ambig.  So
> theoretically it doesn't need to be traced.

Then we're good, sorry for the noise.

What made me suspicious is that we have this in fix_frame:

	Lisp_Object *nle = &FRAME_DISPLAY_INFO (f)->name_list_element;
	IGC_FIX12_OBJ (ss, nle);

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Pip Cet via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

On Saturday, November 30th, 2024 at 10:55, Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
> Helmut Eller eller.helmut@gmail.com writes:
> 
> > On Fri, Nov 29 2024, Gerd Möllmann wrote:
> > 
> > > > Not sure if that is used in your build, but in x_display_info (xterm.h)
> > > > I see a number of struct frame pointers that are not fixed in fix_frame,
> > > > starting with
> > > > 
> > > > struct frame *x_focus_frame;
> > > > 
> > > > And if it's not that display info that is being used, I'd bet a small
> > > > amount that whatever is actually used (pgtk_display_info?) has a similar
> > > > problems.
> > > > 
> > > > (Can't fix this myself, sorry, I only have macOS.)
> > 
> > I think the x_display_info struct (I guess usually only one exists) is
> > allocated in x_term_init (or pgtk_term_init) with igc_xzalloc_ambig. So
> > theoretically it doesn't need to be traced.
> 
> 
> Then we're good, sorry for the noise.

So it turns out X input method handling is somewhat complicated!

I've tried installing fcitx, but it seems to be working the same here with and without MPS.

It would help to establish the value of x-gtk-use-native-input, since that determines whether we use the GTK or X method for communicating with fcitx.

I've attached a patch which logs some debugging info to stderr (because displaying messages using X while debugging X code is a bad idea, IME). If you could apply it and reproduce the output around a keypress that's handled incorrectly, that might help us track this down.

Pip

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-fcitx.patch --]
[-- Type: text/x-patch; name=0002-fcitx.patch, Size: 1923 bytes --]

diff --git a/src/xterm.c b/src/xterm.c
index ebcd3a786e2..066d3828bcf 100644
--- a/src/xterm.c
+++ b/src/xterm.c
@@ -6914,6 +6914,7 @@ x_display_info_for_display (Display *dpy)
     if (dpyinfo->display == dpy)
       return dpyinfo;
 
+  fprintf(stderr, "couldn't find display info for %p\n", dpy);
   return 0;
 }
 
@@ -13027,6 +13028,7 @@ x_dnd_begin_drag_and_drop (struct frame *f, Time time, Atom xaction,
       event_display
 	= x_display_info_for_display (next_event.xany.display);
 
+      fprintf(stderr, "event_display %p\n", event_display);
       if (event_display)
 	{
 #ifdef HAVE_X_I18N
@@ -17913,7 +17915,11 @@ x_filter_event (struct x_display_info *dpyinfo, XEvent *event)
       && !dpyinfo->prefer_native_input)
     {
 #endif
-      return XFilterEvent (event, f1 ? FRAME_X_WINDOW (f1) : None);
+      bool result;
+      result = XFilterEvent (event, f1 ? FRAME_X_WINDOW (f1) : None);
+      fprintf(stderr, "result %d (not GTK) for event %d, frame %p dpyinfo %p\n", result, event->type,
+	      f1, dpyinfo);
+      return result;
 #ifdef USE_GTK
     }
   else if (f1 && (event->type == KeyPress
@@ -17941,9 +17947,13 @@ x_filter_event (struct x_display_info *dpyinfo, XEvent *event)
 	   exercise the wire to make pselect return.  */
 	XNoOp (FRAME_X_DISPLAY (f1));
 
+      fprintf(stderr, "result %d for event %d, frame %p dpyinfo %p\n", result, event->type,
+	      f1, dpyinfo);
       return result;
     }
 
+  fprintf(stderr, "result 0 (no frame) for event %d, frame %p dpyinfo %p\n", event->type,
+	  f1, dpyinfo);
   return 0;
 #endif
 }
@@ -17965,6 +17975,7 @@ event_handler_gdk (GdkXEvent *gxev, GdkEvent *ev, gpointer data)
 
       dpyinfo = x_display_info_for_display (xev->xany.display);
 
+      fprintf(stderr, "dpyinfo %p\n", dpyinfo);
 #ifdef HAVE_X_I18N
       /* Filter events for the current X input method.
          GTK calls XFilterEvent but not for key press and release,

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Gerd Möllmann]

Pip Cet <pipcet@protonmail.com> writes:

> On Saturday, November 30th, 2024 at 10:55, Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
>> Helmut Eller eller.helmut@gmail.com writes:
>> 
>> > On Fri, Nov 29 2024, Gerd Möllmann wrote:
>> > 
>> > > > Not sure if that is used in your build, but in x_display_info (xterm.h)
>> > > > I see a number of struct frame pointers that are not fixed in fix_frame,
>> > > > starting with
>> > > > 
>> > > > struct frame *x_focus_frame;
>> > > > 
>> > > > And if it's not that display info that is being used, I'd bet a small
>> > > > amount that whatever is actually used (pgtk_display_info?) has a similar
>> > > > problems.
>> > > > 
>> > > > (Can't fix this myself, sorry, I only have macOS.)
>> > 
>> > I think the x_display_info struct (I guess usually only one exists) is
>> > allocated in x_term_init (or pgtk_term_init) with igc_xzalloc_ambig. So
>> > theoretically it doesn't need to be traced.
>> 
>> 
>> Then we're good, sorry for the noise.
>
> So it turns out X input method handling is somewhat complicated!
>
> I've tried installing fcitx, but it seems to be working the same here with and without MPS.
>
> It would help to establish the value of x-gtk-use-native-input, since that determines whether we use the GTK or X method for communicating with fcitx.
>
> I've attached a patch which logs some debugging info to stderr
> (because displaying messages using X while debugging X code is a bad
> idea, IME). If you could apply it and reproduce the output around a
> keypress that's handled incorrectly, that might help us track this
> down.
>
> Pip

Searching for "closure" and "user_data" turns up this in gtkutil.c:

  static void
  xg_im_context_commit (GtkIMContext *imc, gchar *str,
                        gpointer user_data)
  {
    struct frame *f = get_glib_user_data (user_data);

That's a Gtk signal handler, or whatever they are called, which
gets set, also in gtkutil.c

  g_signal_connect_data (G_OBJECT (imc), "commit",
			 G_CALLBACK (xg_im_context_commit),
			 glib_user_data (f), free_glib_user_data,
			 G_CONNECT_DEFAULT);

Looks to me like a struct frame * might be "hidden" by this in some Gtk
data structure so that it can be passed to the handler at some point.

Don't know if that's relevant.

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Pip Cet <pipcet@protonmail.com> writes:
>
>> On Saturday, November 30th, 2024 at 10:55, Gerd Möllmann <gerd.moellmann@gmail.com> wrote:
>>> Helmut Eller eller.helmut@gmail.com writes:
>>> 
>>> > On Fri, Nov 29 2024, Gerd Möllmann wrote:
>>> > 
>>> > > > Not sure if that is used in your build, but in x_display_info (xterm.h)
>>> > > > I see a number of struct frame pointers that are not fixed in fix_frame,
>>> > > > starting with
>>> > > > 
>>> > > > struct frame *x_focus_frame;
>>> > > > 
>>> > > > And if it's not that display info that is being used, I'd bet a small
>>> > > > amount that whatever is actually used (pgtk_display_info?) has a similar
>>> > > > problems.
>>> > > > 
>>> > > > (Can't fix this myself, sorry, I only have macOS.)
>>> > 
>>> > I think the x_display_info struct (I guess usually only one exists) is
>>> > allocated in x_term_init (or pgtk_term_init) with igc_xzalloc_ambig. So
>>> > theoretically it doesn't need to be traced.
>>> 
>>> 
>>> Then we're good, sorry for the noise.
>>
>> So it turns out X input method handling is somewhat complicated!
>>
>> I've tried installing fcitx, but it seems to be working the same here with and without MPS.
>>
>> It would help to establish the value of x-gtk-use-native-input, since that determines whether we use the GTK or X method for communicating with fcitx.
>>
>> I've attached a patch which logs some debugging info to stderr
>> (because displaying messages using X while debugging X code is a bad
>> idea, IME). If you could apply it and reproduce the output around a
>> keypress that's handled incorrectly, that might help us track this
>> down.
>>
>> Pip
>
> Searching for "closure" and "user_data" turns up this in gtkutil.c:
>
>   static void
>   xg_im_context_commit (GtkIMContext *imc, gchar *str,
>                         gpointer user_data)
>   {
>     struct frame *f = get_glib_user_data (user_data);
>
> That's a Gtk signal handler, or whatever they are called, which
> gets set, also in gtkutil.c
>
>   g_signal_connect_data (G_OBJECT (imc), "commit",
> 			 G_CALLBACK (xg_im_context_commit),
> 			 glib_user_data (f), free_glib_user_data,
> 			 G_CONNECT_DEFAULT);
>
> Looks to me like a struct frame * might be "hidden" by this in some Gtk
> data structure so that it can be passed to the handler at some point.
>
> Don't know if that's relevant.

It probably isn't relevant because of this

  #ifdef HAVE_MPS
  void free_glib_user_data (gpointer data, GClosure *closure)
  {
    igc_xfree (data);
  }
  #else
  void free_glib_user_data (gpointer data, GClosure *closure)
  {
    return;
  }
  #endif

Don't know where the allocation takes place.

I should shut up, I guess :-).

bug#74590: 31.0.50 [scratch/igc branch]; key input sometimes skip fcitx input method preedit box

[Pip Cet via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> It probably isn't relevant because of this
>
>   #ifdef HAVE_MPS
>   void free_glib_user_data (gpointer data, GClosure *closure)
>   {
>     igc_xfree (data);
>   }
>   #else
>   void free_glib_user_data (gpointer data, GClosure *closure)
>   {
>     return;
>   }
>   #endif
>
> Don't know where the allocation takes place.

It's this code in gtkutil.h:

#ifdef HAVE_MPS
INLINE gpointer
glib_user_data (void *o)
{
  gpointer p = igc_xzalloc_ambig (sizeof (o));
  memcpy (p, &o, sizeof (o));
  return p;
}

INLINE void *
get_glib_user_data (gpointer p)
{
  return *(void **)p;
}
#else
INLINE gpointer
glib_user_data (void *o)
{
  return (gpointer)o;
}

INLINE void *
get_glib_user_data (gpointer p)
{
  return (void *)p;
}
#endif

Does that look correct to you?

My understanding is that the GTK input method code is only used if
x_gtk_use_native_input is true (which we'll have to wait for the OP to
confirm or deny), but xg_create_frame_widgets always calls
gtk_im_multicontext_new, so the problem might be in the GTK code...

Pip